Modern open source alternative to LWT, lingq etc

General discussion about learning languages
languagedev
White Belt
Posts: 12
Joined: Fri Nov 26, 2021 3:32 pm
Languages: English (N), Spanish(C1), French(B1)
x 20

Re: Modern open source alternative to LWT, lingq etc

Postby languagedev » Fri Nov 26, 2021 8:56 pm

Beli Tsar wrote:Decent conjugation/declension and being able to download lists of learned words in their dictionary form (e.g. for Anki) with relevant sentences attached. That would take everything a real step beyond the main offerings.

But, obviously, that's hard in many languages.


That will be quite difficult, especially if I design the app in a way that people can add their own languages. But I will see what I can do. What exactly do you mean by decent conjugation/declension? Do you mean that words like learn/learned/learns would be grouped together? There are natural language processing programs out there that I can look into if this is something people want.

In the beginning, I think it will treat all words as separate (due to complexity) but I may be able to add in the feature later for more popular languages.

Edit: I plan on adding in export to Anki support.
1 x
Language enthusiast, co-founder of https://tryalexandria.com/ - A modern, free and open-source alternative to LWT

languagedev
White Belt
Posts: 12
Joined: Fri Nov 26, 2021 3:32 pm
Languages: English (N), Spanish(C1), French(B1)
x 20

Re: Modern open source alternative to LWT, lingq etc

Postby languagedev » Fri Nov 26, 2021 9:01 pm

rdearman wrote:
languagedev wrote:
TSS42 wrote:Completely cloud-based, no local installation.
Text marking and labelling.
Easy dictionary integration out of the box.
Doesn't go mad and marks everything as new, like 'yes', 'no', 'but',...


I was wondering between using a local version and a cloud version. The cloud version is much more interesting in a lot of ways, e.g. shared translations. The problem I see with it is that if it became very popular in the future, I would have to pay server and database costs. .

That is nothing compared to the costs you are going to pay for copyright infringement when somebody uploads Harry Potter, Dune, and a thousand other copyrighted works. :)

Edit, LWT will let you export to anki as well. The lwt guy used to post here, and I think it is still in active development.

Rather than a cloud installation, I think you should go for a docker instance.


Haha don't worry, all texts would be private (which should be enough to avoid copyright claims right?). I was thinking of having a global database of translations so you could see how other people had translated a particular word, while also having the dictionary option.

I have limited experience with Docker, I've only used it as a development environment, how would a docker instance work? Would that mean anybody could download the image and run it locally? If so that would be a high priority for me!

I thought LWT and another program disappeared overnight last year. It's a great project, but I think it's quite difficult for an average person to set up and it's terrible on mobile. I just want something I can use for myself and I would like to share it with others too.
3 x
Language enthusiast, co-founder of https://tryalexandria.com/ - A modern, free and open-source alternative to LWT

languagedev
White Belt
Posts: 12
Joined: Fri Nov 26, 2021 3:32 pm
Languages: English (N), Spanish(C1), French(B1)
x 20

Re: Modern open source alternative to LWT, lingq etc

Postby languagedev » Fri Nov 26, 2021 9:03 pm

TSS42 wrote:
languagedev wrote:When you say dictionary integration, do you mean that the user can define any dictionary that they want to use, like in LWT?


Not every possible online dictionary obviously, but at least the more popular ones that have a standard API.

languagedev wrote:How would you handle making words as new? What if someone is new to a language and everything is new to them? Or would you prefer that words not be marked as new, only words that you choose to translate are highlighted?


Yeah, the latter. I was thinking like just two highlighting colours: a) previously encountered b) things you yourself manually mark as "hard" or "unknown". Everything else is un-highlighted.


I like that idea. It would also make the database smaller keeping costs down :)
0 x
Language enthusiast, co-founder of https://tryalexandria.com/ - A modern, free and open-source alternative to LWT

daniel1234
White Belt
Posts: 11
Joined: Sat Nov 20, 2021 6:02 pm
Languages: Spanish (Native), English (C2)
x 33

Re: Modern open source alternative to LWT, lingq etc

Postby daniel1234 » Sat Nov 27, 2021 12:39 am

languagedev

The developer of https://elreader.com/ stopped publishing on the Social Media accounts and did not reply to my emails at all since several months ago. There are no new features and numerous bug reports filled by email were ignored. :(

He might not have time at all. That's why it is needed to Open Source any project to allow contributions from other developers.

The Anki model is a good one. You can use everything offline, but the developer has a revenue from the Cloud service paid by iPhone users.

How about being able to use a limited amount of Megabytes on the cloud (e.g. 5 MiB) and having to pay for more ? That could make the project self-sustaining. I would like to be a beta-tester. 8-)

You can write me at : REDACTED.

Admin Edit. Just PM the user for the email address. Publication of your email on a website is a very bad idea.
3 x

User avatar
SpanishInput
Yellow Belt
Posts: 97
Joined: Sun Sep 26, 2021 3:11 pm
Location: Ecuador
Languages: Spanish (N), English (C2), Mandarin (HSK 5)
x 469

Re: Modern open source alternative to LWT, lingq etc

Postby SpanishInput » Sat Nov 27, 2021 8:07 pm

rdearman wrote:That is nothing compared to the costs you are going to pay for copyright infringement when somebody uploads Harry Potter, Dune, and a thousand other copyrighted works. :)


I sometimes wonder why that-other-famous-website-similar-to-LWT hasn't ever gotten in trouble yet. AFAIK, their policy is to remove content only when requested. They do not actively monitor for copyrighted content. This is pretty much where YouTube was during its first years, and we know YouTube was forced to create complex systems to clean up the place... And they still can't clean it up completely.
2 x

User avatar
rdearman
Site Admin
Posts: 7231
Joined: Thu May 14, 2015 4:18 pm
Location: United Kingdom
Languages: English (N)
Language Log: viewtopic.php?f=15&t=1836
x 23120
Contact:

Re: Modern open source alternative to LWT, lingq etc

Postby rdearman » Sat Nov 27, 2021 8:31 pm

SpanishInput wrote:
rdearman wrote:That is nothing compared to the costs you are going to pay for copyright infringement when somebody uploads Harry Potter, Dune, and a thousand other copyrighted works. :)


I sometimes wonder why that-other-famous-website-similar-to-LWT hasn't ever gotten in trouble yet. AFAIK, their policy is to remove content only when requested. They do not actively monitor for copyrighted content. This is pretty much where YouTube was during its first years, and we know YouTube was forced to create complex systems to clean up the place... And they still can't clean it up completely.

The "get out of jail" card they are playing is to make YOU responsible for the copyright infringement. If you look at the Terms-of-Service.

Rights Clearances and Assignments. You will be solely responsible for obtaining all rights clearances with respect to all Content created or provided by User

So I suspect that a lot of their users don't realize if that website were to get an audit done by the lawyers of a bunch of publishers, then they the user will be done for copyright theft. I seriously doubt the company will get off Scot free as well, but it depends on what country the service is hosted and registered in. If you were in a country, say.... Canada? Which doesn't have the DMCA like the USA, then you might get a bit of leniency. Don't know, it is a problem for the lawyers.

But for the sake of one lone developer, I would seriously not host other people's stuff, you'd just be asking to be sued. Not to mention the porn uploaders, the scammers, and all the other riff-raff I spend my time deleting here.
9 x
: 0 / 150 Read 150 books in 2024

My YouTube Channel
The Autodidactic Podcast
My Author's Newsletter

I post on this forum with mobile devices, so excuse short msgs and typos.

Cainntear
Black Belt - 3rd Dan
Posts: 3467
Joined: Thu Jul 30, 2015 11:04 am
Location: Scotland
Languages: English(N)
Advanced: French,Spanish, Scottish Gaelic
Intermediate: Italian, Catalan, Corsican
Basic: Welsh
Dabbling: Polish, Russian etc
x 8656
Contact:

Re: Modern open source alternative to LWT, lingq etc

Postby Cainntear » Sat Nov 27, 2021 11:44 pm

SpanishInput wrote:
rdearman wrote:That is nothing compared to the costs you are going to pay for copyright infringement when somebody uploads Harry Potter, Dune, and a thousand other copyrighted works. :)


I sometimes wonder why that-other-famous-website-similar-to-LWT hasn't ever gotten in trouble yet. AFAIK, their policy is to remove content only when requested. They do not actively monitor for copyrighted content. This is pretty much where YouTube was during its first years, and we know YouTube was forced to create complex systems to clean up the place... And they still can't clean it up completely.

YouTube weren't 100% forced -- they were protect by the DCMA and the concept of "safe harbor".

I imagine part of the motivation for Content ID was to prevent anyone campaigning for a change in the law -- at the time the music, TV and movie industry were getting angry about the amount of infringing content that was getting reuploaded whenever it was taken down, and the copyright holders had to constantly keep monitoring for infringing works.

The talk at the time was of mandating something like Content ID, although that still didn't exist. The objection was that it would be unmanageable for smaller sites.

By launching it voluntarily, Google may have prevented new laws that would really have messed up a lot of sites... and they also protected their advertising revenue... maybe even increased it.
3 x

Beli Tsar
Green Belt
Posts: 384
Joined: Mon Oct 22, 2018 3:59 pm
Languages: English (N), Ancient Greek (intermediate reading), Latin (Beginner) Farsi (Beginner), Biblical Hebrew (Beginner)
Language Log: https://forum.language-learners.org/vie ... =15&t=9548
x 1294

Re: Modern open source alternative to LWT, lingq etc

Postby Beli Tsar » Mon Nov 29, 2021 8:43 pm

languagedev wrote:
Beli Tsar wrote:Decent conjugation/declension and being able to download lists of learned words in their dictionary form (e.g. for Anki) with relevant sentences attached. That would take everything a real step beyond the main offerings.

But, obviously, that's hard in many languages.


That will be quite difficult, especially if I design the app in a way that people can add their own languages. But I will see what I can do. What exactly do you mean by decent conjugation/declension? Do you mean that words like learn/learned/learns would be grouped together?

Yes, exactly. And for lots of languages, 'I learn, you learn, she learns, we learn' etc. And yes, it would be complex. Some individual languages possess their own tools that could help (e.g. Latin has Whitaker's words) but presumably you aren't wanting to engineer a different solution for each language, especially minority ones like Latin.
But this is genuinely what I miss in the existing web services.
3 x
: 0 / 50 1/2 Super Challenge - Latin Reading
: 0 / 50 1/2 Super Challenge - Latin 'Films'

languagedev
White Belt
Posts: 12
Joined: Fri Nov 26, 2021 3:32 pm
Languages: English (N), Spanish(C1), French(B1)
x 20

Re: Modern open source alternative to LWT, lingq etc

Postby languagedev » Wed Dec 01, 2021 11:38 am

Beli Tsar wrote:
languagedev wrote:
Beli Tsar wrote:Decent conjugation/declension and being able to download lists of learned words in their dictionary form (e.g. for Anki) with relevant sentences attached. That would take everything a real step beyond the main offerings.

But, obviously, that's hard in many languages.


That will be quite difficult, especially if I design the app in a way that people can add their own languages. But I will see what I can do. What exactly do you mean by decent conjugation/declension? Do you mean that words like learn/learned/learns would be grouped together?

Yes, exactly. And for lots of languages, 'I learn, you learn, she learns, we learn' etc. And yes, it would be complex. Some individual languages possess their own tools that could help (e.g. Latin has Whitaker's words) but presumably you aren't wanting to engineer a different solution for each language, especially minority ones like Latin.
But this is genuinely what I miss in the existing web services.


I agree, that would be a great addition. Once the core app is working I might be able to add something like that for a few base 'supported' languages. Thanks very much for the suggestion!
1 x
Language enthusiast, co-founder of https://tryalexandria.com/ - A modern, free and open-source alternative to LWT

Cainntear
Black Belt - 3rd Dan
Posts: 3467
Joined: Thu Jul 30, 2015 11:04 am
Location: Scotland
Languages: English(N)
Advanced: French,Spanish, Scottish Gaelic
Intermediate: Italian, Catalan, Corsican
Basic: Welsh
Dabbling: Polish, Russian etc
x 8656
Contact:

Re: Modern open source alternative to LWT, lingq etc

Postby Cainntear » Wed Dec 01, 2021 3:17 pm

languagedev wrote:
Beli Tsar wrote:Decent conjugation/declension and being able to download lists of learned words in their dictionary form (e.g. for Anki) with relevant sentences attached. That would take everything a real step beyond the main offerings.

But, obviously, that's hard in many languages.


That will be quite difficult, especially if I design the app in a way that people can add their own languages. But I will see what I can do. What exactly do you mean by decent conjugation/declension? Do you mean that words like learn/learned/learns would be grouped together? There are natural language processing programs out there that I can look into if this is something people want.

It's not quite as complex as it might appear. At the basic level, it doesn't matter how different languages split their words into paradigms/lemmas specifically, the software design just needs to know that lemmas (lists of related words that are inflected variations on each other) exist.

It's not really a difficulty from a software perspective; the only real problem is getting the data. However, having the option to use it doesn't mean it's necessary (after all, there are many singleton words out there that aren't part of a lemma -- the, if, when etc etc), so having it integrated into the system doesn't exclude languages where lemmatisers and/or paradigm lists aren't readily available.
3 x


Return to “General Language Discussion”

Who is online

Users browsing this forum: No registered users and 2 guests