A language learners’ forum

improbablediscussion wrote:

rdearman wrote:EDIT 2: That last once had problems (only downloads the robots files, you need the offline cache to download books) The link below downloads all the french language books.

Code: Select all


wget -H -w 2 -m "http://www.gutenberg.org/robot/harvest?filetypes[]=epub.images&langs[]=fr"



Well, there's another 3200+ books for my Calibre library! Only the Jules Verne books and Voyage d'un Habitant de la Lune à Paris à la Fin du XVIIIe Siècle have gone on the e-reader so far, though. The hundreds of 19th century periodicals will remain on my desktop with the rest of the random stuff I'm hoarding.

There's actually a good amount of French language study materials in here, like dictionaries, readers, and such. Probably too outdated for normal purposes, but not a bad thing to have on hand if everything suddenly became inaccessible.

Actually I was looking at one of the French readers because it was also Dutch and I thought it had got mixed up when I imported them into Calibre, but the introduction was in Dutch, so this is obviously a reader published for Dutch people. Anyway, the first story was actually kind of cute, about a child putting their hand into the sugar jar while the parents were having tea.

Also, I put in filetypes[]=epub.images, therefore you're only getting books with images included. If you want ALL the books then you're better off using the text (txt) setting. But the full list of file types is:

html
txt
epub.images
epub.noimages
kindle.images
kindle.noimages
mp3

rdearman wrote:I don't know if you still can, but you used to be able to download the entire Gutenberg library as a single monolithic zip file.

Edit: got it.

Code: Select all


wget -w 2 -m http://www.gutenberg.org/robot/harvest



Hope you have a good connection, it is 75000+ books.

EDIT 2: That last once had problems (only downloads the robots files, you need the offline cache to download books) The link below downloads all the french language books.

Code: Select all


wget -H -w 2 -m "http://www.gutenberg.org/robot/harvest?filetypes[]=epub.images&langs[]=fr"



Thanks rdearman, this is awesome! I'm assuming we just exchange 'fr' for other language codes such as 'es' for Spanish, 'no' for Norwegian or 'nl' for Dutch if we want to download what's available in these languages, yeah?

That is correct.

That one-liner looks interesting. Is it OS specific? Do I need to install anything? (I'm using Ubuntu.)

Use need the wget command which is available for all is. You just need

For Ubuntu

See docs for other operating system.

http://wget.addictivecode.org/Frequentl ... l#download

You probably have wget by default on your machine, if you don't I recommend you upgrade your Intel Pentium machine to Ubuntu 2 or something.

Also, you may want to skip the images that usually make up the bulk of the file size for little apparent purpose, and you may want to download in a bunch of languages while ignoring the rest, so this example snippet should do the job:

mokibao wrote:Also, you may want to skip the images that usually make up the bulk of the file size for little apparent purpose, and you may want to download in a bunch of languages while ignoring the rest, so this example snippet should do the job:

Code: Select all

for lang in fr en de it es ru se el; do # French, English, German, Italian, Spanish, Russian, Swedish, Greek
    wget -H -w 2 -m "http://www.gutenberg.org/robot/harvest?filetypes[]=epub.noimages&langs[]=$lang" &
done
wait


Beautiful. Thanks!
I wonder what kind of hit Gutenberg's servers take from this.

Mostly they try to get people to download from public mirror sites. But in fact they allow you to set up your own mirror site if you want using "rsynch" so that you'll automatically get any book mirrored to your computer as soon as it is published. You'll want a lot of disk space for that sort of thing.

The servers your harvesting from using this command isn't actually the Gutenberg site itself, but rather a mirror specially dedicated to getting the downloads for robots. http://aleph.gutenberg.org/ <= robot harvest server.

We are having blackouts and such bad power surges that I have lost a fridge and a shortwave radio to them. I keep everything unplugged as much as possible, now, and depend on just the tech that can run off battery power.

I have been doing a lot of problem solving and research.

Goodereader.com is a Canadian company that ships worldwide. They offer some interesting Asian e-ink devices as well some common American made devices. There a lots and lots of video reviews of the their products.

Here is a budget, phone sized e-ink reader.
https://goodereader.com/blog/reviews/ha ... i-e-reader

A high-end giant screen model with full Google-playstore support.
https://goodereader.com/blog/product/onyx-boox-max-lumi

A phone that a lot of people are using as a small tablet. Beware that this phone is becoming increasingly incompatible with T-mobile service as t-mobile continues to discontinue 3G service while offering only the more common American bands for 4G and 5G. This phone does not have a practical and safe way to add Google playstore. It is a powerful device though and with alternative playstores including the Amazon Appstore, it is worth it to some people. You can download the app for Amazon Audible and download an extensive library of language audio resources like Pimsleur and Paul Noble. You can play back your drm protected content with no internet and on battery power. The e-ink screen last longer than a regular screen.

https://goodereader.com/blog/product/hi ... smartphone

An Amazon tablet has apps that allow you to download videos that can not be downloaded onto a desktop or laptop. Some subscription and library items stop working after a specified period of time, but I think any Udemy courses that have been purchased will play back indefinitely. A new 10 inch model is coming out, and the current version is really cheap, right now.

betterworldbooks.com is my favorite vendor for used language books.

A lot of nice solutions, Decidida. Too bad geoblocking is ruining everything. I can buy the physical thing, I've even bought the kindle. But buying ebooks is a problem.

But I'd say that on the eve of doom, nobody will knock on your door, if you use the last gigabytes of internet to get the package of 10000 books in your target language from a piracy site