Parallel audio and text with Audacity, WorkAudioBook and SpeechTexter

All about language programs, courses, websites and other learning resources
User avatar
tommus
Blue Belt
Posts: 957
Joined: Sat Jul 04, 2015 3:59 pm
Location: Kingston, ON, Canada
Languages: English (N), French (B2), Dutch (B2)
x 1937

Parallel audio and text with Audacity, WorkAudioBook and SpeechTexter

Postby tommus » Sun Dec 16, 2018 4:10 pm

It is often the case that the most interesting audio/video that you can find in your L2 does not have matching text. I have found that a combination of WorkAudioBook and SpeechTexter, although not ideal, provides a very effective method to listen to your interesting audio and read along with the text. There are obvious variations of this technique but here is the essence of what I find effective.

1. Find L2 audio or audio/video on the Internet or elsewhere that you find interesting.

2. Play it while recording the audio with something like Audacity.

3. Simultaneously read and capture the text using SpeechTexter.

4. Stop about every 4 minutes and copy/save the text from SpeechTexter to a text file. Let Audacity continue to record even though there is no audio.

5. Stop the process at convenient points (for example; chapters). Edit the spaces out of the audio and save it.

6. At this point, you will have a long text with no punctuation, some capital letters on proper nouns, but basically all run together. Not a real problem.

7. Load the text into a text editor that can also save it in HTML format (such as OpenOffice). Don't worry much about mistakes in the text.

8. For your video/audio, convert it to just audio using Audacity.

9. Open the HTML text in a browser where you have a pop-up translator to your L1 to use with any words you don't know.

10. Play the audio using WorkAudioBook with the Play Next [N] button. That gives you about one short sentence equivalent of audio (this is adjustable in WorkAudioBook).

11. Read along in your HTML text.

12. The importance of WorkAudioBook short segments is that you get to stop, think, digest, study, learn, look up, etc. And you can proceed at whatever pace you find to be effective for that particular material.

Variations:

1. If you already have pre-recorded L2 audio, play it in WorkAudioBook, one bit at a time, into SpeechTexter, capturing the text as described above. SpeechTexter does very well with these short audio segments.

2. SpeechTexter is based on Google Translate but it seems to do better, and I find the format to be better.

3. You could translate all your captured text with Google Translate, DeepL, etc, but it probably won't be too good because the text is all run together. I find the HTML plus pop-up translator on specific words to be more effective. It depends on your own reading level in your L2

So essentially, I find the combination of WorkAudioBook and SpeechTexter, working with short audio segments, to be very effective.

Audacity

WorkAudioBook

SpeechTexter
4 x
Dutch: 01 September -> 31 December 2020
Watch 1000 Dutch TV Series Videos : 40 / 1000

User avatar
MorkTheFiddle
Black Belt - 2nd Dan
Posts: 2114
Joined: Sat Jul 18, 2015 8:59 pm
Location: North Texas USA
Languages: English (N). Read (only) French and Spanish. Studying Ancient Greek. Studying a bit of Latin. Once studied Old Norse. Dabbled in Catalan, Provençal and Italian.
Language Log: https://forum.language-learners.org/vie ... 11#p133911
x 4824

Re: Parallel audio and text with Audacity, WorkAudioBook and SpeechTexter

Postby MorkTheFiddle » Fri Dec 21, 2018 10:24 pm

This idea seems extraordinarily useful, Tommus. Initially, I balked at step number 3, "Simultaneously read and capture the text using SpeechTexter." However, it turns out that I can record the audio and then play it back with a mike next to the speaker. At first, I wasn't sure it would work because Shazam can not pick up the audio from my speakers. Happily enough, SpeechTexter did pick up the sound.
Here is a sample: "On a déjà un petit peu abordé la question de la matière notamment quand on a fait l'épisode sur Mendeleïev rappelez-vous ...*" This is the very beginning of a blog by e-penser . (Be careful, this video is the full 24-minute blog.)
This is all I have tried so far, but I don't see why putting the mike next to a speaker can't work with DVDs and CDs and radio broadcasts as well.
Another possible use for SpeechTexter would be for recording unknown vocabulary items to look up later. Say you're reading a book and come across a word you don't know. Instead of underlining it or writing it down, you could just say it into your mike. SpeechTexter will record and save it, and you can look up the word later, at your leisure.
The only small down side I can see with this is that SpeechTexter needs Google Chrome to work.

* In English, "We've already talked a little bit about the subject, especially when we did the episode about Mendeleyev, remember ..."
1 x
Many things which are false are transmitted from book to book, and gain credit in the world. -- attributed to Samuel Johnson

User avatar
rdearman
Site Admin
Posts: 7231
Joined: Thu May 14, 2015 4:18 pm
Location: United Kingdom
Languages: English (N)
Language Log: viewtopic.php?f=15&t=1836
x 23125
Contact:

Re: Parallel audio and text with Audacity, WorkAudioBook and SpeechTexter

Postby rdearman » Fri Dec 21, 2018 10:39 pm

I used an android app called Voice Notebook (uses google speech to text) and switched the language to French. I tried it on a librox recording of Jule Verne and it picked up most of the words correctly. Still some errors though.
2 x
: 0 / 150 Read 150 books in 2024

My YouTube Channel
The Autodidactic Podcast
My Author's Newsletter

I post on this forum with mobile devices, so excuse short msgs and typos.

User avatar
tommus
Blue Belt
Posts: 957
Joined: Sat Jul 04, 2015 3:59 pm
Location: Kingston, ON, Canada
Languages: English (N), French (B2), Dutch (B2)
x 1937

Re: Parallel audio and text with Audacity, WorkAudioBook and SpeechTexter

Postby tommus » Fri Dec 21, 2018 11:02 pm

MorkTheFiddle wrote:Initially, I balked at step number 3, "Simultaneously read and capture the text using SpeechTexter." However, it turns out that I can record the audio and then play it back with a mike next to the speaker. At first, I wasn't sure it would work because Shazam can not pick up the audio from my speakers. Happily enough, SpeechTexter did pick up the sound.

My step number 3 is perhaps ambiguous. Steps 2 and 3 can be done at the same time.

2. Play it while recording the audio with something like Audacity.
3. Simultaneously read and capture the text using SpeechTexter.

This means that you are playing audio from a website, you are simultaneously recording it with your computer, you are simultaneously running SpeechTexter in another Chrome tab, SpeechTexter is simultaneously turning the audio into text, and you are simultaneously reading the text that is being produced. It may sound complicated but is actually quite easy to do.

As far as the microphone is concerned, I do not use one. With my desktop Windows 7/64 computer running Chrome, SpeechTexter in one tab can hear the audio from a website being received in another tab. You need to have the Playback and Recording devices in the Control Panel Sound to be the same. (something like Realtek High Definition Audio). This will also work with the audio coming from a built-in audio player such as Audacity or WorkAddioBook. That is much better than playing the audio through a speaker and receiving it with a microphone. That approach might be necessary if you have external audio such as a CD player. But even then, it probably would be best to record the audio to an audio file, and then play it internally with something like Audacity.
0 x
Dutch: 01 September -> 31 December 2020
Watch 1000 Dutch TV Series Videos : 40 / 1000

Doitsujin
Green Belt
Posts: 402
Joined: Sat Jul 18, 2015 6:21 pm
Languages: German (N)
x 801

Re: Parallel audio and text with Audacity, WorkAudioBook and SpeechTexter

Postby Doitsujin » Sat Dec 22, 2018 6:28 am

@tommus If you have an iOS/Android device, you might find some of the free ReadBeyond epub3 audio books helpful. (You'll need to install the free Menestrello app to listen to them.)

@rdearman The author of Menestrello has opensourced the tools that he used to synchronize LibriVox recordings with texts. (It's a Python/C library.)
3 x

User avatar
rdearman
Site Admin
Posts: 7231
Joined: Thu May 14, 2015 4:18 pm
Location: United Kingdom
Languages: English (N)
Language Log: viewtopic.php?f=15&t=1836
x 23125
Contact:

Re: Parallel audio and text with Audacity, WorkAudioBook and SpeechTexter

Postby rdearman » Sat Dec 22, 2018 10:16 am

Yes I know I've used that system before to create subtitle files from LibriVox books. I was just using this book because I needed some clear and understandable French audio to test the telephone app.
0 x
: 0 / 150 Read 150 books in 2024

My YouTube Channel
The Autodidactic Podcast
My Author's Newsletter

I post on this forum with mobile devices, so excuse short msgs and typos.

User avatar
MorkTheFiddle
Black Belt - 2nd Dan
Posts: 2114
Joined: Sat Jul 18, 2015 8:59 pm
Location: North Texas USA
Languages: English (N). Read (only) French and Spanish. Studying Ancient Greek. Studying a bit of Latin. Once studied Old Norse. Dabbled in Catalan, Provençal and Italian.
Language Log: https://forum.language-learners.org/vie ... 11#p133911
x 4824

Re: Parallel audio and text with Audacity, WorkAudioBook and SpeechTexter

Postby MorkTheFiddle » Sat Dec 22, 2018 8:16 pm

@Tommus
Thanks for the clarification, but I think I was the one who was muddled. I am unable to get SpeechTexter to hear another tab in Chrome, but that's okay. I don't feel like fiddling with it, and I don't want to waste anyone else's time troubleshooting it.

@RDearman
Thanks for the heads-up about Voice Notebook. I gave it a try and it works pretty well.

@Doitsujin
Thanks for the heads-up about ReadBeyond. I sampled Verne on my PC, and it came it quite well.
0 x
Many things which are false are transmitted from book to book, and gain credit in the world. -- attributed to Samuel Johnson


Return to “Language Programs and Resources”

Who is online

Users browsing this forum: No registered users and 2 guests