2 weeks to build a language app (a music-style language trainer)

All about language programs, courses, websites and other learning resources
User avatar
tangleweeds
Green Belt
Posts: 280
Joined: Sat Jul 18, 2015 7:09 pm
Location: Portland, Oregon, USA
Languages: English (N)
beginner: Irish, Norwegian
clearing cobwebs: Japanese
on the shelf: French, Latin
Language Log: viewtopic.php?t=705
x 580

Re: 2 weeks to build a language app (a music-style language trainer)

Postby tangleweeds » Wed Sep 02, 2020 4:30 pm

ryanheise wrote:One of the ideas I put up earlier in the thread was to record and then compare your voice to the original audio by analysing features of the audio that are important in speech (pitch and IPA sounds). It's definitely not on the short term plan but I think it would be pretty cool to provide this sort of feedback.
Now this would be very interesting, but far more of a challenge to analyze/implement. How about saving that more difficult bit for 2.0, and making a basic record/playback feature part of 1.n? :D
1 x
Progressing from apps back back to books! (neurological injury healing!!!)
Concentrating on Irish (family heritage) & Japanese (my brother lives there)

User avatar
tangleweeds
Green Belt
Posts: 280
Joined: Sat Jul 18, 2015 7:09 pm
Location: Portland, Oregon, USA
Languages: English (N)
beginner: Irish, Norwegian
clearing cobwebs: Japanese
on the shelf: French, Latin
Language Log: viewtopic.php?t=705
x 580

Re: 2 weeks to build a language app (a music-style language trainer)

Postby tangleweeds » Wed Sep 02, 2020 4:50 pm

jeff_lindqvist wrote: As long as you understand what they're saying (or, as I "understand" what they're playing), just repeat aloud. I'd even go so far and say that those who don't understand the music (or the language) are those who need (or think they need?) to memorize it bit by bit.
Wasn't the original motivation that his listening/understanding skills fell apart when the segment of listening material got too long, so breaking it into small chunks and reassembling them made it more understandable, and therefore rememberable & repeatable? Because he's struggling witht longer segments of material, remembering the whole thing is nontrivial, thus less likely to happen automatically. Am I remembering things correctly here, ryanheise?

In a pure listening case, I might revise my earlier comment because starting the "backchaining" from the front when listening for understanding actually makes far more sense, whereas I have most definitely found that in the speaking (& music memorization, as for a classical music recital), working from the end makes everything far less stressful.
1 x
Progressing from apps back back to books! (neurological injury healing!!!)
Concentrating on Irish (family heritage) & Japanese (my brother lives there)

User avatar
ryanheise
Green Belt
Posts: 287
Joined: Tue Jun 04, 2019 3:13 pm
Location: Australia
Languages: English (N), Japanese (beginner)
x 879
Contact:

Re: 2 weeks to build a language app (a music-style language trainer)

Postby ryanheise » Wed Sep 02, 2020 5:55 pm

One of those two experiments was aimed at improving my listening attention span. I noticed in language exchange sessions that I would have these lapses where I would be listening to what the other person was saying, and then after a couple of seconds I would trip up and then my mind would completely lose track of what was being said after that point. Around that time, I had been doing audio-based sentence SRS and noticed that my listening attention span was roughly about the same as the average length of a sentence in my SRS deck. Funny that. (I guess you get good at exactly what you practice.) So this exercise helped my mind get used to listening continuously to much longer segments of audio without losing concentration.

The other experiment was an over learning experiment inspired by something Richard Simcott said in an interview, that language learning is over learning: i.e. repeating things many many times. It occurred to me that in my native language, I am now experienced enough with English that I don't require over learning to pick up new expressions. I'm familiar enough with English that if I hear a new phrase or word or a new way of saying something, or a way of expressing something with a certain intonation, I'll take enough interest in it that I'll be able to hear it in my head and will be able to recall it and use it later in my own conversations. I can't do that in Japanese because my memory and familiarity is not yet at that level, but I think you can compensate for this through over learning, or much greater than normal repetition. It is true that I have been doing mimicking/shadowing as part of getting familiar with the sentences and sounds in the story I'm listening to, although that alone hasn't been enough to get the sentences to stick in my head so that they are readily available to me to use during my conversations. There's something very different about mimicking something right after you hear it vs being able to ultimately recite the story unaided. That is a much higher level of familiarity, and it was after achieving that that I was able to reach the same level of sentence memory that I already have in English without trying, and at that level, I've found I am able to easily use what I remember in conversation. That's the barrier I have been trying to break with the over learning experiment. So the overall process there is to first listen and shadow, but then after gaining general familiarity, to top it off with full memorisation.

By the way, I tried back chaining yesterday just to see what the experience was like, and I have to say I quite liked it. That said, it does make it a bit harder to appreciate the story going in reverse, and I quite like the memory associations I can make when following the story in a forwards direction.

By the way tangleweeds, I would be interested to know your thoughts on how back chaining can scale because I don't think a chain can go on indefinitely and you might want to divide the whole up into sections. I do need to make it scale to fit into my app. I have thought of something like this:

C,BC,ABC,F,EF,DEF,ABCDEF,I,HI,GHI,DEFGHI,ABCDEFGHI

I would also be interested to know what is the average and maximum length of a typical chain.

Anyway, that's it for Day 4. I did get the play/pause button working. We have sound! The rest of my day was spent trying to think about all of these different training routine approaches and how to design a data structure and algorithm that can cope with them all. That work will still be going on in the background as new ideas occur to me. But tomorrow I get into the more difficult work of analysing the audio stream and breaking it up into a tree.
1 x

Cainntear
Black Belt - 1st Dan
Posts: 1742
Joined: Thu Jul 30, 2015 11:04 am
Location: Scotland
Languages: English(N)
Advanced: French,Spanish, Scottish Gaelic
Intermediate: Italian, Catalan, Corsican
Basic: Welsh
Dabbling: Polish, Russian etc
x 4249
Contact:

Re: 2 weeks to build a language app (a music-style language trainer)

Postby Cainntear » Wed Sep 02, 2020 6:48 pm

tangleweeds wrote:
Cainntear wrote:My issue with backchaining has always been that it is a rote memorisation technique -- the reason that you can't remember the whole word or phrase is that it's too complicated and contains too many new variables that you haven't acquired mastery of yet.

I don't use it to memorize, it's more like learning tongue-twisters, learning to hear and reproduce unfamiliar phonemes and cadences. It's about listening and speaking skills, not memorizing content. Procedural not declarative.

The procedural vs declarative distinction isn't what I'm talking about, though -- it's structured language vs arbitrary utterance; meaningful vs rote.

Looking at a typical Pimsleur-style backchain, the reason it's hard to say "my name is" is because you haven't learned "my" "name" "is", basic sentence structure, or most of the phonemes in the phrase. Fluent language involves chunking -- using multiple levels of cognitive structure simultaneously. When we use those structures, memorising language becomes easier. I mean, it's trivially obvious that even if you've never read the book or seen the film, you'll find it easier to repeat "My name is Iñigo Montoya. You killed my father. Prepare to die" than "Me llamo Iñigo Montoya. Mataste a mi padre. Prepárate a morir" because you can already do the chunking.

When you don't have all that structure and can't break it down, you are simply memorising an arbitrary string of sounds, which is hard, because memory fades very quickly when it doesn't activate existing cognitive structures.

Backchaining allows us to memorise things that are too long and complicated for us to actually process linguistically, and that sounds like a bit of a waste of time to me.

The thing about interacting with real-live native speakers is that they invariably throw at you too many new variables that you haven't mastered yet. So that's actually a skill to practice under presssure.

Your defence here could equally apply to any weak teaching technique, because it's essentially "the real world isn't perfect, so bad teaching is good." But once you look at backchaining specifically, the argument undermines itself further:

this.
like this.
speak like this.
never speak like this.
People never speak like this.

Back-chaining exists purely as a pedagogical intervention to help you learn so that you can later function in real-world situations. If your goal is to present students with in-class tasks that teach real-world skills other than just the basic language learning, backchaining is the wrong activity to use.
1 x

Cainntear
Black Belt - 1st Dan
Posts: 1742
Joined: Thu Jul 30, 2015 11:04 am
Location: Scotland
Languages: English(N)
Advanced: French,Spanish, Scottish Gaelic
Intermediate: Italian, Catalan, Corsican
Basic: Welsh
Dabbling: Polish, Russian etc
x 4249
Contact:

Re: 2 weeks to build a language app (a music-style language trainer)

Postby Cainntear » Wed Sep 02, 2020 7:00 pm

ryanheise wrote:
tangleweeds wrote:The back chaining I have used, both in Japanese and in music, goes:

D
CD
BCD
ABCD


I am currently trying to work out how this scales. My other two training routines scale well, but with back-chaining, does this chain just keep growing and growing, or do you break the chain after a certain length and start a new chain? And if so, to you then do some practice joining chains together? Since there will be options for everything, I'm just trying to understand how this idea could scale if people wanted it to.

What I may end up doing is doing what I had in mind first so you can get a feel for how it scales, and that might give you some ideas on how a similar thing could be done for back chaining.

One extra variable to think about: I was taught backchaining as the student listening to the whole thing first, with the backchaining being used to teach them to repeat it. This may complicate things further.

I haven't used backchaining much (as I've rarely worked with beginners in English, so I keep forgetting to use it when teaching beginners Gaelic) but it was only ever something I used with phrases or long words. A sentence that isn't a phrase-book style sentence seems too long to me. (Of course, this may be down to my exposure to Pimsleur -- I used to get the short courses out of my library as casual listening while walking to work.)

To me, the only thing that would make sense would be to pick out phrases to include as backchaining embedded into the musical scheme. Whether that requires user intervention or is done automatically by the software when it identifies phrases of suitable length is a whole nother question.
0 x

User avatar
jeff_lindqvist
Black Belt - 2nd Dan
Posts: 2085
Joined: Sun Aug 16, 2015 9:52 pm
Languages: sv, en
de, es
ga, eo
---
fi, yue, ro, tp, cy, kw, pt, sk
Language Log: viewtopic.php?f=15&t=2773
x 4808

Re: 2 weeks to build a language app (a music-style language trainer)

Postby jeff_lindqvist » Wed Sep 02, 2020 8:50 pm

tangleweeds wrote:
jeff_lindqvist wrote: As long as you understand what they're saying (or, as I "understand" what they're playing), just repeat aloud. I'd even go so far and say that those who don't understand the music (or the language) are those who need (or think they need?) to memorize it bit by bit.


Wasn't the original motivation that his listening/understanding skills fell apart when the segment of listening material got too long, so breaking it into small chunks and reassembling them made it more understandable, and therefore rememberable & repeatable? Because he's struggling with longer segments of material, remembering the whole thing is nontrivial, thus less likely to happen automatically. Am I remembering things correctly here, ryanheise?


You're probably right - even something as short as a 4 minute episode of Peppa Pig is longer than my Assimil dialogue, and for that matter - a lot longer than an Irish tune. And if it's in an alien language it's indeed an arbitrary string of sounds. I'm not saying memorizing is wrong - but if I understand something I can use chunking.

Now that I think of it, I once half-learned a martial arts form through backchaining. It had several parts. I started with the last one, added the second to last and so on. On the other hand I didn't review it often enough for it to stick forever, so whatever advantages backchaining may have given me, I lost the form.
0 x
Leabhair/Greannáin léite as Gaeilge: 9 / 18
Ar an seastán oíche: Oileán an Órchiste
Duolingo - finished trees: sp/ga/de/fr/pt/it
Finnish with extra pain : 100 / 100

Llorg Blog - Wiki

User avatar
Adrianslont
Blue Belt
Posts: 760
Joined: Sun Aug 16, 2015 10:39 am
Location: Australia
Languages: English (N), Learning Indonesian and French
x 1588

Re: 2 weeks to build a language app (a music-style language trainer)

Postby Adrianslont » Wed Sep 02, 2020 10:37 pm

Some of my thoughts on recently raised discussion:

What I would use this app for: for me it would be a listening and pronunciation trainer. I would inevitably have vocabulary and grammar consolidated and pick up some new stuff - the amount depending on what I fed into the app. I like your idea Ryan about using the structure to build up listening stamina by combining - I use srs and sometimes have the same feelings as you about the shortness of the utterances. Emk used to talk about making longer cards but that’s a bit time consuming, rejigging cards. The approach you suggest can be used on the fly and I like the potential.

Memorising versus remembering: I am so much in agreement with Jeff’s views here. The word memorise actually makes me shudder, especially when people can’t see a difference between memorising and remembering. Memorising is a thing but it’s not something I work on much - maybe I should but I don’t enjoy it so I don’t do much of it. I use anki but not to memorise - I use it so I can see and hear vocabulary, utterances and sentences at spaced intervals. Sometimes I use it for pronunciation practice, repeating utterances to train my speech organs to make sounds they are not used to, in combinations they are not used to. If I remember the sentence, that’s great and I happily inevitably will remember some - but I am not trying to deliberately commit sentences to memory - which is memorisation.

Ryan, you seem keen on memorisation if I understand correctly? Totally cool, of course - just not my approach.

How would I foresee backchaining scaling: this is of course Ryan’s app - he’s building it both for himself and others who would appreciate it. Just personally, thinking of my own user case I would be looking at something I’ll call long utterances with multiple clauses as opposed to short single clause utterances and as opposed to whole anecdotes, whole pieces of music, whole poems or paragraphs of books. So something like:

“We’ll take the fixed price menu, thanks.
I’ll have the soup and the steak, and my wife will have the salad and the salmon.”

This would give me pronunciation practice with the sounds and intonation over a real-life length utterance.

As opposed to, the lengthy anecdote about how I broke my ribs on holiday - also real-life, of course - just not what I’m looking for.

Just my case.

Happy building.
3 x

User avatar
tangleweeds
Green Belt
Posts: 280
Joined: Sat Jul 18, 2015 7:09 pm
Location: Portland, Oregon, USA
Languages: English (N)
beginner: Irish, Norwegian
clearing cobwebs: Japanese
on the shelf: French, Latin
Language Log: viewtopic.php?t=705
x 580

Re: 2 weeks to build a language app (a music-style language trainer)

Postby tangleweeds » Thu Sep 03, 2020 5:06 am

Adrianslont wrote:How would I foresee backchaining scaling: this is of course Ryan’s app - he’s building it both for himself and others who would appreciate it. Just personally, thinking of my own user case I would be looking at something I’ll call long utterances with multiple clauses as opposed to short single clause utterances and as opposed to whole anecdotes, whole pieces of music, whole poems or paragraphs of books. So something like:

“We’ll take the fixed price menu, thanks.
I’ll have the soup and the steak, and my wife will have the salad and the salmon.”

This would give me pronunciation practice with the sounds and intonation over a real-life length utterance.

As opposed to, the lengthy anecdote about how I broke my ribs on holiday - also real-life, of course - just not what I’m looking for.
I agree completely, thank you for answering this because I was having trouble formulating a good reply to the OP's question.

What happens to me is that not super often, but regularly enough, I run into sentences that I just can't say right. There can be a pileup of unfamiliar sounds, familiar ones juxtaposed unfamiliarly, or the two combined. I don't even strictly backchain, I just center ever larger chunks around the phoneme trainwreck that interferes with acceptable pronunciation.

And I personally would be using this in an n + 1 situation.
1 x
Progressing from apps back back to books! (neurological injury healing!!!)
Concentrating on Irish (family heritage) & Japanese (my brother lives there)

User avatar
ryanheise
Green Belt
Posts: 287
Joined: Tue Jun 04, 2019 3:13 pm
Location: Australia
Languages: English (N), Japanese (beginner)
x 879
Contact:

Re: 2 weeks to build a language app (a music-style language trainer)

Postby ryanheise » Thu Sep 03, 2020 5:27 pm

Adrianslont wrote:Some of my thoughts on recently raised discussion:

What I would use this app for: for me it would be a listening and pronunciation trainer. I would inevitably have vocabulary and grammar consolidated and pick up some new stuff - the amount depending on what I fed into the app. I like your idea Ryan about using the structure to build up listening stamina by combining - I use srs and sometimes have the same feelings as you about the shortness of the utterances. Emk used to talk about making longer cards but that’s a bit time consuming, rejigging cards. The approach you suggest can be used on the fly and I like the potential.

Memorising versus remembering: I am so much in agreement with Jeff’s views here. The word memorise actually makes me shudder, especially when people can’t see a difference between memorising and remembering. Memorising is a thing but it’s not something I work on much - maybe I should but I don’t enjoy it so I don’t do much of it. I use anki but not to memorise - I use it so I can see and hear vocabulary, utterances and sentences at spaced intervals. Sometimes I use it for pronunciation practice, repeating utterances to train my speech organs to make sounds they are not used to, in combinations they are not used to. If I remember the sentence, that’s great and I happily inevitably will remember some - but I am not trying to deliberately commit sentences to memory - which is memorisation.

Ryan, you seem keen on memorisation if I understand correctly? Totally cool, of course - just not my approach.


Although I obviously wouldn't expect many others to be just as motivated about memorisation as I am, I would at least like to make it easier for people to try different approaches that they may not have tried before. For example, even the approach I was using to expand my listening attention span is perhaps something that is not easy for people to experiment with without the right tools. This particular idea may also have a lot of potential as a delivery style for new sentences and new vocabulary if the algorithm can repeat and alternate the audio and its translation during the learning stages.

How would I foresee backchaining scaling: this is of course Ryan’s app - he’s building it both for himself and others who would appreciate it. Just personally, thinking of my own user case I would be looking at something I’ll call long utterances with multiple clauses as opposed to short single clause utterances and as opposed to whole anecdotes, whole pieces of music, whole poems or paragraphs of books. So something like:

“We’ll take the fixed price menu, thanks.
I’ll have the soup and the steak, and my wife will have the salad and the salmon.”

This would give me pronunciation practice with the sounds and intonation over a real-life length utterance.

As opposed to, the lengthy anecdote about how I broke my ribs on holiday - also real-life, of course - just not what I’m looking for.


That's interesting to know. OK, so in comparison to the first example you gave me (If only | I’d | known) where it was broken down to the individual words almost, what you describe above sounds more like it's at the clause level rather than the word level.

Since I originally conceived of the atomic units in this app being clauses or phrases, this sounds good to me. I mean it would still be possible to manually split down to the word level, although automatic split detection would mostly only discover gaps between sentences and clauses. I would personally choose to split the audio so that each segment contained at most 1 difficult word, and if there were 2 difficult words, I'd try to insert a split somewhere between them, unless the 2 difficult words were adjacent and tightly connected (e.g. one word modifies the other or maybe even if they are just in the same clause and the clause has few words).

As mentioned earlier, my plan is to have this concept of a "chunk" which contains between 1-to-3 atomic segments. Each of those atomic segments would typically be phrases or clauses. There would be 4 chunks in a unit. So we have for example:

ABC|DE|FG|HIJ

Now, within a chunk is perhaps where back chaining can be applied, but then merging the chunks together from that point onwards uses one of the other two routines that I originally had in mind. So there will be two choices for the algorithm used to merge different chunks together, but then within chunks there may also be a choice of algorithm, with back chaining being among them.

One other interesting thing that I haven't mentioned yet, but this concept of a "unit" isn't some arbitrary thing. The plan is also for this to be the unit of review. So if you learn a new unit today, you will review it tomorrow. Each day you review a unit, it's maturity increases, and as adjacent units' maturity increases, they also can get merged together. At the same time, it makes sense that you would want to have more reps of a unit on its first day, and fewer reps on later days (when I was doing Experiment 2, I collected data on the number of reps I needed, and indeed as the segments merged, the number of reps needed also became less because they were also more mature).

End of Day 5 recap: Today, I fell one day behind schedule as I was redesigning my database to factor in all of the considerations discussed so far. But I think it was something important to get right before continuing. Hopefully things will start taking shape over the next 4 or 5 days.
2 x

User avatar
tangleweeds
Green Belt
Posts: 280
Joined: Sat Jul 18, 2015 7:09 pm
Location: Portland, Oregon, USA
Languages: English (N)
beginner: Irish, Norwegian
clearing cobwebs: Japanese
on the shelf: French, Latin
Language Log: viewtopic.php?t=705
x 580

Re: 2 weeks to build a language app (a music-style language trainer)

Postby tangleweeds » Thu Sep 03, 2020 8:03 pm

ryanheise wrote:One other interesting thing that I haven't mentioned yet, but this concept of a "unit" isn't some arbitrary thing. The plan is also for this to be the unit of review. So if you learn a new unit today, you will review it tomorrow. Each day you review a unit, it's maturity increases, and as adjacent units' maturity increases, they also can get merged together. At the same time, it makes sense that you would want to have more reps of a unit on its first day, and fewer reps on later days (when I was doing Experiment 2, I collected data on the number of reps I needed, and indeed as the segments merged, the number of reps needed also became less because they were also more mature).
This sounds interesting and helpful. It's cool how you have all those experiments to guide you in crafting this, instead of just pulling good ideas at the air (and thus risking the dreaded, "Well, it seemed like a good idea at the time...")

ryanheise wrote:End of Day 5 recap: Today, I fell one day behind schedule as I was redesigning my database to factor in all of the considerations discussed so far. But I think it was something important to get right before continuing. Hopefully things will start taking shape over the next 4 or 5 days.
I just wanted to thank you for both asking for our input and then putting in the extra effort to rework you database to implement it. I've been noticing how all of us who do listening/speaking practice like this have discovered different methodologies that work for us, and I really appreciate the fine-grained attention you're giving to understand and incorporate as many of these ideas as possible.

Even the apps I love most either limit you to the one specific study strategy that the creator was inspired to implement and share [thank you, app creators, nonetheless], or else are so open-ended they become just that much more labor-intensive to work with , in that there's a small and unconscious but very steady leeching of time and mental energy which would have otherwise gone into studying [e.g. manually selecting your clips as in AudioStretch].
2 x
Progressing from apps back back to books! (neurological injury healing!!!)
Concentrating on Irish (family heritage) & Japanese (my brother lives there)


Return to “Language Programs and Resources”

Who is online

Users browsing this forum: No registered users and 2 guests