Experiments: Gold List vs Anki vs ....

General discussion about learning languages
User avatar
zenmonkey
Black Belt - 2nd Dan
Posts: 2528
Joined: Sun Jul 26, 2015 7:21 pm
Location: California, Germany and France
Languages: Spanish, English, French trilingual - German (B2/C1) on/off study: Persian, Hebrew, Tibetan, Setswana.
Some knowledge of Italian, Portuguese, Ladino, Yiddish ...
Want to tackle Tzotzil, Nahuatl
Language Log: viewtopic.php?f=15&t=859
x 7030
Contact:

Experiments: Gold List vs Anki vs ....

Postby zenmonkey » Tue Jan 23, 2018 5:20 pm

Based on a discussion in the Gold List Challenge I'm going to run a little test for myself. Feel free to post up your own tests and results here.

I think I'm going to try a test using both Anki and GM.

1) Generate a list
2) at least 3 x per week do an add of at least 40 words - every other word goes into Anki or a Goldlist. But only do either one at a time.
3) Count time to write out or enter words into Anki and rep time.
4) 60, 120, 180 days in, do a recall test on a limited sample from the original list. Sample test should not interfere significantly with the following tests.

Thanks @Brun Ugle and @smallwhite for your input so far.

I'll report on total words learn and words learnt vs time spent. Maybe some other metrics.

Feel free to join or critique or do your own Gold List vs Memrise, Cloze Master or whatever...

My objective is to get a sense if the GLM is effective versus Anki for me.
17 x
I am a leaf on the wind, watch how I soar

User avatar
Iversen
Black Belt - 4th Dan
Posts: 4768
Joined: Sun Jul 19, 2015 7:36 pm
Location: Denmark
Languages: Monolingual travels in Danish, English, German, Dutch, Swedish, French, Portuguese, Spanish, Catalan, Italian, Romanian and (part time) Esperanto
Ahem, not yet: Norwegian, Afrikaans, Platt, Scots, Russian, Serbian, Bulgarian, Albanian, Greek, Latin, Irish, Indonesian and a few more...
Language Log: viewtopic.php?f=15&t=1027
x 14962

Re: Experiments: Gold List vs Anki vs ....

Postby Iversen » Wed Jan 24, 2018 12:06 am

Good idea, and I'm looking forward to seeing your report about the results.

I once did something similar: I chose a bunch of Russian words - or rather three bunches. I did absolutely nothing with the first bunch except putting them on a list without translations. The second one got the standard goldlist treatment (except that I wrote the lists on plain white paper), and I ran the third one through my own wordlist system with a couple of repetitions. When I had finished the last goldlist destillation I checked the results for each group of words, and for me the result was that the goldlist and the no treatment gave the same result (about a third known words as far as I remember, which ironically wasn't too far from my percentages back then with random words taken directly from a Russian dictionary). With my own system I scored two thirds, which definitely is more than a third.

However these results cannot be taken at face value. I compared the system I was using daily and knew in and out with a system I hadn't ever used before and regarded with an unholy mixture of incredulity and interest. No wonder that my own system yielded the highest recall rates! And precisely therefore I'm very interested in hearing what others can get out of using the goldlist method.
6 x

User avatar
zenmonkey
Black Belt - 2nd Dan
Posts: 2528
Joined: Sun Jul 26, 2015 7:21 pm
Location: California, Germany and France
Languages: Spanish, English, French trilingual - German (B2/C1) on/off study: Persian, Hebrew, Tibetan, Setswana.
Some knowledge of Italian, Portuguese, Ladino, Yiddish ...
Want to tackle Tzotzil, Nahuatl
Language Log: viewtopic.php?f=15&t=859
x 7030
Contact:

Re: Experiments: Gold List vs Anki vs ....

Postby zenmonkey » Wed Jan 24, 2018 12:42 am

Iversen wrote:However these results cannot be taken at face value. I compared the system I was using daily and knew in and out with a system I hadn't ever used before and regarded with an unholy mixture of incredulity and interest. No wonder that my own system yielded the highest recall rates! And precisely therefore I'm very interested in hearing what others can get out of using the goldlist method.


Thanks!
I'm a big fan of your system and have used it quite a bit - I moved over to Anki for the portability of it and stuck to it for a while.

I think you hit the nail on the head that the results I come up with should not be taken at face value and require interpretation of attitudes and past experience and preferences. (and language and level...)

-- notes on first batch of words --

I've created a references list in an Excel doc which will be my reference dictionary for the words I'm putting into each method. For Hebrew, copying and pasting from Wyner's Word List and correcting to remove the nikkud takes about 20-30 minutes for 50 words. I type very slowly in Hebrew (some of my stickers have worn off). These 20-30 minutes are experiment prep and don't really count for either test method as the words will then be split in groups of 5 and go into each method.

It's late and I'll start fresh tomorrow.
2 x
I am a leaf on the wind, watch how I soar

User avatar
smallwhite
Black Belt - 2nd Dan
Posts: 2386
Joined: Mon Jul 06, 2015 6:55 am
Location: Hong Kong
Languages: Native: Cantonese;
Good: English, French, Spanish, Italian;
Mediocre: Mandarin, German, Swedish, Dutch.
.
x 4876

Re: Experiments: Gold List vs Anki vs ....

Postby smallwhite » Wed Jan 24, 2018 2:48 am

zenmonkey wrote:Based on a discussion in the Gold List Challenge...

Cross-referencing the discussion:
Goldlist Challenge
1 x
Dialang or it didn't happen.

User avatar
smallwhite
Black Belt - 2nd Dan
Posts: 2386
Joined: Mon Jul 06, 2015 6:55 am
Location: Hong Kong
Languages: Native: Cantonese;
Good: English, French, Spanish, Italian;
Mediocre: Mandarin, German, Swedish, Dutch.
.
x 4876

Re: Experiments: Gold List vs Anki vs ....

Postby smallwhite » Wed Jan 24, 2018 3:14 am

Brun Ugle wrote:
zenmonkey wrote:
smallwhite wrote:
zenmonkey wrote:
smallwhite wrote:But my biggest question mark is: how do you decide how much time to spend when you goldlist or iversenlist? When I tried them, I found that it was totally up to me whether I wanted to chew my batch of words for 20 or 30 minutes, and that's a 50% difference already.


... you should just write them out slowly...

How slowly? You're trying to compare time spent so How Slowly is critical.


I have no ideas, but slow enough to pay attention to your handwriting and be mindful of the moment. Not slow enough that you look like an Ok Go video.

That's probably a personal parameter one would need to figure out.

The author suggests that you can do a headlist of 25 words in 20 minutes with time to read through the list once after writing it.

I think that was what I was doing, 20 to 30 minutes. And when I looked at the words again 1 or 2 days later (as Iversen had mentioned once), I could only recall around 20% of them L2->L1, and less if L1->L2, and revising them took 20-30 minutes again. That exceeded the 90 seconds per word Flashcard time already and I felt confused. I'm glad zenmonkey's doing a comparison for us because I can't do it myself :?
1 x
Dialang or it didn't happen.

User avatar
zenmonkey
Black Belt - 2nd Dan
Posts: 2528
Joined: Sun Jul 26, 2015 7:21 pm
Location: California, Germany and France
Languages: Spanish, English, French trilingual - German (B2/C1) on/off study: Persian, Hebrew, Tibetan, Setswana.
Some knowledge of Italian, Portuguese, Ladino, Yiddish ...
Want to tackle Tzotzil, Nahuatl
Language Log: viewtopic.php?f=15&t=859
x 7030
Contact:

Re: Experiments: Gold List vs Anki vs ....

Postby zenmonkey » Wed Jan 24, 2018 1:18 pm

smallwhite wrote:I think that was what I was doing, 20 to 30 minutes. And when I looked at the words again 1 or 2 days later (as Iversen had mentioned once), I could only recall around 20% of them L2->L1, and less if L1->L2, and revising them took 20-30 minutes again.


For info, with the GLM you shouldn't be reviewing them in 1-2 days but leave them alone for 2 weeks. Which obviously sounds counterproductive, hence this "study".
0 x
I am a leaf on the wind, watch how I soar

User avatar
zenmonkey
Black Belt - 2nd Dan
Posts: 2528
Joined: Sun Jul 26, 2015 7:21 pm
Location: California, Germany and France
Languages: Spanish, English, French trilingual - German (B2/C1) on/off study: Persian, Hebrew, Tibetan, Setswana.
Some knowledge of Italian, Portuguese, Ladino, Yiddish ...
Want to tackle Tzotzil, Nahuatl
Language Log: viewtopic.php?f=15&t=859
x 7030
Contact:

Re: Experiments: Gold List vs Anki vs ....

Postby zenmonkey » Wed Jan 24, 2018 1:33 pm

A few notes needed about what I'm working on here.

I'm going to do this across my three current active study languages. GL headlists and adding words to the deck at least 3 times a week in the following manner.

Hebrew
From Wyner:
Goldlist: Mostly single words and definition with pronunciation,
Anki: English, Hebrew (print and cursive), Sound, Image - 2 cards per word

German
From Basic German Vocabulary using 2000-4000 level phrases:
Goldlist: short phrases with one word English translation,
Anki: Cloze sentences 1 card per word

Tibetan
Probably the "Guide de conversation tibétain" from Assimil and pick and pull phrases from there. Still need to run this at least once.

---
Tracking - already the GLM is fast, basically about 20 minutes to write down 20 items. Done. While with Anki it takes me about 20-30 minutes to create the cards and then the time to run through them.
1 x
I am a leaf on the wind, watch how I soar

User avatar
Iversen
Black Belt - 4th Dan
Posts: 4768
Joined: Sun Jul 19, 2015 7:36 pm
Location: Denmark
Languages: Monolingual travels in Danish, English, German, Dutch, Swedish, French, Portuguese, Spanish, Catalan, Italian, Romanian and (part time) Esperanto
Ahem, not yet: Norwegian, Afrikaans, Platt, Scots, Russian, Serbian, Bulgarian, Albanian, Greek, Latin, Irish, Indonesian and a few more...
Language Log: viewtopic.php?f=15&t=1027
x 14962

Re: Experiments: Gold List vs Anki vs ....

Postby Iversen » Wed Jan 24, 2018 2:31 pm

The retention after 1-2 days (or a month) depends very much on what you did on day one, and I made my own system in such a way that that the first repetitions take place already while you make the wordlist - hence the need for three columns, the idea that you should work with 5-7 words as a group (to give yourself the chance to forget each of them while you try the remember the others) and the idea that you first test that you remember the translations and later the original words with the translations as a cue. Because of these early repetitions the tests I have done myself have consistently shown retention rates of at least 60-70% ... but take care: you can't just blindly trust those figures!

I was my own judge as to whether I could remember (or guess) the meanings after a certain time, and I was also the one who chose the words that went into the lists. I could in principle have cheated myself by choosing unknown, but fairly easy words. Or be being too lenient on how well I remembered the meanings. I would for instance without hesitation accept a word as learnt if I could translate it into something suitable in any language or come up with a synonym - it didn't have to be exactly the same translation that figured in the original list. But some researchers might take a rather dim view on this because it opens up for a greyzone of interpretations that effectively makes the test unquantifiable. So you have the choice between the use of strict, but quantifiable criteria and criteria that are more in line with the way languages actually function out there in the real world. The usual, but dubious way out of this dilemma has been to use multiple choice forms, but then you only test recognition and NOT real knowledge of the meanings of the test words - and then you get artificially high scores.

The goldlist method obviously functions on totally different principles. As far I can see it must rely on a implicit mechanism that makes you remember words better when you just have discarded them (or at least told yourself not to look at the list where you keep them) - just as you remember things better when you discover that you have lost them. But can you actually apply this logic to words on a list? I have a lot of confidence in mr. James aka 'Huliganov' aka 'uncle Davey', who is both knowledgeable and an original thinker, but it would be nice to have some hardcore experimentation concerning his goldlist method. Alas, you run into exactly the same problems with these experiments as I ran into during my own experiments with my three-column wordlists: you are involved as both guinea pig and researcher, you have the choice between using too lenient and too strict criteria for recall, and multiple choice forms would as a minimum have to be concocted by somebody else since the whole idea of testing the goldlist method would go pouff away if you made them yourself.

Science ain't easy...

By the way: zenmonkey has made one very clever move by choosing Hebrew, German and Tibetan as his test languages. Let me guess: he knows German very well, Tibetan not nearly as well and then maybe Hebrew somewhere in the middle. I have noticed that my own percentage of active words is much higher in strong languages than in weak languages, and it is also easier for me to memorize words in the strong ones. By comparing results from these three languages using the same memorization systems on them you may be able to test whether strong and weak languages function differently when running GM versus Anki on them.
7 x

User avatar
zenmonkey
Black Belt - 2nd Dan
Posts: 2528
Joined: Sun Jul 26, 2015 7:21 pm
Location: California, Germany and France
Languages: Spanish, English, French trilingual - German (B2/C1) on/off study: Persian, Hebrew, Tibetan, Setswana.
Some knowledge of Italian, Portuguese, Ladino, Yiddish ...
Want to tackle Tzotzil, Nahuatl
Language Log: viewtopic.php?f=15&t=859
x 7030
Contact:

Re: Experiments: Gold List vs Anki vs ....

Postby zenmonkey » Wed Jan 24, 2018 11:38 pm

Iversen wrote:By the way: zenmonkey has made one very clever move by choosing Hebrew, German and Tibetan as his test languages. Let me guess: he knows German very well, Tibetan not nearly as well and then maybe Hebrew somewhere in the middle. I have noticed that my own percentage of active words is much higher in strong languages than in weak languages, and it is also easier for me to memorize words in the strong ones. By comparing results from these three languages using the same memorization systems on them you may be able to test whether strong and weak languages function differently when running GM versus Anki on them.


Pretty much - German is B2, Hebrew A1 and Tibetan is A0.

I'm also looking at my sourcing and types of cards for Anki. I have sound, etc. included in Hebrew for now, German are cloze cards and Tibetan is simple Tibetan/Translation/pronunciation cards.

The pre-work for the Tibetan is terribly slow - it takes me about an hour to copy my 40 reference texts to the Excel sheet (which is the source of the gold list and the Anki cards) so that I can use it as a testing base later. That's an hour lost before the 20 min GLM writing or the making of Anki cards. I hope it gets much better as my typing in Tibetan improves.
0 x
I am a leaf on the wind, watch how I soar

User avatar
smallwhite
Black Belt - 2nd Dan
Posts: 2386
Joined: Mon Jul 06, 2015 6:55 am
Location: Hong Kong
Languages: Native: Cantonese;
Good: English, French, Spanish, Italian;
Mediocre: Mandarin, German, Swedish, Dutch.
.
x 4876

Re: Experiments: Gold List vs Anki vs ....

Postby smallwhite » Thu Jan 25, 2018 4:40 am

Thanks zenmonkey and Iversen for the tips. I know I was not doing exactly Goldlist or Iversen-style word lists. I already have a method that produces 90%+ recall and is convenient to do (flashcards on phone), so my main focus was whether the general process of doing pen and paper wordlists would be faster (than 90 seconds per word). I also realise that results would improve with time and experience. But I guess it was hard for me to act naturally when I knew I was testing myself, and I wasn't able to do my paper wordlists at a natural speed.

I've found my paper lists. With one batch of 19 German words, I got 8 correct (42%), 2 half-correct (11%), 9 blank (47.3%). It looks like the testing was done 1 or 2 days after, and for L2->L1 recognition. One batch of 19 words took 26 minutes and another batch 25 minutes, which is 81 seconds per word so approaching 90 seconds already.

Btw, does goldlisting require you to read or otherwise have contact with the language? That is, is the exposure you get from reading part of the method? Or does it work / will you have 70% words to carry forward even if you don't read at all? If reading is required then reading time should be counted. With flashcards you don't need further exposure or revision, you'll simply fail more cards and have to do more reps, but the system takes care of everything for you.
0 x
Dialang or it didn't happen.


Return to “General Language Discussion”

Who is online

Users browsing this forum: Kraut, nathancrow77 and 2 guests