Can one calculate comprehension?

General discussion about learning languages
User avatar
Serpent
Black Belt - 3rd Dan
Posts: 3657
Joined: Sat Jul 18, 2015 10:54 am
Location: Moskova
Languages: heritage
Russian (native); Belarusian, Polish

fluent or close: Finnish (certified C1), English; Portuguese, Spanish, German, Italian
learning: Croatian+, Ukrainian; Romanian, Galician; Danish, Swedish; Estonian
exploring: Latin, Karelian, Catalan, Dutch, Czech, Latvian
x 5179
Contact:

Re: Can one calculate comprehension?

Postby Serpent » Sun Apr 30, 2017 10:24 am

s_allard wrote:
Xmmm wrote:
s_allard wrote:The part I find interesting is more the low end. How can one understand 50% or less of something?


I don't know, this must be a trick question.

Let's say a TV show consists of two scenes, each one minute long and that's it. I will give the English transcriptions for this show:
...


Okay, so I understood 100% of scene number 1, and basically 0% of scene 2. So I say "I understood 50% of this show".

You're saying 50% is wrong? We should use 0% instead? 0% is more accurate? Why?

I stand corrected. I have to admit that this sort of calculation makes sense. Right now I'm reading a collection of nine short stories by Marcel Aymé, called Derrière Chez Martin. Let's assume the stories are all the same length. I can read eight without the slightest difficulty. I don't have to look up the least word. All the idioms are crystal clear. The antiquated tense uses are not a problem. But when it comes to the ninth short story, I can't understand a single word. In this case I can say that I understood 89% of the book.

I have to say that I've never encountered a situation like this when it comes to understanding a target language but it may exist. It's hard for me to imagine perfectly understanding the first episode of a TV series and not a single word of the next
nine episodes of the 10-episode season.

What I have always experienced, and I don't think I'm unique, is more like the entire work is a mixture of known and unknown. Some pages or episodes contain more unknown words or meanings and others less.
The example was about scenes, not entire stories. The percentage referred to content, not to individual words.
You can be familiar with the words I bolded but still remain clueless:
A: I cannot believe that yet again that bastard has gotten involved and caused a fiasco. I said last time I was going to slit his gizzard and this time I mean it.
B: Word.

though you can comprehend about 30% of the content if you get the parts "I can't believe", "I said last time I was going to..." "and this time I mean it". if you know what happened, you can figure out that A is angry with the person, has previously claimed they would do something nasty and is still saying they're willing to do it. (ironically my description is harder to understand than the story itself)
you might even understand that person B agrees and after it comes up a few times figure out more or less what "word" means here.

also if there are L2 subs, native speakers of most European languages will understand fiasco. but orally they might not recognize the pronunciation, even if they've seen the word in writing but automatically read it as in L1.

Basically it's an estimate of how well you understand the scenes, paragraphs, individual sentences, with the help of the context and visual clues. Of course sometimes you'll understand the content despite not knowing some words, sometimes you'll know every word and still not understand 100%, and everything in between. A lot of native content also has multiple story lines, some of which you'll understand better than others. Generally it all evens out.
2 x
LyricsTraining now has Finnish and Polish :)
Corrections welcome

s_allard
Blue Belt
Posts: 969
Joined: Sat Jul 25, 2015 3:01 pm
Location: Canada
Languages: French (N), English (N), Spanish (C2 Cert.), German (B2 Cert)
x 2305

Re: Can one calculate comprehension?

Postby s_allard » Sun Apr 30, 2017 11:44 am

leosmith wrote:
s_allard wrote:What I'm saying is that below a certain threshold, whatever that is, there's no point reading the book or watching a program other than out of masochism.

Some learners like to do this, so I don't think it's right to say there's no point to it. If graduated (i+1) material was always available, I would probably always use it, but I find myself more aggressively studying harder material when I want to "level up" in languages these days.

I think it's a great idea to 'aggressively study harder material' but the question is how harder. That's why I said "... below a certain threshold, whatever that is...". I'm pretty sure most people wouldn't finish a 300-page novel when they can't recognize a quarter of the words on the page. What's the pleasure out of that.
0 x

User avatar
Serpent
Black Belt - 3rd Dan
Posts: 3657
Joined: Sat Jul 18, 2015 10:54 am
Location: Moskova
Languages: heritage
Russian (native); Belarusian, Polish

fluent or close: Finnish (certified C1), English; Portuguese, Spanish, German, Italian
learning: Croatian+, Ukrainian; Romanian, Galician; Danish, Swedish; Estonian
exploring: Latin, Karelian, Catalan, Dutch, Czech, Latvian
x 5179
Contact:

Re: Can one calculate comprehension?

Postby Serpent » Sun Apr 30, 2017 12:22 pm

That's where emk's cheating and consolidation comes into the picture. There are lots of tools/methods that can help, such as parallel texts, popup dictionaries, LR/audiobooks (i find i can follow much harder texts along with the narrator - the intonation and emphasis help, and often things get easier if you simply keep going)... Choosing the content is also an important tool, e.g. picking BD's and manga to make the most out of the visual clues, or reading books/authors you're familiar with, or those where you've seen the movie.

And that threshold can't be discussed without attempting to measure the comprehension in some way, whether using percentages or vague words or any system one can come up with.
2 x
LyricsTraining now has Finnish and Polish :)
Corrections welcome

s_allard
Blue Belt
Posts: 969
Joined: Sat Jul 25, 2015 3:01 pm
Location: Canada
Languages: French (N), English (N), Spanish (C2 Cert.), German (B2 Cert)
x 2305

Re: Can one calculate comprehension?

Postby s_allard » Sun Apr 30, 2017 1:17 pm

Serpent wrote:...
And that threshold can't be discussed without attempting to measure the comprehension in some way, whether using percentages or vague words or any system one can come up with.

But that's the whole point. There is no need to attempt to measure comprehension. It is even doubtful if one can measure comprehension at all. One can measure all sorts of things, such as words on a page, and even that is fraught with complications, but that's not comprehension. Some people speak of basically two levels of comprehension: adequate and inadequate or satisfactory and unsatisfactory. I prefer three.

Without resorting to measuring anything one can certainly detect a level of satisfaction for purposes of determining whether to continue reading, watching or listening to a document in one's target language
0 x

User avatar
Serpent
Black Belt - 3rd Dan
Posts: 3657
Joined: Sat Jul 18, 2015 10:54 am
Location: Moskova
Languages: heritage
Russian (native); Belarusian, Polish

fluent or close: Finnish (certified C1), English; Portuguese, Spanish, German, Italian
learning: Croatian+, Ukrainian; Romanian, Galician; Danish, Swedish; Estonian
exploring: Latin, Karelian, Catalan, Dutch, Czech, Latvian
x 5179
Contact:

Re: Can one calculate comprehension?

Postby Serpent » Sun Apr 30, 2017 1:34 pm

As leosmith said, not everyone is learning English/FIGS where you can pick from a lot of materials (even out of these, Italian has less of a choice for beginners/low intermediate learners). Sometimes you need to use whatever you have, or good materials exist but they're too expensive, etc. If people find the percentages useful, why does it bother you?

You didn't define that threshold you brought up. How would you explain it to a beginner? An experienced learner can certainly learn without HTLAL, that's not the point ;)
3 x
LyricsTraining now has Finnish and Polish :)
Corrections welcome

s_allard
Blue Belt
Posts: 969
Joined: Sat Jul 25, 2015 3:01 pm
Location: Canada
Languages: French (N), English (N), Spanish (C2 Cert.), German (B2 Cert)
x 2305

Re: Can one calculate comprehension?

Postby s_allard » Sun Apr 30, 2017 3:09 pm

Serpent wrote:... If people find the percentages useful, why does it bother you?
...

This is an important point. Does it really bother me that people use percentages to estimate their level of comprehension? Certainly not. I bring the question up only because I think it warrants debate in a forum where we are discussing issues of language learning.

I happen to like clear vocabulary and precise thinking. That's exactly why I waged a losing battle against what I considered the misuse of the term "fluency". I also waged a little battle over the calculation of vocabulary size.

These are all elements of discussion in the hope of bringing clarity to issues. So when someone says they watched 150 hours of TV and went from 45% to 90% comprehension, my simple and naive question is what does that mean and how does one arrive at these figures. Did the person do a before and after sample analysis or simply pick some figures out of the air to give their post an air of scientific precision? How many people here have actually done an analysis of the word coverage of a sample of written or spoken speech? I have and it's a lot more work than just counting the number of words on a page.

I may be the only one asking these question. Everyone else seems to say: How foolish are you s_allard, 45% to 90% means twice as much comprehension. What's the problem?
0 x

User avatar
leosmith
Brown Belt
Posts: 1341
Joined: Thu Sep 29, 2016 10:06 pm
Location: Seattle
Languages: English (N)
Spanish (adv)
French (int)
German (int)
Japanese (int)
Korean (int)
Mandarin (int)
Portuguese (int)
Russian (int)
Swahili (int)
Tagalog (int)
Thai (int)
x 3104
Contact:

Re: Can one calculate comprehension?

Postby leosmith » Sun Apr 30, 2017 3:36 pm

s_allard wrote:I bring the question up only because I think it warrants debate

I'm all for debate, but I wonder what your goal is regarding this topic. Do you want people to stop saying things like "I understand about 55% of this movie"? Do you want us to round off to the nearest 10%? Are vague comments like "study something challenging but not too challenging" acceptable to you? Do you want us to all use a common method of of calculating comprehension? Or are you just practicing your English?

s_allard wrote:I think it's a great idea to 'aggressively study harder material' but the question is how harder.
I assume you meant "how much harder". If you don't want us to estimate comprehension, then how can you answer this question?

s_allard wrote:I'm pretty sure most people wouldn't finish a 300-page novel when they can't recognize a quarter of the words on the page. What's the pleasure out of that.
I assume you meant "What's the pleasure in that?" Who are we to judge what's pleasurable?
4 x
https://languagecrush.com/reading - try our free multi-language reading tool

s_allard
Blue Belt
Posts: 969
Joined: Sat Jul 25, 2015 3:01 pm
Location: Canada
Languages: French (N), English (N), Spanish (C2 Cert.), German (B2 Cert)
x 2305

Re: Can one calculate comprehension?

Postby s_allard » Sun Apr 30, 2017 4:09 pm

leosmith wrote:
s_allard wrote:I bring the question up only because I think it warrants debate

I'm all for debate, but I wonder what your goal is regarding this topic. Do you want people to stop saying things like "I understand about 50% of this movie"? Do you want us to all use a common method of of calculating comprehension? Or are you just practicing your English?

Casual reading of my first post would show that I propose a three level scale for describing degree of comprehension of language (and not movies): nothing, something and full. I really doubt that anybody will change their usage here because of me but I do think that some people will pause and think about how they talk about comprehension.

As I pointed out in my previous post, most people don't care how they use the word fluency and are not going to change because of my objections. When I suggest that passive skills should be called receptive skills because these are not by any means passive, some people think this is silly nitpicking.

In fact, I don't want anybody to stop using percentages. I'm simply curious as to how people arrive at them. Maybe it's something I could use.

In passing, I should note that the usage of the CEFR proficiency scale is now so widespread that very few people remember what it was like to talk about "fluency" on the old HTLAL forum before. Who today talks about "high intermediate fluency"?

Edit: This post refers to an earlier version of leosmith's post. I have not made any changes.
0 x

Cainntear
Black Belt - 3rd Dan
Posts: 3469
Joined: Thu Jul 30, 2015 11:04 am
Location: Scotland
Languages: English(N)
Advanced: French,Spanish, Scottish Gaelic
Intermediate: Italian, Catalan, Corsican
Basic: Welsh
Dabbling: Polish, Russian etc
x 8667
Contact:

Re: Can one calculate comprehension?

Postby Cainntear » Sun Apr 30, 2017 4:39 pm

s_allard wrote:Casual reading of my first post would show that I propose a three level scale for describing degree of comprehension of language (and not movies): nothing, something and full.

That's all well and good, but that still doesn't give us any means of grading material relatively -- there's no way of identifying "harder" or "easier" material on that scale. And you clearly wouldn't propose grading difficulty by vocabulary coverage, as you've already made it clear you think that's a non-starter.

As I pointed out in my previous post, most people don't care how they use the word fluency and are not going to change because of my objections. When I suggest that passive skills should be called receptive skills because these are not by any means passive, some people think this is silly nitpicking.

I agree with you on passive vs receptive, but I disagree about fluency. The term "passive" gives the wrong message, as it assumes no effort or involvement on the part of the listener, whereas "receptive" has no major connotations, so doesn't mislead.

"Fluent", however, has lost its etymological meaning in modern English, which is why the colloquial term has diverged from it so far.
1 x

aaleks
Blue Belt
Posts: 884
Joined: Thu Apr 13, 2017 7:04 pm
Languages: Russian (N)
x 1910

Re: Can one calculate comprehension?

Postby aaleks » Sun Apr 30, 2017 5:04 pm

s_allard wrote:In fact, I don't want anybody to stop using percentages. I'm simply curious as to how people arrive at them.

Well, in my case it’s just a way of speaking. I’ve never really made such calculation. I think the more important thing here is not a number of known words itself but what you know about these words. Your overall knowledge of the subject plays its role too.
0 x


Return to “General Language Discussion”

Who is online

Users browsing this forum: No registered users and 2 guests