Considerations on using dubbed material and the McGurk Effect

General discussion about learning languages
User avatar
smallwhite
Black Belt - 2nd Dan
Posts: 2386
Joined: Mon Jul 06, 2015 6:55 am
Location: Hong Kong
Languages: Native: Cantonese;
Good: English, French, Spanish, Italian;
Mediocre: Mandarin, German, Swedish, Dutch.
.
x 4876

Re: Considerations on using dubbed material and the McGurk Effect

Postby smallwhite » Mon Jun 26, 2017 4:50 am

zenmonkey wrote:while it is weaker for Japanese and Chinese listeners... Chinese listeners are less susceptible to visual cues...

Chinese me heard ga-ga-ga when looking, whether it was on my phone with not-quite-sufficient volume, on my phone at max volume or on my PC. I don't think I was biased by the instructions & descriptions because I didn't really understand them when I listened :oops: :P
1 x
Dialang or it didn't happen.

DangerDave2010
Orange Belt
Posts: 214
Joined: Sun Feb 14, 2016 5:10 am
Languages: gibberish (N)
x 291

Re: Considerations on using dubbed material and the McGurk Effect

Postby DangerDave2010 » Mon Jun 26, 2017 9:35 am

This illusion is really spooky in that it persists even after watching the video many times. But it seems that the sound and the lip movements need to be in perfect sync for it to happen, in a dubbed tape the sound will be always a little off, hence no effect. I suppose context could play a role in neutralising the effect too.

Anyway this proves that there is a hard wired mechanism for lip reading, so I suppose it is a good idea to do one's fair share of listening with video that allows you to pay attention on the lips, so that this circuitry gets activated. I remember when I was learning English I found that my listening comprehension would improve if I were able to pay attention on the lips.
2 x

User avatar
leosmith
Brown Belt
Posts: 1341
Joined: Thu Sep 29, 2016 10:06 pm
Location: Seattle
Languages: English (N)
Spanish (adv)
French (int)
German (int)
Japanese (int)
Korean (int)
Mandarin (int)
Portuguese (int)
Russian (int)
Swahili (int)
Tagalog (int)
Thai (int)
x 3101
Contact:

Re: Considerations on using dubbed material and the McGurk Effect

Postby leosmith » Wed Jun 28, 2017 4:08 am

It didn't work for me using that video. This one did though:
https://www.youtube.com/watch?v=PWGeUztTkRA
2 x
https://languagecrush.com/reading - try our free multi-language reading tool

User avatar
Ani
Brown Belt
Posts: 1433
Joined: Mon Mar 14, 2016 8:58 am
Location: Alaska
Languages: English (N), speaks French, Russian & Icelandic (beginner)
x 3840
Contact:

Re: Considerations on using dubbed material and the McGurk Effect

Postby Ani » Wed Jun 28, 2017 8:05 am

leosmith wrote:It didn't work for me using that video. This one did though:
https://www.youtube.com/watch?v=PWGeUztTkRA


Ahh yes that one is much better!

I do believe we lose a lot of data on dubs, but I don't think we mis-encode data because the lips are SO far off that it just comes up as -no data-. Dubs are still awesome for what they are.
1 x
But there's no sense crying over every mistake. You just keep on trying till you run out of cake.

User avatar
luke
Brown Belt
Posts: 1243
Joined: Fri Aug 07, 2015 9:09 pm
Languages: English (N). Spanish (intermediate), Esperanto (B1), French (intermediate but rusting)
Language Log: https://forum.language-learners.org/vie ... 15&t=16948
x 3631

Re: Considerations on using dubbed material and the McGurk Effect

Postby luke » Wed Jun 28, 2017 9:36 am

zenmonkey wrote:Try it, play this short video while watching the person closely:


It's quite amazing. I never really thought of myself as a lip reader, but the video makes it apparent that I do that. Also amazing is opening and closing the eyes during the brief video. From one utterance to the next the sound changes depending on whether my eyes are open or closed.

This also makes me think of the value of video while learning pronunciation and of babies watching their mommies speak.

High resolution does appear to help in the later posted video. What's so interesting about the first video is that it is not high resolution and yet the mouth movements are easily and imperceptibly processed as changing the sound.
1 x

User avatar
Systematiker
Blue Belt
Posts: 823
Joined: Tue May 10, 2016 6:09 pm
Languages: ENG (N); DEU (C2+) // SWG (~C1); BAR (~C1); SPA (4/3); FRA (~C1); SCO (~C1); NLD (~B2*); LAT (Latinum Bavaricum); GRC (Graecum Bavaricum); CAT (~B2*); POR (~B2*); SWE (~B2*); HBO (Hebraicum); DAN (~B1*); RUS (~A2); KOR (~A1); FAS (still a raw beginner)
*Averaged for high receptive skill
Language Log: https://forum.language-learners.org/vie ... =15&t=7332
x 2070

Re: Considerations on using dubbed material and the McGurk Effect

Postby Systematiker » Wed Jun 28, 2017 1:52 pm

On the first two videos in the thread, I hear tha-tha both watching and with eyes closed.

In the higher resolution link posted, I heard mostly the Spanish "b", sometimes drifting to the English "b" or "v", but I couldn't figure out what would make me drift in hearing.
1 x

User avatar
jeff_lindqvist
Black Belt - 3rd Dan
Posts: 3135
Joined: Sun Aug 16, 2015 9:52 pm
Languages: sv, en
de, es
ga, eo
---
fi, yue, ro, tp, cy, kw, pt, sk
Language Log: viewtopic.php?f=15&t=2773
x 10462

Re: Considerations on using dubbed material and the McGurk Effect

Postby jeff_lindqvist » Wed Jun 28, 2017 1:58 pm

Ani wrote:I do believe we lose a lot of data on dubs, but I don't think we mis-encode data because the lips are SO far off that it just comes up as -no data-.


This is an interesting thought - if what you hear is a coherent sentence or two but the lips are off, you can probably guess the meaning from context. Dada(ism) and Lady Gaga.
3 x
Leabhair/Greannáin léite as Gaeilge: 9 / 18
Ar an seastán oíche: Oileán an Órchiste
Duolingo - finished trees: sp/ga/de/fr/pt/it
Finnish with extra pain : 100 / 100

Llorg Blog - Wiki - Discord

User avatar
zenmonkey
Black Belt - 2nd Dan
Posts: 2528
Joined: Sun Jul 26, 2015 7:21 pm
Location: California, Germany and France
Languages: Spanish, English, French trilingual - German (B2/C1) on/off study: Persian, Hebrew, Tibetan, Setswana.
Some knowledge of Italian, Portuguese, Ladino, Yiddish ...
Want to tackle Tzotzil, Nahuatl
Language Log: viewtopic.php?f=15&t=859
x 7030
Contact:

Re: Considerations on using dubbed material and the McGurk Effect

Postby zenmonkey » Wed Jun 28, 2017 9:53 pm

Ani wrote:
leosmith wrote:It didn't work for me using that video. This one did though:
https://www.youtube.com/watch?v=PWGeUztTkRA


Ahh yes that one is much better!

I do believe we lose a lot of data on dubs, but I don't think we mis-encode data because the lips are SO far off that it just comes up as -no data-. Dubs are still awesome for what they are.


The thing is, that you may be actively untraining the 'listening eye' in your L2. Dubs are great, just be conscious of how you use them, along with other media (podcasts, audio only) that might effectively reduce your fine tuned listening (which also involves the vision field).

I'll continue to use dubbed material, I'll just try to be conscious of not staring at the speaker's mouth. It might be a minor point, but the demo highlights this aspect of cognitive science that we usually ignore.
1 x
I am a leaf on the wind, watch how I soar

User avatar
Ani
Brown Belt
Posts: 1433
Joined: Mon Mar 14, 2016 8:58 am
Location: Alaska
Languages: English (N), speaks French, Russian & Icelandic (beginner)
x 3840
Contact:

Re: Considerations on using dubbed material and the McGurk Effect

Postby Ani » Wed Jun 28, 2017 10:22 pm

zenmonkey wrote:
Ani wrote:Ahh yes that one is much better!

I do believe we lose a lot of data on dubs, but I don't think we mis-encode data because the lips are SO far off that it just comes up as -no data-. Dubs are still awesome for what they are.


The thing is, that you may be actively untraining the 'listening eye' in your L2. Dubs are great, just be conscious of how you use them, along with other media (podcasts, audio only) that might effectively reduce your fine tuned listening (which also involves the vision field).

I'll continue to use dubbed material, I'll just try to be conscious of not staring at the speaker's mouth. It might be a minor point, but the demo highlights this aspect of cognitive science that we usually ignore.



I don't know if you meant you-general or me specifically, but I know I rely on visual cues for a huge percentage of my listening even in L1. I have a hard time with phonemic processing in general (and passed it down to at least a couple of my kids, hence the reason we work with the Lindamood-Bell LIPS). I often need to see faces to understand in L1, although my hearing is fine. I just can't make heads or tails of the sounds coming out of people's mouths.
So for me, I am certain I am not "untraining my listening eye" as you put it, because I know both consciously and subconsciously that I need that information.
As far as I can tell, the McGurk Effect doesn't tell us whether people can actually get out of the habit of watching or processing visual cues. I know many people "listen with the eyes" much less in their L1 than I do. I wonder if people who watch speech less could suffer from such a rejection of information in their L2? And then I wonder if dubs could actually teach such a thing. I want to say I doubt it since most of us aren't watching dubs for the majority of our waking hours, and watching mouths move during speech is something you literally do from the day you are born.
And one third musing, I wonder if the McGurk effect could cause more significant problems with dubs into closely related languages. -- perhaps Italian and Spanish -- as I would think mouth movements could occasionally be close enough to trick the eye into continuing to look for language clues when there really are none. Maybe this is more what you were thinking of.
2 x
But there's no sense crying over every mistake. You just keep on trying till you run out of cake.

User avatar
zenmonkey
Black Belt - 2nd Dan
Posts: 2528
Joined: Sun Jul 26, 2015 7:21 pm
Location: California, Germany and France
Languages: Spanish, English, French trilingual - German (B2/C1) on/off study: Persian, Hebrew, Tibetan, Setswana.
Some knowledge of Italian, Portuguese, Ladino, Yiddish ...
Want to tackle Tzotzil, Nahuatl
Language Log: viewtopic.php?f=15&t=859
x 7030
Contact:

Re: Considerations on using dubbed material and the McGurk Effect

Postby zenmonkey » Thu Jun 29, 2017 5:05 pm

Ani wrote:I don't know if you meant you-general or me specifically, but I know I rely on visual cues for a huge percentage of my listening even in L1. I have a hard time with phonemic processing in general (and passed it down to at least a couple of my kids, hence the reason we work with the Lindamood-Bell LIPS). I often need to see faces to understand in L1, although my hearing is fine. I just can't make heads or tails of the sounds coming out of people's mouths.


Sounds like auditory processing stuff. No, I meant you in a general way.
I think your post points out some of the limits or possible areas for someone to investigate. Excellent food for thought.

I don't think that dubbed material would significantly (or even insignificantly) undermine a person's 'visual reading' in an L1 but given that we basically have to relearn the phonology of L2 when starting a new language it may hamper this a little, if like me you focus a lot of energy on dubbed material early on.

Now you have me wondering if a LIPS-like method would be useful in early learning of a new language.
1 x
I am a leaf on the wind, watch how I soar


Return to “General Language Discussion”

Who is online

Users browsing this forum: Google [Bot], iguanamon, Zomxilla and 2 guests