The pros and cons of expert academic opinion

General discussion about learning languages
User avatar
emk
Black Belt - 1st Dan
Posts: 1708
Joined: Sat Jul 18, 2015 12:07 pm
Location: Vermont, USA
Languages: English (N), French (B2+)
Badly neglected "just for fun" languages: Middle Egyptian, Spanish.
Language Log: viewtopic.php?f=15&t=723
x 6727
Contact:

The pros and cons of expert academic opinion

Postby emk » Mon Jan 15, 2018 4:05 pm

(Responding to this great post by Cainntear, but in another thread to avoid de-railing.)

Cainntear wrote:Having detailed discussions about the minutiae of language learning isn't some kind of navel gazing, it's a necessary thing if we want to fully understand the process. We have a tendency to talk about the superficial details of every activity, which is all well and good, but while the superficial details of what we do is very, very different, I'm sure there's more to be learned from what we're doing while we're doing the activities than there is from what the activities are.

...That's why I personally like to go back to expert opinion -- even when I disagree with it. If I find something I disagree with, I'll keep it in mind and try to find a reason that it's wrong, or at least not applicable to my situation... but quite often I end up realising that I'm wrong myself.

Expert academic opinion is good.

One of the most valuable lessons I learned from my overpriced, "elite" university education is that:

  1. The experts have an amazing breath and depth of knowledge, and they ought to be respected and taken very seriously.
  2. But some of the experts are still wrong. Sometimes it's just an individual bozo (I knew a couple of professors...), and sometimes it's an entire field's conventional wisdom.
For example, I don't get the impression that the huge, multi-decade obsession with string theory actually helped physics much, though I suppose some would disagree. Similarly, a huge number of influential and famous results in experimental psychology have proven impossible to reproduce. And for a so-called "soft science", experimental psychology is surprisingly rigorous! They have some of the best statisticians of any experimental science outside of actual physics. One of the best applied mathematicians I ever knew would get really enthusiastic about the statistical techniques used in the psychology department. And so if experimental psychology is struggling, what field doesn't need to go back and re-check their most famous papers?

Which brings us to language acquisition research. I actually have read a fair number of papers (though less in a last few years). And it's not very pretty. Methodological problems abound, sample sizes are tiny, some researchers are actually monolingual (which doesn't mean they can't do good work, but I've seen them reach some really weird conclusions), and too much research is based on people spending 5 hours with a toy language instead of studying the entire learning process from beginning to end. Just to pick an example, I feel that there are major weaknesses in Paul Nation's research on vocabulary sizes, and most people misinterpret his results.

I even feel this way even about research I like. For example, I find a lot Dr. Stephen Krashen's work interesting. But if you actually look through his research, it's almost entirely individual case studies, classroom reports, and other forms personal experience without any experimental controls. Compared to all those psychology studies that are getting ripped to pieces by the replication crisis, Krashen's work is far less rigorous.

So, yeah, the research is useful. But I think it needs to be taken with a grain of salt. And there are significant benefits from examining how successful polyglots learn languages. In too many ways, serious language-learning is a still a pre-scientific field. Many of the most useful research I've seen actually came from experimental psychology, such as the forgetting curve. And for things which are specific to language-learning, I've found that a lot of the most interesting work comes from FSI, and not from universities.

(None of this intended to criticize actual linguists writing about specific languages. That's a hugely impressively field, with tons of interesting things to offer.)
21 x

User avatar
MamaPata
Brown Belt
Posts: 1019
Joined: Tue Jun 21, 2016 9:25 am
Location: London
Languages: English (N), French (C1*), Russian (B1), Spanish (B1).

Long lost: Arabic and Latin.
Language Log: viewtopic.php?f=15&t=3004
x 1808

Re: The pros and cons of expert academic opinion

Postby MamaPata » Mon Jan 15, 2018 5:44 pm

While I definitely agree with some of the points made in the original post, I too have my reservations with the idea that expert opinions should be given greater priority/importance than personal experience. Mostly because you can do both! Articles have been linked on the forum that discuss the advantages or disadvantages of x or y, and while I may fully respect those conclusions, I also may not apply them to my own life. Just because something may be well established, doesn't mean it will work for me.

In some cases, that may mean that my method is less effective or more problematic. But I do what I do because I know it fits with my life. Another method might get better results, but if it doesn't suit me and my priorities, I won't actually implement it and then I won't get any results!
5 x
Corrections appreciated.

User avatar
Xenops
Brown Belt
Posts: 1451
Joined: Mon Nov 30, 2015 10:33 pm
Location: Boston
Languages: English (N), Danish (A2), Japanese (rusty), Nansha (constructing)
On break: Japanese (approx. N4), Norwegian (A2)
Language Log: https://forum.language-learners.org/vie ... 15&t=16797
x 3583
Contact:

Re: The pros and cons of expert academic opinion

Postby Xenops » Mon Jan 15, 2018 6:11 pm

emk wrote:
  1. The experts have an amazing breath and depth of knowledge, and they ought to be respected and taken very seriously.
  2. But some of the experts are still wrong. Sometimes it's just an individual bozo (I knew a couple of professors...), and sometimes it's an entire field's conventional wisdom.
...So, yeah, the research is useful. But I think it needs to be taken with a grain of salt.


Something I've encountered as I have applied to research assistant positions is that some researchers, despite the gospel of objectivity in science, still have agendas. The example I'm thinking of is the debate if exercise, specifically FNDC5/irisin (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5419585/ ), a product of muscle usage, positively affects the cognitive function of the brain and learning. In spite of the numerous papers papers stating that yes, this product exists and yes, it has a positive impact on the brain and learning, there are researchers that deny that muscles can have any effect on the brain. I'm trying to find the paper that denies the research of all of these people (when I find it, I'll post it here). Essentially this research group used old instruments and gave the attitude of "all of you researchers think this product affects the brain? Well, based on our research, you are all wrong!"

So yes, take research with a grain of salt.

Edit: Found the relevant papers:

Pro paper for irisin: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4802359/

Con paper for irisin: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4352853/
1 x
Check out my comic at: https://atannan.com/

User avatar
CarlyD
Blue Belt
Posts: 610
Joined: Sat Jul 18, 2015 5:44 pm
Location: California, USA
Languages: English (N), Spanish--A2, German--now studying, A2
x 1337

Re: The pros and cons of expert academic opinion

Postby CarlyD » Mon Jan 15, 2018 6:23 pm

I've read many papers and books about how people learn (non-language related.) Most of them seem to agree that different people take in and learn information in different ways--some can hear a lecture and learn it, another needs to take lots of notes and re-write them before it all sinks in, another needs to explain it back in their own words. None of the experts feel any of these people are wrong, just different in how they learn.

Then I started reading books about language learning. Most focus on a "this is the way you learn a language" and don't take individual differences into account, or even act like they exist. Many are so invested in their opinions that they denigrate any method that doesn't fit in--written flashcards are "old school" or "dinosaur" and shouldn't be used--just looking at the word should be enough.

I'd love to find a language learning expert that also understood the basics of learning styles, but so far that hasn't happened. So I continue to read various articles and books by the experts and pretty much consider them to be personal opinions.
7 x
2024 15,000 pages Reading Challenge--pages: 1180 / 15000

User avatar
emk
Black Belt - 1st Dan
Posts: 1708
Joined: Sat Jul 18, 2015 12:07 pm
Location: Vermont, USA
Languages: English (N), French (B2+)
Badly neglected "just for fun" languages: Middle Egyptian, Spanish.
Language Log: viewtopic.php?f=15&t=723
x 6727
Contact:

Re: The pros and cons of expert academic opinion

Postby emk » Mon Jan 15, 2018 7:03 pm

CarlyD wrote:I've read many papers and books about how people learn (non-language related.) Most of them seem to agree that different people take in and learn information in different ways--some can hear a lecture and learn it, another needs to take lots of notes and re-write them before it all sinks in, another needs to explain it back in their own words. None of the experts feel any of these people are wrong, just different in how they learn.

Ironically, the science supporting the existing of learning styles is supposedly not very rigorous. :-(

The report, authored by a team of eminent researchers in the psychology of learning—Hal Pashler (University of San Diego), Mark McDaniel (Washington University in St. Louis), Doug Rohrer (University of South Florida), and Robert Bjork (University of California, Los Angeles)—reviews the existing literature on learning styles and finds that although numerous studies have purported to show the existence of different kinds of learners (such as “auditory learners” and “visual learners”), those studies have not used the type of randomized research designs that would make their findings credible.

Nearly all of the studies that purport to provide evidence for learning styles fail to satisfy key criteria for scientific validity. Any experiment designed to test the learning-styles hypothesis would need to classify learners into categories and then randomly assign the learners to use one of several different learning methods, and the participants would need to take the same test at the end of the experiment. If there is truth to the idea that learning styles and teaching styles should mesh, then learners with a given style, say visual-spatial, should learn better with instruction that meshes with that style. The authors found that of the very large number of studies claiming to support the learning-styles hypothesis, very few used this type of research design. Of those that did, some provided evidence flatly contradictory to this meshing hypothesis, and the few findings in line with the meshing idea did not assess popular learning-style schemes.

That sounds complicated, but it's pretty simple. If you want to say that learning styles are real, first you need some way to classify students into different styles. Then you need to try different teaching techniques with each group of students. Do the students classified as auditory learners actually do better when taught using auditory techniques? You can find the full paper here, which says that, nope, as far as the authors could tell, you don't get better results by classifying students by learning style and then offering them a specialized curriculum. Which either means that we can't classify people by learning style, or that doing so doesn't appear to help.

Should you believe that paper? Well, that's a tricky question.

However, there are other factors worth considering. Maybe some students find certain approaches frustrating. For example, I hate hate hate classic group audiolingual drills, and would rather stop learning languages entirely than do them. The Rassias method drove me away from language learning for an entire decade. Similarly, I find it almost impossible to "shadow" the way Alexander Arguelles suggests. This may be because I can't simultaneously repeat what I'm hearing in English (which means I could probably never be a simultaneous interpreter). But if I first listen to the audio enough times that it becomes an "earworm," then I can actually shadow it, and yes, it does help. So maybe certain students have a weird inability to apply certain techniques, or certain techniques make them profoundly miserable. And I'm not aware of any research that has been done on this, though I'm sure some exists. (I mean, we know that dyslexic students often learn better from audiobooks than the written word.)

I think the best approach to learning a new skill is to go find people who are good at it, and who take success for granted. Look for the people saying, "Oh, yeah, learning languages takes time, but it's not that hard, and pretty much anybody can do it. Here's how I did it." Or the people saying, "Bench pressing your own bodyweight isn't difficult with the right training program." Then pay attention to those people. You may find that those "experts" disagree. (Benny says speak from day 1! Khaztumoto says don't speak until it happens naturally! Krashen says read lots of books! Idahosa says perfect your accent first!) So at that point, just pick an expert whose techniques seem agreeable, try them, and if they don't work, try another expert's techniques.

The important thing is to never ask advice from people who've never succeeded, or who say it's impossible. Do not ask your monolingual coworkers what language course you should buy! How would they know?

Now, would it be a good thing if somebody would study language students from day one until "professional fluency", for thousands of language learners, and carefully measure the results? Yes! And the FSI has done this. Ironically, they reached the following conclusion:

Lesson 3. There is no “one right way” to teach (or learn) languages, nor is there a single “right” syllabus.

Students at FSI and in other government language training programs have learned and still do learn languages successfully
from syllabi based on audio-lingual practice of grammatical patterns, linguistic functions, social situations, task-based learning, community language learning, the silent way, and combinations of these and other approaches. Spolsky (1989:383) writes, “Any intelligent and disinterested observer knows that there are many ways to learn languages and many ways to teach them, and that some ways work with some students in some circumstances and fail with others.” This matches our experience precisely.

It is also clear, as many have reported, that learners’ needs change over time—sometimes rapidly. Types of activities that worked very well for certain learners at an early stage in a course may be almost completely useless a couple of weeks later for those same learners (Larsen-Freeman 1991: 336–37). At the same time, the lesson plan that works beautifully for “Class B” on Monday morning may not work at all for a “Class C” that is at exactly the same stage in a course. Learning is more efficient when the focus is on providing each learner with what he or she needs in order to learn right now, not on teaching a preset curriculum.

Now, the FSI can pretty ruthless about teaching languages, and they are fully expected to produce specific levels of fluency very quickly. (ILR 3/3 in French in 24 weeks, starting from scratch, which is roughly equivalent to C1. That's absolutely amazing.) And they have a huge sample size.

So personally, I would listen very carefully to what the FSI says. Certainly their insights are more trustworthy that some study that involved 20 people studying a language for a total of 5 hours each. Here, the true experts aren't necessarily the people who study language acquisition at a university. They're the people who produce fluent speakers, reliably, on an industrial scale.
11 x

Cainntear
Black Belt - 3rd Dan
Posts: 3533
Joined: Thu Jul 30, 2015 11:04 am
Location: Scotland
Languages: English(N)
Advanced: French,Spanish, Scottish Gaelic
Intermediate: Italian, Catalan, Corsican
Basic: Welsh
Dabbling: Polish, Russian etc
x 8809
Contact:

Re: The pros and cons of expert academic opinion

Postby Cainntear » Mon Jan 15, 2018 8:46 pm

emk wrote:I actually have read a fair number of papers (though less in a last few years). And it's not very pretty. Methodological problems abound, sample sizes are tiny, some researchers are actually monolingual (which doesn't mean they can't do good work, but I've seen them reach some really weird conclusions), and too much research is based on people spending 5 hours with a toy language instead of studying the entire learning process from beginning to end. Just to pick an example, I feel that there are major weaknesses in Paul Nation's research on vocabulary sizes, and most people misinterpret his results.

I completely agree. Much of this is down to the fact that stats are all too often an afterthought in social science courses, with a brief overview given in a course with a title like "Research Methods". I don't know if the situation is quite as bad in North America, though -- the liberal arts degree with its broad curriculum might mitigate this. Certainly, the last place I taught was an international university modelled on the US system, and one of my friends was a stats teacher, and his favourite classes were teaching stats to arts majors, as they were more interested in the topic than scientists and engineers, weirdly enough.

One thing that really made me distraught moving into education was the number of questionnaires used in the research. When I was studying English about a dozen years ago, I remember being told that questionnaires had come to be considered as utterly useless for social science research because people's answers were never particularly valuable -- they never closely reflected reality because people aren't generally aware of their own thinking. And yet they're not uncommon in SLA research.

I even feel this way even about research I like. For example, I find a lot Dr. Stephen Krashen's work interesting. But if you actually look through his research, it's almost entirely individual case studies, classroom reports, and other forms personal experience without any experimental controls. Compared to all those psychology studies that are getting ripped to pieces by the replication crisis, Krashen's work is far less rigorous.

See, with other researchers, I find it a forgivable oversight -- a result of ignorance. But Krashen's work barely qualifies as research, and there's really no excuse for such a lack of rigour.

Extraordinary claims require extraordinary evidence, and Krashen did little more than assert that he was right.

So, yeah, the research is useful. But I think it needs to be taken with a grain of salt.

Quite right. As I've said in the past: I don't want people to assume I'm right, but to think about whether I'm right or not. I look at research the same way -- I take research that makes a strong case, and compare what it says with what I see. Sometimes I find myself noticing myself doing or saying something and realising that what the research was talking about was something I do, but didn't know I did (and this has happened with both good and bad things!).

MamaPata wrote:I too have my reservations with the idea that expert opinions should be given greater priority/importance than personal experience. Mostly because you can do both! Articles have been linked on the forum that discuss the advantages or disadvantages of x or y, and while I may fully respect those conclusions, I also may not apply them to my own life. Just because something may be well established, doesn't mean it will work for me.

Which is perfectly OK, but the big problem with anecdote is that it is rarely a complete or accurate picture (see also what I say about questionnaires above).

But what good research will always provide is another way to look at and reason about language.
When you're learning a language, it's often much easier to notice (for example) a feature of pronunciation in input if you have been told it exists. The same is true in learning theory -- if you know the idea, you'll be more likely to become aware that you do it yourself.

Xenops wrote:Something I've encountered as I have applied to research assistant positions is that some researchers, despite the gospel of objectivity in science, still have agendas.

Yes, but people who aren't researchers often have agendas too. Maybe they've got a product to sell, maybe they've just got an ego to maintain.
6 x

User avatar
aokoye
Black Belt - 1st Dan
Posts: 1818
Joined: Sat Jul 18, 2015 6:14 pm
Location: Portland, OR
Languages: English (N), German (~C1), French (Intermediate), Japanese (N4), Swedish (beginner), Dutch (A2)
Language Log: https://forum.language-learners.org/vie ... 15&t=19262
x 3310
Contact:

Re: The pros and cons of expert academic opinion

Postby aokoye » Mon Jan 15, 2018 9:05 pm

emk wrote:Which brings us to language acquisition research. I actually have read a fair number of papers (though less in a last few years). And it's not very pretty. Methodological problems abound, sample sizes are tiny, some researchers are actually monolingual (which doesn't mean they can't do good work, but I've seen them reach some really weird conclusions), and too much research is based on people spending 5 hours with a toy language instead of studying the entire learning process from beginning to end. Just to pick an example, I feel that there are major weaknesses in Paul Nation's research on vocabulary sizes, and most people misinterpret his results.

*sigh*
Ok so a few things:
  • Qualitative research is a thing. A really common thing in fact. It might not be common in your field, but it is in various sub-disciplines of linguistics and a number of other fields.
  • Linguistics isn't, by in large, about the researcher learning languages. "How many languages do you speak?" is not a logical question to ask a linguist (despite the question being common enough that linguists who don't know eachother use their frustration of that question as an ice breaker) because that's not what linguistics is about. Moreover, how are you deciding that some of the researchers are monolingual (speaking more than one language isn't something most people advertise). Despite that, I've said it before and I'll say it again, most of the linguists that I know or have met are multilingual.
  • SLA doesn't generally focus on autodidactic learning, it focuses on classroom learning.
  • SLA is only one of the many subdisciplines of linguistics.
7 x
Prefered gender pronouns: Masculine

User avatar
aokoye
Black Belt - 1st Dan
Posts: 1818
Joined: Sat Jul 18, 2015 6:14 pm
Location: Portland, OR
Languages: English (N), German (~C1), French (Intermediate), Japanese (N4), Swedish (beginner), Dutch (A2)
Language Log: https://forum.language-learners.org/vie ... 15&t=19262
x 3310
Contact:

Re: The pros and cons of expert academic opinion

Postby aokoye » Mon Jan 15, 2018 9:23 pm

Cainntear wrote:Much of this is down to the fact that stats are all too often an afterthought in social science courses, with a brief overview given in a course with a title like "Research Methods". I don't know if the situation is quite as bad in North America, though -- the liberal arts degree with its broad curriculum might mitigate this. Certainly, the last place I taught was an international university modelled on the US system, and one of my friends was a stats teacher, and his favourite classes were teaching stats to arts majors, as they were more interested in the topic than scientists and engineers, weirdly enough.
I was at AAAL last year and went to a talk on statistics in linguistics. Needless to say it ended up being standing room only (though part of that was the room choice on the part of the organizers - that was by far not the only talk in a room that was too small) and the primary takeaways were, "take more stats classes" and "we need to teach stats courses in our depts so students have to go to different depts to take these classes."
0 x
Prefered gender pronouns: Masculine

User avatar
Serpent
Black Belt - 3rd Dan
Posts: 3657
Joined: Sat Jul 18, 2015 10:54 am
Location: Moskova
Languages: heritage
Russian (native); Belarusian, Polish

fluent or close: Finnish (certified C1), English; Portuguese, Spanish, German, Italian
learning: Croatian+, Ukrainian; Romanian, Galician; Danish, Swedish; Estonian
exploring: Latin, Karelian, Catalan, Dutch, Czech, Latvian
x 5181
Contact:

Re: The pros and cons of expert academic opinion

Postby Serpent » Mon Jan 15, 2018 10:11 pm

aokoye wrote:Linguistics isn't, by in large, about the researcher learning languages. "How many languages do you speak?" is not a logical question to ask a linguist (despite the question being common enough that linguists who don't know eachother use their frustration of that question as an ice breaker) because that's not what linguistics is about.

Image
:evil:
8 x
LyricsTraining now has Finnish and Polish :)
Corrections welcome

User avatar
emk
Black Belt - 1st Dan
Posts: 1708
Joined: Sat Jul 18, 2015 12:07 pm
Location: Vermont, USA
Languages: English (N), French (B2+)
Badly neglected "just for fun" languages: Middle Egyptian, Spanish.
Language Log: viewtopic.php?f=15&t=723
x 6727
Contact:

Re: The pros and cons of expert academic opinion

Postby emk » Mon Jan 15, 2018 10:23 pm

aokoye wrote:Ok so a few things:
  • Qualitative research is a thing. A really common thing in fact. It might not be common in your field, but it is in various sub-disciplines of linguistics and a number of other fields.
  • Linguistics isn't, by in large, about the researcher learning languages. "How many languages do you speak?" is not a logical question to ask a linguist (despite the question being common enough that linguists who don't know eachother use their frustration of that question as an ice breaker) because that's not what linguistics is about. Moreover, how are you deciding that some of the researchers are monolingual (speaking more than one language isn't something most people advertise). Despite that, I've said it before and I'll say it again, most of the linguists that I know or have met are multilingual.
  • SLA doesn't generally focus on autodidactic learning, it focuses on classroom learning.
  • SLA is only one of the many subdisciplines of linguistics.

First, I would just like to clarify one point: My posts above are specifically about SLA research. I hinted at this here:

emk wrote:(None of this intended to criticize actual linguists writing about specific languages. That's a hugely impressively field, with tons of interesting things to offer.)

Just picking a few subfields of linguistics at random, I have huge respect for descriptive linguistics. I'm fascinated by cognitive linguistics, but I have trouble following it if it gets into neuroscience (because brain anatomy is a slog). Unfortunately, I'm not qualified to comment on large parts of theoretical linguistics, but what little I can understand looks very interesting. (I often find things like basic formal semantics and type theory to be fairly approachable, but only at a beginner level.) And of course, I enjoy papers in computational linguistics and natural language processing. Sure, I'm not particularly a fan of some of Chomsky's work (I agree with much of Norvig's critique, even though statistics is playing slightly less of role in recent computational linguistics), but Chomsky's not the be-all and end-all of linguistics.

So, I'm really not trying to complain about linguistics as a whole. I like linguistics.

Above, I was specifically griping about SLA research. Again, I'm not an academic. But I do occasionally need to read scientific papers in other fields, and I have some idea of what rigorous work actually looks like. Krashen often isn't even trying to be rigorous, of course (for example, this interesting article is basically just pure personal experience). But I've seen plenty of other published papers in SLA that are trying to be rigorous, but that are way below the standards accepted in other fields. These are often the same studies that wind up all over the international headlines.

So maybe I just distrust SLA research because I tend to see the worst examples of it most frequently.

And unlike in many other areas of linguistics, I do believe that multilingual researchers have some advantages in SLA research. I mean, learning a language isn't that hard, and if somebody spends their career studying how people learn languages, then I would hope that they'd try to learn one themselves at some point. Obviously, this is completely irrelevant in many other subfields of linguistics, and there are plenty of amazing monolingual linguistic researchers out there. And I know that many professional linguistics would find this opinion to be laughably naive. :-)
1 x


Return to “General Language Discussion”

Who is online

Users browsing this forum: No registered users and 2 guests