The 'driving license' approach to learning (languages)?

[mrwarper]

Where I live, people must sit an exam to get their driving licenses. The exam is generally deemed so easy that the 'standard' way people prepare for it instead of studying for real is, simply do a bunch of mock exams, and then move on to the real one. It basically works, because the exam is easy.

Now, I see what to me is a totally inordinate amount of students that think such an approach is THE way to go (whereas in reality the one above is practically the only situation where it can work), both to simply pass future exams like level certifications and, even worse, as an actual way to 'learn' their stuff, f.e. languages, or how to solve calculus problems.

The only explanation I see to this apparent explosion in numbers is that they have had a number of early experiences similar to the driving license exam, which would provide a false validation, and from there they simply assume everything must be more or less the same (until reality catches up with them).

I am curious if you see this in other places or fields, and if so, whether you may have found any alternative explanations for the phenomenon.

[golyplot]

It's funny that you should mention calculus, since my AP Calculus teacher made the students take numerous past exams for practice. Of course, that isn't meant to teach the material, just to get people used to taking the tests and to know what kinds of answers are expected and to identify any gaps they need to study more.

[mrwarper]

The 'pasts exams' thing has another down side to it: it may make evident (to students who can't be bothered to study the subject at hand for real) that there's some mechanics involved in solving the exercises, that they didn't notice before. Then they'll go to incredible lengths to learn said mechanics in order to avoid understanding the subject. I seem to recall Richard P. Feynman mentioning the same thing about elementary grade algebra in one of his non-technical works.

Interestingly enough, though, I thing this extra dark side is about the only thing that I haven't seen regarding languages. Other than that, Calculus, languages, or quantum mechanics, it doesn't really matter: I can tell you similar stories about pretty much anything I have taught

[ewomack]

Another factor, though definitely not the only one, is that we seem to be judged more and more by the tests we pass. In many cases, passing a test remains the quickest way to an internship, a job, or some other opportunity. In a civilization that seems to increase its pace rapidly generation by generation, a test, or a certification involving some kind of test, often provides the quickest path to a payoff. Ultimately, the quickest way to pass the test is to study the test. This way of thinking gets reinforced quickly and easily, perhaps beginning with drivers' licenses.

I've seen this repeatedly, almost ad nauseum, in technology fields. I have taken classes geared to passing a test, despite whether one has experience or knowledge of the test's subject matter. I've known people who have passed tests simply because they took such a course and, based at least partially on passing that test, have landed jobs (long term results do vary, but some do make it). I currently hold a certification (which will remain nameless) that involved taking a course and then a test. The course focused on passing the test. This test was the easiest test I have ever taken in my life and I could have easily passed it without the course. Yet people put this certification on their resumes, on their social media profiles, and at the end of their names like a religious title. Seeing it so valued, and thinking it must require severe preparation, I completely over-studied and finished the test embarrassingly far, far within the time-frame allowed. Now I have a recognized and celebrated mark on my resume, which I'm happy about, but only because I took a class that prepared me for a test. It's not hard to imagine how this behavior gets dramatically reinforced.

People who have succeeded via this "I am the tests that I pass" creed, which seems to have grown in recognition in the past years, will probably approach language learning the same way. "Gamification" plays to this and Duolingo, though it has its uses, provides a salient example of "I learned a language because I scored points." The message: you can prove proficiency by passing an exam, so why not study the exam and get moving? Though exams and knowledge don't always correlate, as most of us come to realize, the process probably makes things easier for evaluators. Another example: volumes of books on how to pass the Japanese Language Proficiency Exams (N1, N2, etc.) exist and they likely find a wide market because of the "I just need to pass the test" outlook. Some include walkthroughs of old exams and nothing else. One site I found, years ago, had past N5 exams going back years for download.

The attitude that "to pass the exam and earn the badge equals success" has become ubiquitous as a quick way to get ahead. But, as most people also know, passing an exam sometimes involves a slightly different, or arguably an additional, skill set than the subject matter itself. In my experience, passing some tests requires knowledge of the exam's guiding philosophy, the prevalent test psychology used when composing the exam and knowing what the exam's administrators think a passing grade entails. So, for better or worse, studying the exam has, in many cases, become a prerequisite for passing an exam. The best exams also require a solid grounding in the knowledge studied, but certainly not all exams.

So, if people ask me about a certain test or certification, I definitely tell them not only to study the subject matter, but to study the test itself. I think this thinking will become even more prevalent. Do I think this provides a good base for evaluating someone's skills? All by itself, no, especially not for language learning. But, it seems to be the way things currently stand in many fields.

[Xenops]

I have worked both in the medical and academic research spheres in the past year. For medical, obviously: pass your boards, pass your nursing exams, etc. before you can work in a hospital. A bit more flexible for medical technologists, but many hospitals expect you to pass the ASCP shortly after hire.

For academic research, a PhD is most desired, but work experience counts for a lot. I often see for U.S. job postings requesting a master's +1-2 years experience, or an undergraduate with +4 experience. Industrial research seems to prefer candidates with industrial experience, but if you have enough work experience in general they will consider you.

[BeaP]

I think this phenomenon is connected to the goal-focused, achievement-oriented, self-help-filled (how to be more productive and successful) and competitive wave that has seem to become predominant in studying. It has some benefits, to be honest. When I was at university, we tried to study everything connected to a certain topic, and we might say that it was more 'in depth', but I'm not sure. In most cases it was just impossible, undoable for anyone. It just lead to frustration and the feeling of being overwhelmed. By focusing on important things (for the exam) we might have benefitted more.

You can't pass an exam with zero knowledge. Yes, some mechanisms can be recognised, but in the case of a language exam this familiarity can only bring you some points. (Which might mean a pass in some cases.) But you can't base everything on this, you still have to learn the language, although just a relatively limited part of it, if you know what's expected. At the driving licence test we had situations with pictures and we had to state who can go first, where can we park or what's the speed limit. Things like that. Yes, answering questions at home was more beneficial than reading the whole book, but it lead to useable knowledge.

From a company's point of view certificates give a lot of information that shouldn't be overlooked. Passing certain exams means that a candidate could get prepared for a deadline in something that is connected to the field the company is working on. So they'll probably be able to stand up to other challenges. Those who pass these exams prove that they can work towards goals without questioning them, can focus on the end-point and not get distracted, can deal with the stress connected to these exams, often including interaction with an examiner.

[Le Baron]

From a company's point of view certificates give a lot of information that shouldn't be overlooked. Passing certain exams means that a candidate could get prepared for a deadline in something that is connected to the field the company is working on. So they'll probably be able to stand up to other challenges. Those who pass these exams prove that they can work towards goals without questioning them, can focus on the end-point and not get distracted, can deal with the stress connected to these exams, often including interaction with an examiner.

Yes, this is a major value of exam output for employers and bureaucracies. From a knowledge point-of-view it says very little. When I was an undergraduate there weren't any major movements of 'study strategies' as you might see now, but tutors and others gave advice. Studying with an eye on the exam has probably always existed, but studying solely to pass an exam is a phenomenon you noted:

...connected to the goal-focused, achievement-oriented, self-help-filled (how to be more productive and successful) and competitive wave that has seem to become predominant in studying.

Part of the technocratic approach to producing measured outcomes and easily machine-readable results. Frankly I hate it. All its promoters tend to be technocratic blockheads who don't use information analysis alongside less measurable 'quality'; only measurements with predefined simple goals and tasks to be achieved. 'Do this, but don't think about how or why.'

If you want to actually learn something, testing is useful, but the concept of the big exam is less meaningful to the learning process, especially in the earlier stages. It's something that, if at all ever applicable, should come into focus only after one is far down the path of learning.

[Cainntear]

Where I live, people must sit an exam to get their driving licenses. The exam is generally deemed so easy that the 'standard' way people prepare for it instead of studying for real is, simply do a bunch of mock exams, and then move on to the real one. It basically works, because the exam is easy.

Same here. Actually, my older brother turned 17 the first year there was a separate theory test for the UK driving license. Prior to that, theory was tested by interview during the driving test. There was a worry at the time that the introduction of a multiple-choice theory test would result in a decline in standards -- the response was that it wasn't instead of the interview but as well as the interview. The interview, they said, went into a lot of depth, but not much breadth. The multiple-choice exam would augment it because while it couldn't go into much depth, it would cover the breadth. The two, they said, were complementary; people who claimed that standards would decline were being unnecessary alarmist.

There is no theory interview in the UK practical driving test now. Alarmist my a**e.

Now, I see what to me is a totally inordinate amount of students that think such an approach is THE way to go (whereas in reality the one above is practically the only situation where it can work), both to simply pass future exams like level certifications and, even worse, as an actual way to 'learn' their stuff, f.e. languages, or how to solve calculus problems.

The only explanation I see to this apparent explosion in numbers is that they have had a number of early experiences similar to the driving license exam, which would provide a false validation, and from there they simply assume everything must be more or less the same (until reality catches up with them).

I am curious if you see this in other places or fields, and if so, whether you may have found any alternative explanations for the phenomenon.

I don't really think this is anything new. There's a saying: "what gets measured gets managed", and it's well understood that however much we talk a good fight about how we "teach the subject, not the test", in practice, as soon as there is a test, every teacher is obliged to make sure their students pass it.

There's something of an arms race in testing and teaching: a bunch of academics do a study that shows that a particular type of test is a good proxy for overall proficiency, but as soon as they do that, teachers focus on teaching the test, and it's no longer a good proxy.

One example of this is cloze tests -- the cloze test as originally envisioned measured language ability because it was a task that can be completed without training, and your ability on it is a direct view into your internal language model... until you start teaching it, at which point it ceases to be an unconscious expression, but instead becomes a conscious activity based on processing declarative rules.

Then there's the WebCAPE used for testing out of required language credits in US universities. It doesn't measure all skills, but it's considered a good proxy, because anyone who's learned in a balanced way will do well in it. But Duolingo's claim to be worth 2 semesters of college per X hours is based on performance in the WebCAPE test, which features a series of questions in formats that are basically a subset of what Duolingo uses -- the moment the teaching approximates the measure, the measure ceases to be a proxy measure for the teaching, but a direct measure of the teaching. And if it was supposed to be a proxy measure, then teaching to it means not teaching the subject.

[rdearman]

Prior to that, theory was tested by interview during the driving test.

I had to do this. The guy interrogated me about all kinds of stuff, then he got a little flip book out with all the street signs. He flipped one over and asked what is this one, this one, this one, on and on and on until the end of the book. I was freaking out. At the end of the test, he said; You got them all right. I'm only supposed to show you five, and you need to get three out of five, but I figured I would just keep going until you got one wrong, but you never did.

I had basically memorised the highway code book. I knew how hard the UK test was from all the people I knew who failed it. I was pretty happy to pass first time.

[spoon]

The last two to three years of my high school experience were essentially a full-on preparation to final standardized state-organized 'license' exams in several subjects (some were compulsory for everyone, the rest one could choose). The better your performance on the exams was, the higher the chance of matriculation in a college program of your liking would become; as a result, pretty much anybody who planned to carry on with his/her education was very much interested in getting as good a performance on the exams as they could. Now on the institutional side of things, the school administration displayed a vested interest in making sure the students perform well because of some benefits tied to average scores and passing rates across the entire student body.

What it meant in practice were lots of drills. Tons and tons of drills, and then some. Mock exams, mock exercises, you name it. I think it sucked out all the fun from the process - we weren't learning things because they were interesting, or good to know, or anything; we were learning things and factoids because they might appear in the finals, and what's more, the way they would appear in the finals, while nuance, in-depth knowledge of something you might particularly care about, and alternative approaches to, say, solve a math problem were tossed aside for the most part as barely relevant for improving your overall score.

I can't say that was widely effective for everyone. After all, the point wasn't to teach us math, or history, or geography anymore. The point was to train us to perform in the dreaded finals. Some things you crammed to forget immediately thereafter. Some things you had to relearn because they lacked depth or were simply untrue. As a common side effect of all this mock learning, I would imagine, one could form an expectation that this is the way to go about learning anything else later in life - well, at least if there's an exam involved.

Now as for language learning, I reckon there's a lot of students who pick up studies in order to eventually pass an exam because they need to get that job where they ask you for a certificate, or enroll in a university in a foreign country, or something along these lines. And frankly I wouldn't expect to see these people represented much on forums like this one as the need to attain some L2 proficiency for them is an obstacle rather than a passion; I would imagine such language students want to get it over with ASAP and become functional enough to get certified - which results in a certain min-maxing attitude towards studies. Enter the mock exams: one way to get a better score is to teach yourself the exam.

The irony here is that exams of this kind become easy if you know the subject; it's just that people approaching the subject from the test-taking perspective would learn (and teach) the respective exams instead.