Kaufmann: ChatGPT: Should Language Teachers Be Worried?

Ask specific questions about your target languages. Beginner questions welcome!
Kraut
Black Belt - 2nd Dan
Posts: 2568
Joined: Mon Aug 07, 2017 10:37 pm
Languages: German (N)
French (C)
English (C)
Spanish (A2)
Lithuanian
x 3189

Kaufmann: ChatGPT: Should Language Teachers Be Worried?

Postby Kraut » Wed Apr 19, 2023 8:51 pm

ChatGPT: Should Language Teachers Be Worried?
https://www.youtube.com/watch?v=mFdCD6AxyIM
Download with VideoDownloadHelper

CC subtitles available in multiple languages.: Esperanto, Greek, Hungarian ... Download them here:

https://downsub.com/?url=https%3A%2F%2F ... FdCD6AxyIM

There are various ways ChatGPT can assist in language learning: Duolingo seems to have enlisted ChatGPT mostly for error correction and role playing. These are not areas of interest to me. However, these activities could help lessen the workload of teachers.

0:00 What can langauge students do with ChatGPT? 2:19 They key role of a language teacher it to motivate students. 3:45 ChatGPT isn't going to change the fact that language is social. 5:31 ChatGPT could be an assist to the language teacher. 6:25 How I think ChatGPT will benefit language learners the most.
1 x

User avatar
dylan413
Yellow Belt
Posts: 52
Joined: Thu Jun 10, 2021 5:58 am
Languages: English (N)
Russian (C2)
Italian (B2)
Ukrainian (B1)
Azeri (A2)
French (A2)
Turkish (A2)
Uzbek (A2)
x 221

Re: Kaufmann: ChatGPT: Should Language Teachers Be Worried?

Postby dylan413 » Wed Apr 19, 2023 9:29 pm

I don't think that language teachers should be worried for now because most students, especially young ones, really need a teacher for the sake of motivating them and making sure they study consistently. I'm currently a history teacher, but I also teach SAT classes, and I have taught English in the past. For most languages anyone would conceivably study in any normal scenario there are more than enough resources out there to reach an incredible level of proficiency without a teacher. However, most students still prefer to work with teachers anyway. Learning is a skill in itself. Tell someone to go learn Korean and see how far they will get without outside support vs. with the help of a teacher. In most cases, they'll be completely overwhelmed and not even know where to begin. Of course, in the future, ChatGPT will be able to make better lesson plans and help students structure their learning more effectively than in its current state, but even without that, human contact is where much of the value comes for most people. Khan Academy is an extremely valuable resource for SAT English, for example. It can provide students with just about all of the knowledge that I myself can transfer to them, but my students still prefer to have me teach them rather than learn it on their own.

I myself am working much less with tutors ever since I started using ChatGPT, but I like learning on my own, and I already have a great deal of experience learning languages. Like most independent learners, I structured my approach to each language on my own before ChatGPT. So I was already well prepared to take complete advantage of it when it came out. This is likely the case for a number of people on this forum, but it certainly isn't for most people.

Also, ChatGPT is definitely an asset for language teachers. My colleagues frequently use ChatGPT to help them design custom exercises, tests, worksheets, etc. It saves quite a lot of time on that front providing them with more time to focus on actually preparing their lesson. As a teacher who has made hundreds and hundreds of worksheets on myself, this is a huge timesaver.
4 x

User avatar
Iversen
Black Belt - 4th Dan
Posts: 4732
Joined: Sun Jul 19, 2015 7:36 pm
Location: Denmark
Languages: Monolingual travels in Danish, English, German, Dutch, Swedish, French, Portuguese, Spanish, Catalan, Italian, Romanian and (part time) Esperanto
Ahem, not yet: Norwegian, Afrikaans, Platt, Scots, Russian, Serbian, Bulgarian, Albanian, Greek, Latin, Irish, Indonesian and a few more...
Language Log: viewtopic.php?f=15&t=1027
x 14805

Re: Kaufmann: ChatGPT: Should Language Teachers Be Worried?

Postby Iversen » Wed Apr 19, 2023 10:51 pm

If teachers should be worried then it is mostly because their pupils are as incompetent as a chatbox - and how do you then recognize their own honest contributions?
2 x

User avatar
emk
Black Belt - 1st Dan
Posts: 1511
Joined: Sat Jul 18, 2015 12:07 pm
Location: Vermont, USA
Languages: English (N), French (B2+)
Badly neglected "just for fun" languages: Middle Egyptian, Spanish.
Language Log: viewtopic.php?f=15&t=723
x 5753
Contact:

Re: Kaufmann: ChatGPT: Should Language Teachers Be Worried?

Postby emk » Thu Apr 20, 2023 1:52 am

Iversen wrote:If teachers should be worried then it is mostly because their pupils are as incompetent as a chatbox - and how do you then recognize their own honest contributions?

Oh, I can guarantee you that the chatbot will be more coherent than many of the pupils. The paid version of ChatGPT 4 is capable of passing quite a few college and professional exams. Consider this table:

gpt-4-test-results.png


That's compared against native speakers who would already be taking those exams, who are often better than the median student.

Let's try a writing question from the DALF C2. (I didn't feed it the associated texts, because they were long and included pictures.)

emk wrote:Est-ce vous pouvez répondre à cette question, s'il vout plaît ?

ÉPREUVE ÉCRITE

Sujet 1 : En tant qu’étudiant(e) en sciences technologiques, vous envoyez votre contribution à une
revue scientifique francophone qui, en vue d’un numéro spécial, a lancé un appel à ses lecteurs sur le
thème « 2050 : avec la robotique, une société de loisirs ? Quelle place pour les robots dans la société du futur ? Révolution ou menace ? »

À l’aide du dossier joint et d’arguments personnels, vous rédigez un texte structuré dans lequel vous
prenez clairement position sur la question et proposez des solutions concrètes, en adoptant un style
approprié et convaincant.

700 mots minimum

ChatGPT 4 (paid version) wrote:Titre : La robotique en 2050 : entre révolution et menace, quel équilibre pour notre société ?

Introduction

La robotique et l'intelligence artificielle (IA) sont en plein essor, transformant en profondeur notre société et notre mode de vie. Les progrès technologiques réalisés ces dernières années nous laissent entrevoir un avenir où la robotique serait omniprésente, offrant de nouvelles opportunités tout en soulevant des inquiétudes quant à son impact sur nos vies. La question se pose alors : la société de 2050 sera-t-elle une société de loisirs grâce à la robotique, ou devons-nous craindre l'essor des robots comme une menace pour notre existence ? Dans cet article, nous prendrons position sur cette question et proposerons des solutions pour tirer le meilleur parti de cette révolution technologique.

Révolution technologique et société de loisirs

La robotique a le potentiel de transformer notre société en une société de loisirs, où les robots effectueraient les tâches répétitives, fastidieuses et dangereuses, laissant ainsi aux humains plus de temps pour s'adonner à des activités plus gratifiantes et épanouissantes. En effet, dans le secteur industriel, l'automatisation a déjà remplacé nombre de postes, permettant une production plus rapide et une meilleure qualité. De même, dans le domaine médical, la robotique assiste les chirurgiens pour des interventions plus précises et moins invasives.

De plus, la robotique pourrait contribuer à la lutte contre les problèmes environnementaux. Par exemple, les robots pourraient être utilisés pour nettoyer les océans, surveiller la qualité de l'air et l'eau, et participer activement à la préservation de la biodiversité. En outre, les robots pourraient assister les personnes âgées et handicapées, améliorant ainsi leur qualité de vie et leur autonomie.

Menace et inquiétudes

Cependant, cette révolution technologique soulève également des inquiétudes légitimes. L'automatisation de certaines tâches pourrait entraîner une augmentation du chômage, exacerbant les inégalités sociales et économiques. De plus, la dépendance croissante aux robots et à l'IA pourrait rendre la société plus vulnérable aux cyberattaques et aux dysfonctionnements techniques.

En outre, la question éthique se pose quant à l'utilisation de robots autonomes, notamment dans le domaine militaire, où des "robots tueurs" pourraient être utilisés pour mener des guerres sans intervention humaine. Enfin, l'éventuelle création de robots dotés d'une conscience pourrait soulever des questions sur leurs droits et leur statut au sein de la société.

Solutions et propositions

Afin de tirer le meilleur parti de la robotique et de l'IA, il est essentiel de mettre en place un cadre réglementaire adapté pour encadrer ces technologies. Voici quelques propositions :

  1. Soutenir la formation et la reconversion professionnelle : il est crucial d'anticiper les changements sur le marché du travail et de soutenir les travailleurs dans l'acquisition de nouvelles compétences pour s'adapter à l'évolution des métiers.
  2. Garantir une transition équitable : les gouvernements et les entreprises doivent travailler ensemble pour veiller à ce que les bénéfices de la robotique et de l'IA soient partagés de manière équitable au sein de la société. Cela pourrait inclure des mesures telles que la mise en place d'un revenu de base universel ou la redistribution des gains de productivité.
  3. Renforcer la cybersécurité : afin de protéger la société contre les cyberattaques et les dysfonctionnements techniques, il est nécessaire d'investir dans la recherche et le développement de solutions de cybersécurité robustes et fiables.
  4. Encadrer l'utilisation de robots autonomes : l'élaboration de réglementations internationales pour encadrer l'utilisation de robots autonomes, en particulier dans le domaine militaire, est essentielle pour prévenir les abus et garantir la paix et la sécurité mondiales.
  5. Promouvoir l'éthique et la responsabilité : les concepteurs et les fabricants de robots doivent être tenus responsables de l'impact de leurs créations sur la société et l'environnement. L'intégration de principes éthiques dans la conception des robots et des systèmes d'IA est nécessaire pour garantir le respect des droits de l'homme, de la vie privée et de la dignité humaine.
  6. Impliquer la société civile : les décisions concernant l'adoption et l'utilisation de la robotique et de l'IA doivent être prises en consultation avec les citoyens, les organisations non gouvernementales, les universitaires et les experts, afin de garantir que ces technologies répondent aux besoins et aux préoccupations de la société dans son ensemble.
Conclusion

La robotique et l'intelligence artificielle ont le potentiel de transformer notre société en une société de loisirs, offrant de nombreuses opportunités pour améliorer notre qualité de vie et résoudre les défis environnementaux et sociaux. Cependant, cette révolution technologique soulève également des inquiétudes légitimes concernant le chômage, la cybersécurité et les questions éthiques. En adoptant un cadre réglementaire adapté et en impliquant l'ensemble de la société dans le processus décisionnel, il est possible de tirer le meilleur parti de cette révolution tout en minimisant les risques et les menaces qu'elle peut représenter.

This took it about a minute for it to write. It responds to the prompt (though not, perhaps, precisely the way the examiners would prefer—DALF C2 graders are very picky about structure). It's more of a survey plus recommendations, rather than a single argument. And it certainly shows off the vocabulary and connectors you'd want from a C2 student. You could almost certainly get better results if you fed it a sample question & response pair, so it knew what kind of answers the examiners liked.

I'd need to spend a week reading Le Monde and writing practice essays before I could have any hope of writing in that style. My academic French is rusty from disuse.

This means that you can no longer give students unproctored written exams, unless you have great faith in your honor code.

As for actual teaching, I've found it quite useful for a number of subjects. But you have to really be self-motivated, and you have to know when you can trust it. Which isn't always obvious if you don't already know something about the subject.
You do not have the required permissions to view the files attached to this post.
5 x

User avatar
Iversen
Black Belt - 4th Dan
Posts: 4732
Joined: Sun Jul 19, 2015 7:36 pm
Location: Denmark
Languages: Monolingual travels in Danish, English, German, Dutch, Swedish, French, Portuguese, Spanish, Catalan, Italian, Romanian and (part time) Esperanto
Ahem, not yet: Norwegian, Afrikaans, Platt, Scots, Russian, Serbian, Bulgarian, Albanian, Greek, Latin, Irish, Indonesian and a few more...
Language Log: viewtopic.php?f=15&t=1027
x 14805

Re: Kaufmann: ChatGPT: Should Language Teachers Be Worried?

Postby Iversen » Thu Apr 20, 2023 6:54 am

So AI boxes already write better essays than human pupils, and if they come up with blatant errors that's the result of being fed humanmade nonsens and lies (which basically also is something humanmade humans are prone to do). We say that the boxes don't understand what they utter, but the question is what it actually is that we do when we think that we understand something (pattern recognition and juggling patterns, maybe - I wouldn't exclude the possibility that the machines could be taught a reasonable simulacrum of that). Google translate already learns new languages faster than I do, and the stock markets are based on actions initiated by robots. Hail our new masters...

Which reminds me about Voltaire's dictum at the end of Candide: "Il faut cultiver son jardin". And now I have got one, and for the time being it eats up half my allotted study time. It looks like a sinister plan..
3 x

User avatar
emk
Black Belt - 1st Dan
Posts: 1511
Joined: Sat Jul 18, 2015 12:07 pm
Location: Vermont, USA
Languages: English (N), French (B2+)
Badly neglected "just for fun" languages: Middle Egyptian, Spanish.
Language Log: viewtopic.php?f=15&t=723
x 5753
Contact:

Re: Kaufmann: ChatGPT: Should Language Teachers Be Worried?

Postby emk » Thu Apr 20, 2023 11:50 am

Iversen wrote:So AI boxes already write better essays than human pupils,

Under a great many circumstances, yes. That essay really is quite a work of art, in a very specific way. It includes a brief discussion of environmental issues, which is a favorite trick of students who take these exams. The examiners like students who talk about the environment. And detouring to talk about the environment offers the test taker a chance to build "islands" on a predictable subject. So the wise test taker practices making plausible connections between the question asked, and a topic they feel more comfortable talking about.

But the phrase "En adoptant un cadre réglementaire adapté et en impliquant l'ensemble de la société dans le processus décisionnel," is really quite exquisite. You could use it in almost any essay of this sort. And it shows off some lovely academic vocabulary. If you can master a hundred stock phrases like that, you will make your C2 (or baccalauréat) examiner quite happy.

And the essay has a clear structure and goes about 10% over the minimum word count. None of the early sections are too long or too short, and the conclusion arrives when it should, referring back to points made earlier. It's not the best possible essay you could write about robots. Rather, it's finely-tuned to guarantee good marks on a generic exam.

These models are trained by quite literally doing thousands of "cloze" exercises. And so their greatest strength is creating a plausible-sounding text in a specified style. But in a final phase of training, the models are also trained to be as helpful and inoffensive as possible. So the models would rather write optimistic, well-socialized essays. If you ask them to write a scene from a film noir, they will gently balk at all the smoking and drinking. Generated stories will wrap up nicely, with reconciliations and new-found wisdom all around, and with the smallest plausible number of dead bodies. No Montagues or Capulets will be harmed in a street brawl if there's a way to avoid it.

Iversen wrote:and if they come up with blatant errors that's the result of being fed humanmade nonsens and lies (which basically also is something humanmade humans are prone to do).

Oh, no, these models will happily come up with errors of their own. They're extremely gifted at providing plausible answers, but less skilled at providing correct ones. So they fall back on something called "hallucination", but which you could equally well call "confabulation." GPT 3.5 hallucinates constantly. GPT 4 cuts the hallucination by half, I believe.

This is an advantage of working with a skilled human tutor, rather than using the model as a tutor! The human tutor is unlikely, one would hope, to make up plausible nonsense.

Iversen wrote:We say that the boxes don't understand what they utter,

Some researchers recently did a remarkable experiment where they trained a much smaller GPT model on written series of Othello moves, recorded in standard notation. And from nothing but those written transcripts, the model actually learned to play. And a very clever examination of the model revealed a set of "neurons" which actually recorded the state of an Othello board. During training, the model had realized that it was playing a game, on a square board. And it inferred the size of the board, the spatial relationships, and the legal moves. (Though not perfectly.) And it learned all of this by filling in "clozes" and guessing the next word in a text. Clozes, as it turns out, are far more powerful than anyone ever imagined.

A far more sophisticated model like GPT 4 can actually "reason". Not always well, and not always consistently. But when a father and son tried to teach it the son's conlang, it inferred quite a few grammatical rules from a short parallel corpus, and produced a few attempts at translations to and from the conlang. The conlang was linguistically exotic, so nothing in the model's training could have taught it to "parrot" the answers. Rather, the model was actually listening to instructions, and trying to follow them by doing a translation "manually."

The harder one pushes the models to perform genuine reasoning, the more obvious it becomes that they still have significant weaknesses. Certainly, a completely novel task like translating to and from a brand new conlang will produce results that are clearly better than mere "parroting" of the training text (which after all included no examples of the conlang). But the results are still worse than what a diligent and capable human could do. Of course, a typical human would also do poorly translating a conlang. Humans are also better at regurgitation than genuine reason.

Iversen wrote:Which reminds me about Voltaire's dictum at the end of Candide: "Il faut cultiver son jardin". And now I have got one, and for the time being it eats up half my allotted study time. It looks like a sinister plan..

Well, there was always someone out there who learned languages faster than I could, and someone who wrote wittier essays, and someone with a better grasp of math. There are even programmers who consistently make me realize that I am still the barest novice after decades in the field. I was never bothered that Deep Blue could beat me at chess, any more than I was bothered by the fact that Kasperov could.

If the process is enjoyable in itself, there's no need to be world class. And yes, I could use Google Translate to speak to my in-laws, if they'd sit still for such a barbarity, but I rather prefer being able to do it myself.

But to return to the topic, human tutors have little to fear from ChatGPT. Except, of course, for the ease with which dishonest students can now cheat.

It is certainly possible to get ChatGPT to act as a tutor. But actually getting good results requires quite a bit of cleverness, as well as a keen eye for misleading or incomplete answers. Some students will be able to figure out how to do it on their own, just like some people walk into a gym and teach themselves weightlifting with nothing but books and YouTube videos. And yet, there's still a robust market for personal trainers and coaches to hold people's hand, to provide motivation, and to (very occasionally) provide true expert insight into what's subtly wrong with someone's deadlift. Automated tutoring with current tools will no more replace human tutors than a well-stocked library ever replaced schools and professors. The number of students who would walk into a library and teach themselves was only ever a tiny fraction of the population.

And by the time that these models can entirely replace a teacher, well, we'll have bigger issues to worry about. Perhaps we ought to take ChatGPT's advice, and adopter un cadre réglementaire adapté et impliquer l'ensemble de la société dans le processus décisionnel. Preferably before someone builds a model a lot more clever than this one.
4 x

User avatar
Iversen
Black Belt - 4th Dan
Posts: 4732
Joined: Sun Jul 19, 2015 7:36 pm
Location: Denmark
Languages: Monolingual travels in Danish, English, German, Dutch, Swedish, French, Portuguese, Spanish, Catalan, Italian, Romanian and (part time) Esperanto
Ahem, not yet: Norwegian, Afrikaans, Platt, Scots, Russian, Serbian, Bulgarian, Albanian, Greek, Latin, Irish, Indonesian and a few more...
Language Log: viewtopic.php?f=15&t=1027
x 14805

Re: Kaufmann: ChatGPT: Should Language Teachers Be Worried?

Postby Iversen » Thu Apr 20, 2023 12:11 pm

emk wrote:The human tutor is unlikely, one would hope, to make up plausible nonsense.

I'm not optimistic at all about that. Plausible nonsense goes under different names when humans do it, like politics, wishful thinking or jumping at conclusions, but it is not something new. And inventing nonsense yourself comes as an addition to trusting untrustworthy sources.The more I learn about AI the less I trust the way humans think. There are just too many likenesses already..
6 x

User avatar
Le Baron
Black Belt - 3rd Dan
Posts: 3431
Joined: Mon Jan 18, 2021 5:14 pm
Location: Koude kikkerland
Languages: English (N), fr, nl, de, eo, Sranantongo,
Maintaining: es, swahili.
Language Log: https://forum.language-learners.org/vie ... 15&t=18796
x 9178

Re: Kaufmann: ChatGPT: Should Language Teachers Be Worried?

Postby Le Baron » Thu Apr 20, 2023 1:48 pm

Might as well retire the human race and just leave the AI to it. We'll see if they're any better at politics.

To be honest the excitement around this topic is very tiring now. It's such a one-dimensional view of 'learning'. The deep core of language learning is interaction and the communication/understanding of human feelings, ideas and cultural experience. Not simply 'learning 'facts' and being 'efficient' and getting corrections. There's some of that, especially at the beginning, then it starts fading away.

The faith that somehow the better the AI, the better people will always learn is a curious faith. Any tool is only as good as the user and in this case, because the AI is powerful enough to do something doesn't mean this magically transfers to the user's capacities or benefit! The advent of the internet and smartphone with information and learning tools everyone's fingertips doesn't seem to have made the greater mass of people particularly smarter, perhaps even more stupid. Thinking effort and memory has already been partly abandoned in favour of search engines and cutting and pasting from Wikipedia (or maybe that was already there and was easily transferred). I always thought it was about methods for improving human interaction and understanding.

Teacher's should, as usual, be worried about one major thing: 'service providers' eyeing up AI to eliminate their employment costs and individual students being misled into doing the same.

Mark me down as a naysayer.
7 x

galaxyrocker
Brown Belt
Posts: 1115
Joined: Mon Jul 20, 2015 12:44 am
Languages: English (N), Irish (Teastas Eorpach na Gaeilge B2), French, dabbling elsewhere sometimes
Language Log: viewtopic.php?f=15&t=757
x 3309

Re: Kaufmann: ChatGPT: Should Language Teachers Be Worried?

Postby galaxyrocker » Thu Apr 20, 2023 3:17 pm

Le Baron wrote:Might as well retire the human race and just leave the AI to it. We'll see if they're any better at politics.

To be honest the excitement around this topic is very tiring now. It's such a one-dimensional view of 'learning'. The deep core of language learning is interaction and the communication/understanding of human feelings, ideas and cultural experience. Not simply 'learning 'facts' and being 'efficient' and getting corrections. There's some of that, especially at the beginning, then it starts fading away.

The faith that somehow the better the AI, the better people will always learn is a curious faith. Any tool is only as good as the user and in this case, because the AI is powerful enough to do something doesn't mean this magically transfers to the user's capacities or benefit! The advent of the internet and smartphone with information and learning tools everyone's fingertips doesn't seem to have made the greater mass of people particularly smarter, perhaps even more stupid. Thinking effort and memory has already been partly abandoned in favour of search engines and cutting and pasting from Wikipedia (or maybe that was already there and was easily transferred). I always thought it was about methods for improving human interaction and understanding.

Teacher's should, as usual, be worried about one major thing: 'service providers' eyeing up AI to eliminate their employment costs and individual students being misled into doing the same.

Mark me down as a naysayer.



You need to read Technopoloy by Neil Postman. Written 30+ years ago, never more relevant. I'm after finishing it myself a few days ago. I also agree about what you said about learning and would go so far to argue they don't learn. I really wonder how much input they require, and how much input that is compared to a human that 'learns' to talk; I'd wager the human is doing a lot more with a lot less (like, actually understanding semantics, idioms, etc and not giving back hallucinations), which shows a huge fundamental difference between LLM 'learning' and human learning.

That said, I'm a naysayer myself. Why? Simply because ChatGPT can't detect what is actually correct. This matters a lot for languages like Irish, where most Irish is you see online, and even most Irish found in EU documents, etc., has lots of mistakes and is simply wrong. Sadly, I've already seen people trying to build apps to teach Irish via AI, and it's set to be a mess. Garbage in, garbage out.

Disclaimer: I am very against AI chatbots in general and think they don't solve any worthy problems and will actually just make humanity worse off in pretty much every way. I hate this whole idea of pushing forward with it without regard to societal consequences that everyone seems to have currently.
8 x

Monty
White Belt
Posts: 47
Joined: Sun Mar 28, 2021 12:09 pm
Languages: ...
x 108

Re: Kaufmann: ChatGPT: Should Language Teachers Be Worried?

Postby Monty » Thu Apr 20, 2023 3:38 pm

When AI takes away the jobs of 90% of the people (including 90% of the members of this forum), and AI is used everywhere for surveillance and repression, we'll see how much enthusiasm is left for it.

Don't have a job? Retired already? Good luck talking to a chatbot trying to fix an error in your state pension. Good luck with "I'd like to talk to your supervisor".

At that time the standard refrain will be "But we didn't know!"

Yes, you did.
8 x


Return to “Practical Questions and Advice”

Who is online

Users browsing this forum: Amandine, Dragon27 and 1 guest