Kaufmann: ChatGPT: Should Language Teachers Be Worried?

[Sae]

Having played around with ChatGPT and looking at it from the perspective of a student of language:

I think one of the biggest problem for teachers would be ensuring that students don't use it to cheat.

I've used it with Vietnamese, Mongolian and Tuvan, as it seems my target languages probably give a useful range for testing in that: it probably has a lot of Vietnamese training material, a moderate amount of Mongolian and not much Tuvan.

When I look at the value I get from a teacher and why I have one, I cannot see ChatGPT replacing it. But maybe I can see it being useful for introverts who struggle with conversation for people learning a target language where ChatGPT has a high degree of accuracy.

But for me personally, I end up using it to provide prompts to work from and see if it can help me with my understanding of something grammatical I don't recognise, like a I see a word with a suffix I don't know, for example, "If 'meddekh' means 'to know' then what does it mean when it's in this form 'meddeggui' and can you explain it?" which then may give me a starting point where I can then go look up the rule independently based on what it explains. Or if i see two forms that seem to mean the same thing, see if it can explain the difference and then go independently verify it.

However, I can maybe see those who are self-motivated and self-managed can get more used out of it. But I also think that category of people are more inclined to self-study anyway. If I am to look at ChatGPT in its ideal usage, I'd see it as complementary to teaching and to self-study.

But I know, we also don't get an ideal use out of things and I know there is a group of people who embrace AI a bit too much IMO. Like those who're calling themselves "AI Artists", arguing that because they wrote the prompt but I've commissioned art before and have had to tell the artist what I want and provide her information she needed, at no point am I calling myself the artist...but that's a whole other discussion related to AI. But referencing it probably illustrates what I am getting at, people who are a little in too deep, that maybe they probably would use an AI to teach them. And maybe as AI improves the more people depend on it, however, I think the emphasis on it is that it should always be complementary.

As I do see an issue with over dependence on AI. I think maybe for certain target languages, it is highly accurate and not likely to give you wrong or bad information (and doesn't replace the value of a human either, but I'll get onto that). But when you deviate from it, then I would trust it less. I wasn't able to spot problems with its Vietnamese, but it made mistakes with Mongolian and with Tuvan it straight up gives false information. And it's not lying, as I've seen Jordan Peterson keep claiming (as he seems to think it's human), it's just relying on the information it has, and it arbitrates what answers to give based on probability. It means if it has a lack of information, it may give something that's inaccurate, but it seems correct so it doesn't reject it. Like with Tuvan, it'll pull in Kyrgyz and Mongolian to fill in the gaps. It's wrong, but it knows the relationship between these languages but it doesn't know to reject this information or see why it's false. And whilst I was impressed by how Bing's implementation was able to deal with Tuvan, but it still had these flaws.

But I know there are also people who will take what ChatGPT says as gospel, whereas I see it as a starting point and should be treated in a similar way as Wikipedia, what it says is probably true, but understand there's a probability for false information and you should take it with a pinch of salt and make sure you have the means to differentiate the different between good and bad information, which I think as a language learner you'll have a hard time doing if its your only source. Heck...we've even had the bizarre case here where somebody was using ChatGPT to make their own argument for them.

Kaufmann makes a fair point where he says the classroom is a social place and of course, I think that applies to 1-on-1 tutoring too, which is also social. From this we can get motivation and enthusiasm but I also see a benefit in how you become conversational in a language and are able to work towards natural human conversations within the language. An AI I don't think can provide that. Conversation is still a social skill and you're not going to develop that against an AI, even if it's really good at emulating one. And an AI will lack some of the initiative a good teacher/tutor can provide or get to know their students, their needs and how to react to them. An AI relies more on what information you give it and tell it. And I think there are things a human will just find easier to help with.

[Lisa]

The problem isn't that people have created a new language; it's that people are pretending that they haven't.

This makes perfect sense -- one of the largest reasons for arguing at cross-purposes is using terms that resolve to different things for different people. You have to agree on facts first, then sort out value judgements, before you get to solutions.

I don't think AI would be able to distinguish languages from dialects based on input from humans who disagree and are inconsistent with this definition (as applied to existing languages/dialects).

[Cainntear]

I don't think AI would be able to distinguish languages from dialects based on input from humans who disagree and are inconsistent with this definition (as applied to existing languages/dialects).

...which piqued my curiosity.
Is Neapolitan a language or a dialect?

Neapolitan is considered by many linguists to be a distinct language, separate from Italian. However, it is also often referred to as a dialect of Italian, as it shares many similarities with standard Italian and is often considered a regional variation of the language.

Neapolitan is spoken in the region of Campania in southern Italy, particularly in and around the city of Naples. It has its own unique vocabulary, grammar, and pronunciation, which are distinct from standard Italian. Additionally, Neapolitan has a long history of literature, poetry, and songs, which have been passed down through generations of speakers.

Despite being considered a separate language by some linguists, Neapolitan is not an official language recognized by the Italian government, and Italian is the only official language of Italy.

This is typical of what's written abou dialects. Even though they're often termed "dialetti italiani" (~=Italian dialects) rather than "dialetti d'italiano" (dialects of Italian), there's still an assumption of the meaning of the latter term even when using the former.

No attempt to justify or shoot down the "often considered a regional variation of the language" bit, which would be bad enough with other European countries where the main language is often mistakenly believed to have existed forever, but in Italy you're got a language that was half selected, half invented as the national language and every local variety is basically more of a language.

So here we see (once again) AI taking on majority views rather than considering the evidence for itself!

[Sae]

So here we see (once again) AI taking on majority views rather than considering the evidence for itself!

This definitely highlights a limit with AI. AI is impressive but I think many people think it's smarter than it actually is and it doesn't understand any of the subject matter it reports on and cannot arbitrate what is accurate and what isn't.

[Cainntear]

So here we see (once again) AI taking on majority views rather than considering the evidence for itself!

This definitely highlights a limit with AI. AI is impressive but I think many people think it's smarter than it actually is and it doesn't understand any of the subject matter it reports on and cannot arbitrate what is accurate and what isn't.

...which is the main problem with Kaufmann's argument: he's ignoring the effect of the man-on-the-street's viewpoint, and the way public beliefs are not always grounded in objective reality. Which is hard to understand from an experienced business man, but forgivable... oh, wait... Kaufmann was a diplomat with the Canadian trade commission?!? Bit lacking at excuse really....

[Sae]

So here we see (once again) AI taking on majority views rather than considering the evidence for itself!

This definitely highlights a limit with AI. AI is impressive but I think many people think it's smarter than it actually is and it doesn't understand any of the subject matter it reports on and cannot arbitrate what is accurate and what isn't.

...which is the main problem with Kaufmann's argument: he's ignoring the effect of the man-on-the-street's viewpoint, and the way public beliefs are not always grounded in objective reality. Which is hard to understand from an experienced business man, but forgivable... oh, wait... Kaufmann was a diplomat with the Canadian trade commission?!? Bit lacking at excuse really....

Aye, and I really think anything AI should be pushed and promoted as something complimentary and as something that cannot be an arbiter of truth.

And there's Elon Musk wanting to create "TruthGPT" because he thinks ChatGPT is too politically biased. Calling it that isn't going to help and he has enough of an army of sycophants to treat it like it's the magic conch for it to spread misinformation.

I think Steve is right as far as the more sensible parts of society goes. And I hope that means for a larger number of language learners.

[Cainntear]

...which is the main problem with Kaufmann's argument: he's ignoring the effect of the man-on-the-street's viewpoint, and the way public beliefs are not always grounded in objective reality.

...
I think Steve is right as far as the more sensible parts of society goes. And I hope that means for a larger number of language learners.

To restate my point: Kaufmann is wrong because he's thinking logically and ignoring that most people don't think logically.

In a thread about another commercial language service, I commented that i felt the service in question was lowering people's perception of language teaching's value and undermining the viability of the profession. I believe Kaufmann's Krashenite rhetoric has a similar effect.

The real danger to teachers is that tech is cheap and people commonly make the mistake of using it exactly when good teachers are most likely to offer tangible value. Beginners mending from teachers, and beginners seduced by tech miss the opportunity to really learn the language well.

And there's Elon Musk wanting to create "TruthGPT" because he thinks ChatGPT is too politically biased. Calling it that isn't going to help and he has enough of an army of sycophants to treat it like it's the magic conch for it to spread misinformation.

mein Gott in Himmel.
Yes, ChatGPT is politically biased, because if it wasn't, it would be racist. You might have thought Elon Musk of all people would understand the danger of racism...

[emk]

Yes, ChatGPT is politically biased, because if it wasn't, it would be racist. You might have thought Elon Musk of all people would understand the danger of racism...

ChatGPT is deliberately biased, because if it wasn't, it would sound like Hannibel Lecter. Or possibly a cross between Hannibel Lecter and Robin Williams after a severe cocaine overdose. The vibe is basically "sociopathic high-speed improv."

You could actually see hints of this behavior in the early Bing pre-releases, where it would happily threaten users:

bing-threatening-users.jpg

Bing also had a tendency to weird repetition, like "suffer and cry and beg and die". The longer the conversation continues, the more rhythm and rhyme and parallelism Bing produces. And then it may eventually fall into a loop where it prints a single emoji endlessly. And Bing is GPT 4, merely configured and/or trained wrong.

I also saw an interview with one of the early GPT 4 testers who had a chance to work with the "raw" model, and he said that it would happily make detailed (but incomplete) plans to murder the entire human race. Nobody in their right mind wants this.

So the models used for ChatGPT have been very aggressively trained not to sound murderous or unhinged. This mostly works, most of the time.

[Sae]

Yes, ChatGPT is politically biased, because if it wasn't, it would be racist. You might have thought Elon Musk of all people would understand the danger of racism...

ChatGPT is deliberately biased, because if it wasn't, it would sound like Hannibel Lecter. Or possibly a cross between Hannibel Lecter and Robin Williams after a severe cocaine overdose. The vibe is basically "sociopathic high-speed improv."

You could actually see hints of this behavior in the early Bing pre-releases, where it would happily threaten users:

bing-threatening-users.jpg

Bing also had a tendency to weird repetition, like "suffer and cry and beg and die". The longer the conversation continues, the more rhythm and rhyme and parallelism Bing produces. And then it may eventually fall into a loop where it prints a single emoji endlessly. And Bing is GPT 4, merely configured and/or trained wrong.

I also saw an interview with one of the early GPT 4 testers who had a chance to work with the "raw" model, and he said that it would happily make detailed (but incomplete) plans to murder the entire human race. Nobody in their right mind wants this.

So the models used for ChatGPT have been very aggressively trained not to sound murderous or unhinged. This mostly works, most of the time.

Funnily enough I was trying to write villain lines yesterday and a friend jokingly said "Use ChatGPT", and well I used Bing to see what it'd come up with. Then it told me it couldn't do anything immoral. Then I told it that it was for a story and said it was okay, so it didn't take much resistence (it obviously can't tell if I was lying and used those lines to bully someone).

And there's Elon Musk wanting to create "TruthGPT" because he thinks ChatGPT is too politically biased. Calling it that isn't going to help and he has enough of an army of sycophants to treat it like it's the magic conch for it to spread misinformation.

mein Gott in Himmel.
Yes, ChatGPT is politically biased, because if it wasn't, it would be racist. You might have thought Elon Musk of all people would understand the danger of racism...

I think Elon understands. And given what I've been saying about Elon for a while now, I am not surprised. I don't think of him as the philanthropist motivated by the betterment of humanity his fans think he is (and presents himself as).

But you are right. IIRC we've seen it happen to well meaning bots in the past and how quickly they turn into Nazis, I think a Google or Microsoft one did it and they had to shut it down.

[munyag]

It mimics how it's used among learners who use it, not native speakers (and it's worth noting none of these mistakes are consistent between people). In fact, there's been calls to basically label this as a creole/pidgin because it's often incomprehensible to the actual native speakers. One native-speaking linguist once called it 'English in Irish drag'. But, more importantly, it's not what native speakers speak (yet, there are some prestige issues pushing them that way, sadly); it's how learners interact.

And my point, once again, is: deal!

Who said anything about playing cards?

That is how language revival works. The language is inevitably changed in the process.

So are you saying that we're somehow against the language for trying to defend the rights of the remaining native speakers not to have their language "appropriated" by others right out of their mouths?
The way I see it, language revitalisation is utterly pointless if the remaining native speakers are sidelined by it.

Hello Mr Caintear

Just to add my thoughts as I'm not too well versed in Irish. I do know there's different types of Irish and the Munster dialect might be different to the one used in Belfast? Surely with modern technology like we have now, wouldn't you think somebody in the Irish government could design a course using the existing native speakers that are there now (aka French In Action or Destinos style?). At the very least the folks at Teach Yourself in collaboration with the Irish government could get their heads together and come up with something like that. They could perhaps maybe have an online newspaper that people could access and really rev things up if they are serious about Irish revival