Not even AI chatbots can distinguish between present perfect and past simple...

[battlegirl]

Every time I hear of a new AI chatbot I test it with this prompt:

Act like an expert ESL teacher. Create a fill-in-the-gap exercise to practice the difference between past simple and present perfect.

What I've been getting is exercises with 9 sentences in the present perfect, and one in the past simple. I try to give it extra prompts, pointing out that fact, and asking for a more balanced exercise, with 50-50 ratio.

No chatbot has been able to do it!

Today I tried it with Bard, and here are my additional prompts to try and correct it:
- All the answers use present perfect, except for one. Can you make it 50-50?
- I don't know what your idea of 50-50 is, but only 2 out of 10 sentences are in the past simple...
- I think you need to review the meaning of past simple.
(and it did, and even pointed out how it had been wrong...)
- Can you now rewrite the exercise to have 5 sentences in the past simple, 5 in the present perfect?
- No, only 3 are in the past simple. Try again.
- You haven't corrected your mistake. It's still wrong.

... And at this point I gave up. It just couldn't do it.

Every chatbot I try this with does the same thing.

Am I doing the prompts wrong? Is there an innate flaw in the language model where present perfect is used so much more than past simple, and so it just can't imagine a more balanced exercise..??

I'd love to use these chatbots to help me create additional exercises for my students, but the few times I've tried I've had such half-assed results, I always end up just doing it myself, as it has always been.

Any tips would be appreciated!

[diaconia]

I used to use sentences with key words so that foreign students can easily recognize when to use Simple Past and when to use Present Perfect. These are supposed to be for beginning to intermediate students. Eventually they start to read more and develop a feel for the langauge and how we use it to express time relations between actions. You might be able to teach the bot the key words so that it can construct simple sentences like these. I'd personally rather teach kids than AI

She ____________________________________ you to call her yesterday. (want)
She ____________________________________ you to call her for days. (want)

Answer, the first sentence is Simple Past "wanted" and the second is Present Perfect "has wanted"

If you'd like more of the sentences I still have the exercises, so PM me and I can send you a PDF

[battlegirl]

Thank you so much for the tips and the help offer! But... that wasn't my point.

I have been teaching for a while, so I have tons of material. My testing is not about the grammar point of present perfect vs past simple.

It's whether a chatbot can actually teach me grammar, any grammar point, or a language altogether, with accuracy.

I keep hearing about Duolingo or various language platforms offering services that are integrated with an AI chatbot. Or many people here suggesting using ChatGPT to create exercises, or write stories, or have a conversation... But, as proven by my testing, the AI currently still sucks at it.

Conversations are usually ok, but in addition to the present prefect vs past simple test, I've also tried asking the chatbots to write stories in English using certain vocabulary, or certain tenses. I often have to revise many details before I get something I would hand over to a student of mine.

So, until I get a chatbot to reply correctly to this testing, I won't consider it as a tool for me to learn any other language, as I'd be afraid it'd teach me something wrong.

...And that was my point

[battlegirl]

Addendum: My question still stands. Do chatbots really suck, or am I writing the prompts wrong?