Italki Language Test

[lysi]

I got an email from Italki a few days ago about their new online automated language assessments. I know they had an English test for a long time but I haven't seen any information at all about their new test for 7 languages.

https://www.italki.com/languageassessment

Here's some quotes from the site:

Reading, Grammar, and Vocabulary Test
Our multiple-choice adaptive placement assessment is designed to provide individualized evaluation through a combination of grammar, reading, and vocabulary questions. As students proceed through the assessment, the difficulty of the following questions will be adapted based on the previous responses. By offering multiple questions targeted at the student’s level, the assessment can accurately evaluate students’ abilities very quickly. Additionally, the psychometrics used in the scoring algorithm can control for lucky guesses or inadvertent errors that deviate from the expected response model. Once the algorithm is confident in the student’s ability, the assessment ends and the student will receive a score.

Speaking Test
Our fully-automated speaking assessment utilizes artificial intelligence to evaluate students’ speaking ability through the use of innovative linguistic methods, such as elicited imitation (EI). Elicited imitation, which is a fancy way to say “listen and repeat”, is a research-proven method to efficiently predict test-takers' oral language proficiency. It measures how effectively test-takers' brains can process or “chunk” language information. Higher proficiency in the language enables a test-taker to process longer and more complex sentences.

Has anyone done it? Any thoughts on this?

I'm surprised that I haven't seen this mentioned anywhere and I really can't find any talk about it online, other than 1 post from a month ago on the language learning subreddit that has 7 replies. Looking at the internet archive it seems they added it on the 22nd of January of this year.

[Cavesa]

I've just taken it.

I got the email too a few days ago. I was curious. And as the readers of my log know, I've been trying to push myself to finally take a Spanish exam for years! It is also good, because I had no clue about my real level after a longer gap (that's why I paid for this).

The testing method looks rather interesting, even though limited in some ways. Btw here is what italki writes about the exam on the support board https://support.italki.com/hc/en-us/art ... ssessment- but it is not that informative either. It is a very new thing, so you won't probably find many reviews online. I haven't found any.

So, what was it like and what were the results. The overall grade was C1, with C1 for the "grammar part" and B2 for the "oral part". As you see, my main question "should I prepare for DELE B2 or C1?" rests unresolved

The first exam is on speaking. As a short video explains, the first part is based on research showing that an average person holds 7 items in the short term memory. When you're beginning, it's just 7 sounds in the language. As you improve, you remember bigger and bigger chunks and are able to repeat them. The test serves you longer and longer, and more complex, sentences to repeat. Not easy, but I still see how the AI grades that.

The second part of the first exam are 60s answers to open questions. I did well in the first two, in my opinion, but really badly in the third one. I will not tell you the questions, to not spoil it and affect your results, should you decide to take it. This is something I am not really sure about, how does the AI grade your speaking, what is being evaluated and how.

I got 7.6 exam points, which is B2. They also offer equivalents on other scales, namely Advanced Low - Advanced Mid ACTEL, 6.5 - 7
IELTS, 22-25 Toefl, 170-180 TOEIC. As you can see, the exam is originally meant to test English and only later adapted for the other languages. Not sure whether it affects the quality.

I think this exam was surprisingly correct, I think I am B2ish, even though with rather unusual set of strengths and weaknesses. I think it serves well for one's orientation, but of course a normal oral DELE exam is the golden standard.

The second exam is on grammar and reading. Multiple choice. The website tells you it will take 15-30 minutes. I finished in 2 minutes.

They say it is well done and talk about how it eliminates guesses and so on. Well, I had expected a much longer and more complex exam, truth be told. I don't know, whether I should trust the result, which was 897.0 points and therefore C1. No idea, whether the test goes up to C2 (for example SIELE doesn't), or whether the questions are selected based on the result from the first exam (to not annoy an advanced learner with basic questions and the opposite). I found the reading bits very easy, and most of the grammar questions too. No idea where I lost points, or what is the maximum (so how many things I had probably wrong).

I don't know how reliable this result is. I know my writing (of normal texts) is most probably not C1, no idea whether it is even B2. My comprehension is C1 or the listening one even C2. My grammar and vocab? Not sure.

I had expected to be disappointed with the oral exam (we all know that oral computer graded exams are rather controversial) and satisfied with the grammar one. It's the opposite. The oral exam is a very pleasant surprise and in my opinion does have value for your orientation. I have more doubts about the grammar one. I'd say the best online exam for Spanish and French grammar is on Kwiziq, but it doesn't test other stuff, and isn't available for other languages (but it is free!).

No regrets concerning the money, I think I got the value, especially in the speaking part, and have ideas on how to continue. In spite of all the previous failed plans, I'd like to take a real Spanish exam this year. Finally.

[jmar257]

I'd say the best online exam for Spanish and French grammar is on Kwiziq, but it doesn't test other stuff, and isn't available for other languages (but it is free!).

No regrets concerning the money, I think I got the value, especially in the speaking part, and have ideas on how to continue. In spite of all the previous failed plans, I'd like to take a real Spanish exam this year. Finally.

Great, in-depth review. I have some time in a couple of weekends and I may spring for the iTalki test, just to see where I'm at. I've tried Kwiziq before and liked it but didn't love it, but may be convinced to shell out for a few months to see how much I think I will stick with it long term (although I have enough on my plate at the moment, really).

Edit: How long did it take? I saw you said one portion took 2 minutes, but overall.

[Cavesa]

The oral part took me 16 minutes, which is what it was supposed to take (more or less). As it is more structured (you listen to stuff and then record stuff), it's impossible to do it much faster. Really, I think my fast completion of the grammar test is a flaw in design, it is really suspiciously short.

I'd like to hear from someone testing at the A2 or B1 levels, as I'd say the exam is likely to be targeted towards that public. I could try that, but I don't feel like spending the same amount of money to test either my Italian nor my German right now.

But comparing my experience to someone at those levels might give us a much more complete picture.

[jmar257]

The oral part took me 16 minutes, which is what it was supposed to take (more or less). As it is more structured (you listen to stuff and then record stuff), it's impossible to do it much faster. Really, I think my fast completion of the grammar test is a flaw in design, it is really suspiciously short.

I'd like to hear from someone testing at the A2 or B1 levels, as I'd say the exam is likely to be targeted towards that public. I could try that, but I don't feel like spending the same amount of money to test either my Italian nor my German right now.

But comparing my experience to someone at those levels might give us a much more complete picture.

I would bet my French would be at those levels but I wanted to see where it graded my Spanish--I wouldn't be shocked if my oral skills were B1 or so, but since I can read novels without a dictionary I wouldn't expect reading to be that low. That said, I also don't feel like spending $60 on that in a short span, and in general don't see the utility in (paid) tests at the lower levels. I'll definitely post here in a couple weeks when I get around to doing the test, or before if I get the chance.