Dialang validity and reliability?

All about language programs, courses, websites and other learning resources
User avatar
tiia
Blue Belt
Posts: 735
Joined: Tue Mar 15, 2016 11:52 pm
Location: Finland
Languages: German (N), English (?), Finnish (C1), Spanish (B2??), Swedish (B2)
Language Log: viewtopic.php?t=2374
x 1997

Re: Dialang validity and reliability?

Postby tiia » Tue Dec 12, 2017 9:24 pm

MorkTheFiddle wrote:FWIW, I answered 25 of 28 questions correctly on the English exam. As a native English speaker, perhaps I should have done better. The test consumed a bit of time, so I did not try the others.
I did score 1000 on the placement test.

Was it the writing test or another one?
0 x
Corrections for entries written in Finnish, Spanish or Swedish are welcome.
Project 30+X: 25 / 30

User avatar
trui
Orange Belt
Posts: 111
Joined: Sun Jun 04, 2017 5:54 pm
Languages: Native: English
Good enough: Dutch
Not there yet: German
Just starting: French
We'll see?: Russian
Language Log: https://forum.language-learners.org/vie ... 3&start=10
x 215

Re: Dialang validity and reliability?

Postby trui » Wed Dec 13, 2017 1:20 am

Don't have time to do all of the tests now, but here's what I got for Dutch:

503 on the placement test
Listening - C2 (26-30).
Writing - C2 (23-29).

If this is what C2 is, then I agree that there should be D1 and D2 levels :lol: Given that I still have trouble understanding lots of what's said on the radio and on lots of tv programs, I highly doubt I'm C2 in listening and as for writing, the test was basically a grammar and vocabulary/reading test. Now to be fair, it's very difficult to do a multiple choice test for writing but I still think it could have been better. Also, I don't think I'm C2 in writing. Maybe if you only consider grammar, but even then I'd say B2 or C1, not C2.

But hey, who cares about the validity. This means I don't have to study anymore, right, right? :D
Last edited by trui on Wed Dec 13, 2017 3:19 am, edited 1 time in total.
2 x
All comments and corrections welcome.

User avatar
trui
Orange Belt
Posts: 111
Joined: Sun Jun 04, 2017 5:54 pm
Languages: Native: English
Good enough: Dutch
Not there yet: German
Just starting: French
We'll see?: Russian
Language Log: https://forum.language-learners.org/vie ... 3&start=10
x 215

Re: Dialang validity and reliability?

Postby trui » Wed Dec 13, 2017 3:08 am

Did the other tests because I like praise, even if said praise is very unreliable. :lol:

Reading: C2 (25/29)
Listening: C2 (26/30)
Writing: C2 (23/29)
Grammar: C2 (25/30)
Vocabulary: C2 (28/30)

Am I good at Dutch? Sure. Am I C2, well, I think not.
1 x
All comments and corrections welcome.

Ingaræð
Orange Belt
Posts: 170
Joined: Sat Nov 26, 2016 9:34 pm
Languages: English (N), German (heritage)
Learning: Russian, French, German, Mandarin, Arabic, Spanish.
Mostly forgotten: Italian, Welsh.
x 377

Re: Dialang validity and reliability?

Postby Ingaræð » Wed Dec 13, 2017 5:13 pm

Just over a year ago I did the Dialang (Beta) reading test for French and German, and got a C1 and A2 respectively. At the time, I felt they were both over-estimations of my actual level (especially the French). Reading this thread, I'm now wondering whether Dialang wasn't actually that far off, but I was misunderstanding exactly what it was testing.

For example, imagine you're at a busy playground, some kids are having an argument about a ball, or whose turn it is next on the swings or something. You can understand everything they're saying. This understanding may only require an A2 vocabulary, but the manner in which these kids are speaking (fast, overlapping, with a higher pitch than normal) and all the other kids being loud - maybe some background traffic noise too - require a B2 or C1 for full aural comprehension.

Conversely, if you're listening to a non-fiction audio book about economics or engineering, then maybe you'd need C1 vocabulary, but only B1 listening skills to fully understand everything (because the speaker enunciates clearly and is talking at an average speed).

I didn't believe the test results I got because I knew my vocabulary was pretty poor, and in real life I wouldn't be able to understand the majority of what I was trying to read. But maybe my processing of the vocabulary I did know was better than I realised, and I couldn't make an accurate judgement myself because I was conflating the two.

Or maybe it's just wishful thinking.

(Of course, Dialang will make mistakes sometimes, but then 'real world' testing won't be 100% accurate all the time either. Also, I haven't taken any CEFR exams: my examples/conclusions are based upon things said by those who have.)
4 x

User avatar
MorkTheFiddle
Black Belt - 2nd Dan
Posts: 2114
Joined: Sat Jul 18, 2015 8:59 pm
Location: North Texas USA
Languages: English (N). Read (only) French and Spanish. Studying Ancient Greek. Studying a bit of Latin. Once studied Old Norse. Dabbled in Catalan, Provençal and Italian.
Language Log: https://forum.language-learners.org/vie ... 11#p133911
x 4824

Re: Dialang validity and reliability?

Postby MorkTheFiddle » Wed Dec 13, 2017 6:30 pm

tiia wrote:
MorkTheFiddle wrote:FWIW, I answered 25 of 28 questions correctly on the English exam. As a native English speaker, perhaps I should have done better. The test consumed a bit of time, so I did not try the others.
I did score 1000 on the placement test.

Was it the writing test or another one?

The listening test.
0 x
Many things which are false are transmitted from book to book, and gain credit in the world. -- attributed to Samuel Johnson

User avatar
trui
Orange Belt
Posts: 111
Joined: Sun Jun 04, 2017 5:54 pm
Languages: Native: English
Good enough: Dutch
Not there yet: German
Just starting: French
We'll see?: Russian
Language Log: https://forum.language-learners.org/vie ... 3&start=10
x 215

Re: Dialang validity and reliability?

Postby trui » Wed Dec 13, 2017 10:10 pm

tiia wrote:It felt like the responses in Finnish had larger differences than in English, therefore the answer in Finnish was often more than obvious, even if you might not have been able to produce the sentence yourself.


I really do think that the dialang tests vary by language in terms of difficulty and reliability. After all, it'd be pretty impressive to be able to devise accurate tests for so many languages so it's no surprise that they haven't met that goal.

Ingaræð wrote:Just over a year ago I did the Dialang (Beta) reading test for French and German, and got a C1 and A2 respectively. At the time, I felt they were both over-estimations of my actual level (especially the French). Reading this thread, I'm now wondering whether Dialang wasn't actually that far off, but I was misunderstanding exactly what it was testing.

For example, imagine you're at a busy playground, some kids are having an argument about a ball, or whose turn it is next on the swings or something. You can understand everything they're saying. This understanding may only require an A2 vocabulary, but the manner in which these kids are speaking (fast, overlapping, with a higher pitch than normal) and all the other kids being loud - maybe some background traffic noise too - require a B2 or C1 for full aural comprehension.


I see what you're saying, but while I can't speak for the French and German tests, I know that the ones for Dutch are off. My listening and vocabulary aren't C2 right now no matter which way you stretch it. C2 grammar--maybe? but I doubt it. There's still constructions that I'm not comfortable with and thus usually avoid. C2 writing? Fat chance, unless you only consider writing as just grammar and even then, I don't think so, but then why have a separate writing test if it's just about grammar?

For Dutch, I'd say that the reading part felt the most well-made, but having taken and passed the reading part of a sample B2 Dutch exam (NT2 programma II), I would say that the test was around a B2 level, though I'd have to go over the NT2 exam again to say so with more confidence.

The listening part was also pretty well done, but too easy (assuming I was allowed to read the questions before listening to the text, which I did). I'd guess that it'd be similar to a B1 listening test but I'm not sure.

In any case, while you can make accurate tests that test reception, I think that at least for Dutch, dialang falls into the category of most internet tests--that is, the highest level (C2) is often in reality B2 or lower.

That doesn't stop it from being fun and giving a sense of accomplishment though. Even though I know it's inaccurate, I was proud to get through the entire thing. :D It certainly took a decent amount of effort.
1 x
All comments and corrections welcome.

User avatar
reineke
Black Belt - 3rd Dan
Posts: 3570
Joined: Wed Jan 06, 2016 7:34 pm
Languages: Fox (C4)
Language Log: https://forum.language-learners.org/vie ... =15&t=6979
x 6554

Re: Dialang validity and reliability?

Postby reineke » Thu Dec 14, 2017 5:49 pm

trui wrote:
...while I can't speak for the French and German tests, I know that the ones for Dutch are off. My listening and vocabulary aren't C2 right now no matter which way you stretch it. C2 grammar--maybe? but I doubt it. There's still constructions that I'm not comfortable with and thus usually avoid. C2 writing? Fat chance, unless you only consider writing as just grammar and even then, I don't think so, but then why have a separate writing test if it's just about grammar?

For Dutch, I'd say that the reading part felt the most well-made, but having taken and passed the reading part of a sample B2 Dutch exam (NT2 programma II), I would say that the test was around a B2 level, though I'd have to go over the NT2 exam again to say so with more confidence.

The listening part was also pretty well done, but too easy (assuming I was allowed to read the questions before listening to the text, which I did). I'd guess that it'd be similar to a B1 listening test but I'm not sure.

In any case, while you can make accurate tests that test reception, I think that at least for Dutch, dialang falls into the category of most internet tests--that is, the highest level (C2) is often in reality B2 or lower.

That doesn't stop it from being fun and giving a sense of accomplishment though. Even though I know it's inaccurate, I was proud to get through the entire thing. :D It certainly took a decent amount of effort.


trui wrote:I'm happy with my progress. I've been learning Dutch for the past 3 years and while I could have done more in the past year or so, my reading is up to a B2 level now and this is the first language I've got that far in. Before learning Dutch, the best I had done was get to an A2 level in French through school and a bit at home--which I promptly forgot.

My goal is to get my Dutch to a native-like level. I am planning on moving to the Netherlands as soon as that's possible and that will help a lot, but before then I'd like to improve my Dutch as much as I can. Sadly, I seem to have lost a lot of the motivation that I used to have for studying and I'm just reading extensively, chatting with people online, and watching the occassional tv show nowadays which isn't bad, but I feel like I should be doing more. It really feels like a waiting game now--read, write, listen, speak and eventually I'll get there? I miss the days where improvement was as simple and easily quantifiable as learning a new aspect of word order.


trui wrote:LANGUAGE: Dutch
CURRENT LEVEL: I'm guessing B2 reading and B1+? listening (I can watch the average show without subtitles although I still don't understand everything).
COMMENTS: Planning on not counting what I've read/watched up until now with the exception of De Erfenis van Richard Grenville by Natalie Koch. I started reading it a few days ago and I'm only a couple chapters in.


The OP believes in his Spanish skills yet he scored low. You think you scored too high. You've left a fair amount of writing samples on this site. We have native Dutch speakers hanging around here and I think it wouldn't hurt if you contacted someone for an assessment. Don't think of C2 as the summit of language mastery.

I'll just add that recently I ran into a post by a Dutch language tutor who observed that the Dutch test was on the easy side.Not sure if this update is of any value but...
Last edited by reineke on Wed Oct 31, 2018 2:46 am, edited 2 times in total.
1 x

User avatar
trui
Orange Belt
Posts: 111
Joined: Sun Jun 04, 2017 5:54 pm
Languages: Native: English
Good enough: Dutch
Not there yet: German
Just starting: French
We'll see?: Russian
Language Log: https://forum.language-learners.org/vie ... 3&start=10
x 215

Re: Dialang validity and reliability?

Postby trui » Thu Dec 14, 2017 7:44 pm

reineke wrote:The OP believes in his Spanish skills yet he scored low. You think you scored too high. You've left a fair amount of writing samples on this site. We have native Dutch speakers hanging around here and I think it wouldn't hurt if you contacted someone for an assessment.

Don't think of C2 as the summit of language mastery.


Again, like I said, all I can comment on are the Dutch tests (well, and the English ones but I haven't tried those yet). The more popular languages such as English and Spanish could very well have more reliable tests. I don't know.

As for me possibly being C2 in writing (which I think is ridiculous but that's neither here nor there), I expected people might protest my assessment which is why I focused on things that I knew 100% for sure I wasn't C2 on, like listening. If people think being able to listen to a radio and understand a lot of what's said if the topic is familiar = C2 then either they're not being honest with themselves or the CEFR descriptions are off.

From https://rm.coe.int/168045bb52

C2: I have no difficulty in understanding any kind of spoken language, whether live or broadcast, even when delivered at fast native speed, provided I have some time to get familiar with the accent.

C1: I can understand extended speech even when it is not clearly structured and when relationships are only implied and not signalled explicitly. I can understand television programmes and films without too much effort.

B2: I can understand extended speech and lectures and follow even complex lines of argument provided the topic is reasonably familiar. I can understand most TV news and current affairs programmes. I can understand the majority of films in standard dialect.

B1: I can understand the main points of clear standard speech on familiar matters regularly encountered in work, school, leisure, etc. I can understand the main point of many radio or TV programmes on current affairs or topics of personal or professional interest when the delivery is relatively slow and clear.

According to this, I'd say I'm somewhere between B1 and B2. Maybe I'm even B2 already, I don't know. By the way, when I said that the listening test was easy, I meant relative to what I'd expect from a C2 test. I still had to work to get through it and make sure to pay close attention and even then I got a few questions wrong.

As for writing, I'm still going to echo my earlier statement that grammar isn't everything. Yes, I can write about a lot of topics when I make use of my two quite extensive vandale dictionary apps. In addition to using my dictionaries fairly regularly, I also occassionally ask native speakers for help (such as asking which preposition to use). When writing without those aids, I make errors more frequently and I'm able to write about a lot fewer topics.

I don't think of C2 as language mastery but if people think of C2 as 'knows the grammar fairly well and thus can write about a lot of stuff clearly when using a dictionary' then I'd like to respectfully disagree.
0 x
All comments and corrections welcome.

User avatar
Jar-Ptitsa
Brown Belt
Posts: 1000
Joined: Sun Jul 19, 2015 1:13 pm
Location: London
Languages: Belgian French (N)

I can speak: Dutch, German, English, Spanish and understand Italian, Portuguese, Wallonian, Afrikaans, but not always correctly.
x 652

Re: Dialang validity and reliability?

Postby Jar-Ptitsa » Thu Dec 14, 2017 8:26 pm

trui wrote:According to this, I'd say I'm somewhere between B1 and B2. Maybe I'm even B2 already, I don't know. By the way, when I said that the listening test was easy, I meant relative to what I'd expect from a C2 test. I still had to work to get through it and make sure to pay close attention and even then I got a few questions wrong.

As for writing, I'm still going to echo my earlier statement that grammar isn't everything. Yes, I can write about a lot of topics when I make use of my two quite extensive vandale dictionary apps. In addition to using my dictionaries fairly regularly, I also occassionally ask native speakers for help (such as asking which preposition to use). When writing without those aids, I make errors more frequently and I'm able to write about a lot fewer topics.

I don't think of C2 as language mastery but if people think of C2 as 'knows the grammar fairly well and thus can write about a lot of stuff clearly when using a dictionary' then I'd like to respectfully disagree.


Trui, you're wrong: you are not between B1 and B2. You are about C1 or higher.

I think that my Dutch is about C1 or C1.5 and I have never seen a mistake at all in the dutch that you have written, and it seems very natural. Your word order seems correct with separable verbs, inversions, the german-style infinitive at the end etc etc , vocabulary, grammar everything.

Your dutch is the same level as mine or better. I mean written, I can't say how you can speak it or listen.

Tarvos told you that you are not B1 as well.
2 x
-w- I am Jar-ptitsa and my Hawaiian name is ʻā ʻaia. Please correct my mistakes in all the languages. Thank you very much.
: 1 / 50 Spanish grammar
: 5 / 50 Spanish vocabulary

User avatar
Jar-Ptitsa
Brown Belt
Posts: 1000
Joined: Sun Jul 19, 2015 1:13 pm
Location: London
Languages: Belgian French (N)

I can speak: Dutch, German, English, Spanish and understand Italian, Portuguese, Wallonian, Afrikaans, but not always correctly.
x 652

Re: Dialang validity and reliability?

Postby Jar-Ptitsa » Thu Dec 14, 2017 8:55 pm

I tried the ducth test but it was too difficult so I stopped.

When they put some sentences and you must say the correct order i didn't know at all. I understood all the sentences, but how can you know the order that the writer had written? all the sentences were possible to understand.
0 x
-w- I am Jar-ptitsa and my Hawaiian name is ʻā ʻaia. Please correct my mistakes in all the languages. Thank you very much.
: 1 / 50 Spanish grammar
: 5 / 50 Spanish vocabulary


Return to “Language Programs and Resources”

Who is online

Users browsing this forum: No registered users and 2 guests