Dialang validity and reliability?

All about language programs, courses, websites and other learning resources
User avatar
trui
Orange Belt
Posts: 111
Joined: Sun Jun 04, 2017 5:54 pm
Languages: Native: English
Good enough: Dutch
Not there yet: German
Just starting: French
We'll see?: Russian
Language Log: https://forum.language-learners.org/vie ... 3&start=10
x 215

Re: Dialang validity and reliability?

Postby trui » Thu Dec 14, 2017 9:26 pm

Jar-Ptitsa wrote:Trui, you're wrong: you are not between B1 and B2. You are about C1 or higher.

I think that my Dutch is about C1 or C1.5 and I have never seen a mistake at all in the dutch that you have written, and it seems very natural. Your word order seems correct with separable verbs, inversions, the german-style infinitive at the end etc etc , vocabulary, grammar everything.

Your dutch is the same level as mine or better. I mean written, I can't say how you can speak it or listen.

Tarvos told you that you are not B1 as well.


I'm an active member of a Dutch language learning server and one of the members is a native Dutch speaker who teaches Dutch professionally. They've seen a lot more of my written Dutch than you guys have and the institute where they teach further divides up the various levels. So there's B1.1 B1.2 and B2.1 and B2.2 etc. instead of just B1 and B2. They've rated my written Dutch as B1.2 a couple months ago and I asked them again today and they said the same.

If that's not good enough for you guys then I'm sorry. B1 is a very good level with which you can work a job in said language. I honestly think people constantly underestimate the B levels. I'm proud of what I've achieved but I'm definitely not C1 or C2 yet.

If I'm B1, then does that mean you're B1 as well and thus have misjudged your level, Jar-Ptitsa? Well the tendency of people to think in such a way is why I don't like these discussions in the first place. It's really hard to judge someone's level from a few posts online, especially if you're not a professional. Honestly I regret commenting on tommus's level because I had no good basis to say anything.

Likewise, I can't judge your level. To properly compare ourselves we'd have to write about a series of topics without the help of dictionaries and the like and get a professional to rate our writing. Personally, I constantly have to reach for a dictionary when I get out of my comfort zone, which is discussing language learning, my fiance, and so forth and even within my comfort zone I reach for a dictionary fairly often. I do think that the main barrier between me and the higher levels is vocabulary. You may have a much larger active vocabulary than I do but since all you see are my posts where I use a dictionary, things appear more equal. Or maybe you also reach for a dictionary just as often as I do and you're also B1.2. I don't know. That's a possibility as well.
1 x
All comments and corrections welcome.

User avatar
Jar-Ptitsa
Brown Belt
Posts: 1000
Joined: Sun Jul 19, 2015 1:13 pm
Location: London
Languages: Belgian French (N)

I can speak: Dutch, German, English, Spanish and understand Italian, Portuguese, Wallonian, Afrikaans, but not always correctly.
x 652

Re: Dialang validity and reliability?

Postby Jar-Ptitsa » Thu Dec 14, 2017 10:02 pm

trui wrote: I'm an active member of a Dutch language learning server and one of the members is a native Dutch speaker who teaches Dutch professionally. They've seen a lot more of my written Dutch than you guys have and the institute where they teach further divides up the various levels. So there's B1.1 B1.2 and B2.1 and B2.2 etc. instead of just B1 and B2. They've rated my written Dutch as B1.2 a couple months ago and I asked them again today and they said the same.

If that's not good enough for you guys then I'm sorry. B1 is a very good level with which you can work a job in said language. I honestly think people constantly underestimate the B levels. I'm proud of what I've achieved but I'm definitely not C1 or C2 yet.

If I'm B1, then does that mean you're B1 as well and thus have misjudged your level, Jar-Ptitsa? Well the tendency of people to think in such a way is why I don't like these discussions in the first place. It's really hard to judge someone's level from a few posts online, especially if you're not a professional. Honestly I regret commenting on tommus's level because I had no good basis to say anything.

Likewise, I can't judge your level. To properly compare ourselves we'd have to write about a series of topics without the help of dictionaries and the like and get a professional to rate our writing. Personally, I constantly have to reach for a dictionary when I get out of my comfort zone, which is discussing language learning, my fiance, and so forth and even within my comfort zone I reach for a dictionary fairly often. I do think that the main barrier between me and the higher levels is vocabulary. You may have a much larger active vocabulary than I do but since all you see are my posts where I use a dictionary, things appear more equal. Or maybe you also reach for a dictionary just as often as I do and you're also B1.2. I don't know. That's a possibility as well.



ok, please don't be angry with me.

I passed the 4 official government B2 exams (writing, reaidng, speaking, listening), 11 years ago when I was 14 (or maybe 13, I'm not sure now, but it wasn't later becuase after 14 i didn't do more exams) and my Dutch is much better now.

I can't be B1.

If you want to be b1 then ok.

I don't use dictionaries at all.
2 x
-w- I am Jar-ptitsa and my Hawaiian name is ʻā ʻaia. Please correct my mistakes in all the languages. Thank you very much.
: 1 / 50 Spanish grammar
: 5 / 50 Spanish vocabulary

User avatar
trui
Orange Belt
Posts: 111
Joined: Sun Jun 04, 2017 5:54 pm
Languages: Native: English
Good enough: Dutch
Not there yet: German
Just starting: French
We'll see?: Russian
Language Log: https://forum.language-learners.org/vie ... 3&start=10
x 215

Re: Dialang validity and reliability?

Postby trui » Thu Dec 14, 2017 10:15 pm

Jar-Ptitsa wrote:ok, please don't be angry with me.

I passed the 4 official government B2 exams (writing, reaidng, speaking, listening), 11 years ago when I was 14 and my Dutch is much better now.

I can't be B1.

If you want to be b1 then ok.

I don't use dictionaries at all.


That's what I figured and with that we have the solution to our dilemma. ;) You saw my Dutch and thought, how can she be B1? I'm C1 and her Dutch looks as good as mine.

Well, the answer is, like I've been saying this whole time, vocabulary! My grammar is good and so I can make really good posts if given a dictionary and time to look things over. But that's completely different from being able to do it without a dictionary like you can.

As for my Dutch flowing nicely, that can't be easily learned from a grammar book and I'm able to do that from hours of practice chatting online. But that doesn't change my lack of vocabulary.

Fortunately, vocabularly isn't that complicated to learn (it just takes time) and so I'm sure I'll get to C1 and even C2 eventually. :)
1 x
All comments and corrections welcome.

User avatar
tiia
Blue Belt
Posts: 735
Joined: Tue Mar 15, 2016 11:52 pm
Location: Finland
Languages: German (N), English (?), Finnish (C1), Spanish (B2??), Swedish (B2)
Language Log: viewtopic.php?t=2374
x 1997

Re: Dialang validity and reliability?

Postby tiia » Fri Dec 15, 2017 12:41 am

@reineke there went something wrong with your post. You quoted me with a sentence that was written by someone else.
0 x
Corrections for entries written in Finnish, Spanish or Swedish are welcome.
Project 30+X: 25 / 30

User avatar
reineke
Black Belt - 3rd Dan
Posts: 3570
Joined: Wed Jan 06, 2016 7:34 pm
Languages: Fox (C4)
Language Log: https://forum.language-learners.org/vie ... =15&t=6979
x 6554

Re: Dialang validity and reliability?

Postby reineke » Fri Dec 15, 2017 2:01 am

Thanks, tiia.

DIALANG tests and results

Why did I not get the test result I expected?
"You may get a different test result from what you expected. There may be many reasons for this, some of which are described in more detail in the ‘About self-assessment’ part of feedback after the test. Some of the tests are still under development: they have been designed on the basis of expert judgment on task difficulty and empirical data is still being gathered to improve their accuracy.
Some of the reasons for a different test result may be more technical. For example, in many easy or even intermediate versions of tests, it is not possible to be placed at the highest level(s) because the tests are simply too easy to produce such a result. To make sure, therefore, that you get the test that is suitable for you, always take the placement test or the self-assessment statements before taking the test you have chosen.
It is always possible that you under-estimate or over-estimate your ability, and the feedback suggests possible reasons.

In the Placement Test you decide which of the "words" you see are real, and which are not. Do not take risks in this test: mark a word as real only if you are certain. Guessing is often a good strategy when using a language, but in this test claiming false words as real words will heavily lower your score, even as low as zero. However, remember that the Placement Test is only one (optional) step in the DIALANG system. Its main purpose is to help the system to choose the best test for you. If you have a very low score in the Placement test and your test result is much lower than expected, try again skipping the Placement test and see if you get a different result."

https://www.lancaster.ac.uk/researchent ... esults.htm

Dialang Google Group

Hello,

I have recently asked my students to complete three of the Dialang tests in English (Reading, Structure and Vocabulary) and am now in the process of analyzing their results. I am particulary interested in seeing how closely their CEF level in Reading is linked to the levels they have achieved in the Structure end Vocabulary tests. I have a couple of questions that I hope you can help me with with regard to how the levels have been 'calculated'.

For an example, I have two students who have similar scores in the placement tests (375 and 360), so they are in the same bracket. However, one of them is more confident in her reading abilities and has ticked off more of the boxes in the self assessment questionaire. This means that one has been given the easy test and the other one has been given the intermediate test. When it comes to correct answers, they both have 21/30 in Reading, but the student who has taken the easy test achieves level A2, whereas the other student achieves level B2.
What I am wondering is whether there is an inbuilt limit to the level they can achieve with each test - would a student who manages a 30/30 score in the easy test be given a C2 result?

Furthermore, it seems that for the Structure test it only takes one mistake to go from C2 to C1 and only a further mistake to go to B2, whereas the Reading tests have different cut off points. Is there some sort of overview over the different tests that shows where these cutt off points are and whether all types of mistakes count the same?

The answer

Each test consists of a set of items which were calibrated individually and the 3 tests for each language/skill are made up of these. There's no guarantee of an easy test in English being as easy as the easy test in French, although such parity was attempted. There's no overview of these differences, there's just a set of tables with the discrimination values for each individual item, and these values are what are used to provide the final CoE level. You'd have to run some comparative analysis across the dataset.

So, to summarise: you can get a C2 in an easy test in the case of a language/skill that just happens have a set of relatively difficult items. That's a valid scenario. There is no absolute relationship between test difficulties between languages or skills.


https://groups.google.com/forum/?fromgr ... lang-users
4 x

User avatar
Jar-Ptitsa
Brown Belt
Posts: 1000
Joined: Sun Jul 19, 2015 1:13 pm
Location: London
Languages: Belgian French (N)

I can speak: Dutch, German, English, Spanish and understand Italian, Portuguese, Wallonian, Afrikaans, but not always correctly.
x 652

Re: Dialang validity and reliability?

Postby Jar-Ptitsa » Fri Dec 15, 2017 4:29 pm

trui wrote:
Jar-Ptitsa wrote:ok, please don't be angry with me.

I passed the 4 official government B2 exams (writing, reaidng, speaking, listening), 11 years ago when I was 14 and my Dutch is much better now.

I can't be B1.

If you want to be b1 then ok.

I don't use dictionaries at all.


That's what I figured and with that we have the solution to our dilemma. ;) You saw my Dutch and thought, how can she be B1? I'm C1 and her Dutch looks as good as mine.

Well, the answer is, like I've been saying this whole time, vocabulary! My grammar is good and so I can make really good posts if given a dictionary and time to look things over. But that's completely different from being able to do it without a dictionary like you can.

As for my Dutch flowing nicely, that can't be easily learned from a grammar book and I'm able to do that from hours of practice chatting online. But that doesn't change my lack of vocabulary.

Fortunately, vocabularly isn't that complicated to learn (it just takes time) and so I'm sure I'll get to C1 and even C2 eventually. :)


Yes, exactly, i read your Dutch and I didn't see any mistakes. It is correct, sound natural with good vocabulry and great grammatical accuracy.

In some other people's Dutch I can see many mistakes and I know which mistakes are basic, intermediate, advanced ones. Then I get a feeling of the person's level.

I know the B1, b2 etc system, it's the normal european langauge system, and I know what level matches the levels in the Dutch writing of the people.

Tarvos is a native speaker of Dutch and I think that she teaches it. You had the advice of one person on a forum that you are B1, maybe tarvos can give you a second opinion, if you and she want to do that. If you prefer to feel b1, then this is of course ok.

I meant to give you a compliment when i said that you are C1 or higher. and it's the true impression that I had after I had read your posts.

But yes, if you use a dictionary so much, then it's different. i didn't consider this point. I never use a dictionary, only once on this forum I think, to find out "octrooi's" meaning when tristano said that he worked in the octrooiburo.

I didn't overestimate my own level. One time I did an official test (not certified exam, but test with a person who knows the exams) for german and my result was C1, but I feel more B2 because of my grammar insecurity in german. My Dutch is much, much better than my german. And of course I got the B2 exams before, and B2 is very easy.
1 x
-w- I am Jar-ptitsa and my Hawaiian name is ʻā ʻaia. Please correct my mistakes in all the languages. Thank you very much.
: 1 / 50 Spanish grammar
: 5 / 50 Spanish vocabulary

User avatar
tarvos
Black Belt - 2nd Dan
Posts: 2889
Joined: Sun Jul 26, 2015 11:13 am
Location: The Lowlands
Languages: Native: NL, EN
Professional: ES, RU
Speak well: DE, FR, RO, EO, SV
Speak reasonably: IT, ZH, PT, NO, EL, CZ
Need improvement: PO, IS, HE, JP, KO, HU, FI
Passive: AF, DK, LAT
Dabbled in: BRT, ZH (SH), BG, EUS, ZH (CAN), and a whole lot more.
Language Log: http://how-to-learn-any-language.com/fo ... PN=1&TPN=1
x 6093
Contact:

Re: Dialang validity and reliability?

Postby tarvos » Sat Dec 23, 2017 9:34 am

Your writing is good, trui, but I would have to combine it with a full assessment of your speaking to get a feel of the level you're at. Also keep in mind that language schools will evaluate you conservatively, because it gets them more money.
5 x
I hope your world is kind.

Is a girl.

User avatar
BalancingAct
Orange Belt
Posts: 117
Joined: Thu Jan 12, 2017 6:37 am
Languages: Mandarin, Cantonese, English (Prof.), French (Adv. - Prof.), Italian (Adv.), German (Adv. receptive), Spanish (Int. receptive)
x 182

Re: Dialang validity and reliability?

Postby BalancingAct » Sat Dec 23, 2017 10:06 am

tarvos wrote:Also keep in mind that language schools will evaluate you conservatively, because it gets them more money.


That's very true.
2 x

User avatar
Jar-Ptitsa
Brown Belt
Posts: 1000
Joined: Sun Jul 19, 2015 1:13 pm
Location: London
Languages: Belgian French (N)

I can speak: Dutch, German, English, Spanish and understand Italian, Portuguese, Wallonian, Afrikaans, but not always correctly.
x 652

Re: Dialang validity and reliability?

Postby Jar-Ptitsa » Sat Dec 23, 2017 1:00 pm

tarvos wrote:Your writing is good, trui, but I would have to combine it with a full assessment of your speaking to get a feel of the level you're at.


Yes, of course the speaking must be in the full assessment as well, I agree.


tarvos wrote: Also keep in mind that language schools will evaluate you conservatively, because it gets them more money.


So that is why, this explains it.

I knew the Dutch person on the forum where trui wrote had lied about trui's level, it's completely wrong to say that her writing is b1, but I didn't know why the person told her that.

I thought maybe because dutch people like to say that Dutch is impossible for the foreigners becuase it's so difficult. But the langauge schools need more students, and a student who is C1 or C2 won't pay them, but a B1 student will think that they must learn more so they will pay. I understand it. It's very nasty and deceitful, I'm happy that I've never paid a langauge school.
0 x
-w- I am Jar-ptitsa and my Hawaiian name is ʻā ʻaia. Please correct my mistakes in all the languages. Thank you very much.
: 1 / 50 Spanish grammar
: 5 / 50 Spanish vocabulary

User avatar
trui
Orange Belt
Posts: 111
Joined: Sun Jun 04, 2017 5:54 pm
Languages: Native: English
Good enough: Dutch
Not there yet: German
Just starting: French
We'll see?: Russian
Language Log: https://forum.language-learners.org/vie ... 3&start=10
x 215

Re: Dialang validity and reliability?

Postby trui » Sat Dec 23, 2017 6:23 pm

Jar-Ptitsa wrote:
tarvos wrote: Also keep in mind that language schools will evaluate you conservatively, because it gets them more money.


So that is why, this explains it.

I knew the Dutch person on the forum where trui wrote had lied about trui's level, it's completely wrong to say that her writing is b1, but I didn't know why the person told her that.

I thought maybe because dutch people like to say that Dutch is impossible for the foreigners becuase it's so difficult. But the langauge schools need more students, and a student who is C1 or C2 won't pay them, but a B1 student will think that they must learn more so they will pay. I understand it. It's very nasty and deceitful, I'm happy that I've never paid a langauge school.


Okay, I find this accusation disgusting. So because you can't believe someone who knows the grammar of a language well and uses a dictionary a lot to write stuff can be B1 and well on their way to B2, you decide to accuse the person who said so as lying?

There'd be absolutely no profit motive for them to lie about my level. They're in Belgium and I'm in Canada and there's been no talk about me flying over to Belgium (which would be cost prohibitive for me) and getting lessons from them. But noo, you know better than them because you've seen... a bunch of polished posts made by me on this forum where I made frequent use of a dictionary and occassionally asked people for help.

Honestly. I knew that there was a problem with level inflation based on people's various skewed perceptions of things on the internet but I didn't think it was this bad.

Anyways. I just took an official NT2 listening sample exam for B2 online and passed, so I've included that along with my reading sample test result in my profile. For writing/speaking I've put a question mark and I'm no longer going to put an educated guess of my overall level since it will apparently lead to stuff like this.
2 x
All comments and corrections welcome.


Return to “Language Programs and Resources”

Who is online

Users browsing this forum: No registered users and 2 guests