How do people feel about surveys?

Discuss the LLORG's and HTLAL forum's past and its future here.

How do you feel about language-learning surveys from new users?

I enjoy filling out surveys! It helps scientists and students learn more about language learners.
10
26%
They're not my thing, but sure, why not?
13
33%
I only like some surveys. For example, ones that are meant for independent learners, advanced learners and/or polyglots. (Feel free to explain below.)
9
23%
We've had more than enough surveys, thank you, and I'd prefer the mods to be a lot more choosy about what they approve.
7
18%
 
Total votes: 39

User avatar
Iversen
Black Belt - 4th Dan
Posts: 4768
Joined: Sun Jul 19, 2015 7:36 pm
Location: Denmark
Languages: Monolingual travels in Danish, English, German, Dutch, Swedish, French, Portuguese, Spanish, Catalan, Italian, Romanian and (part time) Esperanto
Ahem, not yet: Norwegian, Afrikaans, Platt, Scots, Russian, Serbian, Bulgarian, Albanian, Greek, Latin, Irish, Indonesian and a few more...
Language Log: viewtopic.php?f=15&t=1027
x 14962

Re: How do people feel about surveys?

Postby Iversen » Sat Sep 09, 2017 10:03 am

I don't mind filling out questionnaires and surveys as long as they are well thought through and make sense. But there is one problem with surveys posted on a forum like this one, namely that the answers don't come from a clearly defined population - not even the population consisting of the users of language-learners.org. As the answers above show some of us are likely to answer anything within sight, others don't touch the stuff, and those in between use selection criteria which are totally unknown. So doing anything like a statistical analysis is absurd.

You can use questionnaires as a way to get standardized answers to some relevant questions, but they will be just about as reliable as the interviews with people in some random street which TV people seem to like.

In principle it should be possible to find relations between certain answers, using other questions to subdivide the 'population' of answers. They might for instance show that people who like to watch soaps in their target language aren't keen on reading grammar books (just an example - and maybe wrong). But the number of answers should be quite high even to yield statistically valid results, and you would still have to live with the suspicion that the people who answered the questionnaire represent a totally different study profile from those who didn't.

I personally prefer in-depth studies based on individuals representing different learner types. Whether or not you see these types as deeprooted patterns (maybe even with a genetic component), or you just see them as preferences based on random previous experiences is less important than the fact that we do have different study patterns. But since I doubt that you can get sufficiently large representative populations for serious factor analysis I doubt that you can arrive at any kind of the statistically reliable results even from such studies.

And the quality of answers is also a big problem. Even large-scale research like the one that gave us the supposed percentages of monoglots, diglots etc in the EU are dubious because the researchers couldn't really check the veracity of the answers. Or in other words: if people stated that they could speak three languages nobody checked whether they really sucked in two of them.
2 x

Cavesa
Black Belt - 4th Dan
Posts: 4960
Joined: Mon Jul 20, 2015 9:46 am
Languages: Czech (N), French (C2) English (C1), Italian (C1), Spanish, German (C1)
x 17566

Re: How do people feel about surveys?

Postby Cavesa » Sat Sep 09, 2017 11:29 am

PeterMollenburg wrote:I don't like surveys much in general as I'm suspicious as to the motives behind many of them. Not to mention, I believe that often they are used as an excuse to support biased arguments in media... but did such surveys really take place? In other words surveys, real or not, have a motive behind them often of which i'm distrustful of.

However, on the forum, i feel I'm for the most part pretty clear on what the motives are behind surveys and these surveys, in my opinion, have no hidden agenda. For such surveys, I don't mind filling them out here and there, particularly if I know it's going to assist someone, or clarify a point relative to language learning and language learners (such as providing insight on effective or ineffective study methods or learning techniques).


Winston S. Churchill — 'I only believe in statistics that I doctored myself'

We all know surveys always come with a bit or a lot of a bias. Serious research counts with it and the researchers admit it in their paper. We have a huge election this autumn in the Czech Republic, so there are plenty of funny polls everywhere, pretending to be serious. Some people clearly differentiate among them, based on what company is making it. Some don't. But we all see how different the results are.

For example the mentioned european research on number of languages has obviously had rather small samples. And it would be a huuuuge difference, whether you'd ask people in cities or those in villages, people with a university degree or those with basic education. And it is not the only problematic one, I have written about a worse one, somewhere on the forum, where the researches made very sloppy translations, so the individual nations were actually being asked different questions, yet the answers were counted as if the questions had been the same. Too bad such crap is being paid by all of us.

Yes, some surveys about languages obviously just strenghten stereotypes, as we have been discussing. But I would say it is usually not the intention, in case of surveys about languages that are not paid by a company (Duolingo "surveys" and their interpretations tend to be hilarious). It is just not that easy to avoid suggestive questions and options, especially when the "researcher" simply thinks their prejudices are the truth that doesn't need to be questioned (like "autodidacts are just too poor to pay teachers, otherwise they definitely would" or "autodidacts don't get to advanced levels).
2 x

taryn
White Belt
Posts: 19
Joined: Thu Sep 07, 2017 10:55 am
Languages: Italian (N)
English (C1)
Chinese (B1)
Spanish (A1)
x 9

Re: How do people feel about surveys?

Postby taryn » Sat Sep 09, 2017 1:05 pm

Then, if you'd like, you can share your opinion on this survey: https://goo.gl/forms/WnPv8m9ExKwRx8TF2 That is, of course, if you've ever taken an online course.
The reason why I chose to use this site was because I thought I could collect authentic data on the basis of someone's actual experience.

lavengro wrote:This thread ( a survey about surveys) is easily the most meta thing I have encountered all week!

Personally, I love surveys and polls. The idea that someone is interested in what I may think heartens me considerably, and is a refreshing change from work (where, frankly, few care what I think). I think it shows careful discernment and excellent judgment on the surveyor's or pollster's part in recognizing me, Lavengro, amongst other humans, as someone whose opinion is worth soliciting. I will give it freely, mostly because I cannot pass by an opportunity of skewing the results of a survey or poll by just making up crazy answers.
0 x

User avatar
Iversen
Black Belt - 4th Dan
Posts: 4768
Joined: Sun Jul 19, 2015 7:36 pm
Location: Denmark
Languages: Monolingual travels in Danish, English, German, Dutch, Swedish, French, Portuguese, Spanish, Catalan, Italian, Romanian and (part time) Esperanto
Ahem, not yet: Norwegian, Afrikaans, Platt, Scots, Russian, Serbian, Bulgarian, Albanian, Greek, Latin, Irish, Indonesian and a few more...
Language Log: viewtopic.php?f=15&t=1027
x 14962

Re: How do people feel about surveys?

Postby Iversen » Sat Sep 09, 2017 1:48 pm

You will certainly be able to collect authentic responses. The problem is that they aren't representative of any clearly defined group, and therefore you can't draw conclusions from your results. But as is already has been suggested you can find a lot of authentic testimonies from single individuals about their learning methods here - just not material that is suitable for quantitative analysis.

I would however point out that your focus on e-learning isn't a weakness - it is actually a strength as it clarify which forum users you are expecting to answer. Making that even clearer would have removed one source of 'noise'. Ideally you would want two types of persons to answer: heavy e-learning users and those who haven't ever tried it.
1 x

Cavesa
Black Belt - 4th Dan
Posts: 4960
Joined: Mon Jul 20, 2015 9:46 am
Languages: Czech (N), French (C2) English (C1), Italian (C1), Spanish, German (C1)
x 17566

Re: How do people feel about surveys?

Postby Cavesa » Sat Sep 09, 2017 1:54 pm

taryn wrote:Then, if you'd like, you can share your opinion on this survey: https://goo.gl/forms/WnPv8m9ExKwRx8TF2 That is, of course, if you've ever taken an online course.


This is the problem! You don't define what type of online course or elearning do you mean!

Duolingo is an online course. There are online courses on the Deutsche Welle website and all over the internet.
Paying 1 on 1 skype lessons is an online course.
There are group courses online, for example some MOOCs work that way.
There are email based courses.
The Language Transfer is an example of yet another online course.

The last two options may be regarding as "less online", since you can do the majority of the work offline, just connecting from time to time, but they still fall within the wider category. And the first three are definitely fitting in, unless you give us a more narrow definition.

I have taken a lot of online courses. But in the middle of your survey, I was suddenly asked questions that simply didn't fit any of those I had taken, the few of my experiences that would fit had been too short to be valuable (like one lesson). The problem is not the forum members needing the bold letters, we can read just fine, it's you not telling us accurately what you are looking for. :-)
1 x

User avatar
emk
Black Belt - 1st Dan
Posts: 1620
Joined: Sat Jul 18, 2015 12:07 pm
Location: Vermont, USA
Languages: English (N), French (B2+)
Badly neglected "just for fun" languages: Middle Egyptian, Spanish.
Language Log: viewtopic.php?f=15&t=723
x 6330
Contact:

Re: How do people feel about surveys?

Postby emk » Sat Sep 09, 2017 2:00 pm

taryn wrote:The reason why I chose to use this site was because I thought I could collect authentic data on the basis of someone's actual experience.

No worries! You are totally welcome to run your survey here. :-)

I didn't start this thread because of your survey, but because we've had a lot of surveys from different researchers over the last year or two.

However, I do agree that for future surveys on this forum, we (the moderators) should at least encourage the survey to include a clear description about who the intended audience of the survey is, so that people don't waste time.

Iversen: I agree that it's almost impossible to get a statistically valid sample of the forum. What researchers can get here is an unusual sample with a strong bias towards successful language learners. This might be especially useful in the exploratory phases of the study.
2 x

taryn
White Belt
Posts: 19
Joined: Thu Sep 07, 2017 10:55 am
Languages: Italian (N)
English (C1)
Chinese (B1)
Spanish (A1)
x 9

Re: How do people feel about surveys?

Postby taryn » Sat Sep 09, 2017 4:55 pm

The reason why I put it in bold was because I had already acknowledged my mistake in the description of the survey and explained it to you, not because I think you can't read, at least not in this forum. ;) Thank you for your advice.

Cavesa wrote:
taryn wrote:Then, if you'd like, you can share your opinion on this survey: https://goo.gl/forms/WnPv8m9ExKwRx8TF2 That is, of course, if you've ever taken an online course.


This is the problem! You don't define what type of online course or elearning do you mean!

Duolingo is an online course. There are online courses on the Deutsche Welle website and all over the internet.
Paying 1 on 1 skype lessons is an online course.
There are group courses online, for example some MOOCs work that way.
There are email based courses.
The Language Transfer is an example of yet another online course.

The last two options may be regarding as "less online", since you can do the majority of the work offline, just connecting from time to time, but they still fall within the wider category. And the first three are definitely fitting in, unless you give us a more narrow definition.

I have taken a lot of online courses. But in the middle of your survey, I was suddenly asked questions that simply didn't fit any of those I had taken, the few of my experiences that would fit had been too short to be valuable (like one lesson). The problem is not the forum members needing the bold letters, we can read just fine, it's you not telling us accurately what you are looking for. :-)
0 x

taryn
White Belt
Posts: 19
Joined: Thu Sep 07, 2017 10:55 am
Languages: Italian (N)
English (C1)
Chinese (B1)
Spanish (A1)
x 9

Re: How do people feel about surveys?

Postby taryn » Sat Sep 09, 2017 5:04 pm

The thing is that I hadn't realised the inaccuracy of my description until some of the members made me notice. Especially since I had showed it to an expert and was told it was fine! But that's okay, I can edit it! I'm sorry if I wasted five minutes of someone's time, it was really not my intention.
Anyway, thank you for your kind welcome. I guess I just have to get used to this site. I don't mind being rightly criticised, I only get upset when I read sarcastic comments when I was not even trying to be nasty or anything.
I hope your survey results were more satisfactory !

emk wrote:
taryn wrote:The reason why I chose to use this site was because I thought I could collect authentic data on the basis of someone's actual experience.

No worries! You are totally welcome to run your survey here. :-)

I didn't start this thread because of your survey, but because we've had a lot of surveys from different researchers over the last year or two.

However, I do agree that for future surveys on this forum, we (the moderators) should at least encourage the survey to include a clear description about who the intended audience of the survey is, so that people don't waste time.

Iversen: I agree that it's almost impossible to get a statistically valid sample of the forum. What researchers can get here is an unusual sample with a strong bias towards successful language learners. This might be especially useful in the exploratory phases of the study.
0 x

User avatar
MorkTheFiddle
Black Belt - 2nd Dan
Posts: 2114
Joined: Sat Jul 18, 2015 8:59 pm
Location: North Texas USA
Languages: English (N). Read (only) French and Spanish. Studying Ancient Greek. Studying a bit of Latin. Once studied Old Norse. Dabbled in Catalan, Provençal and Italian.
Language Log: https://forum.language-learners.org/vie ... 11#p133911
x 4824

Re: How do people feel about surveys?

Postby MorkTheFiddle » Sat Sep 09, 2017 6:27 pm

taryn wrote:Then, if you'd like, you can share your opinion on this survey: https://goo.gl/forms/WnPv8m9ExKwRx8TF2 That is, of course, if you've ever taken an online course.
ss by an opportunity of skewing the results of a survey or poll by just making up crazy answers.

One of the questions on this survey forced me to stop immediately. The question is, "Whenever I feel doubtful, I turn to my teacher and ask for guidance. *" This is a starred question that has to be answered, but there is no way for me to answer it honestly, because I have never taken an online course that has a teacher nor have I ever taken an online course at the suggestion of a teacher of a different course.

The only explanation for the question that I can think of is that either the survey pertains to a specific kind of on-line course that I have never taken, but which is not specified, or that the survey does not consider the kind of courses that I would call an on-line course to be a true on-line course. And since the word "teacher" is used, this seems restricted to elementary, middle and high school students, private or public.

For the record, I have taken on-line courses--but not with an instructor--in Old Norse, Old English, and Modern German. All were limited in scope, but allowing for that, excellent.
4 x
Many things which are false are transmitted from book to book, and gain credit in the world. -- attributed to Samuel Johnson

User avatar
aokoye
Black Belt - 1st Dan
Posts: 1818
Joined: Sat Jul 18, 2015 6:14 pm
Location: Portland, OR
Languages: English (N), German (~C1), French (Intermediate), Japanese (N4), Swedish (beginner), Dutch (A2)
Language Log: https://forum.language-learners.org/vie ... 15&t=19262
x 3309
Contact:

Re: How do people feel about surveys?

Postby aokoye » Sat Sep 09, 2017 6:35 pm

taryn wrote:Anyway, thank you for your kind welcome. I guess I just have to get used to this site. I don't mind being rightly criticised, I only get upset when I read sarcastic comments when I was not even trying to be nasty or anything.

Remember that it is often hard to read someone's tone on the internet especially when you aren't used to their posting style.
4 x
Prefered gender pronouns: Masculine


Return to “LLORG & HTLAL discussion area”

Who is online

Users browsing this forum: No registered users and 2 guests