Re: How do people feel about surveys?
Posted: Sat Sep 09, 2017 10:03 am
I don't mind filling out questionnaires and surveys as long as they are well thought through and make sense. But there is one problem with surveys posted on a forum like this one, namely that the answers don't come from a clearly defined population - not even the population consisting of the users of language-learners.org. As the answers above show some of us are likely to answer anything within sight, others don't touch the stuff, and those in between use selection criteria which are totally unknown. So doing anything like a statistical analysis is absurd.
You can use questionnaires as a way to get standardized answers to some relevant questions, but they will be just about as reliable as the interviews with people in some random street which TV people seem to like.
In principle it should be possible to find relations between certain answers, using other questions to subdivide the 'population' of answers. They might for instance show that people who like to watch soaps in their target language aren't keen on reading grammar books (just an example - and maybe wrong). But the number of answers should be quite high even to yield statistically valid results, and you would still have to live with the suspicion that the people who answered the questionnaire represent a totally different study profile from those who didn't.
I personally prefer in-depth studies based on individuals representing different learner types. Whether or not you see these types as deeprooted patterns (maybe even with a genetic component), or you just see them as preferences based on random previous experiences is less important than the fact that we do have different study patterns. But since I doubt that you can get sufficiently large representative populations for serious factor analysis I doubt that you can arrive at any kind of the statistically reliable results even from such studies.
And the quality of answers is also a big problem. Even large-scale research like the one that gave us the supposed percentages of monoglots, diglots etc in the EU are dubious because the researchers couldn't really check the veracity of the answers. Or in other words: if people stated that they could speak three languages nobody checked whether they really sucked in two of them.
You can use questionnaires as a way to get standardized answers to some relevant questions, but they will be just about as reliable as the interviews with people in some random street which TV people seem to like.
In principle it should be possible to find relations between certain answers, using other questions to subdivide the 'population' of answers. They might for instance show that people who like to watch soaps in their target language aren't keen on reading grammar books (just an example - and maybe wrong). But the number of answers should be quite high even to yield statistically valid results, and you would still have to live with the suspicion that the people who answered the questionnaire represent a totally different study profile from those who didn't.
I personally prefer in-depth studies based on individuals representing different learner types. Whether or not you see these types as deeprooted patterns (maybe even with a genetic component), or you just see them as preferences based on random previous experiences is less important than the fact that we do have different study patterns. But since I doubt that you can get sufficiently large representative populations for serious factor analysis I doubt that you can arrive at any kind of the statistically reliable results even from such studies.
And the quality of answers is also a big problem. Even large-scale research like the one that gave us the supposed percentages of monoglots, diglots etc in the EU are dubious because the researchers couldn't really check the veracity of the answers. Or in other words: if people stated that they could speak three languages nobody checked whether they really sucked in two of them.