Josquin wrote:Sarcasm aside, I honestly would like to know how you can have the guts to criticize three sciences for their methodology? Do you really understand string theory that well that you can fully appreciate its merits and shortcomings?
So I do believe in backing up my statements if somebody calls me on them. So here you go, though I don't necessarily think you'll be convinced (or even that you should be convinced).
But at least it will help explain what I was thinking, which I feel is a basic obligation of anybody who makes arguments.
I do not understand string theory. My concerns about string theory are largely based on the comments of physicists that I respect, and on the failure of the LHC to actually provide evidence for things like extra dimensions or super-symmetry, which were often claimed to be the major testable predictions of string theory. And as I understand it,
people seem to have given up on string theory producing any testable science anytime soon:
The upshot is that string theory today includes much that no longer seems stringy. Those tiny loops of string whose harmonics were thought to breathe form into every particle and force known to nature (including elusive gravity) hardly even appear anymore on chalkboards at conferences. At last year’s big annual string theory meeting, the Stanford University string theorist Eva Silverstein was amused to find she was one of the few giving a talk “on string theory proper,” she said.
Now, this isn't to say that the mathematics of string theory haven't proven useful elsewhere. From the same article:
Using the physical intuition offered by strings, physicists produced a powerful formula for getting the answer to the embedded sphere question, and much more. “They got at these formulas using tools that mathematicians don’t allow,” Córdova said. Then, after string theorists found an answer, the mathematicians proved it on their own terms. “This is a kind of experiment,” he explained. “It’s an internal mathematical experiment.” Not only was the stringy solution not wrong, it led to Fields Medal-winning mathematics. “This keeps happening,” he said.
I've been hearing versions of both these claims for years: String theory proper seems to has failed to make testable predictions, and it has gradually become less popular among theoretical physicists as an actual description of reality. But the underlying math has proven to be very useful elsewhere, and it genuinely interesting just as pure mathematics.
However, the experts certainly disagree on these issues, which is why I said "some would disagree." I was merely using string theory as an example of a multi-decade movement in a hard science that doesn't seem to have been as successful as researchers originally hoped. Reasonable people could clearly disagree, of course!
Josquin wrote:How firm is your grasp on psychology, which may be a "soft" science, but is a science nevertheless.
In general, as I said, I'm an enormous fan of some of the mathematical work done in experimental psychology. In this matter, I was significantly influenced by
Hany Farid, a researcher in image processing and machine learning, who sometimes took the time to talk about mathematical techniques used in other fields. In some cases, I have a working knowledge of a rudimentary mathematical technique (such as principal component analysis), and psychologists use a much more sophistic version of the same technique (which generalizes the basic idea to, IIRC, non-linear hypersurfaces), and I don't understand the more sophisticated version. My whole point in my original post is that experimental psychology has some amazingly sophisticated mathematical methods. And yet, experimental psychology has been badly hammered by the replication crisis. This is not a controversial claim on my part. The replication crisis is generally acknowledged by many researchers in the field. You can
start here if you're interested.
Josquin wrote:And what's your stance on the humanities, pray?! Where there are no statistical and mathematical approaches possible?
I assume this is a rhetorical question. But I approve of literature and art, and I enjoy them? Math is only useful for certain sorts of tasks, including things like predictions, modeling the physical world, and evaluating alternative methods of doing things. To choose an example from linguistics, I can't personally imagine any way that Egyptian grammar could be made more useful by adding a lot of math.
As for my complaints about the SLA papers I've seen, I'd love to list a few examples, with specific criticisms. But doing this would take a while, because I lost my old bookmarks, and it will take a while to track down the specific papers that I read, re-read them, and make sure that I still agree with all my earlier concerns.
But just to give another example, let me pick on poor Krashen again, because many of his papers are nicely indexed online, and I can find them again easily. Now, I really like Krashen's work, but much of it is just case studies from various classrooms. Sometimes
he'll publish a p-value, but as he points out, "Obvious flaws in these studies are that no comparison group was used, and there were only six to seven participants." But just as often, he'll publish a case study of three students, or summarize his own personal experiences as a learner. I think these papers are interesting, but they don't really prove very much.
Or if we look at the benefits of language learning (not technically SLA, I believe?), a lot of the
benefits of bilingualism have recently become controversial after a failure to reproduce some of the original studies.
Like I said, I'm certainly not an expert. My sample of papers is undoubtedly biased towards ones that get written up the media, which means I'm probably not looking at the best papers in the field. But my fundamental intuitions say that things like "no comparison group" or "only six to seven participants" are warnings that a study may be suggestive but not conclusive.
But at the end of the day, given the sheer number of hugely influential and widely respected papers in experimental psychology that have proven difficult to reproduce, I feel like it's worth being careful in other fields. I'm really not trying to be a jerk about this or anything, and I'd always be happy to look at links to SLA studies that people think are particularly important or solid. But I hope I've at least clarified the reasons for my earlier remarks. And thank you all for the discussion, which has definitely been interesting!
