Bad statistics? Or bad criticism of statistics?

A few weeks ago, I posted an article about bad statistics in some fMRI work onto my gchat message. Their train of logic was as follows: the reliability of measurements in fMRI experiments should be no higher than about 0.7 (where reliability is the test-retest consistency). Since the correlation of variables is related to the reliability of the measurements, we have a ceiling on the observable correlations. Given that, there were a large number of studies with suspiciously large correlations. They believe that this arises because certain researchers are selecting voxels which cross some threshold of correlation between behavior and brain activity, then using only these voxels for analysis. Worryingly, this biased voxel selection leads to inflated across-subject correlations and can even find significant measures from pure noise. This is a non-independence error – one could be selecting noise exhibiting the effect being searched for.

Some of the criticized authors have posted a rebuttal defending their methods. They have a variety of rebuttals, most of which seem to rest on their assurance that they are performing corrections for using multiple-comparisons and therefore what is described above is okay. Many of the rest of their rebuttals seems to be, mystifyingly, more about defending ‘social neuroscience’ and suggests that the paper offers a biased view of how fMRI studies are performed. It surely doesn’t do that – remember how I said that only a subset of the population seemed to be implementing a flawed statistical method? – but they are pretty defensive anyway. It’s a funny read because they are clearly very peeved throughout the whole rebuttal.

Then the original authors offered a rebuttal-rebuttal. I don’t feel confident enough in my knowledge of statistics to properly evaluate it. It certainly seems reasonable. For instance, choosing a threshold necessarily truncates the noise distributation; correcting for multiple comparisons does not seem like it would help fix the problem at all. Anyone out there with more knowledge than me like to weigh in on this?

The moral of the story? Be careful with statistics! It is really a shame that people are not put through a rigorous year-long series of statistics courses that they must pass when in graduate school. Statistics underlies all our data analysis and is one of the most misunderstood parts of science. Hopefully this back-and-forth will bring a good discussion of the relevant statistical techniques so the problem isn’t propagated any further.

Advertisements

Mind control

In possibly the strangest news I have heard in a long time, there is evidence that semen has mind control properties! Apparently drosophila semen have proteins that fight off rival male sperm and also alter the hormone balance in the female. The more semen the female is exposed to, the more the male ‘controls’ her sex drive.

Also, I’d read the other blog posts by the author of that article. They are, uh, pretty awesome?

tess-and-her-pink-purple-things_mBut gender is pretty interesting! How about the question of boys liking blue and girls liking pink? It seems pretty obvious to me that that is entirely socially transmitted, but some wacky folks want to claim it is biological. Of course, if it isn’t biological maybe we shouldn’t be dressing little girls up in pink all the time. The Straight Dope has an article discussing the history of pink as a ‘girls’ color and blue as a ‘boys’ color. A great quote:

A 1905 Times article said so, and Parents magazine was still saying it as late as 1939. Why pink for boys? Some argued that pink was a close relative of red, which was seen as a fiery, manly color…A color preference study of Caucasian and Chinese men and women showed both Caucasian and Chinese women strongly preferred red and pink, while Caucasian men strongly preferred blue and green. However, the Chinese men showed a broader range, with many picking red and pink — possibly because in China red is considered lucky.

Yup, definitely biological…for the last 100 years in the West. I am curious what an art-historical study would show? Well, let’s soothe everything down with some pictures of kids with their color-coded toys.