Tuesday, December 9, 2008

Follow-Up on Differences in Test Score Variability

In an earlier post, I described a study by Hyde et al. in Science showing that the gender gap in math has disappeared, as measured by average scores on state accountability exams. However, in that article there was also evidence that boys are overrepresented at the high end of performance, due to greater variability in their scores.

When the Hyde et al. paper came out, the press did a good job of making the point that the scores of boys and girls are now equal on average. Most reporters ignored the overrepresentation of boys at the high end. Some may have wanted to tell a simple, positive story; some may have been influenced by Hyde et al.'s rhetorical strategies. (A reporter from Reuters was an exception, but I can't find her story to link to.)

Today I'm just following up because the variance ratio has now been observed in another study: "Global Sex Differences in Test Score Variability," by S. Machin and T. Pekkarinen, Science Vol. 322, 28 November 2008. Click here to go to the article (subscription required).

The new study, based on PISA results, is both weaker and stronger than the study by Hyde et al. It is stronger in that it includes more countries than just the U.S. But it is weaker in that it only looks at 15 year olds. (Hyde et al. considered students in 2nd through 11th grades.)

Also unlike Hyde et al., the study by Machin and Pekkarinen looks at both math and reading. Interestingly, the variance ratios turn out to be similar in the two subjects, but with different consequences or for different reasons. In reading, boys are overrepresented at the low end of performance, whereas in math boys are overrepresented at the high end.

As of this writing, I've only seen one news article on the study, a Washington Post article by Ruth Marcus. Her article is more about Larry Summers than about the research itself.


danimal said...

Hey Jason,

I guess it's possible that the larger variation in boys' math scores could be a cultural/expectations effect. For example, it may be more socially isolating for a girl to dedicate the time necessary to enter the 99th percentile in math/science, and to become known as the "brainy science girl", than it is for a boy.

It's an interesting question whether there's some genetic component to the greater variability for boys. I wonder if there could be more genetic variability in the males of species generally, and this might be a manifestation of that. In sexual reproduction, a given male can mate with a lot of females, while each female usually produces a limited number of offspring. The mean number of offspring of males and females is equal (by definition), but I'm pretty sure that the variation is larger among males. Then it would make sense for nature to generate more variability among males, with those males with the most desirable traits passing them on to the next generation.

Just a speculative idea, and I'm sure it is more complicated than that. Just for fun though, I looked up heights of US citizens, and men are 69 +/- 2.9 inches (standard deviation over average is 4.2%), while women are 65.5 +/- 2.5 inches (standard deviation over average is 4.2%). Not a huge difference, but it does have a real impact at large deviations from the average (i.e., there are more really tall men).

danimal said...

Oops, correction -- in the previous comment, I meant to type standard deviation over average for heights of US women is 2.5/65.5 =
** 3.8% **

Anonymous said...

Well, at least there are enough women in science to even conduct such a comparative study. I'm more worried about the lack of minorities in math and science. Try conducting a study on that. Once you get to the higher levels of math and physics forget about it.