Tuesday, December 9, 2008

Follow-Up on Differences in Test Score Variability

In an earlier post, I described a study by Hyde et al. in Science showing that the gender gap in math has disappeared, as measured by average scores on state accountability exams. However, in that article there was also evidence that boys are overrepresented at the high end of performance, due to greater variability in their scores.

When the Hyde et al. paper came out, the press did a good job of making the point that the scores of boys and girls are now equal on average. Most reporters ignored the overrepresentation of boys at the high end. Some may have wanted to tell a simple, positive story; some may have been influenced by Hyde et al.'s rhetorical strategies. (A reporter from Reuters was an exception, but I can't find her story to link to.)

Today I'm just following up because the variance ratio has now been observed in another study: "Global Sex Differences in Test Score Variability," by S. Machin and T. Pekkarinen, Science Vol. 322, 28 November 2008. Click here to go to the article (subscription required).

The new study, based on PISA results, is both weaker and stronger than the study by Hyde et al. It is stronger in that it includes more countries than just the U.S. But it is weaker in that it only looks at 15 year olds. (Hyde et al. considered students in 2nd through 11th grades.)

Also unlike Hyde et al., the study by Machin and Pekkarinen looks at both math and reading. Interestingly, the variance ratios turn out to be similar in the two subjects, but with different consequences or for different reasons. In reading, boys are overrepresented at the low end of performance, whereas in math boys are overrepresented at the high end.

As of this writing, I've only seen one news article on the study, a Washington Post article by Ruth Marcus. Her article is more about Larry Summers than about the research itself.