Both TalkLeft and Atrios link to this post on Conclusive Evidence, in which journalist David Cullen, discussing the Air Force survey (see my previous post), writes:

So Dave’s point is that women are in more danger of being raped in four years of a regular college (20%) than at the Air Force Academy (12%).

Goodness knows I don’t want to slag on Dave Cullen, a good journalist whose heart is obviously in the right place. He’s absolutely right to think that the dangers of rape even at mainstream colleges is terribly underreported by the press. But he’s mistaken about these particular statistics.

First of all, the particular statistic Dave is relying on from “The Sexual Victimization of College Women” (SVCW) is very dubious. SVCW basically measured the rape incidence for college women for a seven-month period (which was 2.8%, including both completed and attempted rape). To extrapolate from that to four years, the authors multiplied 2.8% by seven (because seven months is approximately one-seventh of four years).

As DCH writes in the comments, “You can’t just multiply probabilities like that. If you do, then you predict, for example, that if a woman attended the Air Force Academy for 36 years, she’d have a 108% chance of being raped.”

Indeed, even the authors of SVCW say this procedure is “problematic for a number of reasons.” As the authors explain in a footnote, it would take a different kind of study “to assess accurately the victimization risk for women throughout a college career.” So it’s doubtful that the authors of SVCW would agree with the emphasis Dave is placing on this statistic.

Secondly, even if we ignore all that, David is still using the wrong statistic for his comparison. As I explain in the post prior to this one, the Air Force study used an outdated survey method - in which very few, non-detailed screening questions are asked - which is widely believed to underestimate rape incidence. As it happens, the SVCW study included a comparison componant, which - like the Air Force study - used a small number of non-detailed screening questions. Since this is the portion of the SVCW study with a methodology closest in design to the Air Force study, the results from this portion of the SVCW are the most appropriate to use for a comparison.

So what are the results? Using the outdated methodology, the SVCW found that 0.34% of college women experience completed or attempted rape in a seven-month period. Multiplying by seven to cover four years worth of college yields 2.4% (I know, this is bad methodology, but I’m ignoring that for the sake of the argument). So - using the closest comparison possible, given the datasets - it turns out that about 2.4% of women at ordinary colleges are raped (including attempted rapes) in a four-year college career, compared to about 12% at the Air Force Academy.

(Just to be clear - the 2.4% number is the best number to use for comparison to the deeply flawed Air Force survey, because it shares similar flaws. However, both numbers are almost certainly underestimates.)

I’m pointing this out because I don’t want Dave - or other progressives and feminists - to attach themselves to an easily-refuted mistake. However, let me emphasize again that although Dave’s mistaken about this one statistical comparison, he’s correct overall: the problem of rape in college - and, I’d add, in high school - is horribly underreported by the mainstream press.

UPDATE: Edited to add quote from DCH.

The New York Times reports:

It’s good news that the Defense Department is finally beginning to take this issue seriously. I can’t say I’m surprised by the results - any time groups of young men are brought together and taught to be “masculine” (whether it’s in the army, a frat house, or a football team), chances are that some of the men will rape. Unfortunately, the DOD survey is very badly designed and so almost certainly underestimates the problem.

Why? Well, there are two basic approaches to surveying people about rape: either you just ask if a respondent has been raped, or you ask a whole bunch of very graphic “behaviorally specific questions,” such as “has anyone ever made you have sexual intercourse by using force or threatening to harm you… by intercourse I mean putting a penis in your vagina…,” with similar-but-not-identical questions asked to elicit reports of other kinds of rape (anal rape or oral rape, for example). Years of experience has taught social scientists that many rape victims will only report their experience in a survey when the latter approach is used.

As far as I can tell, the Air Force survey - described in unfortunately sparse detail in this pdf file - uses the earlier, outdated method of designing their rape survey. Instead of using behaviorally specific questions, they instead just asked respondents if they had been raped (the survey defined rape as “an act of sexual intercourse with a female, by force and/or without her consent [conscious or unconscious]“).

How big a difference could this make? A recent US department of Justice study directly compared the two methods, and found that 11 times as many rape victims reported their experiences when surveyors used behaviorally specific questions.

The Department of Defense says it intends to do further, more thorough surveys. Let’s hope they update their survey methodology to the present tense - the survey method they’re using now has been obsolete for at least a decade. Until their methodology catches up, however, the DOD’s 12% figure - shocking as it is - should be taken as an extremely low estimate of the problem.

Links via TalkLeft and Conclusive Evidence.