The New York Times reports:
It’s good news that the Defense Department is finally beginning to take this issue seriously. I can’t say I’m surprised by the results – any time groups of young men are brought together and taught to be “masculine” (whether it’s in the army, a frat house, or a football team), chances are that some of the men will rape. Unfortunately, the DOD survey is very badly designed and so almost certainly underestimates the problem.
Why? Well, there are two basic approaches to surveying people about rape: either you just ask if a respondent has been raped, or you ask a whole bunch of very graphic “behaviorally specific questions,” such as “has anyone ever made you have sexual intercourse by using force or threatening to harm you… by intercourse I mean putting a penis in your vagina…,” with similar-but-not-identical questions asked to elicit reports of other kinds of rape (anal rape or oral rape, for example). Years of experience has taught social scientists that many rape victims will only report their experience in a survey when the latter approach is used.
As far as I can tell, the Air Force survey – described in unfortunately sparse detail in this pdf file – uses the earlier, outdated method of designing their rape survey. Instead of using behaviorally specific questions, they instead just asked respondents if they had been raped (the survey defined rape as “an act of sexual intercourse with a female, by force and/or without her consent [conscious or unconscious]“).
How big a difference could this make? A recent US department of Justice study directly compared the two methods, and found that 11 times as many rape victims reported their experiences when surveyors used behaviorally specific questions.
The Department of Defense says it intends to do further, more thorough surveys. Let’s hope they update their survey methodology to the present tense – the survey method they’re using now has been obsolete for at least a decade. Until their methodology catches up, however, the DOD’s 12% figure – shocking as it is – should be taken as an extremely low estimate of the problem.