Presentation made to a panel discussion on "Civilian Deaths in Iraq: Quantitative Estimates and Policy Implications," held at the United States Institute for Peace (USIP), Washington DC, 10 Jan 2007.
Other explanations for Lancet figures:
- Main Street Bias
- Impossible time contraints
- Respondent self-selection
How did Lancet arrive at such high numbers?
One explanation proposed by a group of UK-based researchers is “Main Street Bias”, which argues that the study’s household selection methodology favoured streets crossing “main streets”. This would lead to an upwards bias because people living in these areas are more exposed to violence, particularly bombings targeting military vehicles or busy areas.1
1 Conflict Mortality Surveys Johnson et al. 2006-7.
2 Mortality after the 2003 invasion of Iraq: Were valid and ethical field methods used in this survey? (PDF) Households in Conflict Network, 1 Dec 2006.
Another paper, by Dr Madelyn Hicks of London’s Institute of Psychiatry, looked at the amount of time available to travel to the randomly-chosen locations, select the first house and then the 39 “nearest” or “adjacent” homes, and interview their households. Based on information made available by the authors she came to the conclusion that the allotted average time was disturbingly short and inadequate to meet certain ethical standards (such as privacy of respondents and obtaining informed consent).2
These conditions, and others described by the Lancet authors, may have allowed respondent self-selection.
A cardinal rule of representative surveys is that you don’t survey those who “think they should be included” in a survey and present themselves for inclusion, no matter how strongly they feel this. The sample must be selected by rigorous rules under the strict control of the survey team and no one else.
Consider then, this description by the authors of how the survey was conducted: