Misrepresenting Iraq Body Count

Basic errors by Media Lens

• One of the main premises of Media Lens's campaign against IBC is that "IBC is not primarily an Iraq Body Count, it is not even an Iraq Media Body count, it is an Iraq Western Media Body Count" (Media Lens 14/3/06, my emphasis).

This is entirely false. IBC use non-Western media sources and non-media sources (eg hospital, morgue and NGO data). They are able to monitor around 70 major "non-Western" sources on a daily basis, along with 120 "Western" sources. (IBC)

Many incidents/deaths in IBC's database are from the major wire agencies. This reflects the real-world fact that, for example, Reuters covers by far the highest percentage – approximately 50% of documented incidents, compared to 35% from Al Sharqiyah TV (another IBC source), and much lower coverage by other media sources, "Western" or "non-Western" (IBC). Note also that at the level of reporting utilized by IBC, the dichotomy of "Western" vs "non-Western" is false, as agencies such as Reuters employ (for example) Iraqi journalists in covering Iraqi incidents ("We mainly use local reporters, Arab reporters can go out and talk to people" – Reuters’ Baghdad bureau chief).

• In their first article criticising IBC, Media Lens wrote:

"Whereas the Lancet report estimated around 100,000 civilian deaths in October 2004, IBC reported 17,000 at that time."

This is incorrect in two ways. First, the Lancet study didn't estimate "civilian" deaths as Media Lens claim (its estimate includes "combatants" as well as civilians). Second, IBC record only violent deaths, so the comparison should be between 57,600 and 17,687 (57,600 being the Lancet study's estimate of violent deaths, according to Lancet co-author Richard Garfield). But even that isn’t comparing like with like, since IBC do not include combatant deaths, whereas the Lancet study does.

• Medialens also wrote (in the same article):

"But anyway, as we have seen, the IBC figure is selective in its sources, is the lowest estimate of eight serious studies, and relies on 'professional rigour' in the Western media that does not exist."

The claim that IBC provides the "lowest estimate" of "eight serious studies" has been widely circulated but is completely mistaken. It's based on a collection of errors and misconceptions (which were exposed in detail by IBC in 2006), including an error from Les Roberts (Lancet study co-author) which Roberts has acknowledged to be an error ("I said the IBC count was 17 deaths per day over the period 3/1/03 - 2/1/05. That was wrong." – Les Roberts, email to Gabriele Zamparini, June 2006).

It should also be noted that Media Lens's reference to IBC's figure as an "estimate" is misleading. IBC does not provide an "estimate" of a total number of deaths – it provides a running tally of documented violent civilian deaths. Media Lens apparently later realized their mistake, as they no longer refer to IBC's count as an "estimate" or imply that it's directly comparable to projected totals from sample surveys. (Indeed they now criticise journalists for making this mistake, although they haven't, to date, corrected the mistake in their own material).

• One of Media Lens's main claims is that IBC captures only "5-10% of the true death toll". One can see immediately that this isn't supported by their comparison of Lancet 2004 and IBC. (IBC's count of violent civilian deaths is 30% of Lancet 2004's estimate of total violent deaths. In other words, IBC is capturing much more than 30% of the "true death toll" of violent civilian deaths, given Lancet 2004 as a measure).

(It's interesting to note that later estimates, eg from IFHS and CRED, show that IBC is capturing at least around a third of violent civilian deaths – contrary to the claims of Media Lens. An earlier estimate, from ILCS, shows IBC capturing well over a half of violent civilian deaths.)

Latest errors by Media Lens

Media Lens continue their attack on IBC in a more recent article, Iraq Body Count: "A Very Misleading Exercise". This contains several misrepresentations and errors, which I list below:

1. Media Lens write: "IBC's response to the suggestion that violence prevents journalists from capturing many deaths has been, in effect, 'Prove it!'"

This is plainly false. IBC have always stated that "many if not most civilian casualties will go unreported by the media. That is the sad nature of war." Media Lens are aware of this (they've quoted IBC's statement) and cannot claim ignorance.

2. Media Lens: "It is striking that IBC link to a high-profile media report that so badly misrepresents its figures".

This is misleading. The purpose of IBC's link (titled "Lists of victims or victim categories to signal the pervasive impact on every sector of Iraqi society") is to provide an example of how media have used IBC's data on individual victims (see lower section of the cited article, which is clearly titled "Victims' Stories"). Whether Media Lens's assertion that the article "misrepresents" IBC figures has any merit or not is irrelevant to the point of the link. IBC doesn't endorse misrepresentations of its figures.

(Given Media Lens's advocacy for the Lancet studies on Iraq mortality, it's "striking" that they fail to mention a similar misrepresentation of IBC's figures by the Lancet study's authors, in an article for Slate magazine: "Today, IBC estimates there have been 45,000 to 50,000 violent deaths").

3. Media Lens: "Whereas IBC have responded vigorously, indeed tirelessly, in responding [sic] to the 2004 and 2006 Lancet studies..."

In fact IBC released only two documents commenting on Lancet 2006 (both mildly critical) and one on Lancet 2004 (uncritical):

http://www.iraqbodycount.org/analysis/beyond/lancet100000/
http://www.iraqbodycount.org/analysis/beyond/reality-checks/
http://www.iraqbodycount.org/analysis/beyond/state-of-knowledge/ (only part of this document deals with Lancet 2006).

That Media Lens is now condemning IBC for "responding" to the Lancet studies is itself an ironic turn of events. One of the main complaints of Media Lens's earlier articles targeting IBC was that IBC were "Refusing to Respond".

4. Media Lens: "It was [Marc] Herold's Afghan Victim Memorial Project that inspired John Sloboda to set up IBC. Herold's 'most conservative estimate' of Afghan civilian deaths resulting from American/NATO operations is between 5,700 and 6,500. But, he cautions, this is 'probably a vast underestimate' [...] There is no reason to believe that the application of the same methodology in Iraq is generating very different results."

Again this is mistaken and misleading. IBC use the same general approach as Marc Herold has used for Afghanistan, but they don't use the same methodology. One of three reasons listed by Herold in support of his comment in the same article is that his count includes civilian victims directly killed by US/NATO bombings and military action, while excluding victims of the Taliban or other perpetrators. IBC of course includes killings by any perpetrators in Iraq. There are several other differences in the methodologies, and there are also reasons to believe the approach in Iraq is generating somewhat different results than in Afghanistan. But it is unlikely that Media Lens have looked into the matter in enough depth to know the reasons. They have not looked into the matter closely enough even to know that there are differences in the methodologies, or even to know that it is not Herold's "Afghan Victim Memorial Project" (begun in 2004) that inspired IBC, but rather his "Daily Casualty Count of Afghan Civilians Killed by U.S. Bombing" – begun in 2001), two wholly different projects.

In any case, Professor Herold has now written to ZNet stating that the paragraph written by Medialens has inaccuracies which need to be corrected, and that the inference drawn from it regarding IBC is unwarranted.

5. Media Lens: "...what IBC is doing to promote or reduce the confusion".

This is an unworthy insinuation, suggesting IBC are "promoting" confusion, but providing no examples of this.

6. Media Lens: "Well, the bureau chief of one of three Western media agencies providing a third of IBC's data from Iraq sent this email to a colleague last year (the latter asked us to preserve the sender's anonymity)".

Media Lens also cited an "anonymous epidemiologist" in their earlier pieces targeting IBC. It was noteworthy then, as it is now with this anonymous "bureau chief" and "colleague", that these unnamed sources weren't able to send their comments directly to IBC (who would, of course, have treated them in confidence), or stand behind them publicly. In effect it amounts to 3rd-hand rumour-mongering.

7. Media Lens: "...a new ORB poll revealing that 1.2 million Iraqis had been murdered since the 2003 invasion".

This is inaccurate. ORB estimated 1.2 million murders. They did not "reveal" any number of actual murders. Note also that the ORB poll wasn't peer-reviewed science. According to ORB's publicity literature, the person conducting ORB's poll, Munqith Daghir, began his polling career in 2003, with little in the way of formal training or field experience. The ORB poll doesn't have the scientific standing of major studies such as ILCS, which Media Lens failed to mention.

8. Media Lens: "Why is it important for IBC [...] to challenge the methodology and conclusions of epidemiological studies published in the Lancet...".

IBC didn't "challenge" Lancet 2004 (see IBC's uncritical press release on Lancet 2004), so Media Lens are incorrect to write "studies" (plural). And other leading researchers besides IBC have expressed scepticism over the Lancet 2006 estimates: Jon Pedersen of the UNDP Iraq study, demographer Beth Osborne Daponte, Fritz Scheuren, a past president of the American Statistical Association, Professor Hans Rosling and Dr Johan Von Schreeb at the Karolinska Institute in Stockholm, Oxford physicists Neil Johnson and Sean Gourley, Debarati Guha-Sapir, Director of the WHO Collaborating Centre for Research on the Epidemiology of Disasters (CRED), among many others.

9. Media Lens: "Secondly, while IBC's self-described task does indeed require only "care and literacy", does not the task of challenging peer-reviewed science published by some of the world's leading epidemiologists require very much more? Does it not, in fact 'require statistical analysis or extrapolations,'...".

In fact, it does not require "statistical analysis" to observe that the Lancet 2006 figure implies that half a million death certificates are missing. It does not require "extrapolations" to observe contradictions in the accounts of the Lancet 2006 team's description of sampling, or to note that the sampling methodology as published wouldn't give you "random" street selection. You don't need "world's leading epidemiologists" to appreciate how important random sampling is.

Conclusion

The rhetorical basis of Media Lens's campaign against IBC is: "how dare these data collectors tirelessly and vigorously criticise an epidemiological study". It's a weak and misleading argument, and an appeal to crass credentialism. It's noteworthy that Media Lens don't apply the same credentialist standards to the ORB poll which they endorse (and which, as noted above, is not peer-reviewed science). It's noteworthy also that (to date) Media Lens have ignored a large body of science (from leaders in the fields of demography and epidemiology) which tends to support and confirm the data collected by IBC.

Bear in mind that Media Lens went as far as writing (in a letter to New Statesman magazine, 16/10/06) that, "to our knowledge, IBC has not been able to demonstrate support for its methods from a single professional epidemiologist". Presumably they weren't paying attention to the epidemiologists/demographers who have consistently supported IBC's methods. Here are some recent examples from leading authorities (postdating Medialens's NS letter):

Beth Osborne Daponte (the renowned demographer who produced authoritative death figures for the first Gulf War) has recently written:

While each approach has its drawbacks and advantages, this author puts the most credence on the work that the Iraq Body Count has done for a lowerbound estimate of the mortality impact of the war on civilians. The data base created by IBC seems exceptional in its transparency and timeliness. Creating such a data base carefully is an incredibly time-consuming exercise. The transparency of IBC’s work allows one to see whether incidents of mortality have been included. The constant updating of the data base allows one to have current figures.

Debarati Guha-Sapir and Olivier Degomme, from the Centre for Research on the Epidemiology of Disasters, Brussels, have written:

The Burnham [Lancet 2006] estimates of deaths in the post invasion period are much higher than any other estimate. Even the lower limit of its 95% CI is higher than the highest estimate from any other source (Table 1). Further, weaknesses cited earlier as well as several inconsistencies in their published work undermine the reliability of their estimates. [...]

While IBC is undoubtedly missing some deaths in Baghdad, it is unlikely that they would miss an average of over 100 violent deaths a day, given the level of media coverage in the city. We therefore conclude that their Baghdad mortality estimate is close to complete, further corroborated by the ILCS estimates [...]

Our re-estimation of total war-related death toll for Iraq from the invasion until June 2006 is therefore around 125,000.

Leading epidemiologists from World Health Organisation / Iraq Family Health Survey have written:

Both sources [IFHS & IBC] indicate that the 2006 study by Burnham et al [Lancet] considerably overestimated the number of violent deaths. To reach the 925 violent deaths per day reported by Burnham et al [Lancet] for June 2005 through June 2006, as many as 87% of violent deaths would have been missed in the IFHS and more than 90% in the Iraq Body Count. This level of underreporting is highly improbable, given the internal and external consistency of the data and the much larger sample size and quality-control measures taken in the implementation of the IFHS.