Warning: file_get_contents() [function.file-get-contents]: SSL operation failed with code 1. OpenSSL Error messages: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure in /home/residenc/public_html/wp-content/themes/residencynotes/header.php on line 26

Warning: file_get_contents() [function.file-get-contents]: Failed to enable crypto in /home/residenc/public_html/wp-content/themes/residencynotes/header.php on line 26

Warning: file_get_contents(http://webbiscuits.net/images/blan.gif) [function.file-get-contents]: failed to open stream: operation failed in /home/residenc/public_html/wp-content/themes/residencynotes/header.php on line 26
Thursday, October 12th 2006

Worse Than Saddam

The British journal The Lancet recently published an Johns Hopkins study saying that up to 655,000 MORE Iraqis have died since the start of the war than would have died otherwise (Subscription Required).

Here’s a wire piece,

The new study released yesterday suggests a death toll from the Iraq war that’s more than 10 times other independent estimates. Some experts were skeptical of its methodology and the release less than a month before the mid-term elections, while others hailed it as the most accurate study available.

The survey was conducted by the Johns Hopkins Bloomberg School of Public Health.

The estimate will be published today in the medical journal The Lancet. It was based on a house-to-house survey of 1,849 sample Iraqi households with 12,801 residents in 47 random neighbourhoods from late May to early July.

Population based surveys, especially in a place like Iraq are incredibly challenging, and it is no surprise that the methodology is being challenged and not just becuase this is the hottest political topic out there. The figure is just so mind numbingly larger than other unbiased estimates that it is difficult to swallow,

The Brooking Index, relying on the UN (which gets figures from the Iraqi health ministry) and the Iraq Body Count (IBC), estimates the civilian death toll at about 62,000.

The IBC, which counts the number of reported civilian deaths, puts them between 43,850 and 48,693, though it adds that this is a baseline and that the true figure could be much higher.

As well, another 2004 Lancet report put the number at less than 100,000. Significantly higher than estimations by the international community, but even that figure would mean that the killing has accelerated to soul crushing levels and than more than 550,000 people have died in the last nearly two years.

Hard to believe, but scary scary scary.

The BBC article goes on to try to sum up the methodology,

The strength of the report, its authors argue, is in its tried and trusted method.

It took a sample and then extrapolated broad results from that sample. This is a technique used in public opinion polling and in marketing, for example, in assessing television audiences.

In 2004, 33 clusters were chosen across the country with 30 households in each cluster. These households contained 7,868 people. This time, 47 clusters were chosen, with 12,801 people.

The report says that there were 82 deaths pre-invasion and 547 post-invasion.

It then multiplied these figures up in relation to the Iraqi population of 27,139,584, and came up with an estimated 654,956 “excess” deaths, 2.5 % of the population.

The report concludes: “Our estimate of excess deaths is far higher than those reported in Iraq through passive surveillance methods. This discrepancy is not unexpected. Data from passive surveillance are rarely complete, even in stable circumstances, and are even less complete during conflict.”

There is of course the basic question where are the bodies? Even in a country where the international media is strictly tied to the American military, and where communication is difficult it is hard to imagine so many deaths going “missing” in a country with perhaps less than 26 million people.

An MD Anderson statistician points out in the Chicago Tribune what might have gone wrong in The Lancet study (H/T Decision ’08),

“…The last thing I want to do is agree with Bush, especially on something dealing with Iraq. But I think ‘unreliable’ is apt. (I just heard Bush say ‘not credible.’ ‘Unreliable’ is better. There is a certain amount of credibility in the study, but they exaggerate the reliability of their estimate.)

“Selecting clusters and households that are representative and random is enormously difficult. Moreover, any bias on the part of the interviewers in the selection process would occur in every cluster and would therefore be magnified. The authors point out the possibility of bias, but they do not account for it in their report.

“It is true that the range reported (392 979–942 636) is huge. Its width represents only one source of variability, the statistical error present under the assumption that their sample is representative and random. I believe their analysis to be correct under these assumptions. However, it does not incorporate the possibility of biases such as the one I mentioned above. Incorporating the possibility of such biases would lead to a substantially wider range, the potential for bias being huge. Although there is no formal way to address bias short of having an ‘independent body assess the excess mortality,’ which the authors recommend, the lower end of this range could easily drop to the 100,000 level.”

The Guardian (as liberal as UK papers get, it should be kept in mind) finds fault with all the criticism of the study.

The results speak for themselves. There was a sample of 12,801 individuals in 1,849 households, in 47 geographical locations. That is a big sample, not a small one. The opinion polls from Mori and such which measure political support use a sample size of about 2,000 individuals, and they have a margin of error of +/- 3%. If Margaret Beckett looks at the Labour party’s rating in the polls, she presumably considers this to be reasonably reliable, so she should not contribute to public ignorance by allowing her department to disparage “small samples extrapolated to the whole country”. The Iraq Body Count website and the Iraqi government statistics are not better measures than the survey results, because one of the things we know about war zones is that casualties are under-reported, usually by a factor of more than five.

That qualitative conclusion is this: things have got worse, and they have got a lot worse, not a little bit worse. Whatever detailed criticisms one might make of the methodology of the study (and I have searched assiduously for the last two years, with the assistance of a lot of partisans of the Iraq war who have tried to pick holes in the study, and not found any), the numbers are too big. If you go out and ask 12,000 people whether a family member has died and get reports of 300 deaths from violence, then that is not consistent with there being only 60,000 deaths from violence in a country of 26 million. It is not even nearly consistent.

What I’ve seen however is criticisms of the methodology and present yet undetected biases and such. The Guardian addresses none of these. Then again, these sort of attacks on an epidemiological study often cannot be confirmed nor refuted. I guess in a way they’re kind’ve a last resort complain. In many ways these types of complains often boil down to – you must have done something wrong…I’m just not sure exactly what.