Today’s Must Read

The long-story-short on measuring sectarian violence in Iraq? Don’t expect consistent data across U.S. agencies.

The Washington Post‘s Karen DeYoung gets an in-depth explanation of how the U.S. military’s Iraq command, known as MNF-I, tallies sectarian statistics, and notes the discrepancies between how the Pentagon and U.S. intelligence characterize the same data. On Friday, we published MNF-I’s explanation of what it does — and doesn’t — consider sectarian violence.

The analyst in charge of MNF-I’s sectarianism database is a chief warrant officer named Dan Macomber, and he freely concedes that there’s room for debate over whether or not his methodology is proper or consistent.

In recent months, most of the military’s indicators have pointed in a favorable direction. As with all statistics, however, their meaning depends on how they are gathered and analyzed. “Everybody has their own way of doing it,” Macomber said of his sectarian analyses. “If you and I . . . pulled from the same database, and I pulled one day and you pulled the next, we would have totally different numbers.”

Apparent contradictions are relatively easy to find in the flood of bar charts and trend lines the military produces. Civilian casualty numbers in the Pentagon’s latest quarterly report on Iraq last week, for example, differ significantly from those presented by the top commander in Iraq, Gen. David H. Petraeus, in his recent congressional testimony. Petraeus’s chart was limited to numbers of dead, while the Pentagon combined the numbers of dead and wounded — a figure that should be greater. Yet Petraeus’s numbers were higher than the Pentagon’s for the months preceding this year’s increase of U.S. troops to Iraq, and lower since U.S. operations escalated this summer.

That’s not to say the Pentagon quarterly report’s tabulation is consistent, either. As we’ve noted, the report’s tabulation of sectarian killing has fluctuated between editions. In the September edition, for instance, the Pentagon suddenly decided to include vehicle- and suicide-bombings in its sectarian total. According to Macomber, whenever MNF-I sent the Pentagon its sectarian-incident numbers, it always included car bombs and suicide bombs in the count, but the Pentagon, for unexplained reasons, plucked those incidents out. Why the change? “We regularly review our metrics to determine the most informative way to report what is happening in Iraq,” a DOD spokesman e-mailed DeYoung.

DeYoung doesn’t report that the statistics are being manipulated. Indeed, MNF-I tells her that there’s a “current effort to consolidate multiple databases” in Baghdad underway right now. But if, even for admirable reasons of improving flawed methodology, figures previously reported change between reports, or definitions subtly shift, it becomes difficult to independently assess the course of the war.

Intelligence officials, on the other hand, are attempting to be more cautious in their assessments than the military to guard against self-deception:

The U.S. intelligence community considers more than numbers in making its war assessments. “What the Iraqis perceive” about their country and their daily lives “may be more important than what the numbers are,” said a senior intelligence official, who discussed the subject on the condition of anonymity. Even so, he said, intelligence officials found contradictions in the available statistics as they wrote last month’s National Intelligence Estimate on Iraq, whose conclusions were somewhat less optimistic than the military’s.

“It’s not anybody trying to make it come out one way or another way,” said the official, who sympathized with the military’s need to quantify. But it is important, he said, to determine “what the numbers meant. Who collected them? Why do numbers that come in from this piece of the U.S. government differ from those coming in from another part of the government?”

While both Petraeus and the recent Pentagon report emphasized improved statistics over the past three months, the intelligence community generally declines to declare trends based on data measured in periods shorter than six months to a year. Several senior intelligence officials said last week that most numerical indicators appear to be moving in a uniformly positive direction in the nearly two months since the intelligence estimate’s data cutoff — although they said it is too early to determine definitive trends.

Ultimately, Macomber tells DeYoung, “it’s an analyst making an analyst’s call.” It remains an open question if analysts across the U.S. government can make the same call on the same piece of data.