In it, but not of it. TPM DC

Baucus Issues Public Call To McKinsey To Come Clean On Controversial HCR Survey

Max-baucus
Newscom

Baucus' efforts will be reinforced by top Democrats on the three House committees -- Energy and Commerce, Ways and Means, Education and Workforce -- with jurisdiction over health care reform, who will make similar demands of the consulting giant.

McKinsey stepped, perhaps unintentionally, into politically charged territory last week when it published the results of an in-house study on the impact health care reform will have on the employer-sponsored insurance market. The problem: The materials they used apparently did not meet the standards the firm would normally apply to such an issue -- and the results were wildly at odds with numerous other, more rigorous studies. That led to private infighting at McKinsey, and public pressure from the White House and congressional Democrats for their leaders to release the survey materials. Thus far they have declined to do so on the grounds that the survey is proprietary.

However, one insider told TPM that McKinsey would be loath to make those documents or the methodology public because they would embarrass the firm.

That doesn't satisfy Baucus. "The American Association for Public Opinion Research Disclosure Standards make clear that it is standard professional practice to release essential information on the methodology of publicly disclosed surveys," he writes in the letter. "I urge McKinsey & Company to publicly disclose the methodology behind its survey consistent with these standards."

We'll post the letter from House Dems when we get it.

Late update: Here's the letter from Reps. George Miller (D-CA), Henry Waxman (D-CA), Sander Levin (D-MI), Rob Andrews (D-NJ), Frank Pallone (D-NJ), Pete Stark (D-CA), John Dingell (D-MI), Diana DeGette (D-CO), and John Lewis (D-GA).

June 16, 2011

Mr. Dominic Barton

Managing Director

McKinsey & Company

55 East 52nd Street

New York, NY 10022

Dear Mr. Barton:

Last week, McKinsey & Company released a new survey, "How U.S. Health Care Reform Will Affect Employee Benefits." The results of this survey were markedly different from the conclusions of the non-partisan Congressional Budget Office, and other credible studies or surveys conducted by respected entities including the Rand Corporation and The Urban Institute, all of which projected little change to the number of employers offering health coverage under health reform. However, in the year since the law was passed evidence suggests that the number of small employers offering health care is actually increasing.[1] Finally, the real world experience in Massachusetts also belies the McKinsey findings, as employers largely retained coverage when the state's health care reform law - which is similar to the Affordable Care Act - went into effect.

Indeed, the findings of this survey are so markedly out of sync with other assessments that it has raised legitimate questions about the product, including how and why it was created. The report itself states that McKinsey "educated respondents" about the implications of the Affordable Care Act, with no indication of the content of this "education." Anonymous sources in or close to McKinsey have pointed to potential flaws in or limitations to the study, including the fact that the survey was devised by McKinsey employees without expertise in health care policy.[2] One source noted that "this particular survey wasn't designed in a way that would allow it to be peer review published or cited academically."[3] Another source indicated that "the firm may have reached its outlying conclusions by basing its questions on the firm's own advice to clients on how best to arbitrage the new reforms," while a third stated that release of survey materials would "be damaging" to McKinsey.[4]

Even before these misgivings were made public, our offices and others have asked McKinsey for the survey instrument and methodology, only to be rebuffed and told that the information is "proprietary." This is mystifying, however, as it has also been confirmed by McKinsey staff that the survey was generated by McKinsey and not at the request of a client. Thus, the decision to release associated materials seems to be wholly within the purview of McKinsey.

Refusing to release the underlying questions and methodology undermines the credibility of the findings. We are concerned that, if the survey based its conclusions on a questionable instrument and potentially biased methodology, McKinsey may have provided the American public with invalid information about the impact of the Affordable Care Act.

We are therefore writing to obtain additional information that would allow interested parties to objectively assess the implications of the report's findings. We ask that you provide the following information:

1. A complete copy of the final study methodology and all drafts of the study methodology, including any information on how survey respondents were chosen, and complete scripts of all questions asked to employers surveyed by McKinsey or its contractor(s) as part of the study.

2. Results, in tabular form, of responses to questions asked as part of the survey, including results for questions that were not discussed in the publicly available report.

3. All documents related to education of respondents conducted by McKinsey or its agents as part of the survey, including any scripts or written materials used in this process.

4. Information on the demographics (size, location, and industry type) of all businesses contacted as part of the survey, and all businesses that responded to the survey.

5. The names, roles, and professional qualifications of the McKinsey analysts who developed and conducted the study and wrote the public report.

6. Information on the total cost of the study, any third parties that paid McKinsey to conduct the study, and the amount that these third parties paid for the study.

7. Information on any complete, on-going, or planned consulting work by McKinsey for survey respondents related to either the Affordable Care Act or to the recent study.

We understand that companies may have responded to the survey under the assumption that their individual responses would be kept confidential. We therefore wish to clarify that, with regard to questions 1-4, we are asking only for summary information and are not seeking the names or survey results of any individual survey respondent.

We ask that you provide us the requested information no later than July 5, 2011.

Sincerely,