CDC Director Cohen Must Reject HICPAC’s Shoddily Evidenced, Unreviewable, Statistically Invalid, Non-Performing, Conflicted, and Un-Peer-Reviewed Infection Control “Guidance”

By Lambert Strether of Corrente.

I apologize in advance for this book-length, grab two-cups-of-coffee post, but I think that empowering the Covid-conscious by maximizing the attack surface for assaults on HICPAC’s anti-mask, anti-science infection control “Guidance”[1] (“CDC Advised To Weaken Infection Protections As Mysterious Pneumonia Brews Overseas“) is important, tactically, just now, and so I’m gathering as much information as I can, here, in one place. And for those who love the nuts and bolts of public administration, this post should be a real barn-burner. (For background, previous NC posts on HICPAC: November 9, August 17, August 14, July 20, and July 10.)

Here is a roadmap to the post. I will begin by briefly explaining the HICPAC process, showing the steps by which guidance is developed, where we are in the process, and the documents involved.

Next, I will look at (1) HICPAC’s shoddy evaluation of evidence, show how (2) HICPAC’s “Evidence Reviews” were unreviewable by the public, and assault HICPAC’s evidence in toto by (3) raising statistical issues with every engineering-hating gatekeeper’s favorite tool for minimization, the Randomized Controlled Trial (RCT). Finally, after (4) showing that HICPAC is conflicted, I will examine two institutional mechanisms for ensuring that “Guidance” as sloppy and degraded as HICPAC’s never sees the light of day: (5) Performance Reviews under FACA, and (6) the Office for Management and Budget (OMB)’s “Final Information Quality Bulletin for Peer Review.” (These last two items are, so far as I know, original to this post, so after dumping a suitable truckload of salts, bureaucratic knife-fighters and public-spirited lawyers may wish to see if they can be developed further, if only as diversions.)

The HICPAC Process

Here, from CDC itself, is a handy diagram:

(The same process was used by CDC in developing its guidance for Anthrax.) As you can see, we are in Step 4, “Public Comment, waiting for CDC approval[2]. In prose, again from CDC:

The committee voted on whether the updated (revised) Part 1 draft presented on Day 2 could be sent to CDC for review before posting in the Federal Register. HICPAC met pursuant to the Federal Advisory Committee Act (Public Law 92-463). Part 1 cannot be considered final until posted in the Federal Register for public comment, revised for public comment, and presented and voted on again in 2024.

(Don’t let the stately pace and all the verbiage distract you. This HICPAC guidance is as wired up as it can be.)

The CDC describes the documents involved in developing the draft in Morbidity and Mortality Weekly Report:

A CDC is any document issued under agency authority that contains recommendations for clinical practice or public health policy. Recommendations are statements that describe a specific prevention, treatment, or policy action. The scientific evidence underlying these statements is typically obtained through the of the literature and organized in evidence summaries. These evidence summaries present the causal associations that were critical to the decision-making process used to develop the recommendations.

For the purposes of this post, two documents are relevant. The first is the guidance itself: “2024 Guideline to Prevent Transmission of Pathogens in Healthcare Settings” (in short form, “Guideline”). The second is the systematic evidentiary review: “Healthcare Personnel Use of N95 Respirators or Medical/ Surgical Masks for Protection Against Respiratory Infections: A Systematic Review and MetaAnalysis” (shorter, “Review”).

The “Guideline,” through its recommendations, determines policy. HICPAC’s policy proposals are rancid, lethal, and anti-working class. From National Nurses United:

HICPAC’s draft guidance is aimed at frontline nurses and other health care workers, rather than health care employers, inappropriately shifting responsibility and risk to individual workers to protect corporate profits. The focus of HICPAC’s draft is almost exclusively on personal protective equipment, and it fails to make strong recommendations on other essential measures, such as ventilation and patient screening and isolation.

“HICPAC’s draft is permissive and weak and seeks not just to maintain existing practice — which has been shown to be inadequately protective — but even rolls back the use of some important measures, such as airborne infection isolation rooms,” said NNU President Zenei Triunfo-Cortez, RN. “This draft guidance will only further degrade the already dangerous working conditions of nurses and other health care workers and further contribute to high rates of moral distress, which will only serve to drive more nurses away from the bedside and further deepen the staffing crisis in health care.”

All my previous posts have been focused on the technical foundation of “Guidance,” which is “Review.” In this post, headings (1)-(3) focus on “Review.” Headings (4)-(6) focus on “Guidance,” which incorporates “Review” by reference[3]. More vividly, in (1)-(3) I will dynamite the foundations of the building; in (4)-(6) I will set the entire building on fire. Let’s begin!

(1) HICPAC’s Evidence Is Shoddily Handled

This doesn’t take much. Let’s start with the first set of footnotes in “Review,” 1-5. I have helpfully annotated the screen shot:

[1] Notice the subject-verb agreement issue, perhaps a sign that too many cooks spoiled the soup of the second paragraph of the introduction. This should not surprise us; we are used to careless copy-editing at CDC.

[2] Notes 1-5 are more interesting, because they direct us to note 2:

Note 2 refers to the infamous Cochrane study (“Physical interventions to interrupt or reduce the spread of respiratory viruses”). CDC writes:

Medical or surgical masks and N95 respirators, plays [sic] a critical role in protecting healthcare personnel from exposures to infectious respiratory illnesses in healthcare facilities. In laboratory settings, N95 respirators have been than surgical or medical masks at filtering particles and challenge viruses in lab settings.

First, the Brownstone Institute-affiliated first author of the Cochrane study swanned about the media claiming his own study showed that masks were ineffective, for which Cochrane itself had to issue a statement rebuking him:

Many commentators have claimed that a recently-updated Cochrane Review shows that ‘masks don’t work’, which is an inaccurate and misleading interpretation.

It would be accurate to say that the review examined whether interventions to promote mask wearing help to slow the spread of respiratory viruses, and that the . Given the limitations in the primary evidence, the review is not able to address the question of whether mask-wearing itself reduces people’s risk of contracting or spreading respiratory viruses.

Call me crazy, but I don’t think “proven to be more efficacious” means the same as “results were inconclusive.” The Cochrane debacle was a major scandal. Yet not only does CDC include it as a reference, it misquotes it! I don’t have time to look at the other 91 references, but I don’t think I need to: One roach in a wedding cake is enough. Obviously, however, somebody should, before it reaches the CDC Director’s desk, let alone the Federal Register. (And do note the interesting possibility that somebody at CDC fixes the subject-verb agreement problem, but leaves the footnote untouched.)

(2) HICPAC’s “Evidence Reviews” were Unreviewable by the Public

In my previous post on “Review,” I had only a PowerPoint presentation to work with (CDC having violated FACA by not making drafts available). I criticized the PowerPoint for failing, as claimed, to meet the PRISMA protocol, and for not disclosing Division of Healthcare Quality Promotion (DHQP) scales used to assess the validity of the studies. Those defects in “Review” are remedied in the complete draft.

However, the salient features of the “Evidence Reviews” in “Review” are contained in tables, called “Evidence Snapshots”. Here are the relevant tables in the PowerPoint presentation in August (highlighted by me):

And:

In November’s draft, you can see that August’s two tables are consolidated into one:

Moreover, if you look at the Outcomes, Summaries, and Studies, columns, they all differ.

This matters, because in reviewing the August tables, I disaggregated both tables and ranked the individual studies by strength, precision, confounders, and other negative factors — all from CDC’s data! The “best” studies ranked highest, and lo and behold! They were all pro-respirator (see NC here). CDC had jiggered the tables. More politely:

3. I’d very much like to believe that when I disaggregate Slides 31 and 32, and pro-respirator, anti-Baggy Blue studies — to everyone’s utter surprise! — float to the top, that I have committed an error, and that HICPAC and CDC are on the up-and-up, having learned something from CDC’s initial debacle on mask policy, and its losing tooth-and-nail battle against the science and engineering of airborne transmission…. CDC has form. A hermeneutic of suspicion is fully justified.

National Nurses United comments on the time CDC made available for a public review:

Despite the critical importance of this infection control guidance, the CDC/HICPAC is only accepting written public comment on the draft until 11:59 p.m. on Monday, Nov. 6, giving the public a mere five days to review the 24-page document and 162 pages of supporting documentation and provide feedback.

The tables in the August PowerPoint version of “Review” were the technical foundation of “Guidance”; that is why HICPAC’s working group requested them. And yet, in November’s version of “Review,” the technical foundation of “Guidance” has completely changed. The August tables were already a Gish Gallop. Does CDC really expect the public to run a second Gish Gallop, in only five days?

(3) HICPAC’s Randomized Controlled Trials (RCTs) Are Statistically Invalid

Yaneer Bar-Yam, Jonathan M. Samet, Alexander F. Siegenfeld, Nassim N. Taleb published “Quantitative errors in the Cochrane review on ‘Physical interventions to interrupt or reduce the spread of respiratory viruses’” in October of this year. However, the implications of their article go far beyond their demolition job of Cochrane (reviewed by our own KLG here). From the Abstract:

N95 respirators, and other forms of respiratory protection, have proven efficacy in reducing the risk of infection across various environments. Even though multiple studies confirm their protective effect when adopted in healthcare and public settings for infection prevention, studies on their adoption over the last several decades in both clinical trials and observational studies have not provided as clear an understanding. Here we show that . By correcting these equations, it is demonstrated that conclusions drawn from these studies are heavily biased and uncertain, providing little useful information. Despite these limitations, we show that when outcome measures are properly analyzed, existing results consistently point to the benefit of N95 respirators over medical masks, and masking over its absence. Correcting errors in widely reported meta-analyses also yields statistically significant estimates. These findings have important implications for study design and using existing evidence for infection control policy guidelines.

In short form, every RCT cited in “Review” is invalid until proven otherwise. More work for the hapless intern straightening out the “evidence snapshots”!

(4) HICPAC’s Membership Is Conflicted

From the technical foundations of “Guidance” in “Review,” we now turn to larger, institutional factors. Again from CDC, “Standards Required for the Development of CDC Evidence-Based Guidelines“:

The Federal Advisory Committee Act (FACA) provides the framework for consulting with experts outside the federal government to provide advice on appropriate recommendations (16). FACA delineates how federal advisory committees [like HICPAC] should be operated and managed… . A competing interest exists when professional judgment or actions concerning a primary interest, such as patients’ or the public’s welfare or the validity of research, might be improperly influenced by a secondary interest, such as financial gain, professional advancement, or personal relationships.

Every HICPAC member is affiliated with a hospital, and all these hospitals have already adopted masking policies as weak and lethal as “Guidance” proposes, as I show here. The conflicts are three-fold: Trivially, HICPAC members are conflicted because the cost of respirators will appear as a budget line-item for the infection control departments to which many of them belong (“professional advancement”). From NNU in MedPage Today:

After nearly 4 years of living with the COVID pandemic, we have all learned hard lessons about PPE and how healthcare employers often focus on the bottom line rather than protecting healthcare workers and patients.

Second, HICPAC members would incur reputational damage for adopting, as CDC Guidance, a masking policy more protective of patients and workers than they have already adopted for their own institutions, whether formally, or complicitly (“personal relationships”).

Third (“financial gain”), the avoidance of liability:

As an industrial hygienist, Seminario was extremely critical that there were no experts in respiratory protection on the committee nor did it include engineers who developed ventilation guidelines. She believes that the some oppose respirators “because once you are into recommending respiratory protection, with that comes a full respiratory protection program from OSHA,” with penalties for violations.

An epidemiologist and consultant, Michael Olesen, echoed this, citing “pressure to remove liability from hospitals.” He added, “I take a very clear position that we should be having respiratory protection mandates in all healthcare settings right now.”

And on the Twitter:

One hates to be cynical. But this is the CDC. In any case, HICPAC is clearly riddled with “secondary interests,” and the work product of such an entity should not reach the CDC Director’s desk for approval.

(5) HICPAC’s Treatment of Public Comments Is So Belittling As To Trigger a Performance Review Under FACA

Let’s begin by establishing that public comments are, in fact, valuable. In Academic Medicine, “Why Public Comments Matter: The Case of the National Institutes of Health Policy on Single Institutional Review Board Review of Multicenter Studies,” from the Abstract:

In this case study, public comments were effective in shaping policy as the NIH modified provisions or planned supplemental guidance in response to comments.

And from the Center for Medicare and Medicaid Services, “Public Comments“:

CMS encourages public comments and the submission of additional medical, technical, and scientific evidence, and must respond to these comments in our final determination as required by law. Public participation increases the quality of agency decision making.

However, HICPAC members, by their actions, disagree. Here is a live tweet storm that shows some of their belittling behaviors. Chatting on cellphones:

Making live edits:

Replacing public commenter’s faces with timers on Zoom (priorities!):

HICPAC is governed by Federal Advisory Committee Act (FACA), which is administed by the General Services Adminitration (GSA). The GSA has established “Performance Guidelines“:

The Administrator shall prescribe administrative guidelines and management controls applicable to advisory committees, and, to the maximum extent feasible, provide advice, assistance, and guidance to advisory committees to improve their performance. In carrying out his functions under this subsection, the Administrator shall consider the recommendations of each agency head with respect to means of improving the performance of advisory committees whose duties are related to such agency.

(Bold in the original.) Put aside the issue of only five days for public comment of “Guidance” when it has an entirely new technical foundation in November’s Review. Can anyone really believe that HICPAC members diddling with their cellphones during public comment — not to mention the Committee chair who permitted this discourtesy — does not require intervention by a GSA Administrator committed to improving HICPAC’s “performance” “to the maximum extent feasible”? And clearly, deliverables produced by such a Committee, unserious with respect to public comment, should not reach the CDC director’s desk for approval.

(6) As a “Highly Influential Scientific Assessment,” HICPAC’s Guidance Must Be Vetted Under OMB’s “Final Information Quality Bulletin for Peer Review”

The National Nurses Union explains why “Guidance” (and its technical foundation, “Review”) are influential:

The draft Isolation Precautions Guideline, which was only released to the public yesterday, Nov. 2, is an extremely influential document that directs infection control practices for health care facilities in the United States. This guidance, which has not been updated in 16 years, will be referenced by U.S. employers and government agencies across the country and also by health agencies around the world.

As it turns out, there’s a revview process for “influental” documents. From the Bureau of Reclamation (USBR):

The Office of Management and Budget (OMB) directive, Final Information Quality Bulletin for Peer Review, dated December 16, 2004 (263 KB PDF), requires that there be a “systematic process of peer review planning” and public access to a list of information products for that will be peer reviewed as either influential scientific information or .

The USBR provides a handy of projects that fall under the aegis of “Highly Influential Scientific Assessments.” One of them is the “Anderson Ranch Water Quality Model.” Surely we can agree that if a water modeling study for a review in in Idaho falls under the aegis of OMB’s “Final Information Quality Bulletin,” that “Guidance” (and “Review”), which will affect the health and the lives of millions of hospital patients and workers, should do as well? OMB agrees. From “Final Information Quality Bulletin for Peer Review“:

The term “influential scientific information” means scientific information the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions.

Here are OMB’s requirements for how “influential scientific information” should be peer-reviewed:

In general, an agency conducting a peer review of a highly influential scientific assessment must ensure that the peer review process is transparent by making available to the public the written charge to the peer reviewers, the peer reviewers’ names, the peer reviewers’ report(s), and the agency’s response to the peer reviewers’ report(s). The agency selecting peer reviewers must ensure that the reviewers possess the necessary expertise. In addition, the agency must address reviewers’ potential conflicts of interest (including those stemming from ties to regulated businesses and other stakeholders) and independence from the agency.

A proper peer review of HICPAC’s guidance would find, as we have shown above, conflicts of interest. In addition, if the peer reviewers included, as they should, aerosol and industrial experts, they would surely find that HICPAC’s expertise was insufficient.

Note this requirement from OMB:

Draft influential scientific information presented at scientific meetings or shared confidentially with colleagues for scientific input prior to peer review shall include the disclaimer: “THE FINDINGS AND CONCLUSIONS IN THIS REPORT (PRESENTATION) HAVE NOT BEEN FORMALLY DISSEMINATED BY [THE AGENCY] AND SHOULD NOT BE CONSTRUED TO REPRESENT ANY AGENCY DETERMINATION OR POLICY.”

Here is CDC’s footer on “Review”:

Disclaimer: The findings and conclusions herein are draft and have not been formally disseminated by the Centers for Disease Control and Prevention and should not be construed to represent any agency determination or policy.

Not in all caps, but clearly OMB’s verbiage. I would urge that this verbiage shows that HICPAC is fully aware of its obligations under OMB’s “Final Information Quality Bulletin,” but has chosen to go ahead with a conflicted committee regardless. I don’t know at what point OMB should intervene — Before the Federal Register? After? — but intervene they should, because a draft so conflicted should not be presented to the CDC director for approval.

Conclusion

[Shortly to come.]

NOTES

[1] Sources treat “guidance” and “guideline” as synoyms. I prefer guidance. Public administration mavens correct as needed.

[2] CDC is noticeably opaque on how it — presumably in the person of its Director — approves the guidance, and then forwards the guidance onward to the Federal Register for another round of comments. Perhaps some kind reader will leave a link in comments, or throw something over the transom. (You can find my address at the bottom of Water Cooler just before the plant of the day.)

[3] From “Guideline“:

In order to provide context to the update of the 2007 Guideline’s Transmission-Based Precaution categories, the authors requested three systematic reviews from DHQP that answered questions on the performance of several PPE items. These systematic reviews, which were not conducted to support the development of specific recommendations, answered the following key questions: 1. For healthcare personnel caring for patients with respiratory infections, what is the effectiveness of N95 respirators compared to medical/surgical masks in preventing symptomatic illness or laboratory649 confirmed infection? .

I just love “[cite webpage].” As you can see, CDC is, as it were, pulling the ladder up after itself by concealing the technical basis for its recommendations (and how that basis shifted over time).

Print Friendly, PDF & Email