National Academy of Sciences Finds Gaps in Army’s Study of Long-Term Health Effects on Test Subjects

In 2016, the Army issued a report titled, Assessment of Potential Long-Term Health Effects on Army Human Test Subjects of Relevant Biological and Chemical Agents, Drugs, Medications and Substances: Literature Review and Analysis, following a court injunction directing the Army to provide notice to test subjects on the possible long-term health effects of their exposure during testing. Between 1942 and 1975, the US Army conducted tests on 12,000 human subjects.

The National Academy of Sciences (NAS) created a commission to analyze the Army’s report. In 2018, they issued their report entitled Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects.

What Is the National Academy of Sciences?

The National Academy of Sciences(NAS) is a private, nongovernmental institution established in 1963 to advise the nation on science and technology related issues.

What Did They Study and Why?

From the years 1942 to 1975, the United States Army conducted tests on over 12,000 human subjects in order to study the effects of over 100 different substances, “including chemical warfare agents, biological agents, medications, vaccines, and other substances.” The Army tested the short-term and immediate effects of these exposures, and was not required to notify the test subjects of the potential long-term health effects of their exposures. However, an injunction filed in 2016 with the Ninth Circuit Court of Appeals required the Army to notify test subjects of the potential long-term health effects of their exposures and to issue a report entitled Assessment of Potential Long-Term Health Effects on Army Human Test Subjects of Relevant Biological and Chemical Agents, Drugs, Medications, and Substances: Literature Review and Analysis, referred to hereby as “the Report.”

NAS was tasked with forming a commission to conduct an independent review of the Army’s report to determine if the report “appropriately identified potential long-term health effects that could have resulted from test exposures.”

What Did the Committee Find?

The Report Does Not Provide an Adequate Literature Review

In their 67-page report, the committee found that the Report failed “to provide the literature search strategies, lack of critical analysis of the evidence, unclear criteria on which strength-of-evidence determinations were made, and no information on specific exposures on test subjects.” The committee found that the Report did not follow best practices for literature-based review. A literature review is meant to include the current knowledge surrounding a topic, as well as theoretical and methodological contributions to that topic. However, the commission found that the Report failed to include relevant research and literature on the exposure types, and also failed to provide a reasoning as to why it excluded or included the literature. For the criteria of the literature search, the commission states that the Report only provided date ranges and “two reference lists, one for references cited and one for references consulted” in their report, leaving little indication of what process they used to select the literature included in their review.

This is troubling as a literature review is essential to providing accurate and comprehensive context, and is a way of ensuring that a topic was thoroughly and completely reviewed. Committee members with subject matter expertise were concerned that significant studies were missing or certain agents that the Report discussed. The missing studies raised concerns among the committee that the Report also missed long-term health effects related to specific agents.

Lack of Criteria for Classification of Agents and Heath Effects

The Report identifies 18 agents as having potential long-term health effects, however the commission states that only short summaries were provided for each agent and “individual studies are not critically evaluated” and there is no substantial summary “of the collective evidence to explain how an agent was judged to have adequate or inadequate evidence of long-term health effects.”

In reference to the Report’s criteria for classifying the strength of association between the agents and potential long-term health effects, the committee stated that the Report used classifications of low, medium, and high. However, the Report did not adequately explain the criteria used to make these classifications, nor did the Report reference the specific health effects of the agents using this criteria. Additionally, the commission found that the Report did not give information on the “specific exposures to test subjects…beyond generalizations.”

Preparation of the Report

The contract that called for the Report specifically stated that the Report should be written in a manner that would “allow the public to understand the significance of the information.” However, the commission found that the Report was not written in this manner and uses the example of how many findings were described statistically, with no explanation of the terms or their significance. The commission writes, “…simply classifying the evidence as high, medium, or low for an agent without reference to specific health effects is not useful.”

Additionally, the contract stated that the Report was to be modeled after the format used in previous reports, however it was not clear to the commission what report the Report was modeled after. The commission was therefore unable to place the Report “in the context of previously drawn conclusions,” which could lead to the drawing of “erroneous conclusions.”

Other Notable Findings

The commission made a number of other significant observations regarding the Report, including that the Report provides conflicting definitions of key terms used throughout its analysis. The commission also noted the following deficiencies in the Report:

• The Report failed to “provide an adequate description of the overall objective of the review”
• The Report did not “adequately define the types of populations, exposures, comparators, and outcomes…considered relevant to the objective of the study.”
• The Report did not “provide evidence of expert judgement in drawing hazard identification conclusions.”

What Is the Takeaway?

Overall, the Report lacks the rigor and comprehensiveness necessary for the proper evaluation of the long-term health effects on the 12,000 human test subjects. The committee notes, “the Report should not be used as the sole scientific basis to assess the potential long-term health effects from exposure to biological and chemical agents, drugs, medications, and substances.”