Chapter 12

The Salk Vaccine Trials


Introduction | History | Vaccines | Field Trial | Postscript | Resources and References

Introduction

The Salk polio vaccine field trials constitute one of the most famous and one of the largest statistical studies ever conducted. The field trials are of particular value to students of statistics because two different experimental designs were used. In this project, we will compare the designs of the experiments and analyze the results. 

Poliomyelitis, also known as infantile paralysis, is an infectious viral disease that enters through the mouth and is usually spread by contaminated drinking water or food. The virus passes through the stomach and then replicates in the lining of the intestines. Most healthy people infected with virus experience little more than mild fever or diarrhea. In about 1% of infections however, for reasons that are still unclear, the virus spreads to the bloodstream and central nervous system, causing varying degrees of paralysis, and in extreme cases, death.

Strangely enough, severe polio tends to be rare in communities with poor hygiene. The reason is that the virus is abundant in such communities, so babies are likely to be exposed to the virus early, while still protected with antibodies from their mothers. Later (assuming that they survive other diseases associated with poor hygiene), these children develop their own antibodies to the virus. The net effect is that in communities with poor hygiene, most people have a natural immunity.

  top

History

Polio has probably existed throughout human history. For example, there are references to paralytic polio in Egyptian artifacts dating to about 1000 BCE. However, severe polio was rare in ancient time, because of the hygiene factor discussed above. With improvements in sanitation, the incidence of polio began to rise in the first half of the 20th century, particularly in congested urban areas and particularly in the summer. In the US between 1930 and 1955, polio epidemics seemed to come in waves of ever increasing intensity. The vagaries of polio in terms of time and place, and the haunting images of young children in leg braces and the "iron lung" respirators of the time made polio one of the most feared of all diseases.

The following table gives the approximate number of polio cases in the US between 1930 and 1955. The numbers are adapted from Meier [4]. This type of data set is known as a time series.

Year Cases Year Cases Year Cases
1930 9000 1940 10,000 1950 32,000
1931 17,000 1941 9000 1951 27,000
1932 3000 1942 4000 1952 57,000
1933 5000 1943 12,000 1953 34,000
1934 7000 1944 19,000 1954 37,000
1935 10,000 1945 13,000 1955 27,000
1936 4000 1946 25,000  
1937 9000 1947 11,000
1938 1000 1948 26,000
1939 7000 1949 42,000

1. Draw a line graph of the data in the table above. Note the apparent increasing, yet cyclic nature of the data.

Franklin Roosevelt contracted polio as a young man in 1921, and was partially paralyzed for the rest of his life. As president, Roosevelt declared war on polio and thus beginning in the 1930s, considerable effort and resources were directed towards the treatment of polio victims and the search for ways to prevent the disease. In particular, two scientists headed research teams to find a vaccine: Dr. Jonas Salk of the University of Pittsburgh and Dr. Albert Sabin of the University of Cincinnati. 

  top

Vaccines

There are two general method of producing a vaccine for a virus such as polio. Both methods address, in different ways, the fundamental tradeoff between safety and effectiveness.

In the method favored by Dr. Salk, the vaccine is prepared from a sample of the virus that is first killed using heat and chemicals (such as formaldehyde). In principle, this method results in a vaccine that is very safe, because naturally, the killed virus in the vaccine is incapable of causing the disease. The vaccine can also be quite effective, if the killed virus is sufficiently intact to cause a strong antibody response in the vaccinated person. Thus, the process of killing the virus can be quite delicate and challenging: . If the process is insufficient, the vaccine will contain live virus and will itself be capable of causing the disease. If the process is too severe, the proteins of the killed virus will be destroyed to such an extent that the vaccine will be ineffective.

In the method favored by Dr. Sabin, the vaccine is prepared from a live, but mutated strain of the virus that is no longer capable of causing severe forms of the disease (such a virus is said to be attenuated). This method produces a vaccine that is generally very effective, because the live virus replicates in the body, causing a strong antibody response in the vaccinated person. The vaccine can also be quite safe, if the attenuated strain of the virus really is harmless. Again, the development process is delicate and challenging. 

The difficulty of developing a vaccine that is both safe and effective is illustrated by a polio vaccine developed in the 1930s. This vaccine was abandoned because a number children contracted polio from the vaccine, and several of those died.

  top

The 1954 Field Trial

By the early 1950s, Salk had finished the development of a killed-virus vaccine, administered in a series of three inoculations. The vaccine was believed to be safe (prematurely, it turned out), but a field trial was needed to judge effectiveness. 

Statistical tests of new medical treatments almost always have the same basic format: the responses of a treatment group of subjects who are given the treatment are compared to the responses of a control group of subjects who are not given the treatment. Before we study the designs of the 1954 field trials, let us consider some simple, but flawed designs.

2. Criticize the following design: A sample of children in a specified region are given the vaccine. The rate of polio among these children during a specified period of time is compared to the rate of polio in the same region during the same period of the previous year.

3. Criticize the following design: A sample of children in a specified region are given the vaccine. The rate of polio among these children during a specified period of time is compared to the rate of polio during the same period of time in a different, but similar region.

In general, the treatment and control  groups must be as similar as possible in every respect except the treatment factor. Thus, because of the vagaries of polio epidemics in time and place, it was essential to have the treatment and control groups drawn from the same regions and the same time periods. Moreover, a large number of subjects were needed for the trial for the following reasons:

  1. The disease is rare. The average incidence during the 1950s was about 50 per 100,000 or 0.0005.
  2. Diagnosis is difficult, particularly in mild cases. Mild polio is marked by fever and weakness, which of course, are also symptoms of many other diseases.

4. Suppose that the incidence of polio in a one-year trial period is about 50 per 100,000, and that we want to select a sample large enough so that about 100 persons in the sample would contract polio during the trial period. Assuming perfect diagnoses of the disease, how large should the sample be? 

The field trial was conducted by the National Foundation for Infantile Paralysis (NFIP). The original design called for vaccination of second-graders at selected schools in selected areas of the country (with the consent of the children's parents, of course). The vaccinated second-graders would form the treatment group. The first and third-graders at the schools would not be given the vaccination, and would form the control group. This design was known as the observed control experiment.

5. Why do you think the decision was made to have both first and third-graders as the control group, instead of just first-graders or just third-graders?

6. Can you identify any problems with the observed control experiment?

Hopefully, your answer to Exercise 6 included two serious issues: selection bias and diagnostic bias. Only those second-graders whose parents consented could be vaccinated, so to a certain extent, the treatment group is self-selecting. The consenting parents tended to have higher incomes than the non-consenting parents. This difference might not matter if the incidence of polio were unrelated to income. But in fact, the incidence of polio tends to be slightly lower among children of poorer parents, because of the hygiene factor discussed earlier. Thus, the selection bias is against the vaccine. Also, parents and doctors attempting to diagnose illnesses would know whether the child had been vaccinated. Such persons, even when trying hard to be objective, tend to be biased  in favor of the vaccine--being a bit more likely to diagnose polio in an unvaccinated child and a bit less likely in a vaccinated child. The net affect of the selection and diagnostic biases is unclear and thus makes the interpretation of the results difficult.

A second design was proposed to overcome the deficiencies of the first design. First a sample of children would be selected, all of whose parents consented to vaccination. The sample would be randomly divided into two groups. One group would be given the polio vaccination; the other group would be given a placebo (three injections of inert saltwater that would appear identical to the three injections of the real vaccine). This type of design is known as a randomized control experiment. The randomization tends to nullify all effects (confounding variables) except the treatment effect. Additionally, none of the participants would know the group identity--not the child, not the parents, and not the examining doctors. An experiment of this type, in which both the subjects and the evaluators are ignorant of the treatment/control status, is known as a double-blind experiment. The randomized control, double-blind design is considered the gold standard of statistical designs.

The results of the 1954 field trials are given in the following table, which is adapted from the famous Francis Report [2], the official evaluation of the trials:

Experiment Study Group Population Polio Cases False Reports
Paralytic Non-Paralytic
Randomized

Control 

Vaccinated 200,745 33 24 25
Placebo 201,229 115 27 20
Not Inoculated 338,778 121 36 25
Incomplete Vaccinations 8,484 1 1 0
Observed

Control

Vaccinated 221,998 38 18 20
Controls 725,173 330 61 48
Grade 2 Not Inoculated 123,605 43 11 12
Incomplete Vaccinations 9,904 4 0 0

Table Notes

  1. The False Reports column refers to cases that were initially reported as polio, but were later determined not to be polio.
  2. The Not Inoculated row refers to the children in the designated grades who did not participate in the experiment, and includes 8,577 children who received one or two (but not three) injections of placebo.
  3. The Incomplete Vaccinations row in both experiments refers to children who received one or two (but not three) injections of the vaccine.
  4. The Controls row refers to the total first and third grade populations used as controls in the observed control experiment.

7. Using the results of the randomized controlled experiment, perform an appropriate hypothesis test, at the 0.01 significance level, to see if the incidence of polio is larger for the control group than for the treatment group.

  1. State the hypotheses.
  2. Compute the polio rate for the control group and for the treatment group.
  3. Compute the appropriate test statistic.
  4. State your conclusions

8. Using the results of the observed control experiment, perform an appropriate hypothesis test, at the 0.01 significance level, to see if the incidence of polio is smaller for the treatment group than for the control group.

  1. State the hypotheses.
  2. Compute the polio rate for the control group and for the treatment group.
  3. Compute the appropriate test statistic.
  4. State your conclusions

9. Comparing the observed control experiment with the randomized control experiment, do you think that the observed control experiment is biased for or against the vaccine?

  top

Postscript

After the success of the field trials, the Salk vaccine was put into widespread use. Unfortunately, one improperly manufactured batch of the vaccine caused polio in a number of children, and the vaccination program was temporarily halted (see [4]). By the early 1960s, the Sabin live-strain virus had been perfected and soon replaced the Salk vaccine. 

Salk was at once lionized by the public and the media and criticized by other scientists, who felt that he had rushed the development of the vaccine for the sake of personal glory (and to beat Sabin).

In an invited review article [1] for the Journal of the American Statistical Association in 1955, shortly after the trials, KA Brownlee did not mince words in his scathing criticism of the original observed control experiment, referring to the design as "stupid and futile," and the results as "worthless."

In spite of the problems with the initial design of the statistical experiment, the manufacturing problems, and the excessive media hype, the development of the polio vaccines remains a supreme triumph of the scientific method, a triumph in which statistics played a fundamental role. Because of the vaccines, polio has largely been eradicated in the industrialized world, and countless children and others have been spared untold misery.

  top

References and Resources

Books and Articles

1. K. A. Brownlee. "Statistics of the 1954 Polio Vaccine Trials." Journal of the American Statistical Association 50 (1955), 1005-1013.

2. Thomas Francis, Robert Korn, et al. "An Evaluation of the the 1954 Poliomyelitis Vaccine Trials." American Journal of Public Health 45 (1955), 50 page supplement with a 63 page appendix. 

3. David Freedman, Robert Pisani, and Roger Purves. Statistics 3rd edition, WW Norton, 1998.

4. Paul Meier. "Safety of the Poliomyelitis Vaccine." Science 125 (1957) 1067-1071.

5. Paul Meier. "The Biggest Public Health Experiment Ever: The 1954 Field Trial of the Salk Poliomyelitis Vaccine" in Statistics, A Guide to the Unknown, Judith Tanur, Frederick Mosteller, et al. editors, Holden Day, 1978.

Web Sites

6. Britannica.com  

  1. Biography of Jonas Salk
  2. Biography of Albert Sabin

7. PBS 

  1. A Science Odyssey- People and Discoveries: Salk Produces Polio Vaccine
  2. Biography of Jonas Salk

8. Polio.net

9.  Polio Information Center Online (PICO)

10. March of Dimes > Polio
or From March of Dimes > Professionals and Researchers > Quick References: March of Dimes Fact Sheets > Polio


  top