Statistical method used in study represents major advance
In results from the largest health policy study of its kind, a Mexican health care program created in 2003 has been found effective in reducing crippling health care costs among poorer households. The results reflect the success of the Seguro Popular program, and arise from an evaluation conducted by researchers, including a Princeton University faculty member, in collaboration with researchers in Mexico.
"We were able to scientifically establish that the program achieved its main goal to reduce health care costs," said Kosuke Imai, an assistant professor of politics at Princeton who developed a new statistical method for the study. "This represents an important success not only for health care but also for a larger agenda to encourage evidence-based policymaking." The new method, he said, enables more accurate and efficient evaluation and is now being implemented or considered for evaluations of many other public policy programs around the world.
The results are published in the current issue of The Lancet.
"The success of Seguro Popular in reducing catastrophic health expenditures is remarkable, not least because governmental money spent on the poor in many countries rarely reaches the intended recipients," said Gary King, the lead author on the study and the David Florence Professor of Government and director of the Institute for Quantitative Social Science at Harvard University.
The study, which included 500,000 people, is the largest randomized health policy study ever conducted. The success of Seguro Popular, which covers about as many people as are uninsured in America, could provide lessons for other countries, the authors said.
Seguro Popular was developed to provide health care to 50 million Mexicans who otherwise would lack coverage. Voluntary enrollment in the program, provided at no cost to the poor, offers access to health clinics, medications, regular and preventive medical care, and the money to pay for it. The program's primary goal is the reduction of catastrophic health expenses -- costs that exceed one-third of a household's yearly disposable income.
"In addition to its substantive conclusions, the research offers new insights into how to evaluate policy programs," Imai said. "The challenge for program evaluation is the cost of following up on results for a large number of individuals."
The study monitored health outcomes and expenditures in 118,569 households over 10 months. In order to select comparable groups, the researchers paired up 174 communities based on similarities in background, such as how healthy its inhabitants were, the size of its population and the number of schools that were located there. One community within each pair was randomly chosen to receive treatment. Families in the treatment community were encouraged to enroll in Seguro Popular, health facilities were built or upgraded, and medical personnel, drugs and other supplies were provided. Families in the other community did not receive any change in their health care resources.
Researchers found this "matched pair" design decreased the margin of error to as little as one-sixth of what it would be with traditional experimental methods. In other words, researchers can have more confidence in conclusions with fewer individuals and communities. "The power of pair-matching is incredible," Imai said.
The new statistical method combined this pair matching technique with an approach known as cluster-randomized experiments. For reasons that include privacy, scientists in many disciplines from politics to public health are now opting to randomize clusters of people, rather than individuals, according to Imai. For example, political scientists studying the efficacy of various get-out-the-vote strategies may randomize households, rather than individual voters. Public health investigators may randomize hospitals rather than patients.
But, for statistical reasons, such cluster-randomized experiments require a larger sample size. The costs of conducting a large-scale evaluation can become prohibitive. The new method allows for the use of pair matching techniques in cluster-randomized experiments as a way to make policy evaluation more efficient. Imai is the lead author of a paper to come out later this year in the journal Statistical Science that describes the theoretical basis of the new approach.
Imai is interested in quantitative social science, which uses statistical methods to analyze social science problems. With a graduate student, he has developed a statistical method that can be used to design optimal get-out-the-vote campaigns. The method can inform campaign planners about which types of voters need to be canvassed with which method, such as door-to-door visits, phone calls or postcards.
The researchers were not able to establish in the time frame provided whether the Mexican program improved people's overall health and encouraged them to seek treatment more frequently. They may return for a follow-up study to investigate those longer-term trends, the authors said.
The research was funded by the Mexican Ministry of Health, the National Institute of Public Health in Mexico and the Harvard Institute for Quantitative Social Science.
