Globally, inadequate immunization rates among children are an ongoing issue. Recently, a study carried out in India demonstrates that a cost-effective combination of strategies, such as text notifications and minimal financial rewards, significantly enhances immunization rates.
Directed by economists from MIT, the study reveals that a combination of incentives, text messages, and information shared by local community members leads to a 44 percent improvement in child immunizations at a low cost. Conversely, when financial incentives are not utilized, but text messages and local insights are employed, there is still a 9 percent rise in immunizations at virtually no cost — marking the most cost-efficient increase identified by the researchers.
“The most beneficial combination overall includes incentives, reminders, and engagement of community ambassadors to alert individuals,” states MIT economist Esther Duflo, a key contributor to the research. “The expenses are quite minimal. An even more economical arrangement would be to exclude incentives entirely — immunizations can be increased simply through reminders leveraging social networks. That effectively amounts to a free benefit because it optimizes the existing immunization infrastructure. Thus, the slight cost incurred by the program is outweighed by the reduction in the overall cost of administering immunizations.”
This study is also remarkable due to the advanced methodology the research team created to integrate various strategies within the experiment — allowing them to precisely assess the effects produced by different combinations as well as their individual components.
“The great aspect of this is how it connects and synthesizes all these pieces of evidence,” explains MIT economist Abhijit Banerjee, also a lead on the project. “Our confidence in asserting this is a viable policy strategy is significantly enhanced.”
A new publication that outlines the findings and methodology, “Selecting the Most Effective Nudge: Evidence from a Large-Scale Experiment on Immunization,” will be published in the journal Econometrica. Duflo and Banerjee are among 11 co-authors of this paper, alongside several members from MIT’s Abdul Latif Jameel Anti-Poverty Lab (J-PAL).
Both Duflo and Banerjee are co-founders of MIT’s Abdul Latif Jameel Anti-Poverty Lab (J-PAL), a leading global authority on field experiments regarding poverty alleviation programs. In 2019, they were honored with the Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel, along with Michael Kremer from Harvard University.
Evaluating 75 Strategies Simultaneously
Yearly, approximately 2 million children worldwide perish from diseases that could be prevented by vaccines. In 2016, when this study commenced, only 62 percent of children in India were completely immunized against tuberculosis, measles, diphtheria, tetanus, and polio.
Previous studies conducted by Duflo and Banerjee have affirmed the importance of exploring innovative methods to enhance immunization rates. In a past study, the economists discovered that immunization rates for children in rural Rajasthan, India, surged from 5 percent to 39 percent when families were offered a small quantity of lentils as an incentive. (This insight was referenced in their Nobel citation.) Following this, numerous researchers have explored new strategies to boost immunizations.
For the current research, the team collaborated with the Haryana state government in India to conduct an experiment across more than 900 villages from 2016 to 2019.
The team structured the experiment around three fundamental strategies to encourage parents to immunize their children: financial rewards, text reminders, and information from local “ambassadors,” or well-connected residents. They then formulated various combinations of these elements, sometimes adjusting the quantity of incentives or text messages, and varying exposure to local information.
Ultimately, the researchers generated 75 combinations of these strategies and devised a new method to assess them, named treatment variant aggregation (TVA). Essentially, the scholars created an algorithm that employed a systematic, data-driven methodology to amalgamate identical variations and noted which ones lacked effectiveness. To identify the most effective package, they also adjusted their findings for the so-called “winner’s curse” common in social-science studies, where the policy option that performs best in a specific experiment may do so by chance.
Overall, the researchers believe they have established a technique for evaluating multiple “treatments” — such as financial incentives — within a single experiment, rather than merely testing one approach, like distributing lentils, in each large study.
“It’s not just one experiment where you compare A with B,” explains Banerjee, the Ford Foundation International Professor of Economics. “Here, we assess a blend of elements. Even in cases where no effect is observed, valuable information can be gleaned. It’s possible that one part of a combination is effective while others are counterproductive, yielding a net zero impact, but still providing insight. Thus, it is essential to account for all options as you progress, even though it’s a mathematically intricate task.”
The researchers also discerned that variations among local populations influence the effectiveness of the different strategies tested. Typically, communities with lower immunization rates tend to respond more favorably to incentives for vaccination.
“In a way, we are returning to the conclusions drawn from the lentil study in Rajasthan, where low immunization rates lead to significantly high effects from these incentives,” notes Duflo, who is also the Abdul Latif Jameel Professor of Poverty Alleviation and Development Economics. “We replicated that finding in this context.” However, she emphasizes that the new methodology enables scholars to gather more information about this process more swiftly.
A Practical Path Forward
The research team is optimistic that the new TVA methodology will be widely embraced by scholars and result in more experiments with multifaceted approaches, assessing numerous potential solutions concurrently. This method could extend to research on poverty alleviation, medical trials, and beyond.
Furthermore, they highlight that these insights empower governments and other organizations to anticipate how different policy options might unfold, both in health and financial aspects.
“The primary motivation behind this was to offer the Haryana government a pragmatic route for moving forward,” states Duflo.
She adds: “Previously, it was thought that to assert something with confidence, only one treatment should be tested at a time,” referring to one type of intervention at a time, such as either incentives or text messages. However, Duflo affirms, “I’m pleased to assert that multiple approaches can be utilized simultaneously, and all can be analyzed. Although it involves several steps, such is life: many steps.”
In addition to Duflo and Banerjee, the co-authors of the study include Arun G. Chandrasekhar of J-PAL; Suresh Dalpath of the Government of Haryana; John Floretta of J-PAL; Matthew O. Jackson from Stanford University; Harini Kannan of J-PAL; Francine Loza of J-PAL; Anirudh Sankar from Stanford; Anna Schrimpf of J-PAL; and Maheshwor Shrestha from the World Bank.
This research was facilitated through collaboration with the Haryana Department of Health and Family Welfare.