Impact Science |
The Issue
An extensive literature has established the effectiveness of various behavioral interventions for a range of conditions but this literature often fails to isolate the intervention components that are more or less effective... Behavioral intervention research cannot become a cumulative science that builds upon prior research until intervention studies can answer not only if the intervention changed behavior, but also how it changed behavior, and which intervention components were most effective in changing behavior.
William Riley, Director of the Office of Behavioral and Social Science Research at NIH, 2014
Despite 5.7 million published studies on 'behavior change', and $73 trillion annually on global social spending, we have only made a 3% improvement towards the Sustainable Development Goals since 2014.
Enter Impact Science
Impact Science uses structured data, core components, and probabilistic models to predict the effectiveness of social interventions, and eventually design more effective interventions. This is similar to how econometrics uses statistics and modeling to forecast, which in turn informs policies intended to produce better economic outcomes.
Using data in this way, social impact practitioners can radically improve program design, make better-informed resource allocation decisions, and ultimately, improve the lives of more people without spending more.
Impact Science solves challenges around social spending and social programs, ultimately achieving better outcomes overall.
Using data in this way, social impact practitioners can radically improve program design, make better-informed resource allocation decisions, and ultimately, improve the lives of more people without spending more.
Impact Science solves challenges around social spending and social programs, ultimately achieving better outcomes overall.
The Building Blocks of Impact Science
1. Data Standardization
The lack of standardized data is fundamentally what’s holding our field back. There is no periodic table of elements for social change, no common genes or chromosomes to sequence the DNA of social interventions, and no universal terminology for elements of social programs. Without these, social impact research remains little more than a compilation of well-documented PDFs. Defining the data elements of the social sector in a standardized way is an essential, required step in Impact Science because it allows for the creation of large datasets containing evidence related to program components, outcomes, and the beneficiaries, context, and implementation of programs.
Our research initiative, the Impact Genome Project, is another example of data standardization. Over the past seven years, we have created standardized taxonomies for 132 common social outcomes, the strategies used to achieve them, and the characteristics of beneficiaries and program context. Our taxonomies cover a wide range of areas such as public health, education, workforce development, financial health, social capital, housing, and food security. This standardization allows for apples-to-apples comparisons across programs and studies. This enables aggregation, synthesis, prediction, matching, and benchmarking of data across tens of thousands of research studies, evaluations, and grant reports.
Our research initiative, the Impact Genome Project, is another example of data standardization. Over the past seven years, we have created standardized taxonomies for 132 common social outcomes, the strategies used to achieve them, and the characteristics of beneficiaries and program context. Our taxonomies cover a wide range of areas such as public health, education, workforce development, financial health, social capital, housing, and food security. This standardization allows for apples-to-apples comparisons across programs and studies. This enables aggregation, synthesis, prediction, matching, and benchmarking of data across tens of thousands of research studies, evaluations, and grant reports.
2. Aggregation & Synthesis Methodologies
Once standards exist, they can be consistently applied to evidence and other unstructured data. In the social sciences, this is called coding—essentially, applying standards to a (typically) qualitative source and extracting the meaning it holds in a structured way.
In Impact Science, this process can turn any unstructured data—such as program evaluations, PDFs of randomized control trial studies, or grant reports—into coded, structured datasets that can be used to analyze outcomes, core components, beneficiary characteristics, and contextual factors.
Other sectors have developed data refineries to do just that: make data more valuable and useful. To borrow an analogy from Scientific American, it’s similar to the idea of oil refinement, in that crude oil is not very usable but refined oil is highly usable for multiple applications. See also “Does the Social Sector Need An Impact Registry” published in SSIR.
In Impact Science, this process can turn any unstructured data—such as program evaluations, PDFs of randomized control trial studies, or grant reports—into coded, structured datasets that can be used to analyze outcomes, core components, beneficiary characteristics, and contextual factors.
Other sectors have developed data refineries to do just that: make data more valuable and useful. To borrow an analogy from Scientific American, it’s similar to the idea of oil refinement, in that crude oil is not very usable but refined oil is highly usable for multiple applications. See also “Does the Social Sector Need An Impact Registry” published in SSIR.
3. Prediction, Matching, & Benchmarking
The final frontier of Impact Science is not just structured data—it’s how you use it. Structured data super-charges analytics such as meta-analysis, predictive analytics, benchmarking, and matching. Imagine the possibilities of harnessing the predictive power of data for good, using methods successfully deployed in other sectors like marketing, economics, and even meteorology.
There are a number of organizations—we identified nearly 30 of them—advancing these methodologies.
There are a number of organizations—we identified nearly 30 of them—advancing these methodologies.
- GovLab is looking at innovative ways of encouraging and using open government data for decision-making in developing economies and criminal justice, among other areas.
- Meta-analysts such as Mark Lipsey and Larry Hedges are pushing forward new methodologies to understand linkages between program strategies and outcomes.
- Predictive analytics and recommender engines are becoming more widely used to improve social supports and outcomes, such as in child welfare, college graduation, and healthcare.
- We’ve also been working on this at Impact Genome Project. In 2020, we published the first taxonomic meta-analysis of childhood obesity prevention interventions in collaboration with researchers from NIH, CDC, and several leading universities, which identified specific intervention components—not full interventions—correlated with positive outcomes.