In its simplest form, evaluations help determine what works, what could be improved in a program or process, and what the program or process outcomes are. For complex and adaptive initiatives, like local wellness funds, where there is not a clear yes or no if the fund worked or in situations where desired systems change may take decades to fully realize its objectives, evaluations can yield valuable information to: - Measure interim progress and impact
- Clarify context-specific barriers or drivers of change
- Inform any process improvement
- Contribute to the national knowledge base around local wellness funds
In its simplest form, evaluations help determine what works, what could be improved in a program or process, and what the program or process outcomes are. For complex and adaptive initiatives, like local wellness funds, where there is not a clear yes or no if the fund worked or in situations where desired systems change may take decades to fully realize its objectives, evaluations can yield valuable information to:
- Measure interim progress and impact
- Clarify context-specific barriers or drivers of change
- Inform any process improvement
- Contribute to the national knowledge base around local wellness funds
Assessing Processes and Outcomes
There are two key components to evaluating a local wellness fund:
1) Assessing the structure and process of the local wellness fund (a process evaluation) and
2) Measuring impact of the local wellness fund (an outcomes evaluation).
Assessing the local wellness fund infrastructure can start very early in the development process and can include readiness, partnerships, and capacity evaluations. These evaluations could include some quantitative findings from assessment scales, as well as qualitative information gathered from interviews with key stakeholders.
Assessing the impact of the local wellness fund should include both the impact of the local wellness fund’s investment portfolio as a whole, as well as the impact of individual investments or grants. Start with the end in mind. Design the evaluation before resources are distributed and identify the data and measures that will need to be collected. What measures of population health, well-being, or social determinants is the local wellness fund trying to impact? How are they changing over time? What role does the particular investment play in influencing change? What are the intermediate outcomes that are important markers for longer term impact?
Financial return on investment (ROI) and social return on investment (SROI) can be valuable measures of a local wellness fund’s success. Financial ROI can be used to evaluate an investment’s financial impact and efficiency. SROI measures non-financial value and can be used to evaluate a wellness fund’s impact on stakeholders and communities. While financial ROI and SROI can help measure a local wellness fund’s efficacy, they are not always appropriate and can be challenging to estimate. Barriers to determining financial ROI include:
- Philosophical objections to monetizing public health
- Lack of resources and expertise to reliably calculate ROI
- Difficulty establishing a causal link between the local wellness fund’s activities and any changes in the community given that many social drivers influence health outcomes
- The so-called “wrong pocket” problem where the entity that bears the cost of implementing a practice does not receive a commensurate benefit
Similar challenges exist for conducting a SROI analysis. Often SROI is context and stakeholder specific so defining, measuring, and interpreting SROI can be difficult.
Despite the challenges, when financial and social returns are appropriate and can be accurately measured, they can help local wellness funds demonstrate their impact and their value. Demonstrated impact and value are examples of how data can help “make the case” for local wellness funds and can attract greater investment.
Telling the Story With Data
Taken together, the qualitative and quantitative data from both the process and outcomes evaluations will tell the story of the local wellness fund: how, why, and impact. This story is central to making the case for the local wellness fund, expanding partnerships and resources, and ensuring sustainability of the local wellness fund. Additionally, communicating the story of the local wellness fund, generated through evaluation, will build trust, transparency, and accountability with the community and with funders ensuring prioritized uses are reflective of community wants and needs.
California Accountable Communities for Health Initiative
The California Accountable Communities for Health Initiative (CACHI) is using the Accountable Communities for Health (ACH) model to address population health and health equity through multisector, community-based, public-private partnerships to advance a shared vision of healthier communities. CACHI’s ACHs share many of the principles of local wellness funds, including collective, multisector action; community engagement; and a focus on upstream prevention. CACHI requires ACHs to develop wellness funds as a means to sustain ACH efforts for the long-term. With 13 active ACHs in California, many of CACHI’s learnings are of relevance to local wellness funds.
The CACHI Core Element Assessment Tool is designed to help local collaboratives develop a shared understanding of their strengths, capacities, and gaps across multiple dimensions, which we define as readiness to launch an ACH. Local wellness funds may find it helpful to assess their readiness around some or all of the seven core elements of ACHS, including shared vision and goals, partnerships, leadership, backbone, data analytics and capacity, wellness fund development, and portfolio of interventions.
CACHI: Data-Related Milestones highlights what data and measurements an ACH will need to understand progress towards desired outcomes, the effectiveness of its selected interventions, and the overall impact of the ACH’s infrastructure and presence in the community. This guidance emphasizes connections between outcomes, interventions, and infrastructure, as well as opportunities for alignment around shared measurement, coordinated messages, and incorporating an equity lens. Local wellness funds can consider incorporating these measures into their evaluation plan.
The Guide to Evaluating Collective Impact
Developed by FSG and the Aspen Institute’s Collective Impact Forum, this is a practical handbook that presents a framework for approaching evaluation and includes sample questions, outcomes, and indicators. Evaluation of collective impact efforts require flexibility to accommodate changes in maturity and the need for continual learning and adaptation. Given that shared measurement is a core element of collective impact, local wellness funds may benefit from this webinar and handbook in designing their evaluations.
Return on Investment Calculator for Partnerships to Address the Social Determinants of Health
This calculator, by the Commonwealth Foundation, is designed to help community-based organizations and their health system partners plan sustainable financial arrangements to fund the delivery of social services to high-need, high-cost patients. Local wellness funds may find this a useful tool to explore as they begin to explore how to assess return on investment in their local context. This calculator was created in conjunction with the report Investing in Social Services as a Core Strategy for Healthcare Organizations: Developing the Business Case.
Social Return on Investment: A Comprehensive Guide
Sopact compiled this comprehensive guide that includes definitions, principles of social return on investment methodology, guidance on when to calculate social return on investment, and resources and tools to calculate. Local wellness funds will find this site to be a great portal for resources to explore that may guide their understanding and planning for such analysis.
