If your organization is like most, you have to balance meeting customer expectations for delivering high quality products or services with the hard reality of needing to keep costs in line to maintain profitability. Often, organizations will implement “interventions” aimed to improve business, such as to reduce costs, increase revenue, or retain customers. But since the intervention strategy itself has investment costs, it is necessary to measure the ROI of the intervention to understand whether the goals have been met.
There are many types of analyses that can be used to measure the effects of an intervention. The challenges lie in knowing which type of analysis is appropriate given the unique situation, variables, and data available for each intervention and its affected population. Given the complexities of such an analysis, the process of trying out a variety of methods to find the ideal approach can be inefficient and time consuming due to manual, ad hoc prep work and a lack of a roadmap for consistency from previous intervention analyses. Lost time in figuring out the best measurement framework and data preparation methods exacerbates the risk of unnecessary expense or lost revenues related to continuing an ineffective intervention.
What may appear to be a challenging analysis can be simplified with the right structure and guidance. By employing consistent and appropriate modeling and data prep methods based on the nature of the intervention, organizations can enact thoughtful data-driven decision-making around the continuation or expansion of interventions.
Interventions and their importance
Organizations frequently take actions to achieve their short and long term business objectives. These actions, which we generically call interventions, typically deliver a treatment to a targeted set of subjects and attempt to impact one or more outcome metrics. The need for insightful intervention results is crucial in industries such as healthcare, HR, retail and marketing.
- Within marketing, the impact of specific campaigns or promotions on customer purchasing behavior is crucial to future campaigns.
- Within retail, how changes to store formats, pricing, and website design will impact sales.
- In human resources, the impact of training or incentive programs informs ongoing policy and practices.
- For health insurance and service companies, enrollment in clinical programs – whether behavioral health, palliative care, or chronic disease management – are a powerful lever for controlling costs and improving health outcomes.
What is common for all forms of interventions is the need for a measurement framework that accurately informs the next steps, which might include continuation, roll-back, or expansion of the intervention. Additional needs often include a record of the measurement framework for consistent future applications, and a repository that saves all the evaluations for efficient comparisons.
Importantly, due to increased competition and the modern business environment, companies must accelerate the rate of experimentation with new interventions, putting increased pressure on their ability to evaluate their impact. A key challenge for businesses then is executing and evaluating interventions at scale.
Organizations with frequent intervention evaluation needs often will create a formally defined intervention evaluation process or platform to govern long-term large-scale projects as well as nimble one-offs. The process or platform should include education for the staff on how to evaluate interventions, including guidance on which types of models are appropriate given various scenarios and data availability. The process or platform should also include templates for data preparation and reporting, guidelines for time frames of tests relative to the interventions, and a repository of tests already conducted along with their assumptions, code, and resulting outcomes for benchmark comparisons. Such a process or platform will usually start small, and will grow in breadth and depth as the number of actual intervention analyses grows.
Some organizational considerations for companies centralizing its intervention evaluations include determining ownership over the organization of the process, files, and data, especially as it grows; access and storage location and controls; methodological reviews and oversight; and maintenance of a code repository. Without controls over each of these elements, the platform will be incomplete at best, or will perpetuate inaccurate analysis at worst. If not mandated to be consistently used and updated, it will likely be considered unhelpful and quickly forgotten.
Simplifying the measurement of intervention impacts
We know the timeless truth that measuring the impact of interventions on customer behavior is crucial to making informed decisions. Which is one reason why clients turn to Fulcrum for assistance on small and large scale evaluation needs. Intervention evaluations can be measured in a one-off fashion using our Matched Control Group methodology for smaller companies or smaller projects. Organizations with a high frequency of interventions find our Fulcrum Impact solution to be the ideal tool for efficiently extracting insight and continuously monitoring intervention effectiveness. This solution allows organizations to “DIY” the analysis of interventions with the assistance of pre-built workflows to streamline the process and allow for greater analyst efficiency.
Fulcrum Impact contains a database full of customizable frameworks and reports, reducing the labor needed to prepare data, design models, and build reports. It allows the user to plug in data, benchmark metrics, build perimeters, and analyze the outcomes all through a simple UI. And, to facilitate evaluations at scale, it enforces consistency and provides a repository of past interventions and evaluation results, providing a key knowledge management tool for the organization.
While a less granular analysis may only show a sweeping overview of the statistics for an intervention, our semi-automated models illustrate the answers as to why an intervention “worked” or not, providing input to the optimization of the intervention or the design of future interventions. The benefits to users of Fulcrum Insight include model standardization, data preparation shortcuts, code simplification, data prep automation, and centralized storage as highlighted below:
The steps ahead
Interventions are necessary for organizations to innovate. However, they have costs, both in time and money, and the measurement of intervention ROI can be a difficult task. Whether you build your own evaluation platform or use something like Fulcrum Impact, centralization and organization of evaluation processes makes life easier for analysts responsible for measuring intervention effectiveness. To find out how you can leverage the Fulcrum Impact to speed up and bolster the measurement of your organization’s interventions, click here to learn more.
