3 Key Lessons for Practitioners Using Data to Improve Social Outcomes

In California, about 50% of individuals released from custody are convicted of a subsequent crime within three years of release—a statistic that has remained stubbornly high since the early 2000s. Organizations like Interface Children & Family Services (Interface), often in partnership with local government agencies like the Ventura County Probation Agency (VCPA), are working to reduce recidivism by providing individualized services to formerly incarcerated individuals, promoting family stability, and improving public safety. Social Finance partnered with Interface and Ventura County to establish Ventura County Project to Support Reentry, a program that has already enrolled over 340 individuals into reentry services at Interface. These service recipients are part of a study conducted by UCLA to understand the effectiveness of Interface reentry services compared to probation as usual. The project uses a Pay for Success mechanism through which impact investors provide upfront capital for services and are repaid by Ventura County to the extent that measurable reductions in recidivism are achieved.

Detailed and purposeful data infrastructure and analysis procedures are essential to measure and analyze the results of this project. They have allowed UCLA, the project’s independent evaluator, to set up a rigorous evaluation; VCPA to assess and determine eligibility for clients; and Interface to enroll individuals into the project. Conversations with the project partners from Interface and VCPA reveal the significance of this data infrastructure, and the associated procedures and general orientation toward the importance of data, in making this project a success beyond the results of the evaluation. We learned three important lessons on using data to improve social outcomes through this project, which we hope will help other practitioners.

1. Robust data can improve projects by helping you understand what’s working—and what can be improved

Interface’s internal data system, TIER, tracks a wide variety of metrics that represent clients’ progression through case management and clinical services. Dr. Courtney Stallings, Director of Justice Services at Interface, and Yesenia Munoz, Interface’s data coordinator, set up systems that would identify what was going well as well as areas for deeper focus. They understood that the population served by the Pay for Success project would be new and different from those that Interface served in the past, and they were aware that what had worked in the past might not necessarily resonate. In discussing Interface’s thorough data system, Dr. Stallings explained that “positive results don’t help anything but our ego, but results which point out shortcomings can really allow us to identify how to enhance our programs.”

VCPA has similarly set up a rigorous data infrastructure, which Miguel Guerrero, Ventura County’s IT Manager, and Edith Hernandez, Office Systems Coordinator, helped bring to fruition. Even before clients enter the program, they have already been placed through a detailed assessment and randomization process at VCPA. This assessment system tracks various metrics throughout the pipeline, including how many clients are eligible, how many “opt in” to receiving services, how many are randomized, and how many are ultimately referred to Interface.

At both organizations, this in-depth process has allowed for real-time decision-making. For example, giving visibility to the probation units on how they were performing against targets at every stage of the pipeline helped probation officers understand how they were performing against targets at every step, rather than only a broader number of individuals assessed or the ultimate number of individuals enrolled (see figure below). By doing so, VCPA was able to achieve a more targeted response and ultimately increased the number of randomization.

Service delivery data has allowed project partners to understand how clients were progressing through different types of treatment, and what was resonating with the population. For example, midway through the project, Interface was able to see that some services were less well-attended than expected and that the population they were serving was more in need of substance misuse treatments than prior populations that Interface had served. In response, the project began offering “Staying Quit” classes, which catered to a population with higher substance misuse. Finally, this data infrastructure has been useful for staff performance metrics and professional development. Interface cites being able to use client touchpoint data to provide targeted feedback on how case managers can use their time more efficiently.

2. Get people on board by making data entry processes simple

Having good data to analyze must start with getting good data from those on the front lines. As Dr. Stallings told us, “You want your case managers to be able to deliver the best and highest quality services. We can enable them to spend more time focusing on service delivery if we make their lives as easy as possible when it comes to data entry.” Both Interface and VCPA cited having a “can do” attitude in standing up these processes. They purposefully thought about what data points they were asking for and how it was being entered. For VCPA, this included careful thinking about reducing duplication—VCPA’s system is integrated across five agencies, so some basic information was integrated on the “home screen” to address this. They were also able to seamlessly integrate data with external parties; for example, they found a way to integrate UCLA’s “randomizer” directly into the VCPA system. Finally, with help from Social Finance, they conducted in-depth training with the probation officers to ensure they felt confident using the system.

Interface also tried to make data entry as painless as possible, especially since the level of service progression data they were asking for was unprecedented at the time. Dr. Stallings emphasized that when they had a question about if they could track something in a certain way, they didn’t “build from the current system, but rather wrote out what we wanted, and then asked: ‘Is this possible? If not, how can we get closer?’” This mindset has been crucial in getting the buy-in of the case managers and allowed them to build the system in a way that puts the program at the center.

3. Don’t underestimate the importance of human relationships in ensuring robust data

Finally, both partners emphasized that good data must be built on a foundation of good relationships. The level of data sharing is a unique feature of the Ventura project: Interface and VPCA exchange data weekly to understand the enrollment and discharge status of each client, each partner in the project reports on key monthly metrics up to the project’s Management Committee, and VCPA communicates data on arrests to the project’s independent evaluator each quarter. Social Finance consolidates and analyzes these metrics for the Management and Executive Committees.

Relationships matter, because no matter how seamless the processes are, there are always bumps in the road. The project partners that work with the data at each organization have a close relationship; if things don’t reconcile, they can quickly work through them. Guerrero emphasized, “It is just like customer service—it works both ways. If I provide good service to this person, I will get the same thing in return.” Since data procedure updates were required midway through the project, good relationships allowed the new process to be understood and set up quickly.

Finally, good relationships at the governance committee level matter; when larger issues arose, project partners were able to use data-driven insights to work through them. For example, when randomizations and enrollments were lower than expected, the Management Committee created “Action Plans” to keep partners accountable. While this was a data-driven process, it also relied on a strong relationship and an understanding that all program partners had the shared goal of providing services to as many individuals who needed them.

In the coming year, the evaluator at UCLA will complete the study, which will help project partners understand the impacts of Interface reentry services in Ventura. But the lessons learned on data infrastructure from this project can be acted on immediately and are not unique to the reentry space. By setting up robust data systems, getting the buy-in of those who enter the data, and emphasizing the importance of human relationships, outcomes-based social programs can create an environment for data to unlock far-reaching successes.

Learn more about the Ventura Project to Support Reentry →

Photos: Interface Children & Family Services