What Do We Do With What Clients Tell Us?

Every year, California asks tens of thousands of people in behavioral healthcare how their treatment is going. The Consumer Perception Survey collects their answers, organizes them by demographics and domains, and reports them out months later as a set of aggregate scores. By the time a provider sees the results — if they see them at all — the clients who gave responses may have moved on, and the results are so high-level that it isn’t clear what the provider should do with that information. Nothing about a behavioral health provider’s operations, and certainly not the way it gets paid, is tied to client satisfaction or experience. The state runs a comprehensive and statistically rigorous survey to hear from people, but it is disconnected from anything a local government or service provider can use to act on what they said.

This challenge isn’t specific to California, or to behavioral health. Most social services, delivered either directly by governments or by community nonprofits they contract with, are paid for based on either the cost of delivering the work (with no connection to output or results) or the volume of work delivered (measuring quantity, but not quality). At Social Finance, we have worked for 15 years on helping jurisdictions shift towards outcomes-based contracting approaches, where at least some payment is tied to outcomes that matter to the governments funding and running the programs.

Over the years, we have become more focused on the connection between the experience of the people in a program and the objective results policymakers and agencies typically want to measure. Just as we encourage partners to regularly review operational and outcomes data to adapt and improve program delivery, we think client experience — measured in ways that matter to clients themselves — should be a central, regular, and incentivized part of delivering services. We could do this by creating community and context-specific surveys, delivering them to current clients or service users, and then tying the results of those surveys to payments to service providers.

Client experience, in practice, usually means feedback. The idea of collecting feedback is not novel.¹ Many human services already collect client feedback, some systematically. In addition to the Consumer Perception Survey, California reserves seats for consumers and family members on its county behavioral health boards and its state planning council, and requires community input across its planning process. While these practices are valuable, they tend to have an aggregating quality that makes feedback on the system too far removed from the day-to-day work of the organizations and the front-line staff (case managers, clinicians, eligibility workers) who -work directly with clients. There is no direct feedback loop.

For several years, in Social Finance’s work to help Veterans get jobs through the evidence-based supportive employment program Individual Placement and Support, we included a payment-linked Job Satisfaction metric. Alongside more typical measures, like earnings and days worked, this measure used a validated survey² to check whether veterans were satisfied with their jobs, based on factors like pay, relationships with colleagues, and how they feel on the job. The survey has been shown to connect with longer job tenure; but in the context of the project, it created a clear incentive for all parties, from case managers to the state agencies overseeing the program, to deliver a service that was focused on the individual outcomes of participants. It recognizes that doing this is also in service of program- and policy-level outcome goals. After three years, three out of four sites earned the maximum payment for Job Satisfaction.

There are — of course — risks and unintended consequences to consider. If providers are paid partly based on client experience, two risks need managing: survey administration that skews towards the views of happy clients, and rewarding providers who might serve people with less complex needs who are easier to satisfy. Careful design can guard against both. For example, we would set floors for response rates to be eligible for payment, and we could tie payment to improvement over baseline, rather than absolute scores. Operational challenges and data security can create hurdles: The Veterans CARE project has continued in Massachusetts through an outcomes-based contract. Challenges with survey administration, data security, and individual privacy have complicated the continued use of a job satisfaction or program experience survey. This experience underscores the need for the right partners and clear principles about how data will be used before building one of these payment pools.

Client experience gets overlooked because it is subjective and it can be difficult to measure. Providers have hard jobs already, and they face many demands. But whether a client is satisfied with their experience can influence other outcomes, and in some services, it is a worthy goal in and of itself. We should expect providers to manage for it, just as they do other policy goals and compliance requirements. And we should make that easier to do, both by building the survey infrastructure for regular feedback and by tying payment to the results. Whether through public comment, surveys, focus groups, or long-forgotten case notes, the people in these systems have been telling us about their experiences all along. It’s time to build systems that can hear them fast enough to do something about it.

To learn more, contact Annie Dear, Vice President, Public Sector Practice →