Prioritizing measures to assess performance of drug treatment services: a Delphi process with funders, treatment providers and service-users

Abstract

Background and aim: While many studies have examined outcome measurement as part of clinical trials and routine outcome collection at the person-level in alcohol and other drug (AOD) treatment services, there has been limited attention to measures required to assess performance at the service-level. In Australia, non-government services are primarily funded by government using public funds; however, there is no standardized approach to performance measurement. This study sought to establish a finite list of performance measures that represented consensus between funders, treatment providers and service-users.

Method: A three-round Delphi process was undertaken with (i) funders of treatment (n = 10), (ii) treatment providers (n = 10) and (iii) treatment service-users (n = 10). Participants were asked to rate a range of measures on a 10-point Likert scale on how important they were to be included in contracts with funders. Measures with a median score > 7 and agreement among participants above 70% were the criteria for inclusion in the final set of measures. Qualitative data in the form of text responses provided by participants for their ratings in rounds 1 and 2 were also analysed.

Results: Participants rated 93 measures in round 1, which reduced to 78 measures in round 2 and 32 measures in round 3. Fifteen service-level measures and two system-level measures met criteria for inclusion in the final set of performance measures. The final set of measures cover a range of measurement types: outcomes (n = 5), access (n = 3), structural (n = 3), experience (n = 2), input (n = 2), process (n = 1) and output (n = 1).

Conclusion: In Australia, performance measures for alcohol and other drug treatment services that represent a consensus among service-users, providers and funders focus upon demonstrating accountability for public funds, improving services and communicating key measures of success to future service-users and the broader community.

Keywords: Accountability; Delphi consensus; core measure set; drug treatment; non-government organizations; performance measures.