Aarons, G.A., Sommerfeld, D.H., Hecht, D.B., Silovsky, J.F., & Chaffin, M.J. (2009). The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: Evidence for a protective effect. Journal of Consulting and Clinical Psychology, 77(2), 270-280.
The introduction of empirically supported interventions (sometimes referred to as evidence-based practices or EBPs) in an agency can lead to an increase in structure and a decrease in job autonomy. The disruption of normal working conditions may result in increased staff turnover, which would increase agency costs and limit optimal functioning. This present study was an analysis of the impact on staff retention of an EBP program in the Oklahoma Children’s Services System — part of a state-wide trial, involving 21 teams followed over 29 months.
The program was SafeCare, a home-based intervention designed to reduce child neglect and improve parent-child interactions. Classic behavioral intervention techniques were used, with three components: 1) training parents to use appropriate health care procedures, 2) home safety and cleanliness, and 3) parent-child bonding. Subjects were the service providers (N = 153); a majority were female, Caucasian, and had at least some graduate-level education (social work 40%). Program fidelity was monitored by independent professionals, trained in the relevant service model, who observed sessions, and provided feedback and additional training as needed.
The evaluation involved a 2 x 2 experimental design; SafeCare vs. services as usual (SAU) crossed with fidelity monitoring vs. no monitoring. Data collection (at four points, in web-based surveys) assessed demographics, job tenure, job autonomy, work attitudes, turnover intentions and actual turnover. The only control variable found to be significant was age — older employees had a reduced risk of turnover.
A decrease in job autonomy, expected with introduction of an EBP, was not found.
Contrary to expectations, turnover rates were lowest for the EBP with monitoring — 14.9% per 100 person-years, compared to 33.4% for the EBP without monitoring, 41.5% for SAU with monitoring and 37.6% for SAU without. The authors argued that: 1) an organizational change like this can be successful to the extent that it matches the values of the organization, and 2) monitoring here was a non-punitive supportive coaching model, which may have made it more acceptable than a simple performance checklist.
This study appears to have been carefully planned and conducted. To balance population characteristics, administrative regions were systematically assigned to program conditions (EBPs vs. SAU); teams within each program were then randomly assigned to monitoring conditions. Survival function estimates — retention probability by months in job — were calculated for each condition; in a log-rank test, the survival function for the EBP plus monitoring was significantly better than the combined curve for the other three conditions.
Fidelity monitoring was found to have a positive effect on the impact of the EBP program, but not for SAU. It makes sense that workers could benefit from the monitoring of a new program — especially with a model that incorporated support and coaching. If the effect on turnover had been assessed without monitoring, the conclusion would have been that the EBP had no effect; this is a strong argument for including fidelity monitoring whenever a new program is introduced.