Berzin, S.C. (2006). Using sibling data to understand the impact of family group decision-making on child welfare outcomes. Children and Youth Services Review, 28(12), 1449-1458.
Les études sur la prise de décision en conseil de famille (FGDM) en matière d’impacts en protection de l’enfance ont souffert de petites tailles d’échantillon et d’un manque de groupes témoins adéquats. Depuis que la FGDM est administrée au niveau familial, l’utilisation de données sur les fratries est potentiellement compensatoire pour les petites tailles d’échantillon et les faibles taux d’inscription. Cette étude se sert de données sur les fratries du California's Title IV-E Waiver Demonstration Project Evaluation afin de comparer les impacts en protection de l’enfance dans deux comtés pour des familles recevant la FGDM (comté de Fresno, n = 110; comté de Riverside, n = 87) avec des familles recevant des services de protection à l’enfance traditionnels (comté de Fresno, n = 74; comté de Riverside, n = 52). Après avoir contrôlé pour les effets de regroupement en utilisant des équations d’estimation générales (EEG), les différences de groupe en mauvais traitements envers les enfants, la stabilité de placement et la permanence ont été modelées en suivant une régression linéaire et logistique. Les résultats des deux comtés n’ont montré aucune preuve de différences de groupe dans les indicateurs en protection de l’enfance; ces résultats neutres soulèvent des questions concernant l’efficacité de la FGDM. Cette étude confirme les données précédentes malgré des tailles d’échantillon plus petites sur l’impact du FGDM, mais Berzin insiste sur le fait qu’elle marque une amélioration méthodologique par rapport aux analyses de dossiers individuels ou familiaux.
This seems to be a genuine effort to implement a rigorous experimental design. Families were randomized to treatment conditions, and sophisticated multivariate statistical procedures were used to deal with “the clustering effects for siblings from the same family group,” and to control for child’s time in the study and past occurrences of maltreatment. Three potentially-important outcomes—child safety, placement stability and permanency-related outcomes—were measured. No improvement was found; in fact the impact of FGDM was actually slightly negative. Berzin suggested that a negative effect on maltreatment rates may have been the result of hyper-vigilance by the social worker or higher rates of reporting by other family members.
There were, however, significant problems with actual program delivery: (1) Practice methods, child age ranges and FGDM models differed somewhat in the two counties—there was no consistent intervention with a defined client group; (2) There is no report of any effort to monitor actual program implementation; (3) As some workers were using both methods with different families, there may have been some methodological leakage between groups. Altogether, this suggests a possible “Type III error,” with the planned intervention not fully or consistently implemented.
Berzin argued that the use of sibling data with FGDM is a methodological improvement, because it increases the sample size and compensates for low enrollment. Using children instead of families as the unit of analysis certainly increases N; but the intervention involved family group decision-making, and the actual number of families was unchanged. The issue here seems to be the change in the nature of the family data (inclusion of sibling information) and the corresponding change in statistical procedures, rather than a simple increase in "sample size."
In summary, this project combined a rigorous experimental design and sophisticated data analysis—this last being the one thing that Berzin could actually control. However, the unsatisfactory program implementation, combined with the absence of any measurable impact, raises serious questions about the application of these findings.