Green, B. L., Sanders, M. B., & Tarte, J. (2017). Using administrative data to evaluate the effectiveness of the Healthy Families Oregon home visiting program: 2–year impacts on child maltreatment & service utilization. Children and Youth Services Review, 75, 77–86.
Model(s) Reviewed:
Healthy Families America (HFA)®
| Screening decision | Screening conclusion | HomVEE procedures and standards version |
|---|---|---|
| Passes screens | Eligible for review | Version 1 |
| Rating | Design | Attrition | Baseline equivalence | Compromised randomization | Confounding factors | Valid, reliable measure(s) |
|---|---|---|---|---|---|---|
| High | Randomized controlled trial | Low | Established on race and ethnicity; established on SES; outcomes not feasible to assess at baseline | None | None | Not assessed in manuscripts reviewed before 2021 |
Notes:
This study reported outcomes based on analyses of unadjusted means and proportions, as well as outcomes calculated based on regression analyses. Among the outcomes reported based on analyses of unadjusted means and proportions, all outcomes that are not assessable at baseline rate high. Three outcomes from the analyses of unadjusted means and proportions are assessable at baseline. Two of those outcomes, the percentage of mothers who ever received employment services and the number of days mother received employment services, rate moderate because the study does not establish baseline equivalence or control for differences in the baseline outcome. The third outcome, the percentage of mothers receiving substance abuse treatment services, rates low because we could not calculate attrition, and the study does not demonstrate equivalence or control for differences in the baseline outcome. All outcomes from the regression analyses rate low because we could not calculate attrition and the study does not establish baseline equivalence.
| Study participants | Healthy Families Oregon staff who visited hospitals and clinics serving pregnant women screened families for eligibility. Eligible families were English-speaking, first-time parents with a child under 90 days old and identified as high-risk on a standardized screening tool (the New Baby Questionnaire [NBQ]). Parents who scored positive for substance abuse or depression, or a combination of two other parent and child risks (such as lack of comprehensive prenatal care or single-parent status) were determined to be high risk. The NBQ was used to screen 8,520 first-time parents, and 2,727 of these parents were found eligible and randomized to either the home visiting program or a control group (1,438 and 1,289, respectively). Enrollment took place between February 2010 and February 2012. Families were randomized in seven home visiting program sites in Oregon. At randomization, between 57 and 60 percent of families were white, non-Hispanic; 24 to 27 percent were Hispanic/Latino; and 15 to 16 percent were another race. Parents were, on average, 22 years old, and between 31 and 33 percent lacked a high school diploma or equivalent. Both parents were unemployed in between 35 and 37 percent of families. Outcomes were assessed using state administrative data matched to mother and child at two years following study enrollment for an analytic sample of 1,427 families in the intervention group and 1,280 families in the comparison group. |
|---|---|
| Setting | This study took place in seven Healthy Families Oregon sites that were oversubscribed, three of which served primarily rural communities and four of which served mixed urban and rural communities. |
| Intervention services | Healthy Families Oregon is an accredited state home visiting program based on the HFA model. Families enroll prenatally or up to 90 days after birth and receive home visiting services through the child’s third birthday. Families receive weekly individualized home visits for at least six months that typically focus on child development and positive parent-child interactions. Families also receive case management services that identify their needs and link them to relevant services. Intervention uptake was low; of those assigned to the home-visiting group, only 44 percent received at least one home visit.In addition, all families in the study received brochures on parenting and child development, information about community resources, books, videos, and other small gifts. |
| Comparison conditions | Families assigned to the comparison condition were not eligible to receive home visiting services through the Healthy Families Oregon program. However, they did receive brochures on parenting and child development, information about community resources, books, videos, and other small gifts. |
| Subgroups examined |
This field lists subgroups examined in the manuscript (even if they were not replicated in other samples and not reported on the summary page for this model’s report). • Public benefit receipt (mother received Temporary Assistance to Needy Families, Supplemental Nutrition Assistance Program, child care subsidy, or employment services) |
| Funding sources | The research was funded by the U.S. Department of Health and Human Services, Children’s Bureau (grant number 90CA1782). |
| Author affiliation | None of the study authors are developers of the program model. |
| Peer reviewed | Peer reviewed status is not listed for manuscripts reviewed before 2021. |
Study Registration:
Clinicaltrials.gov Identifier: None found. Study registration was assessed by HomVEE beginning with the 2014 review.
Findings that rate moderate or high
| Rating | Outcome measure | Effect | Sample | Timing of follow-up | Sample size | Intervention group | Comparison group | Group difference | Effect size | Statistical significance | Notes |
|---|---|---|---|---|---|---|---|---|---|---|---|
| High | Number of days on TANF, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 mothers | Unadjusted mean = 176.10 | Unadjusted mean = 169.60 | Mean difference = 6.50 | HomVEE calculated = 0.03 | Not statistically significant, p = 0.51 | Statistical significance is based on HomVEE calculations.
|
| High | Number of days on SNAP, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 mothers | Unadjusted mean = 475.10 | Unadjusted mean = 461.20 | Mean difference = 13.90 | HomVEE calculated = 0.05 | Not statistically significant, p = 0.20 | Statistical significance is based on HomVEE calculations.
|
| High | Number of days with child care subsidy, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 mothers | Unadjusted mean = 19.50 | Unadjusted mean = 19.80 | Mean difference = -0.30 | HomVEE calculated = 0.00 | Not statistically significant, p = 0.92 | Statistical significance is based on HomVEE calculations.
|
| High | Percentage ever received TANF, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 mothers | Unadjusted proportion = 0.42 | Unadjusted proportion = 0.41 | Mean difference = 0.01 | HomVEE calculated = 0.02 | Not statistically significant, p = 0.56 | Statistical significance is based on HomVEE calculations.
|
| High | Percentage ever received child care subsidy, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 mothers | Unadjusted proportion = 0.11 | Unadjusted proportion = 0.11 | Mean difference = -0.01 | HomVEE calculated = -0.04 | Not statistically significant, p = 0.33 | Statistical significance is based on HomVEE calculations.
|
| High | Percentage ever received SNAP, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 mothers | Unadjusted proportion = 0.85 | Unadjusted proportion = 0.82 | Mean difference = 0.02 | HomVEE calculated = 0.10 | Statistically significant, p = 0.01 | Statistical significance is based on HomVEE calculations.
|
| Moderate | Number of days with employment services, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 mothers | Unadjusted mean = 98.90 | Unadjusted mean = 94.40 | Mean difference = 4.50 | HomVEE calculated = 0.02 | Not statistically significant, p = 0.53 | Statistical significance is based on HomVEE calculations.
|
| Moderate | Percentage ever received employment services, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 mothers | Unadjusted proportion = 0.31 | Unadjusted proportion = 0.30 | Mean difference = 0.00 | HomVEE calculated = 0.01 | Not statistically significant, p = 0.77 | Statistical significance is based on HomVEE calculations.
|
| Rating | Outcome measure | Effect | Sample | Timing of follow-up | Sample size | Intervention group | Comparison group | Group difference | Effect size | Statistical significance | Notes |
|---|---|---|---|---|---|---|---|---|---|---|---|
| High | Number of days in out-of-home care, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 children | Unadjusted mean = 15.21 | Unadjusted mean = 12.74 | Mean difference = 2.47 | Not available | Not reported | Negative effect is favorable to the intervention. |
| High | Number of substantiated abuse allegations, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 children | Unadjusted mean = 0.01 | Unadjusted mean = 0.01 | Mean difference = 0.00 | Not available | Not reported | Negative effect is favorable to the intervention. |
| High | Number of substantiated maltreatment reports, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 children | Unadjusted mean = 0.07 | Unadjusted mean = 0.07 | Mean difference = 0.00 | Not available | Not reported | Negative effect is favorable to the intervention. |
| High | Number of substantiated neglect allegations, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 children | Unadjusted mean = 0.07 | Unadjusted mean = 0.07 | Mean difference = 0.00 | Not available | Not reported | Negative effect is favorable to the intervention. |
| High | Percentage with at least one out-of-home placement, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 children | Unadjusted proportion = 0.04 | Unadjusted proportion = 0.03 | Mean difference = 0.01 | HomVEE calculated = 0.10 | Statistically significant, p = 0.01 | Statistical significance is based on HomVEE calculations. Negative effect is favorable to the intervention.
|
| High | Percentage with at least one substantiated maltreatment report, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 children | Unadjusted proportion = 0.06 | Unadjusted proportion = 0.06 | Mean difference = 0.00 | HomVEE calculated = 0.03 | Not statistically significant, p = 0.41 | Statistical significance is based on HomVEE calculations. Negative effect is favorable to the intervention.
|
| High | Percentage with at least one substantiated neglect report, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 children | Unadjusted proportion = 0.06 | Unadjusted proportion = 0.06 | Mean difference = 0.00 | HomVEE calculated = 0.03 | Not statistically significant, p = 0.40 | Statistical significance is based on HomVEE calculations. Negative effect is favorable to the intervention.
|
| High | Percentage with at least one substantiated physical or sexual abuse report, intent-to-treat analysis, unadjusted | FavorableUnfavorable or ambiguousNo Effect |
Oregon, 2010-2012 | 2 years | 2,707 children | Unadjusted proportion = 0.01 | Unadjusted proportion = 0.01 | Mean difference = 0.00 | HomVEE calculated = -0.29 | Statistically significant, p = 0.00 | Statistical significance is based on HomVEE calculations. Negative effect is favorable to the intervention.
|