American Indian or Alaska Native
100%
Pfannenstiel, J. (2015). Evaluation of the i3 validation of improving education outcomes for American Indian children. Unpublished manuscript. Overland Park, KS: Research & Training Associates, Inc.
Study design characteristics contributing to rating
| Design | Attrition | Baseline equivalence | Confounding factors? | Valid, reliable measures? |
|---|---|---|---|---|
| Randomized controlled trial | High |
Established on race/ethnicity and SES; outcomes not feasible to assess at baseline |
None |
Not assessed in manuscripts reviewed under Handbook of Procedures and Standards, Version 1 |
Notes from the review of this manuscript
This study reports findings for both a RCT and a quasi-experimental design (QED). In the RCT, outcomes that were not assessable at baseline received a moderate rating. Outcomes that were assessable at baseline received a low rating because the study did not establish baseline equivalence for these outcomes.
In the QED, all outcomes received a low rating because treatment and comparison groups were not enrolled in the study at the same time, creating a confounding factor.
Child Development and School Readiness
| Outcome Measure | Timing of Follow-Up | Rating | Direction of Effect | Effect Size (Absolute Value) | Stastical Significance | Sample Size | Sample Description |
|---|---|---|---|---|---|---|---|
| Boehm Test of Basic Concepts -3 Preschool, Total Score | 3 years | Moderate | 0.34 | Not statistically significant, p = 0.09 | 67 families | RCT: BabyFACE vs. comparision | |
| Boehm Test of Basic Concepts-3 Preschool - Total Score | 3 years | Moderate | Not statistically significant, p=0.050 | 67 families | RCT: BabyFACE vs. comparision | ||
| Devereux Early Childhood Assessment (DECA) - Attachment | 2 years | Moderate | 0.07 | Not statistically significant, p = 0.69 | 66 families | RCT: BabyFACE vs. comparision | |
| Devereux Early Childhood Assessment (DECA) - Attachment | 3 years | Moderate | 0.15 | Not statistically significant, p = 0.26 | 68 families | RCT: BabyFACE vs. comparision | |
| Devereux Early Childhood Assessment (DECA) - Behavioral concerns | 3 years | Moderate | Statistically significant, p = 0.03 | 68 families | RCT: BabyFACE vs. comparision | ||
| Devereux Early Childhood Assessment (DECA) - Behavioral concerns | 2 years | Moderate | 0.19 | Not statistically significant, p = 0.22 | 66 families | RCT: BabyFACE vs. comparision | |
| Devereux Early Childhood Assessment (DECA) - Behavioral concerns | 3 years | Moderate | 0.47 | Not statistically significant, p = 0.56 | 68 families | RCT: BabyFACE vs. comparision | |
| Devereux Early Childhood Assessment (DECA) - Initiative | 2 years | Moderate | 0.27 | Not statistically significant, p = 0.56 | 66 families | RCT: BabyFACE vs. comparision | |
| Devereux Early Childhood Assessment (DECA) - Initiative | 3 years | Moderate | 1.04 | Not statistically significant, p = 0.19 | 68 families | RCT: BabyFACE vs. comparision | |
| Devereux Early Childhood Assessment (DECA) - Self-control | 2 years | Moderate | 0.38 | Not statistically significant, p = 0.82 | 66 families | RCT: BabyFACE vs. comparision | |
| Devereux Early Childhood Assessment (DECA) - Self-control | 3 years | Moderate | 0.10 | Not statistically significant, p = 0.35 | 68 families | RCT: BabyFACE vs. comparision |
Positive Parenting Practices
| Outcome Measure | Timing of Follow-Up | Rating | Direction of Effect | Effect Size (Absolute Value) | Stastical Significance | Sample Size | Sample Description |
|---|---|---|---|---|---|---|---|
| Frequency of home literacy activity | 3 years | Moderate | 0.64 | Statistically significant, p = 0.01 | 67 families | RCT: BabyFACE vs. comparision | |
| Hours parent reads to child | 3 years | Moderate | 0.49 | Statistically significant, p = 0.04 | 67 families | RCT: BabyFACE vs. comparision | |
| Protective Factors Survey - Nurturing and Attachment | 2 years | Moderate | 0.08 | Not statistically significant, p = 0.55 | 66 families | RCT: BabyFACE vs. comparision | |
| Protective Factors Survey - Nurturing and Attachment | 3 years | Moderate | 0.13 | Not statistically significant, p = 0.56 | 68 families | RCT: BabyFACE vs. comparision | |
| Protective Factors Survey - Parent's Child Development Knowledge | 3 years | Moderate | 0.27 | Not statistically significant, p = 0.13 | 68 families | RCT: BabyFACE vs. comparision | |
| Protective Factors Survey - Parent's Child Development Knowledge | 2 years | Moderate | 0.16 | Not statistically significant, p = 0.21 | 66 families | RCT: BabyFACE vs. comparision |
This study included participants from the following locations:
- Arizona
- Idaho
- New Mexico
- North Carolina
- South Dakota
- Washington
Study Participants
The study recruited families from the catchment areas of 20 Bureau of Indian Education schools that had not previously implemented the FACE program (Baby FACE is the home visiting component of FACE). One of the 20 programs withdrew from the grant after two years and is not included in the study. The study included a QED analysis and a smaller RCT.
The QED analysis sample included families from the 19 remaining schools. The QED study included 1,329 participants: 853 in the treatment group and 476 in the comparison group. Within the QED, 49 percent of children in the treatment group and 55 percent of children in the comparison group were male. In the treatment group, 25 percent of mothers had completed less than a 12th grade education and 37 percent had a high school diploma or GED. Three percent had a bachelor’s degree or higher. In the comparison group, 21 percent of mothers had completed less than a 12th grade education and 32 percent had a high school diploma or GED. Seven percent had a bachelor’s degree or higher. A majority of children were from two-parent households. Seventy-three percent of treatment and comparison groups received Supplemental Nutrition Assistance Program benefits.
The RCT sample included families from one of the 19 Bureau of Indian Education school catchment areas that were included in the QED analysis. The site was selected for the RCT because it had more eligible families than space in the program. Within the RCT, 129 families were randomly assigned, with 63 families assigned to the treatment group (Baby FACE) and 66 assigned to the comparison group (business-as-usual).
Setting
The QED study included 20 Bureau of Indian Education school catchment areas located across the following U.S. states: Arizona, Idaho, New Mexico, North Carolina, South Dakota, and Washington. The RCT took place in one of the QED study sites that was oversubscribed.
Intervention condition
Note: Navigate to the model page for more information about the home visiting model. See the source manuscript for more information about how the model was implemented in this study.
Comparison Conditions
The comparison group was eligible to receive existing business-as-usual services that were available to them within their communities.
Subgroups examined
• Maternal education (high education, level undefined) • Household poverty status (below poverty threshold) • Intergenerational household (yes or no)
Funding Sources
Institute of Education Sciences (IES) U.S. Department of Education, Grant #U396B100189.