Frequently Asked Questions
This page compiles responses to frequently asked questions. For more detailed information, please consult the HomVEE Handbook of Procedures and Evidence Standards. If you do not see the information you are looking for, please feel free to email the HomVEE team at: HomVEE@acf.hhs.gov
Search By
Review Process
How does a model get on the HomVEE list?
The HomVEE “list” refers to models HomVEE has reviewed and found to meet the HHS criteria for evidence-based models. HomVEE’s annual review typically includes review of both models that are not yet evidence based (Track 1) and updates to the research on models that have previously been designated as evidence based (Track 2).
HomVEE uses a systematic process to select models for review. Track 1 models are selected using a prioritization process. HomVEE conducts a thorough literature review and scores models based on a combination of manuscript and model characteristics. Models are ranked by points and the highest-ranking models are selected for review as resources allow. (For more information, see response to the next question “How are models prioritized and reviewed by HomVEE?”) HomVEE reviews manuscripts according to our published procedures and standards. When a model is reviewed, findings from well-designed research are reported in detail in each Model Effectiveness Research Report and used to assess whether a model meets the HHS criteria. Track 2 models are reviewed on a schedule rather than a prioritization process. Regardless of whether the model meets the HHS criteria, information about each reviewed model can be accessed on the HomVEE website via the Model Search Page. A list of models HomVEE has reviewed, and whether they meet the HHS criteria, is also available in the summary brief, available on the HomVEE Resources page.
How are models prioritized and reviewed by HomVEE?
For models that are not yet evidence based (Track 1), HomVEE will select models to review by calculating a prioritization score using a combination of model and manuscript characteristics. Prioritization scoring assigns points based on various factors, including (1) number and design of eligible impact studies, (2) sample sizes and sample characteristics of the impact studies, and (3) characteristics of the model. After a model is selected for review, HomVEE will apply its latest standards to review all research that is unreviewed or was reviewed under prior standards. See the Handbook of Procedures and Evidence Standards located on the Resources page of the website for the most up to date information on these processes.
Models that are already evidence based (Track 2) are updated based on a predetermined schedule instead of using a prioritization process. Previously reviewed research about these models is not re-reviewed for these updates. The purpose of the Track 2 review is to update existing Model Effectiveness Research Reports and Model Implementation Profiles to keep them current; HomVEE does not reassess a model’s evidence rating based on a Track 2 review.
Models
How does HomVEE define an early childhood home visiting model?
For the purposes of the HomVEE review, an early childhood home visiting model is an intervention in which trained home visitors meet with expectant parents or families with young children to deliver a specified set of services through a specified set of interactions. These programs are voluntary interventions that are either designed or adapted and tested for delivery in the home. During the visits, home visitors aim to build strong, positive relationships with families to improve child and family outcomes. Services may be delivered on a schedule that is defined or can be tailored to meet family needs. A model has a set of fidelity standards that describe how the model is to be implemented.
Models reviewed by HomVEE must serve pregnant women or families with children from birth to kindergarten entry (that is, up until their sixth birthday), and the primary service delivery strategy must be home visiting. In addition, the model must have research that examines its effects in at least one of eight outcome domains.
Does HomVEE review research on virtual service delivery?
Yes. HomVEE will review research about models that use entirely in-person home visiting and models with hybrid approaches that use both in-person and virtual home visits. Models that deliver all services virtually are ineligible; a model must be designed or adapted to require at least one in-person (that is, not virtual) home visit. HomVEE defines a virtual home visit as “delivery of an intervention’s home visit content to an individual caregiver or family conducted solely by use of electronic information and telecommunications technologies. The content should be designed or adapted for synchronous delivery. Some content may be delivered asynchronously, but asynchronous delivery cannot be the primary mode of delivery.”
HomVEE applies the same standards to research about virtual home visits as to research about in-person home visits. An evidence-based model may incorporate hybrid service delivery; doing so does not affect the model’s existing evidence rating. Previously reviewed research about evidence-based models is not re-reviewed to assess use of virtual service delivery.
Are maternal/infant health home visiting models, not just early childhood ones, included in the review?
Yes, any model that meets the definition of the early childhood home visiting model is included in the review, and this may include models that enroll people during pregnancy and aim to improve maternal health, infant health, and birth outcomes. For models with well-designed research that meets HomVEE’s published standards, detailed information about the theoretical approach, intended population, and targeted outcomes can be found via the implementation link on the Model Search Page on HomVEE’s website. That page also has a search function that allows the user to filter all models by population served. Detailed information about maternal and child health outcomes can be found in the “Effects shown in research” tab of each model effectiveness research report.
Methods and Standards
Which study designs are eligible for review?
MIECHV’s authorizing statute (Social Security Act, Section 511(d)(3)(A)(i)(1) [42 U.S.C. 711(d)(3)(A)(i)(1)]) requires that models implemented through MIECHV be evaluated using well-designed and rigorous randomized controlled research designs or quasi-experimental designs (QEDs). Therefore, HomVEE only reviews studies using these types of designs. In addition to randomized controlled trials (RCTs), HomVEE reviews three types of QEDs: (1) single-case designs (SCDs), (2) regression discontinuity designs (RDDs), and (3) non-experimental comparison group designs (NEDs). These are the only study designs that are eligible for review by HomVEE.
Why can research with some designs rate higher than research with other designs?
Study designs vary in their relative risk of bias—that is, they vary in the extent to which researchers can be confident that impacts are due to home visiting rather than other factors. HomVEE accounts for risk of bias when assigning ratings to research. Generally, HomVEE assigns manuscripts about impact studies a rating of high, moderate, or low. Research with a higher risk of bias receives a lower rating. HomVEE routinely contacts authors if additional details are needed to assess the appropriate rating to assign to the manuscript. The fourth rating HomVEE can assign is “indeterminate.” An indeterminate rating means that HomVEE reviewers lacked sufficient information on one or more features of the study design to rate the manuscript as high or moderate. A rating of indeterminate is not a statement about the quality of the research or the research design. For more details, please see Chapter 3 of the Handbook of Procedures and Evidence Standards located on the Resources page.
What types of analyses or contrasts are not eligible for review?
HomVEE focuses its resources on reviewing manuscripts about impact studies that answer the review’s core question of whether an early childhood home visiting model is effective. Research that answers other questions is not eligible for review by HomVEE. For example, questions about for whom and under what circumstances a model is effective are valuable to the field but beyond the scope of the HomVEE review.
Research evaluating the impact of an isolated model feature or group of features is generally ineligible for HomVEE’s annual review, as are certain types of findings based on (1) covariate-unadjusted analyses, (2) item-level measures, and (3) binary variables. Additionally, analyses designed to answer questions other than whether a model is effective are generally ineligible for review. A longer discussion of contrasts that HomVEE reviews and ineligible and preferred analyses can be found in Chapter 3 of the Handbook of Procedures and Evidence Standards.
Does HomVEE provide guidelines authors can refer to when writing up their research findings?
HomVEE’s Reporting Guide for Authors provides guidance about how to describe each eligible research design and how to report findings in a way that is clear, complete, and transparent. Reporting the information described in the Reporting Guide for Authors is considered a best practice in general, but it can also help HomVEE reviewers determine the appropriate rating to assign to the manuscript.
Indigenous Peoples and Communities
Are models on HomVEE’s website automatically eligible for Tribal MIECHV funding?
Tribal MIECHV grant recipients may select home visiting models that are either evidence-based among Tribal populations (as determined by HomVEE) or that qualify as a promising approach. Currently, models on HomVEE’s website qualify as either evidence-based or a promising approach and are therefore eligible for Tribal MIECHV funding. Model selection is designed to be driven by the needs of a particular community, as determined by a community needs and readiness assessment. For additional information about the Tribal MIECHV Program, see the ACF website.
MIECHV
Are models on HomVEE's website automatically eligible for MIECHV funding?
A HomVEE designation as an evidence-based model does not guarantee that a model is eligible to be implemented with MIECHV funding. To be implemented using MIECHV funds, a model must meet statutory requirements for a model (as determined by HRSA), including the HHS criteria for evidence of effectiveness (as determined by HomVEE). The list of MIECHV eligible models is provided in the model search. In addition, MIECHV’s authorizing statute allows awardees to utilize a portion of their MIECHV funding for a model that qualifies as a promising approach. For additional information on the MIECHV Program, see the HRSA website.
Study Sample
When does HomVEE report subgroup findings?
HomVEE lists the subgroups examined in each high- or moderate-rated manuscript reported on the HomVEE website. However, HomVEE only reports detailed subgroup findings when the subgroup is replicated, meaning that HomVEE has reviewed at least two well-designed studies that analyzed research about a sample with the subgroup characteristic. The Handbook of Procedures and Evidence Standards provides additional information about HomVEE procedures for subgroup research.
HomVEE’s website provides several resources for users that wish to know more about the characteristics of the research samples in a home visiting model’s evidence base. For more information, see FAQ “What information is available on the study samples in HomVEE's research database?”
Evidence-Based Criteria for Home Visiting Models
What are evidence-based models?
HomVEE uses the term “evidence-based model” to refer specifically to a model that meets HHS criteria developed based on statutory requirements in the authorizing legislation for the MIECHV Program. HomVEE recognizes that other systematic reviews may use different criteria to evaluate evidence of effectiveness. Thus, an evidence-based model in the context of HomVEE might or might not meet requirements for evidence of effectiveness according to other systematic reviews. The HHS criteria for an evidence-based early childhood home visiting service delivery model are designed to ensure confidence in model impacts through replicated findings from well-designed research.
How does HomVEE apply HHS criteria for evidence-based early childhood service delivery models?
All research, including research that includes virtual service delivery, is subject to the same standards and procedures. HomVEE groups together all research on a model to apply the HHS criteria unless the model developer indicates that particular manuscripts are research on an adaptation of their model.
HomVEE assesses whether each model meets the HHS criteria for an “evidence-based early childhood service delivery model.” To operationalize the HHS criteria related to studies, and because study findings may be reported across several manuscripts, HomVEE rates manuscripts based on the highest-rated finding reported in that manuscript. Any high- or moderate-rated finding from a study about a model is considered as part of the evidence base for that model. Notably, for models with research solely from either randomized controlled trial (RCT) or single-case design (SCD) studies, additional criteria apply (see the HHS criteria). For example, the additional criteria for RCTs to be from peer-reviewed journal articles and to have sustained findings align with MIECHV Program statutory requirements. The HHS criteria for an evidence-based model have no additional requirements for regression discontinuity design (RDD) or non-experimental comparison group design (NED) studies.
A model may be evidence based on the strength of subgroup findings alone only if the research about it satisfies all of the subgroup criteria.
For details, please see the HHS criteria for evidence-based early childhood home visiting service delivery models.
Is a model required to have evidence from an RCT in order to be evidence-based according to HHS criteria?
No. A model may be evidence-based, according to HHS criteria, on the strength of either RCT research or quasi-experimental design (QED) research (including non-experimental comparison group, single-case, and regression discontinuity designs). The research must be of high or moderate quality based on HomVEE’s published standards. Additional HHS criteria apply when the research uses an RCT design. For details, please see the HHS criteria for evidence-based early childhood home visiting service delivery models.
Where can I find a list of models that meet the HHS criteria for evidence-based models?
Users can find a filtered list of models that meet the HHS criteria on the Model Search page by clearing the "MIECHV Eligible" filter. The information is also available in the summary brief, available on the HomVEE Summary page. Notably, a HomVEE designation as an evidence-based model does not guarantee that a model is eligible to be implemented with MIECHV funding.