HomVEE periodically updates standards and procedures to address emerging issues or areas in need of clarification. In the coming months, HomVEE will release Version 2.2 of its Handbook of Procedures and Standards. The Administration for Children and Families is committed to a rigorous and transparent review process. HomVEE invites public comment on key proposed changes summarized below.
Please submit comments via email to HomVEE@acf.hhs.gov no later than 5:00 PM on July 28, 2023.
Proposed changes to HomVEE Handbook of Procedures and Standards (Version 2.2)
HomVEE is proposing to make changes in the forthcoming Version 2.2 Handbook to incorporate virtual home visiting research into the review and limit the eligibility of certain findings. The updated handbook will also include minor clarifications and changes that were previously announced or determined in consultation with our expert panel.
HomVEE is seeking public comment on the following key changes:
- Virtual home visiting. HomVEE has previously focused on research on in-person home visiting services. HomVEE is proposing to begin reviewing research on virtual service delivery, defined as “delivery of an intervention’s home visit content to an individual caregiver or family conducted solely by use of electronic information and telecommunications technologies. The content should be designed or adapted for synchronous delivery.” This may include fully virtual or hybrid (virtual and in-person) service delivery. To be eligible for review, interventions must meet all other elements of HomVEE’s definition of an early childhood home visiting model (see Exhibit I.1 on page 2 of the handbook).
- Limiting findings eligible for review. To promote an efficient review that focuses on high-quality research findings, HomVEE proposes to limit certain findings eligible for review. Specifically, HomVEE proposes the following:
- Prioritize covariate-adjusted findings over covariate-unadjusted findings if both are available. Covariate-adjusted estimates are generally more precise than unadjusted estimates. HomVEE will review unadjusted findings only when adjusted findings are not available and the study design meets HomVEE’s standards for a high-quality randomized controlled trial (RCT). HomVEE will not review unadjusted findings from quasi-experimental designs (QEDs) or high attrition or compromised RCT.
- To not review findings based on individual items drawn from scales or subscales. Some manuscripts report scale or subscale scores in addition to the item-level findings drawn from those scales. HomVEE will not review item-level findings unless they are the only outcomes reported and they meet HomVEE’s standards for face validity and reliability.
- Whenever a manuscript reports on multiple measures of similar or related constructs, review only the most comprehensive item. For example, a manuscript may include findings for four separate measures that indicate whether a child’s diet meets dietary recommendations for (1) dairy (2) fruits and vegetables, (3) grain, and (4) protein; and a fifth measure to indicate whether all dietary recommendations are met. In this case, HomVEE will review only findings for the measure for all dietary recommendations because it is the most comprehensive, and would not review the findings for the other four measures.
- To not always review binary variables that are constructed based on continuous scores. Some manuscripts report findings on binary variables alongside continuous scores, such that the binary variable reflects a person’s performance relative to a designated threshold value of that continuous score. HomVEE will review these binary variables only if the manuscript does not present a continuous score, or, if the manuscript presents both types of measures, when authors indicate that the key threshold defining a binary variable is substantively important.
- Revise the face validity requirement. Measures must demonstrate face validity for all ages and groups the study examines, not just for some of the ages or groups. This requirement was not specified in previous versions of the Handbook.
- Definition of subgroup. HomVEE proposes to revise the definition of subgroup to indicate that it will no longer consider analyses by site or cohort as analyses of subgroups. Findings from these analyses will become eligible for review. The revised definition will also clarify that the remaining sample after attrition or sample loss is not a subgroup.
- Subgroup replication. HomVEE proposes to revise the definition of replicated subgroup to indicate that subgroups can be replicated by a subgroup or a full sample, if the full sample has the same characteristic(s) as the subgroup. For example, a subgroup of first-time mothers can be replicated by the full sample from a separate study that focuses on first-time mothers. A subgroup will be considered replicated only if both samples have findings rated High or Moderate. For example, if findings from one study rate Low and findings from the other rate High or Moderate, the subgroup would not be replicated.
When does HomVEE intend to finalize and apply Version 2.2 changes?
HomVEE intends to begin applying the proposed revisions to procedures and standards with the 2024 review. HomVEE will finalize the revisions and publish an updated handbook on the HomVEE website by early 2024. A summary of the final substantive changes will appear at the front of the updated handbook so HomVEE users can easily identify what is new. HomVEE will notify website users by an email blast after it posts the updated handbook. Interested readers should join the HomVEE email list to receive updates.