Multi-colored lines representing a network and the Word Justice Information Resource Network

Objective Research for More Equitable Outcomes

Member Login

Measuring Success in the Criminal Justice System’s Response to Domestic/Dating Violence, Sexual Assault, And Stalking: A Pilot Project

Background

The U.S. Office on Violence Against Women (OVW) funded this project to identify ways to measure the success of the programs it funds to improve law enforcement response to domestic violence, dating violence, sexual assault, and stalking (VAWA crimes). Those grantees have traditionally reported on their activities (outputs, such as numbers of people served or trainings conducted), not the results of those activities (outcomes, the difference those activities make). The project was carried out by the Violence Against Women Act Measuring Effectiveness Initiative (VAWAMEI) at the University of Southern Maine, working in partnership with the Justice Information Resource Network (JIRN).

Project Goals

  • Identify outcome measures that could indicate success in OVW-funded law enforcement programs.
  • Conduct a pilot test to determine whether it is possible to collect data on these outcome measures and the extent to which the participating programs agree that these are appropriate ways to measure success.
  • Produce workable “outcome measures” that law enforcement grantees can use to demonstrate results.
  • Provide recommendations for OVW to improve data collection requirements to measure grantee performance/success in the future.

NOTE: OVW is not requiring grantees to adopt any of these recommended metrics now. Instead, this information is provided to help those who are interested take steps to measure their outcomes – whether or not they are funded by OVW.

Project Components and Methods: Two Phases

Phase 1: Develop a conceptual model for law enforcement grantees based on research and field experience that shows how activities should produce desired outcomes and recommend data measures/metrics for pilot testing.

A conceptual model outlines the pathways to achieving program outcomes, and a theory of change explains how program activities are believed to produce those outcomes.

Phase 1 activities included a literature review, an inventory of existing national statistical collections (datasets) that could be used to measure outcomes in addition to new data collected by grantees, an advisory board of academic and practitioner experts, and the development of a conceptual model that links program activities to outcomes.

view the conceptual model

Phase 2: Develop and pilot test a set of indicators to measure success in the criminal justice system’s response to the VAWA crimes

Twelve law enforcement grantees acted as pilot sites to test the workability of collecting data for 84 possible data points to capture outcomes identified in the conceptual model that might apply to their VAWA-funded work. The purpose of this pilot testing was to identify which measures could feasibly and reliably be collected and reported with the least burden and greatest utility for grantees in their work and for OVW for measuring national outcome trends.

The pilot testing phase also included interviews with sites to assess the data for its usefulness and to discuss the process. Grantees reported that many of the proposed metrics—or data points—were possible to collect and report. However, metrics that relied on other system partners or collecting victim experiences were less feasible. The pilot test did inspire sites to think differently about data, but they needed assistance or tools before being asked to collect more or different information.

“A lot of these questions prompted me to think this wouldn’t actually be hard to track if we just started doing it!”

Pilot Site Grantee

“We would need funding support to make those database changes.”
Pilot Site Grantee

Project Products

Many of the products developed as a result of this project can help victim service providers, law enforcement, and researchers start thinking about measuring the benefits of their work:

  • Model Brief: 4-page summary of our conceptual model process and how you can do something similar for your program.
  • Executive Summary: Two-page project summary.
  • Literature Review: How has previous research measured these outcomes?
  • Dataset Inventory: Inventory of existing national datasets for usefulness in measuring these outcomes. [LIVE LINK]
  • Indicators Matrix: A matrix of metrics (MS Excel document) that law enforcement agencies serving victims of VAWA crimes can use to measure their results. It can be sorted by concept area, provides several options to measure each outcome, and is free for download and use.
  • Research Agenda: to guide researchers and practitioners in needed projects that would fill gaps in our knowledge and advance the field even further.
  • Complete Conceptual Framework Report: Full project report.

 

Webinar

Presentation from the JRSA Webinar – Measuring Outcomes in Response to Victims of Crime (December 8, 2021)

NOTE: OVW is not requiring grantees to adopt any of these recommended metrics now. Instead, this information is provided to help those who are interested take steps to measure their outcomes – whether or not they are funded by OVW.

For more information, contact contact Roger Przybylski at rprzybylski@jirn.org. or Sarah Goan at sarah.goan@maine.edu.

This project was supported by Grant No. 2019-SI-AX-K001 awarded by the Office on Violence Against Women, U.S. Department of Justice. The opinions, findings, conclusions, and recommendations expressed in this publication/program/exhibition are those of the author(s) and do not necessarily reflect the views of the Department of Justice, Office on Violence Against Women.