OPTIONS AND STRATEGIES FOR EVALUATION
OF TRAINING AND IMPLEMENTATION

Click on a link below to jump to its content.

Pupose for SOP Evaluation
Tools to Evaluate Safety Organized Practice
- Practice Profiles
- Case File Review

Pupose for SOP Evaluation

Successful training, coaching and use of Safety Organized Practice require the establishment of an overarching research and evaluation framework. The existing SOP research and evaluation strategies are aimed at assisting counties to create the systemic infrastructure necessary for realizing and sustaining Safety Organized Practice across all levels of the workforce. The information collected is analyzed and reported to ensure consistent practice and program fidelity.

Outcomes to Examine Related to the Training and Implementation of Safety Organized Practice
Outcomes	Measures/Tools	Sources
Improved completion and accuracy of SDM Tools	Identify SDM completion measures	SafeMeasures®
Improved completion and accuracy of SDM Tools	Measure Accuracy/Consistency	Case Review Tool
Utilization of SOP tools and skill enhancement	Measures of SOP tools utilization	Worker Training Survey, Six month follow up survey, Case Review Tool
	Practice Profiles	Practice Profiles
	Worker Satisfaction Survey of Coaching	Worker Satisfaction
Improved identification of danger & safety	Worker Training Survey (skill and knowledge questions)	Worker Training Survey
	Case Review Tool	Case Review Tool
	Practice Profiles	Practice Profiles
Improved parent/family engagement	Survey questions on engagement, involvement in case planning and understanding of CWS involvement	Family Survey
	Practice Profiles	Practice Profiles
	Case Review Tool	Case Review Tool
Increased client advocacy	Practice Profiles	Practice Profiles
Increased client advocacy	Family Survey	Family Survey
Reduced disproportionality	In care and entry rates by race/ethnicity	SafeMeasures
Fewer children in out of home care & long term foster care	Number in out of home care & percentage in permanent placement (excluding guardian)	SafeMeasures
Reduced recurrence of maltreatment	CFSR measure	SafeMeasures
Increased timeliness of reunifications	Percentage of exits that are due to reunifications	SafeMeasures
Increased timeliness of reunifications	Reunification within 12 months (CFSR measure, entry cohort)	SafeMeasures
Reduced re-entry into care	CFSR measure	SafeMeasures

[back to top]

Tools to Evaluate Safety Organized Practice

Practice Profiles

Click here to view/save the Practice Profile (PP).
Click here to view/save the PP FAQs.

FOCUS
UC Davis continues to develop practical approaches to assessing whether staff members are implementing key elements of SOP or some form of safety-oriented practice. Presently we are working on finalizing the development of SOP fidelity of tools for Safety Mapping, Incorporating the Child’s Prospective, Harm and Danger Statements, Safety Planning, Safety Networks and Safety Goals. Following the development of the practice profiles of SOP tools, we will begin defining the critical components of SOP practices. These critical components of SOP consist of the broader themes that make up the core philosophy of SOP. Just as the SOP practice profiles break down developing skill to use a specific tool or method at the micro level, the critical components of SOP capture the various avenues in which a practitioner can enhance their ability in a particular area. The critical components are:

Advocacy/Communication: Such as recognizing individual or group needs, providing intervention on behalf of a client or client group, communicating with decision makers and initiating actions to secure or enhance a needed service, resource, or entitlement; in part accomplished by creating collaborative Safety Goals.

Cultural Humility: Interacting with families holding assumptions, respecting and learning from the unique characteristics and strengths of the family while acknowledging and honoring the diversity within and across cultures, and applying these skills to the partnership with the family and the options made available to them.

Engagement/Involvement: Effectively joining and partnering with the family to establish common goals concerning child safety, well-being, and permanency; could be accomplished by implementing Safety Mapping, Three Houses.

Critical Thinking (Assessment): Gathering information about reported concerns and family needs, evaluating the relevance of that information as well as identifying family strengths and community resources that may be applied to address those concerns and needs; could be accomplished by using Structured Decision Making and in formulating Harm and Danger statements, Three Houses, Safety Mapping.

Evaluation: Monitoring outcomes of service plans and system programs to determine if the desired goals are being achieved and if not, to use this information to reconsider goals and strategies developed in the planning phase or services and resources identified in the implementation stage.

PR0DUCTS
Fidelity measurement tool(s), research brief once the tools are piloted.

TIMELINE
Data collection design and measures:

Practice profiles for tools: completing the last Ppractice Pprofile Safety Goals
Once the final Safety Goal profile is complete we will begin piloting the tool with pPractice lLeaders and sSupervisors to start. The purpose this initial pilot is to obtain feedback on the definitions, scaling systems, and the overall ease of use.

Case File Review

FOCUS
This tool includes key elements of Safety Organized Practice in three areas: Engagement, Critical Thinking and Increased Safety. To complete the case review tool, both computerized case records (e.g. in automated child welfare information systems such as CWS/CMS) and paper files are examined. Before completing the case review tool, the reviewer talks with the caseworker about where information about Safety Organized Practice (SOP) may be found. Case information may be in more than one paper file depending on the case and the county. Different workers also may have different ways of documenting child, family, and safety network input in case notes. Scales are provided for each SOP element to define a progression of skill from a basic or “emergent” level through an advanced or “distinguished” level mirroring the framework used in California’s Practice Profiles. Scale points are defined generally as follows:

Emergent practice - There is evidence in the case files that the worker has begun to use the practice or tool; “it’s there.”
Accomplished practice - There is evidence that the worker has begun to use the element to communicate with others involved with the case and to build shared understanding and agreement among the parties; “it’s shared.”
Distinguished practice - There is evidence that the element is integrated into the worker’s practice as a way of doing business; “it’s integrated.”

PRODUCTS
Case review tool developed, piloted, and disseminated.

[back to top]