March 20, 2015

Page 1 of 2

Colorado Department of Human Services Assessment Contact Review

Denver County Department of Human Services Review Overview and Results

On February 2nd, 2015, the Colorado Department of Human Services (CDHS) notified the Denver Department of Human Services (DDHS) that it would be conducting a review of DDHS assessment practices. This included a process to review and verify contacts occurring as part of child welfare assessments. The following provides a summary of the review process and results.

Review Process

It was determined that the review would verify contacts on assessments that were open at the time of the review. This decision was made so that contact for verification purposes would be occurring with families and collaterals participating in active, open assessments, rather than contacting individuals who may not have had contact with the child welfare system in some time. This would also help minimize individuals not remembering discussions clearly, having moved and being unable to be located, or other issues that might impact the integrity of the review.

Review Sample

The population eligible for the review included assessments open on February 24th, 2015 where at least one contact was recorded in Trails. Contacts eligible for review included any contact documented by the worker as “completed” and identified in Trails as having a contact date between December 24th, 2014 and February 24th, 2015. No further filtering of contact type (e.g., email, telephone call, face-to-face, etc.) occurred. The population of eligible contacts consisted of a total of 1,717 contacts.

Using the software SPSS, a random sample was extracted from the Trails system. Using a 95% confidence level with a 10% interval, a sample size of 92 contacts was determined to be required for review. A 15% oversample (14 contacts) was also pulled in the sampling process in the event that contacts in the primary sample were unable to be reviewed. The application of this sampling technique means that if the entire population was reviewed, the results would be consistent with the findings of this review 95% of the time (with a margin of error of 10%). The 92 contacts identified for review were made by 36 different workers from 11 units in Denver County.

Review Process

The CDHS review team consisted of 13 staff from various Divisions. A CDHS staff member was assigned to each DDHS supervisor who had a worker with a contact selected into the sample. This created a total of 11 pairings that allowed the supervisor to have the direct contact with the individuals as part of the ongoing management of the assessment, while the CDHS staff person was responsible for verification of the sampled contacts. The paired staff were responsible for verifying each contact in the sample for that particular supervisor.

The review started in DDHS on Wednesday, March 4th, 2015. DDHS staff were briefed on the background, scope, and purpose of the review. Each DDHS supervisor and CDHS staff pair were asked to spend time reviewing the contacts needing to be verified in order to create a work plan and schedule for this work. As the DDHS supervisors needed to continue to be available to triage incoming referrals, provide guidance to staff on child safety decisions, etc., CDHS staff were asked to work around the DDHS supervisor’s schedule and responsibilities to make ongoing child safety decisions.

In order to consider a contact verified, the DDHS supervisor and CDHS staff pair looked to confirm both that the contact occurred, and that the individual(s) verified the content (i.e., summary of the contact entered into the Trails system). Contact verification occurred through various processes, often depending on the type of contact and information available at the time of the review. Examples include:

  • Emails: If DDHS staff were able to print a copy of the sent/received email from their email system and provide it to the CDHS staff member, the contact was considered verified. In addition, DDHS worked with their OIT to pull copies of emails directly from the server and provided these to CDHS staff. If emails had been deleted from the system already, then a telephone call was made to the recipient/sender of the email to verify the contact with them verbally.
  • Telephone Calls: The DDHS supervisor and CDHS pair called the individual identified in the contact to verify the telephone call. In instances where an individual was not able to be contacted directly, copies of phone logs were requested, received, and examined to provide verification of the contact. An example includes use of the language line for translation services.
  • Face-to-face: The DDHS supervisor and CDHS pair called the individual(s) identified in the contact to verify the face-to-face contact.
  • Voicemail Message: The DDHS supervisor and CDHS pair called the individual identified as having either left or received the voicemail and verified this information.

Results

Eighty-nine of the 92 contacts in the sample group were reviewed. Three of the contacts needed to be replaced with a contact from the oversample group. One contact was within an assessment that closed between the time the sample was pulled (February 25th, 2015) and when the review started (March 4th, 2015). On another contact, the DDHS supervisor and CDHS staff were unable to locate the individual in order to verify the contact. The third contact needed to be dropped because the DDHS supervisor and CDHS staff were unable to discuss the contact with that individual. When possible, attempts were made to replace the sample contact with an oversample from the same worker, the same supervisor, and of the same contact type.

Of the 92 contacts reviewed, all 92 were verified as having occurred and the summary of the contact in Trails was supported. As such, there was no evidence of systemic concerns of falsification of contacts.

Next Steps

The CDHS will conduct a re-review of open assessment contacts in Denver County in six months and will work with the Denver County Quality Assurance team to develop internal monitoring of contacts.