Pennsylvania’s Court Improvement Project

Data Collection and Analysis Grant

Program Assessment Report for

Federal Fiscal Year 2010

October 1, 2009 - September 31, 2010

Office of Children and

Families in the Courts

Pursuant to the Administration for Children and Families Program Instructions ACYF-CB-PI-07-09, the Office of Children and Families in the Courts (OCFC), Pennsylvania’s Court Improvement Project (CIP) hereby submits its annual Program Assessment Report for the CIP Data Grant for Federal Fiscal Year 2010 to the U.S. Department of Health and Human Services (DHHS), Administration of Children and Families (ACF) Region III. This annual program report addresses all Data Collection and Analysis Activities of Pennsylvania’s 2010 CIP; the outcomes of court improvement activities; and how they help provide for the safety, permanency and well-being of children and youth in Pennsylvania’s foster care system.

I.  Description of needs and activities undertaken with CIP funds to meet those needs during the program period

The following list represents those needs as identified by the OCFC and Children’s Roundtables addressed throughout the fiscal year. These identified needs are a result of the 2005 CIP Reassessment produced by the National Center for Juvenile Justice (NCJJ); the Pennsylvania Children’s Roundtables Initiative; and ongoing meetings with the Department of Public Welfare’s Office of Children, Youth and Families (DPW/OCYF) officials. Some of these identified needs include:

·  Need foran automated statewide capacity to track dependency caseloads at the local level or reliable dependency court data to enable courts to assess its key performance measures;

·  Need for a mechanism to collect court data statewide;

·  Need to implement court performance measures/indicators;

·  Need to enhance knowledge regarding the use of data in system reform efforts;

·  Need for local courts to become more involved in the day-to-day scheduling, operation and management of the dependency docket;

·  Need to track timelines of cases through the system from the initial hearing to TPR; and

·  Need for new/updated equipment to access the CPCMS Data System

II.  Activities and progress towards the goals delineated in the strategic plan and results of the assessment of activities funded under this grant

Implementation of CPCMS – Dependency Data System

The AOPC Judicial Automation Department has implemented a Dependency Data System with OCFC staff providing ongoing programmatic support to the Judicial Automation Department and local jurisdictions. Roll-out to counties commenced in March 2009. As of September 2010, all 67 Pennsylvania counties were using the new system (with Philadelphia going live in August and Allegheny going live September). Case management reports for 65 counties were available with a limited number of statistical reports being developed, released and refined.

Implementation of the statewide dependency data system has been a monumental task. In addition to gathering statistical data, Pennsylvania’s dependency court data module created judicial Findings & Orders documents for the majority of dependency hearings. The forms, based on the Pennsylvania Dependency Judicial Rules of Court, require local jurisdictions to adjust business practices. The forms and court orders that are now part of the system were created using the Pennsylvania Model Court orders, originally developed by the Juvenile Court Judges’ Commission (JCJC), as well as information from other states.

System document changes are updated through a sub-committee of the Pennsylvania Juvenile Procedural Rules Committee in collaboration with DPW/OCYF. Any changes to the forms or the CPCMS system are first reviewed by the Forms Sub-committee with any change that has a potential impact of federal funding requirements also reviewed by DPW/OCYF prior to implementation. This oversight process ensures that no change is made that conflicts with the Juvenile Dependency Rules or jeopardizes much needed financial resources for abused/neglected children and their families.

With all counties now actively entering data into the system, AOPC Judicial Automation and OCFC staff continues assisting counties on issues of data accuracy and business practice realignment. Because counties inputted initial case conversion data differently across the Commonwealth with some counties entering all dependency cases elements as they went live on the system and others adding only enough information to create the case; statewide data collection will not be possible for several years. However counties do have access to their own data and have worked to enhance its accuracy.

As such, AOPC’s Judicial Automation and OCFC staff continue to make county site visits, providing technical assistance as well as additional training sessions. In addition, OCFC staff provided significant business process technical support during 2010. As expected with any new data system during its infancy, data integrity will continue to be a primary focus for several years. Judicial Automation and OCFC staff will continue to collaborate in their efforts to assist counties.

In addition to the 65 counties live on the system, in 2010 special attention and support was provided to Philadelphia and Allegheny counties (the two largest jurisdictions in the Commonwealth). Ironically, their issues were at opposite ends of the spectrum. Allegheny already had an incredibly sophisticated court computer system which allowed for e-filing (something not currently possible in CPCMS) while Philadelphia had a combination of several independently functioning systems. Both Philadelphia and Allegheny were committed to fully implementing the CPCMS system and diligently worked with AOPC staff to implement the module. Simultaneously AOPC staff worked closely with the two counties making system adjustments to accommodate local court needs. Monthly project update conference calls for both counties occurred throughout implementation. These calls included the Administrative Judge of the County (and any county staff the judge may want to include), AOPC’s Judicial Automation Director, AOPC’s Judicial Automation CPCMS Project Manager, and OCFC’s Administrator. The primary purpose of these calls was to identify problems early, create and implement solutions and, to the extent possible, meet the implementation timeline.

Throughout 2010, the initial 65 counties ran their data reports which listed county specific case loads. Upon doing so, the need for system revisions became evident and AOPC staff identified and fixed system flaws. AOPC Judicial Automation staff also released several (not all) statistical reports which pull information from the caseload data reports for each county.

During all three Leadership Roundtable meetings within this reporting period (November, March/April and September) as well as the 2010 State Roundtable, AOPC’s Judicial Automation Department provided an overview of system updates, including forms and court orders. Comments and feedback from each Leadership Roundtable were used to further improve the CPCMS Dependency Module.

OCFC Data Analyst:

In May 2009, at its annual meeting, the State Roundtable approved the addition of a judicial analyst research position to assist counties with the development and comprehension of their county specific reports and with statewide data/research information. This position was subsequently included in the CIP Strategic Plan and Budget. After extensive and thoughtful recruitment, Ms. Justine Taylor was hired to fill this position in August 2010.

Since joining the OCFC, Ms. Taylor has focused her work on analyzing, revising and developing a more robust level of data related to the Permanency Practice Initiative. To this end, Ms. Taylor has: 1) reviewed the initial three quarters of PPI data submitted by counties; 2) identified areas of concern with the current reporting form; 3) refined the reporting form to better collect data; 4) presented the revised form for approval to the PPI Oversight Committee; and 5) created a statistical report for analysis of the PPI data (see more detail below).

Development & Initiation of Data Reports

The OCFC currently has available data from two court data sources related to CIP. These include the Pennsylvania Common Pleas Case Management System (CPCMS) Dependency Module and the Permanency Practice Initiative Quarterly Reports (for PPI counties and discussed in detail below). In addition, the OCFC works closely with the DPW/OCYF utilizing available AFCARS data to create a more robust picture of Pennsylvania’s child dependency system.

CPCMS:

Developing, implementing, organizing and analyzing detailed court dependency data is a new experience for Pennsylvania’s dependency court system. Upon initiation of Supreme Court oversight of the CIP in Pennsylvania, Justice Max Baer led a court reform effort that included the creation and implementation of a statewide court dependency data & outcome monitoring system. Funded entirely by the Pennsylvania Judiciary, the Dependency Module of the Common Pleas Case Management System (CPCMS) was initiated.

The system was developed with county & state child welfare as well as local court input and based on the national court performance measurements. System implementation began in March 2009. By September 2010 all 67 counties/60 judicial districts were utilizing the module for dependency case court action with Allegheny & Philadelphia counties going live August and September 2010 (respectively). As a result, Judicial Automation and OCFC staff provided counties with significant support and technical assistance during 2010.

The system, which was implemented voluntarily by all counties, is a huge change for local jurisdictions but provides a level of uniformity not previously seen in Pennsylvania’s Dependency Courts. The system provides for the scheduling of events, creation of court findings & orders for all major hearings (shelter, adjudication, disposition & permanency reviews) and corresponding case management/statistical reports. Any requested changes to the system are reviewed by a sub-committee of the Pennsylvania Juvenile Procedural Rules Committee to ensure conformity to state statute & procedural rules. In addition, the Department of Public Welfare’s Office of Children, Youth & Families is consulted when proposed system changes have financial or significant practice implications.

Because the system is relatively new and a significant change for local jurisdictions, staff from the AOPC’s Judicial Automation Department and OCFC is working to assist counties with data integrity. To this end, staff from both departments made numerous county visits to assist county review of Case Management reports, initiate data clean-up and support business process clarification. During this reporting period, all counties (with the exception of Allegheny and Philadelphia) were able to print county specific caseload data reports. OCFC staff continues working with counties to ensure data integrity of these reports and recognizes, from consultation with national data experts, the task of assuring data integrity will be an ongoing one.

That said, upon ensuring an acceptable level of data integrity with basic data caseload reports, statistical reports will be more fully implemented. In time, these reports will provide statewide aggregate Dependency data information as well as county specific data. Data will be provided to local jurisdictions as well as the Leadership and Statewide Roundtables. OCFC staff will support counties by sharing data trends within the county and support strategic planning, development of programs and implementation of practices that may address any concerns.

PPI Quarterly Data Reports:

In close consultation with the Permanency Practice Initiative Oversight Team, OCFC staff developed a PPI Quarterly Data Report (Attachment A). Effective April 2010, Phase One & Phase Two counties began reporting PPI data. Upon hire, analyzing and recording data reflected in the reports became the priority task for OCFC Data Analyst Justine Taylor.

To this end, Ms. Tayor evaluated the initial three quarters of data submitted by PPI counties. In so doing, several issues became clear. First, the layout of the reporting form was backwards, with the PPI “Model Outcome” items at the beginning and the PPI “Model Fidelity” items at the end. Ultimately, it was decided to move the PPI Model Fidelity items to the beginning of the reporting form so counties could focus on how well they were implementing the total model before they focused on their subsequent outcomes.

Second, in some instances it appeared the reporting form instructions were unclear or misinterpreted, and as a result, the wrong data was reported, particularly in regards to the outcome measures. To correct this problem and enhance the validity/reliability of future date, supplemental instructions to the reporting form were written.

Third, it became apparent that certain information needed to complete meaningful statistical analysis of the data was not captured on the reporting form. These value blocks have been added to the revised PPI reporting form and will be covered in the 2011 Data Grant Assessment Report.

Finally, it was concluded there are better ways to capture data on Family Finding efforts which would be more reflective of the Kevin Campbell Family Finding Model being utilized in Pennsylvania. This section of the PPI data form was significantly changed.

Despite the concerns noted above, Ms. Taylor did complete a preliminary analysis of the PPI data submitted during 2010 identifying two additional challenges. First, the OCFC never specified that PPI counties had to select only one PPI population and keep that population for the entire year. As a result, some counties had multiple PPI populations over the year (for a multitude of legitimate reasons), making data analysis almost impossible. To overcome this particular challenge, the OCFC will be instructing PPI counties to select one PPI population and keep the same population for the entire 2011 calendar year.

Second, it was evident that children outside of the county identified PPI target population were receiving PPI services since counties were never required to have a “control” group. While this has potentially positive implications for all children in the PPI counties, this lack of control group coupled with the fact that counties were free to choose their own PPI Target Populations, has made it difficult to associate county success specifically with PPI practices. The OCFC and the PPI Oversight Team are currently exploring the best way to answer this question.

Regardless of data imperfections, Ms. Taylor did create a statistical data format for analysis of initial PPI data (Attachment B). These reports have been created for all PPI Phase One & Two counties. OCFC judicial analysts will be meeting with the lead dependency judge and child welfare administrator from each PPI county in 2011 to review the report and address concerns/issues. Overall, it became apparent that counties in which model fidelity was high had much better outcomes than those struggling with model fidelity.

In addition the following general observations can be made after review of the 62 PPI reports submitted by the various PPI counties. These reports gathered data on permanency practice initiatives that occurred between January 1, 2010 and September 30, 2010. Based on the reports, 49 meetings of Local Children’s Roundtables have convened; 1,247 three-month Court Reviews have occurred; and 311 Family Group Decision Making Conferences serving 617 children have been held, resulting in 272 Family Service Plans. Furthermore, 882 Accurint computer searches have been performed, which resulted in approximately 400 new Life Connections for children in the PPI Target population. In addition, 49 children in the PPI Target population have been placed with someone identified during the Family Finding process. Finally, there are currently 111 Family Development Credentialing Workers working with children within the PPI Target populations.