ATTACHMENT 4.11(A): COMPREHENSIVE STATEWIDE NEEDS ASSESSMENT
ATTACHMENT 4.11(a): COMPREHENSIVE STATEWIDE NEEDS ASSESSMENT
BACKGROUND
In May 2010, the federal Rehabilitation Services Administration (RSA) issued guidance, “Developing a Model Comprehensive Statewide Needs Assessment (CSNA) with Corresponding Training Materials for State VR Agency Staff and SRC Members (Guidance). This current CSNA reflects the 2010 Guidance. In its Introduction, the Guidance outlines the scope and focus of the CSNA. The Rehabilitation Act of 1973, as amended (Act) calls for periodic comprehensive statewide needs assessments to be conducted jointly by each state’s vocational rehabilitation (VR) agency and State Rehabilitation Council (SRC) to inform the State Plan. The Act is specific regarding areas that a needs assessment should address. In addition to the overall need for rehabilitation services in the state, the Act focuses on several VR subpopulations and services: individuals with most significant disabilities, including those in need of supported employment; unserved and underserved individuals, including minorities; individuals served by other parts of the statewide workforce investment employment system; and establishment, development or improvement of community rehabilitation programs (CRPs).
In Pennsylvania, the designated state unit, otherwise known as the Pennsylvania Office of Vocational Rehabilitation (OVR), and the Pennsylvania Rehabilitation Council (PaRC), jointly conducted the CSNA. Section 321.69 of the Act outlines the specific topics to examine, including specific populations and the cooperation of stakeholders and providers. The CSNA is an opportunity for combining existing information and new data to inform the State Plan.
It is essential to note that this CSNA, as noted above, encompasses the overall rehabilitation needs of people with disabilities throughout Pennsylvania, including those “individuals served by other parts of the statewide workforce investment employment system.” So, in addition to information about services provided by OVR staff, rehabilitation professionals from other community partner agencies contributed to this document. Consequently, information contained in this document, while intended for use by OVR staff and PaRC to develop the State Plan, may be used by other rehabilitation community partner agencies to inform their own strategies.
However, throughout the document, most of the service references will be ways that OVR can address the services gaps through their funding mandate, keeping within the boundaries of the Rehabilitation Act and the regulations.
HISTORY
OVR submitted, for FFY 2005-2008, a comprehensive needs assessment that used a systems research approach, utilizing data available on the Internet, as well as existing information supplied by staff and constituent activities. In addition, OVR augmented this information with data gathered from staff and stakeholder questionnaires. OVR designed the questions to address the statutory requirements for areas to be addressed in the Needs Assessment.
Attachment 4.11(a): Page 1 of 23
Effective Date: October 1, 2013
In 2008, OVR created an online survey to meet the requirement of the CSNA with questions developed by OVR staff in partnership with the PaRC. PaRC financed and posted the survey on their website for the month of August. The district offices announced the survey to customers, partners, and providers. Approximately 1,000 surveys were completed. PaRC met with OVR to discuss and draft a report that was submitted in 2009 for FFY 2010.
This current CSNA not only reflects the detailed suggestions and models in the 2010 Guidance, but also expands upon the research techniques used in the development of previous CSNA. Most obviously, the use of focus groups directed at specific stakeholder groups, was an attempt to engage those customers more in depth.
DEFINING GOALS & DEVELOPING PLAN
A needs assessment is a systematic exploration of the way things are and the way they should be. These “things” are usually associated with organizational and/or individual performance. There are four basic steps to conducting a needs assessment, (1) gap analysis, (2) identify priorities, (3) identify causes of performance problems and/or opportunities and (4) identify possible solutions and growth opportunities. Although there are many effective methods that OVR currently uses that bring success, this is an opportunity to highlight the areas and possible practices where OVR could take steps to improve service.
Goals
The information goals of the needs assessment were determined and stated in the RSA Guidance. The RSA requested the following information:
1. The rehabilitation needs of individuals with disabilities, particularly the vocational rehabilitation services needs of individuals with most significant disabilities, including their need for supported employment services.
2. The vocational rehabilitation services needs of minorities.
3. The vocational rehabilitation services needs of individuals with disabilities who have been unserved or underserved by the vocational rehabilitation program.
4. The vocational rehabilitation services needs of individuals with disabilities served through other components of the statewide workforce investment system.
5. The need to establish, develop, or improve community rehabilitation programs within the state.
Attachment 4.11(a): Page 2 of 23
Effective Date: October 1, 2013
Steering Committee & Responsibilities
A committee of OVR staff and members from PaRC steered this project. It was the responsibility of the steering committee to recommend participants, decide on the content and questions, and summarize each phase of the process. The Committee further relied on the institutional knowledge of OVR field staff, for example, soliciting their input in the development of the survey questions. This expertise in rehabilitation services of OVR staff and the Council, as well as the familiarity with customers, vendors, counselors, and other possible participants, outweighed the advantages of outsourcing this task.
OVR staff secured assistance from the Pennsylvania Office of Administration’s Office of Strategic Services (OSS). OSS has vast expertise in primary data collection and analysis. In addition, OSS is experienced in meeting state government performance and economic goals. OSS assisted in administering the decisions of the steering committee. OSS analyzed the surveys, assisted in conducting focus groups and individual interviews, and assisted in authoring the final report. OSS services are paid for by 1.5% of all state agencies’ budgets, so the actual services for this project were free of specific charges to OVR.
Process
1. OVR staff reviewed the RSA requirements.
2. The Steering Committee (Attachment A) discussed the requirements, determined what type of information was needed, and who was in the best position to provide the information.
3. OVR staff and OSS took the result of the brainstorming session and drafted the actual survey types and questions.
4. OVR staff (executives and counselors) and PaRC members reviewed and approved the draft surveys
5. The surveys were published on the PaRC website. Announcements of the survey were sent to agencies, organizations, providers, customers and advocates.
6. Results of the surveys, processed in charts and graphs, were distributed to the Committee. The committee decided on the central or common themes of each survey type. With approval from the OVR executive staff, the common themes became the discussion topics for the focus groups.
7. PaRC members and OVR staff nominated potential focus group participants. Participants were invited by letter. OVR and OSS staff conducted the focus group process.
8. After reviewing the results of the focus groups, the committee deliberated on what other detailed information was needed to answer the RSA questions completely.
9. OVR staff and PaRC members recommended potential interviewees who possessed personal knowledge to help close some of the information gaps identified after the focus groups. OVR and OSS staff then conducted the interviews.
10. National and state trends were reviewed to determine whether the conclusions found were unique to Pennsylvania.
Attachment 4.11(a): Page 3 of 23
Effective Date: October 1, 2013
11. OVR and OSS staff wrote the report and submitted it to the OVR executives and PaRC on January 13, 2011. OVR staff and the PaRC membership had numerous meetings and conference calls to discuss the draft report and make changes.
12. The draft document was submitted to the OVR Executive Staff for final review at the end of January 2012.
13. For discussion purposes, terminology throughout this reports was taken directly from the RSA Guidance
GATHERING INFORMATION
The RSA Guidance required that the information be obtained through both primary and secondary data collection. The Committee included the following required components:
1. Surveying VR counselors and consumers of rehabilitation services
2. Key informant interviews
3. Focus groups with people with disabilities, employers, disability navigator/PA CareerLink staff and professional staff from other service providers and community rehabilitative programs (CRP)
4. Public community hearings
5. Secondary data sources including the American Community Survey, Current Population Survey, Behavioral Risk Factor Surveillance Survey, disability population state estimates, population projections and economic forecasts, VR agency data and information, state level data from federal sources, state/local data & reports and literature review.
Customer Satisfaction Surveys
PaRC
In 2011, PaRC conducted a customer satisfaction survey. The SurveyMonkey.com survey was posted on the PaRC website from October 1, 2010, through September 30, 2011. The survey asked questions about the service (response time, individual participation, relationship between OVR and vendors) participants received from OVR. A relatively small sample size of forty people completed the survey. (Attachment B).
OVR
In order to comply with various sections of the Rehabilitation Act, OVR conducts surveys of all successfully rehabilitated customers six and twelve months post-closure (Attachments C). Resources do not permit surveying customers closed in other statuses. The survey questions cover satisfaction with OVR services, as well as current employment and benefits information. An OVR staff and stakeholder workgroup developed this survey instrument and process. The purpose of this survey is to gauge how the OVR services aided in finding and maintaining gainful employment. Bar codes are placed on each survey enabling OVR to track each participant’s response.
Attachment 4.11(a): Page 4 of 23
Effective Date: October 1, 2013
In October 2010, OVR implemented a new survey and process. The new template allowed the survey results to be processed through and integrated into OVR’s Commonwealth Workforce Development System (CWDS). CWDS is an electronic case management system used by, Labor and Industry and Department of Public Welfare partners to aid in job search, job development, and placement services. CWDS also functions as the electronic case record for OVR customers. The OVR portion of the site is secure to protect customers’ confidentiality. The public can use CWDS to self-refer for services, create résumés and match their skills, requirements and backgrounds with job openings. Employers can use CWDS to find qualified job applicants and report their new hires. Service or training providers can use CWDS to receive referrals of those seeking the services, programs or training they provide. And Commonwealth staff use CWDS to track service delivery. This shared system streamlines service provision and provides a comprehensive view of services provided to any individual.
From January 1, 2011, until December 1, 2011, 9,261 six-month surveys and 8,635 twelve-month surveys were mailed to closed OVR case customers, with a return from 5.8% and 4.5% respectively (537 six-month surveys and 389 twelve-month surveys).
Needs Assessment Surveys
The CSNA Committee developed five survey instruments. The survey audiences were customers, advocates, employers, private sector providers and public sector providers (Attachments D through H). The Committee advertised the availability of the survey through e-mail blasts, PaRC and organizational newsletters, and in conference and workshop announcements. The e-mail blasts and announcements encouraged the reader to spread the word about the opportunity to participate. The target audiences included OVR, the Department of Aging, Corrections, Public Welfare, Veterans’ Affairs, Education, county government and the partners and customers of these organizations.
Approximately 2,000 people visited the survey website, and 18.3% completed one of the five surveys. The questions inquired about the relative ease to access services, the types of services requested and received, the organizations providing the services, and the current gap in the services available.
The surveys were posted on the PaRC website from April 1, 2011, to May 31, 2011, via Surveygizmo.com. OVR staff and OSS developed the top themes from the surveys and scheduled the focus groups. The top themes were the topics discussed in the focus groups.
Focus Groups
Focus groups were held in August and September, 2011. OVR staff statewide, OVR executives, and PaRC members recommended participants for the focus groups. Participation was by invitation only. The invitation included the date, time, location of the focus group, the goal of the focus group, as well as the top themes that would be discussed at the focus groups, and included accommodation requests.
Attachment 4.11(a): Page 5 of 23
Effective Date: October 1, 2013
Each focus group was designed for 10-12 participants, followed a pre-set agenda (Attachment O), and lasted approximately two hours. Each group was staffed with a moderator, a note taker, and a flip chart recorder. OVR trained and briefed focus group staff on their duties and the purpose of the focus group. The moderator facilitated and managed the group with the goals of easing and promoting honest and balanced discussion, eliciting useful information from all participants, and keeping people on track and on schedule. The note taker took complete notes of what was being said, as exactly as possible, attributing statements to their source, if relevant. The flip chart recorders captured key statements by participants and made them visible to the group, using colored markers on a 20” x 25” notepad, mounted on an easel. The flip chart was important because it helped the group members to keep track of, and focus on, the discussion as it took place. The recorder worked closely with the moderator to ensure that what he or she recorded matched the intent of the person who said it.
The RSA guidance suggested focus group types and gave each state a choice in the particulars to focus upon. Pennsylvania selected the following groups:
1. Individuals with significant disabilities, as defined in the Rehabilitation Act.
2. Individuals with disabilities from minority groups, as grouped for RSA data collection. (e.g., African-American, Asian-American, Hispanic, Native American, etc.);
3. Unserved and underserved individuals with disabilities
4. Employers who have worked, or are currently working, with OVR.
5. Pennsylvania partners in the statewide workforce investment system, including PA CareerLink staff members and VR counselors from the Pennsylvania Departments of Welfare, Aging, Corrections and Veteran Affairs.