The Survey

In November 2014, following the conclusion of the one-year Bridge Funding grant period (10/01/2013-09/30/2014), a survey was sent to representatives of eight EBI programs that were offered intensive technical assistance (TA) from the EPISCenter during the grant period. TA recipients included five programs that received Bridge Funding and three programs that applied for but did not receive grants. Two MTFC programs that applied for grants but closed early in the TA process were not sent the survey.

The survey was sent to 28 individuals, 11 of whom responded to the survey. Respondents included:

9 representatives from 7 provider agencies (2 FFT programs and 5 MST programs), all of which participated in intensive TA. All five grantees were represented. Respondents included agency administrators, program directors, and clinical supervisors.

2 representatives of MST Services who participated in the TA process alongside providers.

The survey included sections on Impact and Participant Satisfaction.

Impact of Funding and EPISCenter Technical Assistance (TA)

The survey sections on impact were completed by 8 respondents from 6 agencies.[1]Five of the 6 programs represented received a Bridge Funding grant from PCCD.

The Big Picture: Results highlight the impact of TA in three key areas: Program Utilization, Stakeholder Engagement, and Sustainability Planning. In general, providers reported improvement in all threeareas as well as increases inprogram sustainability.

Five of six providers rated their programs as “very likely” to continue operating over the next year.

Half of providers (3 of 6) reported that their programs are operating at a higher level, compared to before the grant period began, while the other half reported the program was operating at the same level.

Three of five providers (60%) reported their programs broke even or made a profit in Q1 of FY 2014/2015. In the case of financial loss, one provider reported, “Growth and expansion did not really start to occur until late summer…there had been no payoff until much later in the quarter.”

Across domains, lower ratings were provided by the two grantees that have not yet met their TA goals and continue to experience challenges related to stakeholder collaboration and funding. EPISCenter support to these sites is ongoing.

Bridge funding grants and the associated technical assistance provided to sites had a significant, positive impact on sustainability. The majority of providers reported improvements in utilization, stakeholder engagement, and program finances. Grantees were able to serve 588 youth during the grant period.

Impact of Grant

Five programs represented in the survey received Bridge Funding grants from PCCD. Grants were most commonly used for therapist salaries (4 programs), following by training costs (2 programs) and developer fees for licensing and consultant (2 programs).

“Our FFT program would not have been able to continue without the bridge the gap funding. The funds helped our site send (a new supervisor) to Externship as well as supervisor training and helped cover the fees for FFT certification based on our phase of implementation.”

“(The grant) allowed us the opportunity to stabilize the program with our current staff and make a plan for when to add a new worker without worry of losing the program.”

“The assistance provided a cushion to support the sustainability efforts in (two counties) and provided opportunities to generalize the successes...to additional counties. The funding provided assistance in (staff) recruitment efforts, which was one of the top factors impacting sustainability.”

Sustainability Planning

Having a clear plan is an important element of program sustainability. All six providers reported that EPISCenter assisted them with developing or revising their sustainability plan. The most common areas in which providers not only developed but also executed plans included:

Identifying key stakeholders who might support the program (4 providers)

A fiscal plan outlining funds needed to sustain the program (4 providers)

Internal agency changes to support quality implementation (4 providers)

Internal agency changes to make program operation more fiscally sound (4 providers)

It is worth noting that the providers who did not report executing plans in most of these areas also had the lowest average ratings for changes in stakeholder collaboration and support and impact on referrals.

Program Utilization

Five of the 6 providers (83%)reported that increasing utilization (number of referrals) was one of their TA goals. Sustainability is impacted by program utilization, as well as the match between team size (staffing) and referrals.

Four out of five providers reported their teams were now well matched to the rate of referrals. Provider responses indicated various ways this occurred:

  • By downsizing the team to match existing referrals (1 provider)
  • True growth as indicated by increasing referrals, cases served, and team size (2 providers)
  • By increasing referrals without changing team size (1 provider; Note that in this case the goal of being able to support a larger team has not yet been met)

One provider reported a decrease in referrals, resulting in low utilization of the existing team and the goal of increasing referrals being still unmet.

“The efforts toward marketing, utilization, retention/recruitment, and county assistance office collaboration have led to an additional team covering Washington county again.”

“Referrals are so unpredictable it is difficult to find a trend and make program decisions. Referral sources surveyed are pleased with the program but as need varies they are not always consistently referring to the program.”

“Program has grown significantly. Increased staff from 2 to 4 due to demand from County.”

Stakeholder Engagement

For most providers, a critical part of the TA process was engaging local stakeholders to address issues related to buy-in, referrals, and/or funding.

On average, providers reported that their relationships with 6 out of 10 different stakeholders somewhat or greatly improved during the grant period. Only one provider reported a relationship (with one stakeholder) had worsened, which is related to staff turnover in the county office and possibly issues between the provider and county. The provider continues to work on the relationship and building engagement.

In many instances, provider reported that outreach to local stakeholders, either by their agency or the EPISCenter, had a positive impact on referrals. The systems for which providers reported the greatest improvement in collaboration are also the systems for which increased referrals were most likely:

Juvenile Probation – Increased referrals reported by 5 of 6 providers

Child Welfare – Increased referrals reported by 3 of 6 providers

Schools – Increased referrals reported by 3 of 6 providers

Four providers also reported that referrals from other treatment providers had increased.

PARTICIPANT SATISFACTION with EPISCENTER TA

The Satisfaction section of the survey was completed by 11 respondents, representing seven provider agencies and MST Services.

Ratings of Technical Assistance

Respondents were asked to rate their opinion on 14 items related to EPISCenter TA. These included items regarding whether the TA was practical, accessible, and convenient among several other aspects. The average on each item ranged from 4.0 to 4.67, the average across all items was 4.36 - indicating agreement or strong agreement with quality and acceptability of TA. Below are a few highlights.

Value of EPISCenter TA and Resources

All respondents indicated that during the past year they used EPISCenter resources on a quarterly to monthly basis and expected they would continue to do so in the coming year.

Three out of four providers reported that EPISCenter facilitation of meetings or conversations with local stakeholders was helpful.

Five out of six providers reported that EPISCenter suggestions, resources, or guidance for provider outreach to stakeholders was helpful.

Does EPISCenter Offer Something Different?

All respondents reported that EPISCenter support differs from the support received from other sources.

“EPISCenter provides support and resources in the form of data collection and organization of reports that we can utilize when meeting with local and state stakeholders…”

“’(EPISCenter provides) ‘lessons learned’ resources which would not be available as readily without the EPISCenter as a resource, insight into the landscape in PA, a more global outlook based on their contacts across the state.”

“Supports with contacting counties and gathering information for us to utilize data to best support the counties we work with.”

“I appreciate the opportunity to join with other MST Providers in the State to network, get ideas, talk about problems and solutions during Provider meetings.”

“Very helpful with ideas or suggestions to maintain staff and referrals.”

“Problem solving on a state and county level.”

[1]Survey results are generally reported below by provider. Where there are multiple respondents from the same agency and differences in reported impact, the less desirable response or the response from the most knowledgeable respondent is counted.