Disseminating on-going surveillance data to public health professionals.

April 2001

Prepared by:

Anne Marie Parkinson

Community Health & Epidemiology

Department of Public Health Sciences

Faculty of Medicine

University of Toronto

The following people have contributed to this project:

Durham Region Health Department (DRHD)

Philippa Holowaty, Epidemiologist

Donna Reynolds, Associate Medical Officer of Health

Joanne Bradley

Angela Cooper-Brathwaite

Ross MacEachern

Patricia Main

Shawn Woods

Lori Ullius

Disseminating on-going surveillance data to public health professionals.

Abstract

The Rapid Risk Factor Surveillance System (RRFSS) is an on-going survey of adult attitudes and health behaviours to enable public health professionals to prioritize health issues, direct operational plans, measure effectiveness and monitor progress at the local level.

A data dissemination model is described that enables integration of data into public health program planning, implementation and evaluation. Qualitative research methods were used to obtain information related to data understanding, dissemination and utilization. Using existing frameworks, values, attitudes, access, supports, skills and settings were examined. The dissemination model, implementation process and evaluation plan were developed as part of the RRFSS initiative at Durham Region Health Department.

Introduction

The Rapid Risk Factor Surveillance System (RRFSS) is an on-going survey of adult attitudes and health behaviours to enable public health professionals to prioritize health issues, direct operational plans, measure effectiveness and monitor progress at the local level. A need was identified to create an environment where RRFSS data would be effectively used in programming.

Research-based practice

Evidence-based decision making is increasingly valued within public health. However effective incorporation of evidence into practice is complex and remains inconsistent [Coyle & Sokop (1990), Funk et al. (1991)]. Therefore, the RRFSS which enables public health professionals to base programming decisions on data, could not be more timely.

Literature directly related to effective incorporation of health surveillance data into practice is lacking, however the broader research utilization literature has described factors that were expected to be applicable to data utilization.

Research utilization is defined as the process by which relevant research is critically examined and applied to validate current practice or to change practice [Royle et al. (1996), McCurren (1995), Camiletti (1998)]. Many authors have described the mediating factors and barriers to implementing research findings into practice [Simpson (1996) , Funk et al. (1995a, 1995b), Stolk & Mayo (1995), Camiletti (1998), Cabana (1999, 2000), Carter et al. (1995), Parahoo (2000), Retsas, (2000)].

Strategies to increase research-based practice

There are many ways that have been identified that can increase an organization’s potential for research-based practice. These include: 1) Creating an environment where administrators show the value of evidence-based practice by example and by expectation, by formalizing the expectation for others through inclusion in staff responsibilities and performance appraisals [Funk et al. (1991, 1995a,1995b)]; 2)Enhancing administrative support and improving availability and accessibility to research findings [Funk et al. (1991), Parahoo (2000)]; 3) Supporting research utilization by providing mechanisms for colleague support through group activities [Funk et al. (1991, 1995b), Parahoo (2000)]; 4) Active dissemination of research findings through informing and educating health professionals about content; 5)Increasing time available for reviewing and implementing research findings [Funk et al. (1991), Parahoo (2000)]; 6) Involving health professionals in the selection of research topics [Burns et al. (1992), Anderson (1993), SIGN (1999)] and 7) Regular reminders relating directly to practice activities [Cabana (1999), Anderson (1993), SIGN (1999)]. These strategies relate to research utilization but may be applicable in promoting the integration of data into public health program planning, implementation and evaluation.

The barriers include a lack of positive attitude toward research, lack of time to read research and implement new ideas, lack of availability of pertinent research findings, resistance to change, limited access to journals, limited ability to understand and interpret research reports, limited exposure to strategies of how to use research, lack of administrative support and concerns about reallocation of staff time [Funk et al. (1995a, 1995b), Stolk and Mayo (1995) Camiletti (1998), Cabana (1999, 2000), Carter et al. (1995), Retsas (2000), Parahoo (2000) ].

It is important to acknowledge however that factors affecting research utilization may not all be directly relevant for data utilization, although the broader categories identified in the literature, that affect research utilization in practice, were expected to be transferable to the use of data for similar purposes. The barriers to research utilization have been classified in the literature into various categories. For the purpose of this paper, health data utilization is examined using a combination of existing research utilization frameworks in terms of values and attitude, access and support, skill, and setting as described by Camiletti et al. (1998) and Champion and Leach (1986, 1989).

Rapid Risk Factor Surveillance System

The RRFSS is an on-going survey of adult attitudes and health behaviours that are directly related to programming efforts of public health professionals. RRFSS enables public health professionals to prioritize health issues, direct operational plans, measure effectiveness and monitor progress at the local level. The surveillance system is a telephone survey that provides data suited for detecting temporal changes, seasonal variation, the effects of a program overtime and data that will allow for before-and-after comparisons when a program is initiated or when other potential impacts occur [MacNeill (1997)].

The questionnaire has three components: 1) core module, a core of questions asked by all participating health units in Ontario; 2) optional modules, sets of questions chosen by individual health units to meet specific needs and 3) health unit specific questions.

As part of the RRFSS initiative, the Epidemiology & Evaluation Unit at Durham Region Health Department, Whitby Ontario identified the need for a system of data dissemination that would assist program managers and staff to begin regularly integrating health data from RRFSS into the program planning, implementation and evaluation cycle. This report is aimed at exploring the research utilization culture of public health professionals and describing a data dissemination model that promotes utilization of data in public health practice.

Methodology

Qualitative research methods were used to explore program managers and staff’s attitude toward using on-going health risk factor surveillance data (RRFSS) in relation to program planning and evaluation, their perceived access to relevant health data and support for the implementation of findings. The stages of the design are shown in Figure 1.

Three focus groups were conducted, one with management and two with program staff. The management focus group participants were purposively selected as representatives from different Health Department program areas. These same managers also formed an Advisory Working Group to oversee the data dissemination and utilization model development.

For the staff focus groups, the health department was stratified by divisions and programs: Environmental Health, Health Promotion-Disease Prevention, Parenting, Injury Prevention, Clinical Services, and Communicable Disease Control and Prevention Program. Dental Health, Infant Development, and Emergency Medical Services were excluded due to availability. Administrative staff were also excluded. Focus group participants were then randomly selected without replacement from each program, proportionate to the number of FTE’s in that area.

The selection criteria for staff focus group participantswas defined as individuals who: i) Occupied a full-time position at Durham Public Health Department in a specific program; ii) Were available for participation on the focus group date; iii) Had management approval, if required.

Six managers and fourteen staff were selected for the focus groups; 5 managers and 11 staff were present. The job titles of focus group participants are presented in Table 1. The focus group questions used are illustrated in Table 2.

The qualitative data obtained in the focus groups was analyzed by hand using a method adapted from the Public Health Research, Education and Development (PHRED) Evaluation Tool Kit [Porteous et al. 1997]. All focus groups were tape-recorded and transcribed. The completed transcripts were read independently by researchers 1 and 2. The focus group results were analyzed inthe context of research utilization frameworks from the literature [Camiletti et al. (1998),Champion & Leach (1986,1989)]. Highlighters were used - a different colour for each of the following four categories that guided the analysis: 1) Value and attitude 2) access and support 3) skills and 4) setting. Value & Attitude refers to the perceived level of commitment to data utilization at the health department and individual’s feelings about incorporating data into program planning. Access & Support refers to perceptions of availability of information and perceptions of support in understanding and effectively using data. Skill refers to the ability to find, read, analyze and apply research to practice and Setting refers to the workplace environment, including time, access to technology and staffing. The highlighters were used to mark the comments that dealt with each category. Using the above categories, opinions, ideas, or feelings expressed in the focus groups were tallied and clustered into themes.

The Advisory Working Group of managers along with the researchers used the focus group results for the development of the data dissemination and utilization model. The model was developed using a logic model format adapted from the PHRED Evaluation Tool Kit [Porteous et al. (1997)].

The long-term outcome objective of the model was to ensure the use of community health status information in assessing local health needs and in the planning and evaluation of programs as set by the Ontario Ministry of Health & Long-term Care Mandatory Requirements and Standards for program planning and evaluation. The intermediate outcome objective was the foundation for this project, that RRFSS data would be used for program planning, implementation and/or evaluation, that is, evidence-based action at the program level. The themes identified in the focus groups were used to determine the activities required in the model to increase utilization of RRFSS data in the programming cycle and short-term outcome objectives formed the link between the activities or process objectives and the intermediateobjectives of data utilization.

Public Health Unit

5 Managers11 Program Staff

1 focus group 2 focus groups

Transcription of tapes
Independent analysis

Researchers 1 & 2

Figure 1: Research design

Table 1: Job title of participants

Program Staff
/
Managers
Focus group 1 Senior Public Health Inspector
Public Health Nurse-Parenting
Public Health Nurse-Reproductive
Health
Public Health Nurse-Injury
Prevention
Public Health Nurse-Clinical
Services
Focus group 2 Public Health Inspector-General
Program
Public Health Nurse-Health
Promotion Disease Prevention
Public Health Nurse-Health
Promotion Disease Prevention
Public Health Nurse –Injury
Prevention
Community Health Nurse–
Communicable Disease Control and
Prevention
Public Health Nurse – Parenting / Focus group 1 3 Managers-Public Health
Nursing & Nutrition
Program Coordinator- Public
Health Nurse
Manager – Environmental Health
Division

Table 2: Focus Group Questions

Topic:
How do we use health data for planning and evaluation, and how can the Epidemiology & Evaluation Unit make this data most available to you?
Questions:
1. What do you think about when I say data?
  1. What do you think the Health Department’s commitment is to using health
data for planning?
  1. How do you feel about using data, yourself, in your program planning?
4. How do you get health data at the moment?
  1. If this information were available from several places and in different forms, how would you most prefer to receive it?
  2. Once you have the data, do you feel comfortable using it?
  3. Would you like there to be regular discussions of RRFSS data and its implications for programming?
  4. How frequently should the surveillance data go out to you?

Results

The themes identified in the focus groups show that most participants valued data as part of the program planning, implementation and evaluation cycle, however, barriers such as lack of time, lack of access to computers, and a skill deficit in data interpretation and applying findings in practice compromise data utilization. The key findings from the focus groups that were applicable to disseminating RRFSS data are presented (Table 4).

There were no comments in the focus group discussions that fell outside the four predetermined categories, however this may have been due to the investigators previous knowledge of the research utilization literature preceding focus group question development. The focus group results were independently analysed by researchers 1 and 2 and minimal to no deviation in agreement was identified between the two analyses.

The data dissemination and utilization model summarizes the key elements of data dissemination, shows which activities are expected to lead to which outcomes and identifies the critical questions for evaluation [Porteous et al. 1997].The evaluation component of the dissemination process is embedded in the intermediate and short-term outcome objectives of the logic model. The intermediate outcome objective measures utilization of the RRFSS data and the short-term outcome objectives measure the outcomes from processes of dissemination that should lead to utilization. The supporting documentation for the model includes the tools necessary for evaluation of the RRFSS dissemination and utilization model.

The Question Feedback Form is the evaluation tool to measure utilization of the RRFSS data at the programming level. The question feedback form collects information on how the data has been used (intermediate outcome objective) and what actions (ie. program changes) have been made as a result of the data. The feedback form is also used to determine which questions are useful and should remain on the questionnaire.

The RRFSS Internal Evaluation Survey is the evaluation tool to measure the dissemination process. The survey measures the short-term outcome objectives of the data dissemination and utilization model. The data dissemination and utilization model with supporting documentation is presented in Appendix A.

1

Table 3:Focus Group Results

Categories
Value & Attitude
/
Access & Support
/
Skills
/
Setting
There is a positive attitude to using data and the value of data is increasing.
There is a desire to be evidence-based.
Data can ensure we are on target with program plans, otherwise changes may need to be made to plans. / Support from the Epidemiology & Evaluation Unit (E&E) is needed, to answer questions and clarify interpretation.
There is confusion on how to access E&E.
The intranet is easy to access but there must be a hard copy as well.
Results in relation to the objectives would be helpful. A text explanation is also needed / Staff competence with computers and looking for resources varies vastly.
Comfort level with computers and the intra/internet is also an issue.
Skills vary in data interpretation and how to apply it to programming in daily practice / There is a lack of computers and intra/internet access.
Lack of time.
RRFSS results need to be discussed at program meetings; it should be a standing item on the agenda, regular discussion is needed.

1

Discussion

The research utilization frameworks used to investigate data utilization were very useful to our investigation. The themes that we identified for data utilization within each category were similar to those described in the research utilization literature.

Most of the public health professionals in this study valued data as part of the program planning, implementation and evaluation cycle and expressed a desire to be more evidence-based. However, barriers such as lack of time, lack of access to research findings and a skill deficit in data interpretation and applying findings in practice compromise data utilization. Similar themes are reported by Rodgers (1994) and May (1998) who used semi-structured interviews to explore the research culture of practitioners. A number of surveys have attempted to determine the factors which affect research utilization among practitioners and also reported similar results [Hatcher (1997), Camiletti (1998), Funk et al. (1991) (1995a) (1995b), Retsas (2000), Parahoo (2000) ].

This suggests that for Durham Region Health Department, the barriers identified in the research utilization literature are interchangeable with the barriers identified in the utilization of data in program planning, implementation and evaluation. Therefore, the strategies that have been identified in the literature to increase an organization’s potential for research-based practice may also be useful in integrating data into public health programming.

Based upon the findings of the focus groups and the literature, the reasons why public heath professionals do or do not use health data for program planning, implementation and /or evaluation are complex and involve a web of individual and organizational factors. Therefore, an effective intervention to increase data utilization in practice must address both the individual and organizational factors.

The data dissemination and utilization model presented here addresses a number of individual and organizational barriers that were identified in the focus groups and that are consistent with the research utilization literature. The model: 1) acknowledges that implementing evidence into practice is an organizational rather than individual issue; 2) acknowledges the need for collaboration between those that use the data and those that generate it; 3) recognizes the skills needed to complete the cycle of combining evidence and practice; 4) recognizes the importance of disseminating data in a variety of ways; 5) acknowledges the need for access to findings; 6) recognizes the need for training and education in research i.e./ interpretation and analyses and 7) recognizes the need for regular discussion.