Intervention Effectiveness Research -
Engineering Design of Safety and Health Programs by Optimizing Intervention Activity
Joel M. Haight, Ph.D., P.E., CSP, CIH
Assistant Professor
Penn State University
University Park, PA 16802
Abstract
Safety and health programs are often implemented without a quantified design. The objective of this study was to determine if a safety and health program could be quantified and optimized through a design that minimizes incident rates and the percentage of available human resources required to implement it. A model was developed for this purpose.
In a two-phase empirical study, an oil production operation was analyzed. Four categories of interventions were studied 1) behavior modification, incentives, awareness; 2) training; 3) job/procedural design; 4) equipment. The percentage of available time spent implementing interventions in these categories was the independent variable and the incident rate was the dependent variable. Findings show a mathematical relationship between interventions and incident rate. The resulting best-fit equation is an intuitively expected exponential function showing a decreasing incident rate with an increasing intervention application rate. This model can be used to analyze the mathematical function for a minimized incident rate, aiding in design of optimized safety and health programs.
In the second phase, an attempt was made to determine whether a designed safety and health program could be optimized to minimize the loss producing incident rate. In this phase, the objective was to use the mathematical function developed in phase 1 to formulate a model that calculates a minimized incident rate. Evaluating 81 application rate level combinations of four intervention categories and subjecting them to management constraints accomplished this. The resulting model provided insight into the design of a safety and health program that prescribes the appropriate amount of human resource time that should be assigned to specific safety and health intervention activity.
Actual verification data were used to test the adequacy and accuracy of the optimization model. Findings indicate that the model predicts with reasonable accuracy, intuitively expected results. The model shows promise as a potential tool to aid safety and health engineers in designing their safety and health programs. It also shows promise in providing some predictability of safety and health performance under given safety and health program designs (Haight, Thomas, Smith, Bulfin and Hopkins, 2001).
Introduction
Workplace injuries and property damage—and the safety programs designed to prevent them—are expensive facets of contemporary industrial, agricultural, and military activities. Indeed, the National Safety Council estimates that the cost of work place injuries totaled $128 billion in 1999, a value approximately equal to the combined profits of the top 30 Fortune 500 corporations (National Safety Council Website). Optimizing intervention strategies to decrease rates of injury and property damage with less costly safety programs would contribute to improved productivity and economic vitality in all activities that involve such risks. The results could lead to improved safety practices and improved profitability in American industry.
It is difficult to predict the future. Even in some of today’s more established sciences, we have trouble predicting what might happen tomorrow and beyond. What will tomorrow’s weather be? What will the stock market do next week or next year? Who will win the NCAA football championship next year? It is even more difficult to predict the future in less established sciences. What will your injury rate be next year? What is your target rate or goal? How will you know? How will you get there? How much manpower will you need (Figure 1)? These are questions that many in the safety engineering profession, would like to be able to answer scientifically.
Figure 1. Available Human Resource Scale (How much human resource will be required?)
Many organizations whose activities involve the risk of injury or destruction of property commit human and financial resources to intervention activities intended to prevent injuries, fires, spills, chemical releases, etc. Many of these “safety-related” interventions, including activities such as safety training and facility inspections, are commonplace enough that, often, little attempt is made to determine whether they are effective, whether they are at optimum levels and whether they might be improved. A model has been developed to help us determine which intervention activities are working to prevent or reduce incidents and to what extent they are working (Haight, et. al., 2001). From this model, we can determine the right “recipe” or correct design for our safety or loss prevention programs (Rinefort, 1977) in terms of usage of human resource power.
The Model
The basis for the model is a mathematical approach to designing an effective safety program developed by Haight, et. al. 2001. The model proposes the rate of incidents be I and Ai, i = 1, 2 … N, be the expenditure rate of human resources in intervention activities. The original model relating I and the Ai is a non- linear relationship:
(1)
in which the p’s are parameters controlling the application of the various intervention activities.
After being determined empirically through statistical analysis of safety activities in an oil and gas production operation, the mathematical relationship was used to design an optimum safety intervention program that minimized expenditure of available human resource time in intervention activities while still minimizing incidents. The model thus represents a safety and health system (Figure 2). (Haight, et. al. 2001)
There is currently a research effort in progress with a Canadian power company to further prove the model and give it wider applicability by extending it to other industries and government organizations. It is expected that as the database size increases and as more industries are represented, model results may become generalizable across other industries.
INPUT (Independent) OUTPUT (Dependent)
Intervention Application Rate Incident Rate
Figure 2. This is a representation of the Safety and Health System Model (Haight et. al. 2001).
In the application of the model, study participants record time spent by employees implementing four categories of interventions and the weekly rate of incidents. The analysis relies on data recorded by the study participants without intervention by the researchers.
The incident rate is the dependent variable and the intervention application rates for four categories of intervention activity are the independent variables. All variables are normalized by worker hours. The observed data thus compare the fraction of available worker-hours applied to implementing interventions to the rate at which incidents occur. The data analysis process takes account of the individual and interactive effects of four main intervention activities, producing 15 independent variables whose contributions are isolated through regression techniques.
The effects of safety intervention activity are neither instantaneous nor permanent. The time-delay and carry-over effects are detected using moving average and exponential smoothing techniques to identify changes in statistical relationships between the dependent and independent variables over the course of the forward projected averaging and smoothing periods.
The resulting empirical version of Equation 1 for the particular firm being studied becomes the objective function in a mathematical programming model that can be used to determine optimum strategies for obtaining minimum incidents at a reduced cost of intervention. If such a strategy is adopted by the firm on which the model is based, then subsequent observations either validate the model or reveal new aspects of that safety program that must be investigated.
The initial application of the model to an oil production operation showed that a strong mathematical relationship exists between the independent and dependent variables and that the function was optimizable using operations research-based mathematical modeling techniques. Post analysis observation of incident rate data indicates that the lowest injury and incident rates occurred when the organization’s safety program was operating in the optimum range of human resource commitment (Haight, et. al. 2001).
Discussion of a Completed Study
A previous two-phase study on intervention effectiveness was done in 1998/1999 in an oil and gas production operation in Central Asia (Haight, et. al. 2001).
Safety programs are often implemented without a quantified design. Research has now shown that a loss prevention system can be quantified, designed, and therefore, optimized as any engineering-based system. In support of this effort, a statistically significant mathematical relationship was shown between the intervention activity implemented to reduce the incident rate (independent variable) and the incident rate itself (dependent variable). Many literature studies are available evaluating the effectiveness of individual intervention activities (examples are Fellner and Sulzer-Azaroff, (1984); McKelvey, Engen and Peck, (1973); Kalsher, Geller, Clark, and, Lehman (1984)). Building on these studies, this initial research evaluated a complete loss prevention system exploring all main effects from a comprehensive set of interventions as well as the interactive effects between interventions. The study integrated all components of a defined loss prevention system in order to establish a mathematical relationship that would allow for the design and optimization of a complete loss prevention program (Haight, et. al. 2001).
Experimental Method and Design of the Original Study
During Phase 1 of the study, the subject organization operated with 130 employees. Collectively, these employees worked approximately 5500 hours per week. For 26 weeks, the employees tracked and reported the amount of time they spent implementing four categories of interventions and the resulting weekly incident rate (both the traditional and total incident rates). “Traditional incident rates” included spills, fires, injuries, toxic releases, etc., while “total incident rates” included the traditional incidents as well as unplanned process upsets or shutdowns and equipment damage, etc. Reported data were used for the research. The researcher did not intervene in the implementation of the program (Haight, et. al. 2001).
The independent variables were quantified each week using the amount of man-hours applied by the work group of 130 employees to the defined intervention activities. These hours were normalized as percentage of total available hours, referred to as the “intervention application rate”. The dependent variable was developed by recording the number of incidents that occurred during each week, multiplying it by 200,000 hours and dividing it by the number of hours worked by the 130 employees, yielding the “incident rate.” This was done for both traditional incidents and total incidents (Haight, et. al., 2001).
Analysis and Results
The combined 26 weeks of intervention application rate and incident rate data were recorded for the four factors (main effects) and the two incident rates. The cross multiplication products of the main effects to account for interactive effects between factors were computed. To integrate all interactive effects from the two- , three- , and four-factor interactions, 15 independent variables resulted. A representation of the spreadsheet is shown in figure 3.
Figure 3. This is a representation of data collection and totaling spreadsheet (Haight, et. al. 2001).
Week / Factor A / Factor B / Factor C / Factor D / AxB / nxn / AxBxC / nxnxn / AxBxCxD / Traditional Incident Rate / Total Incident Rate1 / Xa1 / Xb1 / Xc1 / Xd1 / Xab1 / . / Xabc1 / . / . / Ytr1 / Yt1
2 / . / . / . / . / . / . / . / . / . / .
n / Xan / Xbn / Xcn / Xdn / Xabn / . / Xabcn / . / . / Ytrn / Ytn
To determine a best-fit function for these data, several regression analyses were carried out using the least squares method, considering both linear and non-linear fits. To evaluate how long the effect of an intervention lasts, weekly data points were used. To calculate two- , three- , four- , five- , and six-week effects, forward projected moving average and exponential smoothing techniques were applied. From this, one can determine if any effect from week one carries over to week two, three, four, etc., by assessing the quality of the regression fit. The forward-projected exponential smoothing equation was adapted from Elsayed and Boucher (1994): (Haight, et. al. 2001)
Xt = aXt + a(1-a)*Xt+1 + a(1-a) 2*Xt+2+a(1-a) t+2*X1 + (1-a) t *X0
Discussion of Results
Analysis showed that the best fit occurs when regressing the four-week moving average model and traditional incident rate. The resulting function was exponential, with an R2 = 0.982209, F0 = 25.764 vs. Fa=.01,15,7 = 6.31, and Mean Square Error (MSE)= 0.78069. A strong fit also occurs in:
1. The non-linear four-week exponential smoothing model for the traditional incident rate with an R2 = 0.978429, F0 = 21.16714 vs. Fa=.01,15,7 = 6.31, and an MSE = 0.949857, and;
2. The non-linear five-week moving average and exponential smoothing, traditional incident rate cases, with R2 values = 0.963596 and 0.968286, F0 = 10.587, and 12.21272 vs. Fa=.01,15,6 = 7.56, and MSE values = 1.01647 and 0.89773 respectively.
Figure 4 shows graphically the total intervention application rate curve. As demonstrated, the exponential trend line is fit to the total intervention application rate, and it still generates an R2 value of 0.5317 without all the interactive effects shown. As was noted above, with all interactive effects and variables included, the R2 is 0.98209.
Figure 4. This graph shows the total percentage of available man-hours vs. traditional incident rate, with the exponential function shown (Haight, et. al. 2001)
The resulting function for the four-week case linear transformed function is shown in Figure 5.
Figure 5. This mathematical function shows the relationship between the incident rate and all 15 regressor variables
LnY=Xabcd*lnmabcd+Xbcd*lnmbcd+Xacd*lnmacd+Xabd*lnmabd+Xabc*lnmabc+Xcd*lnmcd+Xbd*lnmbd+Xbc*lnmbc+Xad*lnmad+Xac*lnmac+Xab*lnmab+Xd*lnmd+Xc*lnmc+Xb*lnmb+Xa*lnma+b
LnY=Xabcd*(ln0.00188)+Xbcd*(ln3.826E+16)+Xacd*ln(2014.943)+Xabd*ln(6.966)+Xabc*ln(4.2998E+12)+Xcd*ln(1.85E-13)+Xbd*ln(.001597)+Xbc*ln(7.01E-65)+Xad*ln(0.150024)+Xac*ln(5.82E-11)+Xab*ln(.000517)+Xd*ln(404.4604)+Xc*ln(1.063E+45)+Xb*ln(449.5E+7)+Xa*ln(526.9246)+1.03E-7.
Where Y is the incident rate, Xi is the individual intervention values for each of the 15 factors, and mi is the slope at each represented point on the curve. The function shown here contains actual m value (Haight, et. al. 2001)
Introduction - Phase 2
In the second phase, an attempt was made to determine whether a designed loss prevention program could be optimized to minimize the loss producing incident rate while minimizing the intervention application rate. The primary objective was to use the phase 1 mathematical function to formulate a model that calculates a minimized incident rate. Evaluating 81 application rate level combinations of the four intervention categories and subjecting them to management constraints accomplished this. A theoretical minimum incident rate could be achieved evaluating the objective function, using the 81 different combinations of the four categories of intervention activities. The resulting model provided insight into the design of a safety or loss prevention program that will prescribe an appropriate amount of human resource time that should be assigned to safety intervention activity.