ISM Core Function Teams Assessment Plan

January 2008

Appendix 14–Core Function #5 - Feedback and Continuous Improvement
Recommended Activities and Lines of Inquiry

General to the Assessment

  • Review the JLab ISM Website and provide feedback on content
  • Review the JLab ISMS Program Description and provide feedback on content

Document Reviews

  • Are documents being used the most current version?
  • Are forms being used the most current version and are they filled out properly?
  • Is collected data dispositioned in accordance with the procedure or applicable program document?
  • Are there conflicting requirements between procedures and program documents applicable to the activity being reviewed?

Observing the Conduct of Activities

  • Attend pre-job briefing
  • Review work control document applicable to activity being performed
  • Is it clear which document(s) or procedure(s) is applicable to the activity being conducted?
  • Can the work control document be followed as written?
  • If not, is the activity stopped until the work control document is revised (if necessary)?
  • Is authorization received prior to commencing conduct of activity?
  • Are all hazards addressed in the work control document or hazard analysis document?
  • Are hazard controls specified in the work control document being followed?
  • Record observations and questions and verify facts either during the observation or after with the goal of being able to make a positive statement about what was observed in the report.

Interviews

  • Strive to ask open ended questions rather than yes/no questions
  • Conduct interviews either before observing activities, after observing activities, or both

Assessment Report

  • While reviewing documents, observing activities, and conducting interviews keep in mind the information that will be needed to make a concise and complete statement in the report related to the objectives of the assessment focus area

Core Function Specific

From the U.S. DOE, Office of Independent Oversight, Criteria Review and Approval Document - Work Planning and Control Implementation Inspection Criteria, Approach, and Lines of Inquiry (11/26/07)

Inspection Criteria: Line management has effectively developed and implemented a feedbackand improvement process at the work activity level.

Inspection Activity: Interview facility managers, project managers,and facility organizations, workers, ES&H or other appropriate personnel.

Inspection Activity: Review work planning and control processes and procedures, workpackages, and assessment/oversight/feedback activity documentation. Interview personnelincluding project personnel, subject matter experts, managers, work control managers, foremen,supervisors, environment, safety, and health support personnel, and operations/technicianpersonnel.

Inspection Activity: Observe work, including pre- and post-activity briefs and reviews. Reviewevidence of feedback and improvement such as post job reviews and other correspondence usedto elicit feedback and improve performance.

Inspection Lines of Inquiry:

  • Are formal post-activity review processes (e.g., post-job reviews, operations reviews)established and effectively used?
  • Do subject matter experts, workers, supervisors, and line managersrecognize, report, evaluate, and address accidents, incidents, near misses, injuries, illnesses,exposures and opportunities for improvement in a timely manner and in accordance withestablished procedures?
  • Is feedback from workers effectively solicited and used during work planning, execution, andcloseout?
  • Is worker participation in safety programs (e.g., behavior based safety, safety committees)encouraged and effective?
  • Are lessons learned identified and incorporated into the work planning and authorization process?
  • Do assessment activities by line oversight include observation of workactivities by managers, supervisors, and subject matter experts?
  • Are deficiencies and weaknesses identified during work activities appropriately documentedand managed in accordance with site issues management processes? Are associatedcorrective actions developed and implemented as required?
  • Have findings related to work planning and control from previous Independent Oversightassessments been effectively corrected?
  • For issues identified by the current inspection, what prevented contractor lineoversight activities from identifying and correcting the problems?

Potential assessment activities applicable to this core function

Partial List of JLab Documents/Processes Applicable to this Core Function

  • Daily Planning Meetings
  • Peer Reviews
  • FEList Comments
  • FLOG
  • Weekly Meetings
  • On-Line Forum
  • Various E-Logs
  • Pre-Job Meetings
  • Post Job Closeout
  • Safety Warden Area Inspections
  • ESH&Q Self Assessments
  • STOP
  • CATS
  • ORPS
  • Work Observations
  • Worker Safety Committee
  • Director’s Safety Council
  • INSIGHT Webpage
  • DOE Corporate Lessons Learned

Evaluate whether or not key functions specified by the Independent Assessment Procedure are being performed in accordance with procedure requirements and the effectiveness of these processes to include:

  • The Integrated Assessment Schedule
  • Is a trend analysis on assessment-identified items being conducted quarterly?
  • Are the results of the trend analysis being reported to the Director’s Safety Council?
  • Is an evaluation of the Independent Assessment and Management Assessment programs being conducted semi-annually and reported to the Director’s Safety Council?

Are IA reports being made available on the web?

Review key elements of the process to document, track, and complete identified Corrective Actions. This may include:

  • Determine whether or not all corrective actions for the Lab are tracked in one central system or if there are multiple systems in use
  • If multiple systems are being used, determine if a method is in place to perform collective trending analysis of all identified issues and the resultant corrective actions for all Significance Levels (1-4)

Interviews – Potential Questions

Q: Which procedures / mechanisms are in place to examine the findings of internal and external assessments? / A:
Q: Are the results of the lessons learned reviewed by management? / A: Yes, in fact we have a new procedure that more formally regulates how this system works. Both internal and external lessons learned are routinely collected and disseminated throughout the lab. If they apply to the work we do here, and can help us improve how we operate, we use these lessons learned to make changes. The Director’s Safety Council is routinely briefed on the results of this process.
Q: Do committee meetings provide effective feedback? / A: Yes, the committee members routinely report back to the employees in their home organizations. They also provide a focal point for lab wide efforts where the floor workers need to be involved, such as the Workers Safety Committee involvement in the ISM upgrade program.

Appendix 141

CF#5 Lines of Inquiry Page 1