Principles of rechecking and sample size_High level

PRINCIPLES OF RECHECKING AND SAMPLE SIZE DETERMINATION (National and TOT Level)

Purpose: / To provide participants with an understanding of the principles of rechecking and sample size determination
Module Time: / 3 hours
Learning objectives: / At the end of this module, participants will be able to:
  • Explain the background of rechecking
  • Explain the prerequisites for rechecking
  • Describe the sample concepts
  • Describe the sample size parameters
  • Explain the influence of the prevalence of positive slides on the test performance
  • Explain determination of the sample size
  • List conditions for stratified sampling

Module Overview

Step / Time / Activity/
Method / Content / Resources
Needed
1 / 15min / Presentation / Background of rechecking / Slides 3 - 9
2 / 1 hour 40 min / Presentation / LOT Quality Assurance Sampling (LQAS) andsample size determination / Slides 10 - 43
5 / 5 min / Summary / Key messages / Slide 44
6 / 1 hour / Exercise / Sample size determination / Exercise notes; exercise file.

Material/Equipment Checklists

  • PowerPoint slides
  • Computer w/LCD projector
  • Calculators (ordinary)
  • Handouts:
  • Exercise: Sample Size Determination («Rechecking_Exercise sample size determination.doc»)
  • Exercise answers: Sample Size Determination (“Rechecking_Answers sample size determination.doc”)
  • LQAS tables(Appendix D4, “External Quality Assessment for AFB Smear Microscopy” guidelines)
  • Flipchart

Teaching Guide

Slide Number / Teaching Points
1 / Module: Principles of Rechecking and Sample Size
DISPLAY this slide before you begin the module. Make sure participants are aware of the transition into a new module.
2Flipchart
/ Content Overview
(Suggested technique for presentation)
WRITE the content outline before beginning this session.
REFER to flipchart frequently to orient participants to where they are in the module.
EXPLAIN that these are the topics that will be covered in this module.
3 / Aim Of Rechecking
STATE the purpose of rechecking is to screen for possible problems in AFB microscopy based on a sample of smears examined in routine work.
EXPLAIN that rechecking is not feasible for the confirmation of a diagnosis of an individual patient and this is not the primary aim of this EQA method.
STRESS that high quality AFB microscopy services are highly important for the management of patients. The principal aim of rechecking is to improve the quality of AFB microscopy services by identification of microscopy centers with unacceptable performance.
4 / Principles Of Rechecking (1)
CUSTOMISE this slide by stating the exact designation of the staff and level of the service responsible for various tasks in your programme.
EXPLAIN the principles of rechecking in a simple way: a random and representative number of slides from every laboratory are rechecked blindly (without knowledge of the peripheral result) at a higher level. Discordant smears are again checked by a second controller. Errors can thus be assigned to the laboratory or to the first controller,andthe work of the first controller can also be evaluated.
5 / Principles of Rechecking (2)
STATE that analysis is meant to identify laboratories that may be performing below standard.
STRESS that not all labs that have errors may perform below standard, so an in-depth analysis has to be made according to pre-set criteria to identify those which are more likely to do so.These will be the labs with the poorest performance, and they need a problem-identification and –solving visit by an expert. Labs with only slightly too many errors may not have a real problem, or only a minor one such as slightly superficial reading. These labs will be followed-up by further rechecking, and visits will be made only if the excessive number of errors persists. The aim of the visits must be to probe into the causes of the errors, and it should result in solving them so as to prevent further errors.
6 / Rechecking Merits
STATE the message on the slide.
7 / Rechecking Disadvantages
STRESS that rechecking can not be done properly in the absence of a well established national TB control program and a highly functional laboratory network. There should be personnel, materials and finances available to implement rechecking properly.
8 / Inputs for Rechecking (1)
CUSTOMISE this slide by stating the exact designation of the staff and level of the service responsible for various tasks in your programme.
STRESSthe needs for sufficient manpower for rechecking activities as stated on the slide.
INFORM that the task analysis of the different levels of the microscopy network is presented in the Appendix 1 to this Module (handouts).
9 / Inputs for Rechecking (2)
DESCRIBE material and financial inputs, practical arrangements to be considered when implementing a rechecking program, as stated on the slide.
10 / Sampling Principles
EXPLAIN that the sample should be representative and random. But technical difficulties should also be taken into consideration. Striving for statistical precision usuallyresultsin large numbers of slides to be collected and rechecked and therefore in overloading of controllers, compromising correct execution. There should be a balance between statistical and technical accuracy; hence it is much better to select the smallest possible sample size.
11 / Sample Size Concepts (1)
EXPLAIN that if the sample is to be truly representative, it must be possible to select any examined slide, irrespective of its result and type (whatever it is a diagnostic or a follow-up smear).
EXPLAIN that smears with scanty results (1-9 AFB / 100 HPF) have a special status. It is both easy to miss them and difficult to confirm and thus statistically accurate evaluation is limited.
12 / Sample Size Concepts (2)
STATE the Lot Quality Assurance Sampling (LQAS) is a statistically based sampling method used for rechecking.
EXPLAIN that LQAS is a one-sided test with wide confidence limits except for large areas. If e.g. 3% false negatives are found in a sample of 90 slides per microscopy centre, the confidence limits are 0.8%-9.1% if used for one centre, 2.1%-4.2% if used as the average of 12 centers and 2.9% to 3.1% if used as the average of 850 centers.
13 / Sample Size Concepts (3)
CONTINUE describing sample size concepts as stated on the slide.
STRESS that LQAS is applied for negative slides.
14 / Sample Size Concepts (4)
EXPLAIN considerations with regard to false-positives and false-negative results as stated on the slide.
MAKE SURE participants understand abbreviations used (HFN, LFN, etc.)
15 / Sample Size Concepts (5)
EXPLAIN that any technician, however good, will, from time to time, make a false negative error, and these low rates must be distinguished from an excessive rate due to underlying problems. This needs a statistical system.
This is not so for false positives (except low scanty, reproducibility problem): clear false positives should not happen at all, and are always significant. No statistics are needed. And we are only interested in more frequent false positives due to a problem that must be solved: these can be found easily from small samples.
For convenience and direct comparability, positives and scanty smears are sampled at their frequency in each lab register, by selecting a single random sample regardless of the result recorded. Sample sizes needed for negatives have been inflated to accomodate these positive/scanty smears.
16 / Sample Size System
STATE the message on the slide.
17 / LQAS Parameters (1)
EXPLAIN that a number of parameters are used to determine a sample size based on LQAS method.
18 / LQAS Parameters (2)
STATE the message on the slide.
19 / Influence Of Smear Positive Prevalence On False Negatives (1)
GIVE an example of the influence of the smear positive prevalence on FN. The tables show that in case of equal sensitivity and specificity, there will be more FN when the prevalence of positives is 10% as compared with a prevalence of 5 % (there will be 2.8% and 1.3% false-negatives, correspondingly). DEMONSTRATE all relevant calculations, if needed.
20 / Influence Of a 10% And 20% Prevalence On FN
CONTINUE explaining the influence of smear positive prevalence on false-negatives as graphically shown on the slide.
21 / LQAS Parameters (3)
STATE the message on the slide.
22 / Sample Size Determination (1)
CUSTOMISE this slide by specifying which values for the parameters decisive for sample size (d, relative sensitivity) have been retained by your program.
STRESS thatto make technically correct rechecking possible, d is usually set at zero, and the critical value of relative sensitivity kept low. Explain that this does not mean that in routine work no false negative errors are allowed at all. And also that this low critical relative sensitivity is not to be seen as a target for performance, but just as a warning level. Every laboratory performing below this level will be considered as having too many errors.
23 / Sample Size Determination Example (1)
DESCRIBE Step 1 of the sample size determination based on the example provided on the slide.
24 / Sample Size Determination Example (2)
DESCRIBE Step 2 of the sample size determination based on the example provided on the slide.
25 / Sample Size Determination Example (3)
DESCRIBE Step 3 of the sample size determination based on the example provided on the slide.
26 / Sample Size Determination Example (4)
DESCRIBE Step 4 of the sample size determination based on the example provided on the slide.
27 / Sample Size Determination Example (5)
DESCRIBE Step 5 of the sample size determination based on the example provided on the slide.
28 / Sample Size Determination Example (6)
DESCRIBE Step 6 of the sample size determination based on the example provided on the slide.
29 / Sample Size Determination Example (7)
EXPLAIN how to use the LQAS sample size tables.
STRESS that the sample size only slightly increases with increased workload above 1000 negative slides per year. The more important variable is the SPR.
30 / Sample Size Determination (2)
DESCRIBE an alternative method of the sample size determination – per microscopy center, as stated on the slide.
31 / Sample Size Determination (3)
STATE that it is recommended to have a uniform sample size for all centers. PROVIDE explanations as stated on the slide.
32 / Comparison of Sample Sizes: Old and LQAS Method
STATE this slide shows a comparison of sample sizes using the in the past frequently applied sampling of all positives +10% of negatives, versus LQAS samples sizes taken from the LQAS samplesize tables.
STRESS that LQAS samples are smaller except if positivity rates are very low or laboratories have very low workload. All three laboratories in this example have either high positivity rates (first one)or high workload (other two).
STRESS also that the samplesize required for laboratories with high workload can be very small with LQAS, and more slides are not required to detect poor performance due to problems. However, with low positivity rates more slides need to be rechecked according to LQAS, and if also the workload is low, the samples will represent a large proportion of total smears performed.
33 / Stratified Sampling (1)
STATE that stratified sampling could be a solution for medium prevalence situations (SPR is around 5%), but it can not be used when almost no positives are detected.
34 / Stratified Sampling (2)
EXPLAIN the rationale for the stratified sampling as stated on the slide. EXPLAIN why follow-up smears are given more attention.
35 / Stratified Sampling (3)
CONTINUE explaining the stratified sampling approach as stated on the slide.
36 / Stratified Sampling (3)
STATE in centers with less than 40 follow-up smears annually and less than 3% SPR, it is better not to attempt rechecking. Instead, it is recommended to concentrate on panel testing and intensified supervision.
37 / Stratified Sampling (4)
EXPLAIN that objective and comprehensive analysis is possible along the principles explained in the module: “Reporting and analysis of AFB-smear laboratories performance, interpretation and feedback”.
38 / Conduct of Rechecking (1)
EXPLAIN that here just the requirements are listed. Details are discussed in the module: “Rechecking procedures”.
39 / Conduct of Rechecking (2)
CUSTOMISE this slide by stating the exact designation of the staff and level of the service responsible for various tasks in your program.
STATE implementation of a rechecking program requires collaboration between different levels. Some functions of different levels can be combined.
40 / Conduct of Rechecking (3)
CUSTOMISE this slide by stating the exact designation of the staff and level of the service responsible for various tasks in your program.
EXPLAIN that the supervisor can be a NTP supervisor, a TB supervisor or a general supervisor, depending on the situation in the country. It will usually not be a laboratory professional. The last bullet implies that the supervisors can identify and solve minor practical problems, such as poor staining technique or poor microscope and can identify problems that require a visit by a laboratory professional.
41 / Conduct of Rechecking (4)
CUSTOMISE this slide by stating the exact designation of the staff and level of the service responsible for various tasks in your program.
LIST responsibilities of theintermediate level rechecking coordinator as stated on the slide.
42 / Conduct of Rechecking (5)
CUSTOMISE this slide by stating the exact designation of the staff and level of the service responsible for various tasks in your program.
CONTINUE listing responsibilities of theintermediate level rechecking coordinator as stated on the slide.
43 / Conduct of Rechecking (6)
CUSTOMISE this slide by stating the exact designation of the staff and level of the service responsible for various tasks in your program.
LIST the responsibilities of the high level coordinator as stated on the slide.
44 / Key Messages
STATE the message on the slide. ANSWER any questions the participants may have.
Exercise / MOVE to the exercise section below as well as to the Exercise files.

NOTES ON THE EXERCISE “SAMPLE SIZE DETERMINATION”

The exercise “Sample Size Determination” is designed to strengthen participants’ knowledge on LQAS sampling of AFB smears for rechecking and to practice their skills on sample size determination and planning of controllers’ workload with regard to various practical situations. The exercise requires availability of ordinary calculators and LQAS tables (Appendix D4 of the “External Quality Assessment for AFB Smear Microscopy” guidelines) for each participant.The exercise consists of 11 questions and it is recommended that the questions are discussed in a group one after another. It means that the Trainer provides the participants with some time (a few minutes) to read a question and to find an answer by making the necessary calculations (when appropriate) and analyzing information; then,after all participants are ready to provide an answer, the question is discussed in a group. Subsequently, the group moves to the next question until all 11 questions have beendiscussed. The Trainer should make sure that all trainees actively participate in the discussions.The duration of the exercise is about 1 hour. The exercise consists of two MS Word files – the exercise task file and the answer file. Participants should first be provided with the exercise task sheets only and after the exercise is completed; the participants canbe provided with the answer sheets for reference.

AFB Smear Microscopy EQA WIB TRAINER’S GUIDE

Page 1 of 10