Hatchery sampling analysis – User guide

Hatchery sampling analysis

User guide

Introduction

This utility analyses historical surveillance data for a hatchery, to determine the system sensitivity of past testing and confidence of freedom over time. It also provides estimates of sample size for any additional testing required to achieve a specified level of confidence of hatchery-freedom from disease at the specified design prevalence.

Briefly, the program allows input of data on historical testing of lots from a hatchery, based on sampling of either moribund or healthy fish or both. The sampling data is used to estimate system sensitivity and confidence of freedom on a monthly basis, after allowing for discounting due to the probability of introduction during any time period. Finally, the additional sample size required (moribund or healthy or a combination) to achieve the target confidence of freedom at the completion of the historical surveillance is calculated.

Running the software

This software runs on a portable web-server running on a USB stick. See Appendix (EpiTools on a stick Instructions) for detailed instructions on installing and running the web software and EpiTools. To run the hatchery sampling utility, start the web-server as usual and when the home page opens go to the URL and click on the link.

Inputs

Inputs required and entered on-screen include:

Area description / A brief description of the analysis to be done (eg hatchery name)
Design prevalence / Lot-level design prevalence is specified as one lot infected in the hatchery and cannot be changed
Fish-level design prevalence is specified as a proportion, default value is 0.05
Prior probability of freedom / The prior probability (confidence) of disease freedom (before the start of the surveillance being analysed) is a proportion, specified either as a fixed value or a pert distribution (minimum, most likely and maximum values). Default value is a fixed value of 0.5 (no prior knowledge).
Probability of introduction / The probability of disease introduction during any surveillance time period (month)is a proportion, specified either as a fixed value or a pert distribution (minimum, most likely and maximum values). Default value is a Pert(0.01, 0.02, 0.05).
Target Confidence of freedom / The target value for the probability (confidence) of disease freedom following the planned sampling. This is the level of confidence that you wish to achieve, usually set at the default value of 0.95 (95%)
Number of iterations / This is the number of times to run the model. Default is 1,000 iterations but can be 100, 1000 or 10000. The more iterations the longer it will take to run.
Proposed sampling date / This is the date (month) of the planned sampling to achieve the target confidence of freedom
Date format / Dates in the sampling data can be specified in either American (mm/dd/yyyy) or European (dd/mm/yyyy) format.
Default date format is American format

Testing data

Testing data for previous surveillance undertaken to be included in the analysis must be pasted into the text box on the input screen. Required data columns are:

  • Date (or month) ofsampling (labelled 'Date') and in format dd/mm/yyyy or mm/dd/yyyy (default), as specified in inputs;
  • Lot identifier (labelled 'LotID');
  • Number of moribund fish sampled (labelled 'Moribund') and
  • Number of healthy fish sampled (labelled 'Healthy').

In addition:

  • Columns can be in any order;
  • Additional data columns are optional, but can be included if desired (for example, species, age or other descriptive data);
  • A header row specifying column names (as above for required columns) must be included;
  • Data must be aggregated on a monthly and lot basis, with a single row per month per lot (i.e. if more than one sampling occurred during a month enter a single row of aggregated testing for each lot on a single date);
  • All lots present in the hatchery at each sampling must be represented in the data - any lots with no testing should be entered as 0 in both Moribund and Healthy columns;
  • Do notinclude data for months in which no sampling was undertaken.

An example file of simulated data is provided in the file data/data.xls, or available by clicking on the link on the data entry page.

Model parameters

Additional parameters for the model are set in the file data/params.txt (or you can edit in params.xls and then save as tab-delimited txt file params.txt).

Specific parameters included here are:

  • Population proportions for moribund fish, entered as a pert distribution [default values are Pert(0.0001, 0.0002, 0.0005)];
  • Relative risks for moribund fish, entered as a pert distribution [default values are Pert(2, 310)];
  • Estimated test sensitivities for multiple experts and two tests, as pert distributions

Formatting is important for the parameter file:

  • Name must be params.txt (case sensitive and located in HatcherySampling/data folder;
  • Format is tab-delimited text;
  • Seven columns required, in specified order:
  1. Factor
  2. Level of factor
  3. Variable name
  4. Minimum value for pert distribution
  5. Most likely value for distribution
  6. Maximum value for pert distribution
  7. Weighting for individual experts for each test – can be any positive number and selection of experts for each test is weighted according to relative weights allocated (for example an expert with a weight of 5 will be selected 5 times as often as one with a weight of 1). Weights are not required for relative risks or population proportions for moribund fish.

For test sensitivities, the following data is required:

  • Multiple experts are allowed for each test, one expert per row;
  • There is no real limit on the number of experts (simply add/delete rows as required);
  • Numbers of experts can differ between tests;
  • First column must include test number: “Test_1” or “Test_2” in all rows for the relevant test (case sensitive);
  • Second column is test name. This can be whatever you want to call the test (“IFAT” or “Culture” for BKD example);
  • Third column is identifier for each expert for each test and can also be whatever you want to use;
  • 4th to 6th columns are minimum, most likely and maximum values, respectively, for the pert distribution for each expert;
  • Seventh column is relative weighting to be allocated to each expert.

Run the analysis

Once all of the input fields and parameters are set and the data has been pasted into the text box, click submit to run the analysis. The analysis could take a few seconds to a minute or two, depending on the amount of data entered and the number of iterations selected. Results will be displayed on-screen when the analysis is complete, along with a link to an excel file of results.

Outputs

Two options for output are available: summary output (default) or detailed output.

Summary outputs from the analysis include:

  • Median sample sizes required for each lot for planned sampling to achieve target confidence of freedom, for varying combinations of moribund and healthy fish; and
  • Summary (percentiles, mean, standard deviation and distribution plots) of the output distributions for system sensitivities (probability of detecting disease if it was present at the specified design prevalences) for each time period sampled;

Detailed outputs from the analysis include:

  • Table of input values used;
  • Summary (percentiles, mean, standard deviation and distribution plots) of the output distributions for system sensitivities (probability of detecting disease if it was present at the specified design prevalences) for each time period sampled;
  • Summary (percentiles, mean, standard deviation and distribution plots) of the output distributions for confidence of freedom of the population from disease (at the specified design prevalences) for each time period analysed (includes intermediate time periods when no sampling was undertaken);
  • Median values for sensitivity and confidence of freedom by individual lot for each time period;
  • Median and 95%-ile sample sizes required for each lot for planned sampling to achieve target confidence of freedom, for varying combinations of moribund and healthy fish;
  • A summary of sampling undertaken at each time point, including mean system sensitivity and mean sensitivity for each lot;
  • Summaries of distributions for important parameters calculated during the analysis (combined test sensitivity, effective probability of infection for moribund and healthy fish, unit sensitivity for moribund and healthy fish, confidence of freedom achieved at end of surveillance, additional sensitivity required to achieve target confidence of freedom, relative value of moribund:healthy fish and individual test sensitivities);
  • A summary of probability of introduction by time period;
  • A table of input distribution parameters from params.txt;
  • A table of actual data analysed and lot sensitivity achieved for each lot tested;
  • Plots of system sensitivity and confidence of freedom over time for the hatchery overall and for each lot individually;
  • A plot of median and 95% probability intervals for system sensitivity and confidence of freedom at each time point;and
  • Plots of median sensitivity and median confidence of freedom by lot for each time period.

Graphs

Output tables include thumbnails with links to summary graphs. To open the graphs, right-click on the thumbnail and select option to open graph in new tab or new window. If you left-click on the thumbnail it will open in the same window and you will need to use the back button and refresh the analysis to get back to the main results screen.

Download results

All tabular results can be downloaded as an excel file by clicking on the Download link at the bottom of the results page.

Appendix

EpiTools on a Stick Instructions

Introduction

This system is designed to run from a USB memory stick. In contains all the files necessary to run EpiTools and the Freedom software, including an Apache web-server to run the system from the stick.

The whole system runs from the memory stick and this document describes how to start and stop the web server and run EpiTools.

Contents and structure of stick

The root directory of the stick includes batch files for running the web server and starting EpiTools. The stick also has the following folder structure:

-Resources – workshop resources for epidemiology and surveillance

-R: contains the R program and related files:

-Apache2, icons, PHP, tmp and WWW – these folders contain program files forrunning the webserver and EpiTools and should be ignored.

Running the web server and EpiTools

To install the system in any Windows computer:

1)Plug the memory stick into a spare USB port

2)Using Windows Explorer, browse to the root directory of the memory stick

3)Start the web server by double-clicking on "start.bat" (or just “start”, depending on how explorer is configured). A DOS window will open and stay open while the server is running (do not close this window or the server will stop running).

4)A web browser window will open with theAusVet Surveillance Training Resource Kit home page, with links to Freedom Scenario tree modelling software and EpiTools.

To shut down the system

1)Close your web browser

2)Double-click on “stop.bat”" (or just “stop”)

3)Safely remove the USB stick

Notes:

  1. If you get lost and need to go back to the home page, or accidentally close the browser, double-click on the “Home Page” shortcut in the root directory of the USB
  2. If you close the DOS window the server will stop running
  3. If EpiTools doesn’t work check that the DOS window is open (the server is running)
  4. After some usage, the USB may fill up with temporary files, so that there is insufficient space available to write additional files, causing epitools to crash. If epitools suddenly stops working or appears to behave abnormally, delete all files in the WWW/tmp and WWW/epitools/tmp directories and try again.

AusVet Animal Health Services12 September 2010