Botswana-Harvard HIV Reference Laboratory
(BHHRL)
Tel: +267 3902671;
Fax: +267 3901284 / Summary Report:
Calcium and Phosphorous Validation
BHHRL Cobas Integra 400 plus Analyzer Serial Number 397844

Chemistry Validation Report

Non-Smile Contributor
Author:
BHHRL / Document Number: / Equ30-31
Effective (or Post) Date: / 6-March-09
Document Origin / Company: / BHHRL
SMILE Approved by: / Penny Stevens
Review
by / Heidi Hanes / Review date / 13-Feb-12
SMILE Comments: This document is provided as an example only. It must be revised to accurately reflect your lab’s specific processes and/or specific protocol requirements. Users are directed to countercheck facts when considering their use in other applications. If you have any questions contact SMILE.
Title: / Calcium and Phosphorous Validation
BHHRL Cobas Integra 400 plus Analyzer Serial Number 397844
Prepared By: / Name, Title / Signature, Date
Munyaradzi P. Mangwendeza, QA Manager
Early Manyiwa, Deputy QA Manager
Matshediso Zachariah, Acting Deputy Quality Manager
Reviewed By: / Name, Title / Signature, Date
Sikhulile Moyo, Laboratory Manager
Penny Stevens - SMILE
Approved By: / Name, Title / Signature, Date
Dr. Rosemary Musonda, Laboratory Director
  1. General Policy
  2. It is BHHRL’s policy that before any instrument or machinery is used to test patient samples, its performance is verified against the manufacturer claims and expected performance by the laboratory or against a gold standard method.
  1. Purpose
  2. Calcium and Phosphorus are to be introduced as new analytes tested at BHHRL and hence the instrument used for their testing has to be validated and verify the manufacturer claims of the instruments performance when running the test.
  3. The instrument performance against BHHRL expectations has to be verified.
  1. Scope
  2. The validation process covered the performance of the Cobas Integra 400 plus Roche analyzer serial number 397844 for Calcium and Phosphorus. The parameters considered were:
  3. Precision
  4. Accuracy
  5. Linearity & Analytical MeasurementRange
  6. Sensitivity & Specificity
  7. ReferenceRanges
  1. Methodology
  2. Study performed in accordance with the BHHRL SOP on Method and Equipment Validation/Verification BHHRL/001/PR018.
  3. The following methods were being validated:
  4. Calcium – calcium according to Schwarzenbach with o-cresolphthalein complexone.

Alkaline pH

Ca2+ + o-CPC calcium – o-CPC complex

4.2.2.Phosphorous

Phosphate + Ammonium Molybdate H2SO4 nonreduced phosphomolybdate

4.3.Precision

4.3.1.CAP CN3-C 2007 samples CHM-11 and CHM-12 were used for determining the within day precision of the instrument.

4.3.2.CHM-12 was used as level 2 and CHM-11 used as level 1.

4.3.3.The sample runs were repeated 10 times for each level for precision studies and statistically analyzed using the Simple Precision module of EP Evaluator Release 8 (David G. Rhodes Associates, Inc Kennett Square, PA 19348.)

4.3.4.Acceptable criteria: The manufacturer’s performance specifications will be compared to the allowable error for each assay as defined previously. Each assay is expected to perform equal to or less than the manufacturer’s performance specifications. In the event that an assay does not perform as expected, the performance will be compared to the allowable RE (50% of CLIA) and the Laboratory Director will determine acceptability.

Analyte / Mfg Precision / 50% of CLIA
Calcium / 0.99% / 0.125 mmol/L
Phosphorus / 0.95% / 0.05 mmol/L or 5.35%, whichever is greater

4.4.Accuracy

4.4.1.Accuracy of the instrument in testing for the 2 analytes was determined from the instrument’s performance in the CAP CN3-C 2007 Survey for the analytes in question.

4.4.2.The other part of accuracy was determined using correlation study with the back-up instrument at Lancet Gaborone Laboratory. 5 patient samples were tested at BHHRL and Lancet Gaborone laboratory and correlation determined to verify accuracy.

4.4.3.Acceptable criteria: The Two Instrument Comparison Module of EP Evaluator by D.G. Rhoades will be used to determine if the instruments are clinically equivalent within the Total Allowable Error (TEa) as follows:

4.4.3.1.Correlation coefficient (R) >97%

4.4.3.2.Error Index < 1.00 and > -1.00. The EP comparison module calculates the Systematic Error (PE and CE) by evaluating the ratio of the difference (bias) for each sample and comparing it to the TEa as follows: (Y-X)/TEa. This provides an error index for each measured pair of data. An index greater than 1.00 or less than -1.00 is unacceptable because it means the difference between the methods exceeds TEa. The results will only be accepted if at least 95% of the specimens do not exceed the TEa (Error Index).

4.4.3.3.Slope of 1.00 within a 95% confidence interval as calculated by EP evaluator.

4.4.3.4.Intercept of 0.00 within a 95% confidence interval as calculated by EP evaluator.

4.5.Linearity

4.5.1.Linearity verification and the reportable range will be determined using the Biorad Lyphochek Assayed chemistry Controls. Level 2 will be reconstituted with half the recommended amount of distilled water to create a higher concentration and dilutions to carry out the analysis study.

4.5.2.Linearity will be determined using linear regression.

4.5.3.The reportable range will be determined by selecting a high sample (with a known value) near the upper end of the manufacturer’s stated Analytical Measurement Range (AMR) and a low sample (with a known value) near the lower end of the AMR. If the low sample is not sufficiently low enough to verify the AMR, the lowest sample will be serial diluted to verify the low end of the AMR.

4.5.3.1. 0-5 mmol/L (0-20 mg/dl) for calcium

4.5.3.2. 0-4.8 mmol/L (0-20 mg/dl) for phosphorous

4.5.4.The Manufacturer’s Clinical Reportable Range (CRR) will be accepted if it is sufficiently large enough to cover grade 4 events of the Division of AIDS Toxicity Table. If it is not, high samples will be commercially acquired to verify dilutions greater than those verified by the manufacturer’s recommended CRR.

4.5.5.Acceptability criteria:

4.5.5.1. Linearity:

4.5.5.1.1.Slope of 1.00 within a 95% confidence interval as calculated by EP evaluator

4.5.5.1.2.Intercept of 0.00 within a 95% confidence interval as calculated by EP evaluator.

4.5.5.1.3.The linear regression will be visually evaluated for the best fit line and loss of linearity. The Laboratory Director will make the final determination of acceptability.

4.5.5.2. Reportable Range:

4.5.5.2.1.The measured values must be within the allowable TEa for each analyte.

4.5.5.2.2.The reportable range must be within the manufacturer’s AMR.

4.5.5.2.3.The manufacturer’s upper limit can be accepted if the known sample is within ± 10% of the upper limit and the TEa is acceptable (e.g., Upper AMR = 700 can be validated with a known & measured sample of at least 630, providing the TEa is acceptable). If a sample within 10% cannot be obtained, the highest known sample measured and within the acceptable TEa will be used as the highest reportable undiluted range.

4.5.5.2.4.The manufacturer’s lower limit can be accepted if the known sample is within ± 50% of the lower limit and the TEa is acceptable (e.g., Lower AMR = 5 can be validated with a known & measured sample of at least 7.5, providing the TEa is acceptable). Serial dilution will be used to obtain a sample within the desired range. However, if a sample within 50% cannot be obtained, the lowest known sample measured and within the acceptable TEa will be used as the lowest reportable range.

4.6.Sensitivity & Specificity - The instrument and reagents are FDA approved therefore separate sensitivity & specificity studies were deemed not necessary and the claimed specificity and sensitivity by the manufacturer was adopted for use by the laboratory.

4.7.Reference Values

4.7.1.The issue of reference values is being handled in a separate national wide study and will be given once available.

4.7.2.Manufacturer given reference ranges will be used by the laboratory until the national study on reference ranges if completed.

  1. Raw Data
  2. The raw data for each of the studies is presented in appendix 1.
  3. A summary of the results is given below. Raw data is comprised of the actual machine printouts as well as CAP survey reports used for the validation process.
  4. Linearity and AMR Raw Data
  5. Phosphorus - Linearity & AMR Determination
  6. Initial Concentration 1 = 2.40 mmol/l

Volume Stock Solution (ul) / 0 / 250 / 65 / 35 / 20 / 500
Volume Distilled water (ul) / 500 / 250 / 435 / 465 / 480 / 0
Expected Concentration / 0 / 1.20 / 0.36 / 0.18 / 0.10 / 2.40
Observed Result – 1st Run / 0 / 1.23 / 0.33 / 0.21 / 0.15 / 2.40
Observed Result – 2nd Run / 0.01 / 1.23 / 0.33 / 0.20 / 0.15 / 2.39
Volume Stock Solution (ul) / 0 / 125 / 250
Volume Distilled water (ul) / 250 / 125 / 0
Expected Concentration / 0 / 2.40 / 4.80
Observed Result – 1st Run / 0 / 2.31 / 4.47
Observed Result – 2nd Run / 0.01 / 2.33 / 4.46

5.2.1.1.2.Initial Concentration 2 = 4.80 mmol/l

5.2.1.2.Calcium – Linearity and AMR Determination

5.2.1.2.1.Initial Concentration 1 = 2.97 mmol/l

Volume Distilled water (ul) / 500 / 250 / 435 / 465 / 480 / 0
Expected Concentration / 0.00 / 1.49 / 0.44 / 0.22 / 0.12 / 2.97
Observed Result – 1st Run / 0 / 1.48 / 0.32 / 0.19 / 0.12 / 3.06
Observed Result – 2nd Run / 0 / 1.47 / 0.32 / 0.19 / 0.12 / 3.05

5.2.1.2.2.Initial Concentration 2 = 5.94 mmol/l

Volume Stock Solution (ul) / 0 / 125 / 250
Volume Distilled water (ul) / 250 / 125 / 0
Expected Concentration / 0 / 2.97 / 5.94
Observed Result – 1st Run / 0 / 2.96 / >5.00
Observed Result – 2nd Run / 0 / 2.97 / >5.00

5.2.2.BHHRL vs. Lancet Gabarone Instrument Correlation

5.2.2.1.Phosphorous

Lancet / 0.67 mmol/L / 1.87 mmol/L / 1.26 mmol/L / 1.09 mmol/L / 0.97mmol/L
BHHRL / 0.62 mmol/L / 1.84 mmol/L / 1.28 mmol/L / 1.10 mmol/L / 0.90 mmol/L

5.2.2.2.Calcium

BHHRL / 1.32 mmol/L / 2.64 mmol/L / 3.20 mmol/L / 2.24 mmol/L / 1.48 mmol/L
Lancet / 1.32 mmol/L / 2.62 mmol/L / 3.22 mmol/L / 2.25 mmol/L / 1.51 mmol/L
  1. Results

Result report printouts and data analysis are presented in appendix 2.A summary of the report is given below. All results show the Cobas Integra as performing satisfactorily in testing for calcium and phosphorous.

6.1.Precision

6.1.1.Both analytes passed precision studies.

Analyte / Expected Results / Observed Results / Acceptability
Manufacturer’s Precision / 50% of CLIA / CHM-11 (Low) CV% / CHM-12
CV%
Calcium / 0.99% / 0.125 mmol/L / 0.8% / 0.5% / Acceptable
Phosphorus / 0.95% / 0.05 mmol/L or 5.35% / 1.0% / 0.4% / Acceptable

6.2.Accuracy

6.2.1.This was judged using the results for CAP 2007 C-C, LN2-B and Accutest Event 3 EQA. Results were compared against the peer mean to determine accuracy.

6.2.1.1.Sample LN19 was excluded from the phosphorous study due to the large error identified in the sample. The error was though to be sample related because sample LN19 failed in several international laboratories. This could not be verified due to sample deterioration during storage for all LN2B samples.

6.2.1.2.. Both analytes passed accuracy studies.

Analyte / Slope / Intercept / Correlation Coefficient (R) / ErrorIndexRange / Acceptability
Observed / 95% Confidence Interval / Observed / 95% Confidence Interval / Expected
>0.97 / Expected
-1.0 to1.0
Calcium / 0.946 / 0.914 to 0.979 / 0.1125 / 0.028 to 0.197 / 0.9982 / -0.74 to 0.23 / Acceptable
Phosphorus / 0.974 / 0.950 to 0.999 / 0.0098 / -0.037 to 0.056 / 0.9994 / -0.62 to 0.18 / Acceptable

6.2.2.Accuracy was also assessed using 5 patient samples in comparison with Lancet Gabarone and these passed the accuracy test.

Analyte / Slope / Intercept / Correlation Coefficient (R) / ErrorIndexRange / Acceptability
Observed / 95% Confidence Interval / Observed / 95% Confidence Interval / Expected
>97% / Expected
-1.0 to1.0
Calcium / 0.997 / 0.952 to 1.041 / 0.015 / -0.086 to 0.117 / 0.9997 / -0.08 to 0.12 / Acceptable
Phosphorus / 0.993 / 0.879 to 1.108 / 0.021 / -0.119 to 0.161 / 0.9980 / -0.16 to 0.52 / Acceptable

6.3.Linearity

6.3.1.Both analytes passed linearity studies. Calcium passed within a systematic error budget of 0.125 mmol/L (50% of the TEa of 0.25 mmol/L). Phosphorus passed within a systematic error budget of 5.9% or 0.053 mmol/L (55% of the TEa of 10.7% or 0.097 mmol/L) for all samples except the sample at 4.8 mmol/L concentration. This sample passed within a systematic error budget of 7.49% (70% of the TEa of 10.7%). This is suspected to be due to the sample dilution. The precision (CV) for phosphorus is significantly lower than the remaining allowable error of 3.21%. The 3.21% was determined as follows: 100% -70% = 30% TEa; 10.7% CLIA TEa x 0.3 = 3.21%

Analyte / Slope / Intercept / Visual Evaluation / Acceptability
Observed / 95% Confidence Interval / Observed / 95% Confidence Interval
Calcium / 0.986 / 0.954 to 1.019 / 0.0314 / -0.0429 to 0.1056 / Linear / Acceptable
Phosphorus / 1.071 / 1.023 to 1.118 / -0.0515 / -0.1452 to 0.0421 / Linear / Acceptable

6.4.Analytical MeasurementRange

6.4.1.This was verified to be correct for both analytes as claimed by the manufacturer. The analytical measurement ranges are also sufficient to identify critical values as required by the DAIDS toxicity tables.

Analyte / Mfg’s AMR / Low Value Verified / High Value Verified / Reportable Range / Dilutions / Clinical Reportable Range / DAIDS Toxicity Grade 4
Calcium (mmol/L) / 0 - 5.0 / 0 / 5.0 / 0 - 5.0 / None / 0 - 5 / Adult & Peds (> 7 days):
<1.53 or >3.38 mmol/L
Infants (< 7 days):
<1.38 or > 3.38 mmol/L
Phosphorus (mmol/L) / 0 - 4.8 / 0 / 4.8 / 0 - 4.8 / 1:10 / 0 - 48.0 / Adult & Peds (>14 years):
< 0.32 mmol/L
Peds (<14 years):
<0.48 mmol/L

6.5.Specificity & Sensitivity

6.5.1.The instrument and reagents are FDA approved therefore a separate specificity study was deemed not necessary and the claimed specificity and sensitivity by the manufacturer was adopted for use by the laboratory.

Summary of Manufacturer’s Claims for Sensitivity and Specificity
Analyte / Specificity (Interfering Substances) / Sensitivity
Calcium / Icterus – No significant interference
Hemolysis – No significant interference
Lipemia – No significant interference
Anticoagulant – complexing anticoagulants such as citrate, oxalate and EDTA must be avoided. / 0.01 mmol/L
Phosphorus / Icterus – No significant interferences upto a Bilirubin level of 513umol/l
Hemolysis – Significant effect. Avoid haemolysed specimens since high amount of phosphate is liberated during rapture of erythrocytes.
Lipemia – No significant effect
Anticoagulants – Do not use EDTA, citrate or fluoride as anticoagulants. / 0.017 mmol/L

6.6.Reference Values

6.6.1.The issue of reference values is being handled in a separate national wide study and will be given once available.

6.6.2.Manufacturer given reference ranges will be used for the adult range until the national study on reference ranges if completed. The pediatric reference ranges were selected from the Harriet Lane Handbook, 17th edition. The pediatric ranges will not be verified in the national study due to ethical considerations. However, the ranges will be verified over time as well baby data is collected for the indicated analytes and age categories.

6.6.3.Adult and pediatric reference ranges are as follows:

6.6.3.1. Adult

Analyte / ReferenceRanges
Adult >17 yrs (Male & Female) / % Verified
(Expected >90%)
Calcium mmol/L / 2.20 -2.55 / Not verified
Phosphorus mmol/L / 0.87 - 1.45 / Not verified

6.6.3.2. Pediatric

Analyte / Age (Male & Female) / ReferenceRange / % Verified
(Expected >90%)
Calcium mmol/L / 0 to < 10 days / 1.9 - 2.6 / Not verified
10 days to < 24 Months / 2.3 - 2.8 / Not verified
2 - 12 years / 2.2 - 2.7 / Not verified
> 12 years / 2.2 - 2.5 / Not verified
Phosphorus, mmol/L / 0 to < 10 days / 1.45 - 2.91 / Not verified
10 days to < 24 Months / 1.45 - 2.16 / Not verified
2 - 12 years / 1.45 - 1.78 / Not verified
> 12 years / 0.87 - 1.45 / Not verified

6.7.Comparison with Back-up Laboratory

6.7.1.The two instruments perform comparably within allowable error limits and the back up laboratory instrument at Lancet Gaborone laboratory can be used when the primary instrument at BHHRL cannot be used. Refer to section 6.2.2 for the comparison study results.

  1. Conclusions
  2. The BHHRL Cobas Integra performs as expected by BHHRL and as claimed by the manufacturer in testing for calcium and phosphorus and is recommended for use in testing these two analytes.
  1. Method Approval
  2. There is no single formula that can be used to make the final decision to validate and accept a new methodology. Statistics themselves only provide estimates of certain types of errors and should not be used as the sole criteria for the judgment of the acceptability of a method. The acceptable error limits or performance standards that have been published make acceptability decisions easier for those methods where such standards have been defined. Factors such as sample size, speed of analysis, equipment costs, reagent costs and availability, storage requirements and personnel requirements should also be considered.
  3. The final decision on methodology validation and acceptance is made after a careful review of all the studies performed as part of the complete method validation process. The Laboratory Director shall make the ultimate decision on method validation. Method acceptance is based on the results from the above studies plus an evaluation of the new method’s cost effectiveness, turn-around-time, laboratory staff training needs, and any other relevant operational considerations.
  1. References
  2. NCCLS EP9-A2 Vol.22 No.19 (Method Comparison and Bias Estimation Using Patient Samples), Sep 2002.
  3. NCCLS C28-A Vol. 15 No. 4 (How to Define and Determine Reference Intervals in the Clinical Laboratory), Jun 2000.
  4. NCCLS C10-A2 Vol. 22 No. 29 (Preliminary Evaluation of Quantitative Clinical Laboratory Methods), Dec 2002.
  5. NCCLS C6-A Vol. 23 No. 16 (Evaluation of the Linearity of Quantitative Measurement Procedures: A Statistical Approach), April 2003.
  6. NCCLS C5-A2 Vol. 24 No. 25 (Evaluation of Precision Performance of Quantitative Measurement Methods), Aug 2004.
  7. NCCLS EP21-A Vol. 23 No. 20 (Estimation of Total Analytical Error for Clinical Laboratory Methods), Apr 2003.
  8. CAP Commission on Laboratory Accreditation, Laboratory General Checklist, 2007.
  9. Fawzi, W.W, et al. Vitamins and Perinatal Outcomes among HIV-Negative Women in Tanzania. N Engl J Med 2007; 356: 1423-31.
  10. Roche Cobas package inserts for ALT, AST, ALB, 2006-08 and 2006-05.
  11. HPTN-MTN Lab Manual, Version 1.0, 15 November 2006
  12. EP Evaluator,
  13. MuhimbiliUniversityof Health and Allied Sciences Department of Microbiology and Immunology. Muhas/Harvard Research Laboratory Validation Plan and Protocol for ALT, AST and Albumin 2008.

Page 1 of 11