Whole Genome Sequencing of M.tuberculosis for Detection of Drug Resistance: a Systematic Review
Dimitrios Papaventsis PhD1§, Nicola CasaliPhD2, Irina Kontsevaya PhD2 , Francis Drobniewski PhD2, Daniela Cirillo PhD3, Vlad Nikolayevskyy PhD2,4*§
Supplementary files
APPENDIX S1.Outcome measures and Search methods
The following types of outcome measures were recorded:
-Number of isolates found to be resistant/sensitive to individual drugs, MDR-TB and XDR-TB using phenotypic methods;
-Number of isolates found to be resistant/sensitive to individual drugs, MDR-TB and XDR-TB using WGS;
-Performance characteristics (sensitivity, specificity, PPV, NPV) compared to phenotypic DST.
Search methods
The following search strategies were used:
a) Electronic searches
A compound search strategy (Supplementary files, Tables S1 and S2) was developed to identify all relevant studies regardless of language or publication status using the following electronic databases:
-MEDLINE (OVID)
-EMBASE (OVID)
-Web of Science
b) Other resources
Reference lists of all studies identified by the above methods and editorials were examined; no additional references satisfying original eligibility criteria have been identified.
APPENDIX S2.Quadas-2 Protocol
Review question:
What are the performance characteristics (specificity, sensitivity, PPV, and NPV) of for the detection of resistance to first and second line anti-tuberculosis drugs using phenotypic methods as a gold standard?
Domain 1: Patient Selection
Risk of Bias: Could the selection of patients/specimens have introduced bias?
• Signaling question 1: Was a consecutive or random sample of patients or specimens enrolled? We scored ‘yes’ if the study enrolled a consecutive or random sample of eligible patients; ‘no’ if the study selected patients by convenience, and ‘unclear’ if the study did not report the manner of patient selection or unable to tell.
• Signaling question 2: Was a case-control design avoided? We rated ‘no’ if a case-control design was used, ‘yes’ if prospective or cross-sectional design was used.
• Signaling question 3: Did the study avoid inappropriate exclusions? We scored 'no' if the study excluded samples based on specific characteristics (DNA quality; prior testing etc) and 'unclear' if unable to tell.
Applicability: Are there concerns that the included patients and setting do not match the review question?
We were interested in how WGS performed in TB patients regardless of the country of isolation and/or M. tuberculosis genetic lineage. We made this judgment based on the information on TB genetic lineages reported based on VNTR and other methods.
We scored 'low concern' if study population included two or more genetic lineages, and 'high concern’ if study was confined to one lineage only. We judged applicability to be of ‘unclear concern’ if no data on genetic lineages was reported.
Domain 2: Index Test
Risk of Bias: Could the conduct or interpretation of the index test have introduced bias?
• Signaling question 1: Were the index test results interpreted with knowledge of the results of the reference standard? We answered this question ‘yes’ when WGS data analysis and interpretation were performed in a blind manner, i.e. operators were not aware of the reference test results; ‘no’ if WGS data was analysed and interpreted with knowledge of the results of the reference test, and ‘unclear’ if it was not reported.
• Signaling question 2: If a threshold was used, was it prespecified? Threshold(s) were pre-specified in all versions and types of software used for WGS data analysis so we answered ‘yes’ in all studies.
Applicability: Are there concerns that the index test, its conduct, or its interpretation differ from the review question? Variations in test technology, execution, or interpretation may affect estimates of the diagnostic accuracy of a test.
There are currently no established/widely recognised protocols for WGS data interpretation for drug susceptibility testing, there are no commercially available kits and manufacturers’ recommendations and variety of protocols and online databases are in use. We have therefore answered ‘unclear’ for all studies; however all the technical parameters were carefully recorded and reported in the current review (Table S4).
Domain 3: Reference Standard
Risk of Bias: Could the reference standard, its conduct, or its interpretation have introduced bias?
• Signaling question 1: Is the reference standard likely to correctly classify the target condition?
We scored ‘yes’ if WHO-recommended and/or internationally recognized phenotypic DST methods [35-38] were used for determination of susceptibility for the first and second line drugs, and scored ‘no’ if non-standard methods were in use. If no data on the reference standard was reported, we answered ‘unclear’.
• Signaling question 2: Were the reference standard results interpreted without knowledge of the results of the index test?
We answered ‘yes’ in all studies as all the studies were designed in such a way that index test (WGS) was performed after the phenotypic DST.
Applicability: Are there concerns that the target condition as defined by the reference standard does not match the question?
Phenotypic DST is endorsed by the WHO and is currently considered a gold standard for drug susceptibility testing in TB laboratory diagnostics. We have therefore judged applicability to be of ‘low concern’ for all studies.
Domain 4: Flow and Timing
Risk of Bias: Could the patient flow have introduced bias?
• Signaling question 1: Was there an appropriate interval between the index test and reference standard?
TB is a chronic disease and, as such, several days delay between the index test and reference standard is unlikely to affect diagnosis; importantly in our systematic review we were addressing issues of drug susceptibility testing bot a TB diagnosis. We have therefore answered this question with ‘yes’ for all studies.
• Signaling question 2: Did all patients receive the same reference standard?
We answered this question ‘yes’ for all the studies as the same reference standard (acceptable reference standard as specified as a criterion for inclusion in the review, i.e. phenotypic DST) was in use across all papers.
• Signaling question 3: Were all patients included in the analysis?
In all studies all patients and corresponding specimens were included in the analysis (in 2x2 tables used for specificity and sensitivity calculations) so we have answered ‘yes’ in all studies.
APPENDIX S3.General characteristics of included studies and data available
The characteristics of included studies and relevant outcome data are summarized in Table S3 of the supplementary files. Whole genome data was available for 6,906 MTBC isolates from 20 studies. Importantly, in one study [6], sequences already available in the public domain were used (N=792) so the total number of original sequences was 6,114. The number of isolates included in each study varied from 2 to 3,651. In many studies MTBC isolates were collectively referred to as “M. tuberculosis complex strains” without precise speciation; in some papers [4, 5, 22], authors stated that test samples included M. africanum strains along with M. tuberculosis.
A large proportion of studies (N=17) exclusively or partially included strains collected in high TB and/or DR-TB settings including China, Russian Federation, Haiti, Malawi, Pakistan, Moldova, South Africa, Uzbekistan, Sierra Leone, Thailand and Lithuania. Only three studies [26, 29, 32] were conducted in low TB incidence settings (United Kingdom). Clinical and baseline epidemiologic characteristics of patients (ethnic background; gender; age) as well as the proportion of MTBC cultures isolated from patients belonging to risk groups have not been systematically collected and reported; some studies were conducted in high HIV incidence settings and/or included strains isolated from HIV-coinfected patients [23-25, 33]. Information on new/retreatment cases was available in seven studies [12, 26, 30, 31, 33-35].
Methods used for phenotypic DST varied considerably across reports; in two studies methods have not been specified[6, 26]. The proportion method on solid media (predominantly LJ or 7H10), rapid culture tests on liquid media (BACTEC MGIT 960), and the resistance ratio method were employed in 7, 5, and 1 studies, respectively. In five studies combinations of two (BACTEC MGIT 960 plus proportion method on solid media) or three (BACTEC MGIT 960 plus proportion method on solid media and resistance ratio) methods were used[5, 24, 28-30]; in these studies liquid media was predominantly used for second and third line drugs. In two reports [28, 29], specific types of media for PZA were in use. Drug concentrations used for phenotypic testing were reported in all studies; predominantly standard WHO-recommended methods [37-40] were in use in all laboratories. The number of anti-TB drugs per study varied from 1 to 15; the full list of drugs and corresponding genes and other genomic regions associated with the resistance to specific drugs, as well as WGS calculated performance characteristics are presented in Table 1.
Information on phylogenetic lineages was reported in all but three [14, 29, 30] studies. Phylogenetic characteristics were determined using PCR-based genotyping methods (spoligotyping and/or variable number of tandem repeats [VNTR] typing) often supplemented by the data derived from WGS. MTBC strains belonged to different genetic lineages including Euro-American (e.g. Haarlem, S, LAM, T etc), East African Indian, East Asian (e.g. Beijing), and Indo-Oceanic. In six papers, the analysis was confined exclusively to East Asian (Beijing) isolates [12, 23, 27, 30, 31].
Nearly all studies were designed in a broadly similar manner as retrospective studies conducted on pre-selected sets of M. tuberculosis cultures isolated from routine or research-based primary specimens which had been tested in the past for susceptibility for selected drugs using phenotypic DST. Inclusion criteria across studies were not consistent; commonly isolates with specific drug resistance profiles (e.g. MDR-TB and/or XDR-TB supplemented by fully sensitive specimens for control purposes), and/or isolated in certain geographic regions or belonging to specific genetic lineages have been included in the studies. In all but one study original sets of cultures were used; in the study by Coll et al. [6] novel online tools for rapid determination of drug resistance using WGS data were validated using sequences downloaded from the public domain. Two more small studies [26, 30] with a total of five isolates tested were conducted prospectively on cultures isolated from patients put on anti-TB treatment.
WGS technical data, quality control and polymorphism calling criteria
WGS technical data are summarized in Table S4. All studies reported the sequencing chemistry platform and hardware for sample analysis; in seventeen studies various versions of Illumina platform were utilized (Figure S1). Alternative chemistries included Ion Torrent [12], Pac Bio RS (Pacific Bioscience) [35] and the 454 Roche GS FLX system [26]. Genome coverage and average sequencing depth were reported in 11 and 12 studies, respectively, and varied from 33.2% to >99% and from 27x to 916x, respectively. Lower genome coverage rates (33.2-70.0%) reported in one study [12] could probably be linked to the technology used (Ion Torrent); in other studies coverage was over 88%.
Principal QC criteria applied at analytical stages included Phred quality scores, minimum depth for valid single nucleotide polymorphisms (SNP) call and consensus rate for valid call and exclusion of certain portions of genome depending on their nature and location. SNPs located in repetitive and high GC-rich regions (including PE/PPE/PGRS regions) were excluded in thirteen studies and not reported in the remaining seven papers [5, 27, 29-32, 35]. Minimum depth for valid SNP call and minimum consensus rate for valid SNP call were reported in 9 and 8 studies and ranged from 4x to 20x and from 70% to 90%, respectively; in many studies it was not clear whether these criteria were applied to each DNA strand separately or not. Exclusion criteria and rules applied to polymorphisms located within 12 bp to each other, as well as information on whether DNA was extracted from single colonies, were not systematically reported.
Bioinformatics workflows and software used for bioinformatics analysis, reported in all studies, were extremely diverse. Generally workflows included initial QC analysis of fastQ files (using aforementioned criteria and specific settings within software packages), mapping reads/contigs to the reference annotated M. tuberculosis sequence, and further analysis using specialized software packages and/or online databases (e.g. TBDreamDB [40]) for the detection of specific polymorphisms associated with drug resistance.
APPENDIX S4.Technical, analytical and other parameters to be included in future reports
WGS technical parameters:
-Hardware platform and chemistry used;
-Length of reads;
-Genome coverage and depth;
-Bioinformatics analysis parameters and criteria for calling polymorphisms:
- Exclusion or inclusion of SNPs in repetitive sequences, PE/PPE, ESX and similar genes
- Exclusion or inclusion of SNPs located within certain distance (e.g. 12 bp)
- Minimum depth and consensus threshold for a valid call and whether these apply to each DNA strand
-Software for bioinformatics analysis and principal settings
Detection of polymorphisms associated with drug resistance:
-Selection of specific targets, genes and other relevant genomic regions
-Differentiation between drug resistance and phylogenetic markers, sSNPs and nsSNPs. Exclusion of phylogenetic markers and sSNPs
-Sources of online databases and other tools;
Study design, baseline epidemiological data and settings:
-Study design (prospective/retrospective; population-based; longitudinal etc)
-Target populations, presence of high risk groups;
-Inclusion and exclusion criteria
-TB settings (low-, high etc), country
-Length of the study
Phenotypic DST:
-Method used (absolute concentrations, resistance ratio, proportion etc)
-Media
-Drugs and concentrations
WGS results and performance characteristics:
-Number of phenotypically resistant strains with mutations (true positive)
-Number of phenotypically resistant strains without mutations (false negative)
-Number of phenotypically sensitive strains with mutations (false positive)
-Number of phenotypically sensitive strains without mutations (true negative)
-Performance characteristics (sensitivity, specificity, PPV, and NPV).
Table S1: Search terms for Medline and Embase1
SET / TOPIC / SEARCH TERMS1 / Whole genome sequencing / SEQUENCE ANALYSIS, DNA
2 / WGS
3 / HIGH-THROUGHPUT NUCLEOTIDE SEQUENCING
4 / new adj1 generation adj1 sequencing
5 / whole adj1 genome adj1 sequencing
6 / Set 1-5 were combined with "or"
7 / Tuberculosis / MYCOBACTERIUM TUBERCULOSIS
8 / TUBERCULOSIS
9 / Tb
10 / m tuberculosis
11 / Set 7-10 were combined with "or"
12 / Set 6 and 11 were combined with “or”
1 - Words written in capital letters were used as Medical Subjects (MeSH) headings; the others were used as free text. Search terms which were the same for all databases are only mentioned once under the EMBASE heading.
1
Table S2: Search terms for Web of Science
SET / TOPIC / SEARCH TERMS1 / Whole genome sequencing / Whole Genome Sequencing
2 / Next Generation Sequencing
3 / Set 1-2 were combined with "or"
4 / Tuberculosis / Mycobacterium tuberculosis
5 / Mycobacterium
6 / Tuberculosis
7 / Set 4-6 were combined with "or"
8 / Set 3 and 7 were combined with “or”
1 - Words written in capital letters were used as Medical Subjects (MeSH) headings; the others were used as free text. Search terms which were the same for all databases are only mentioned once under the EMBASE heading.
1
Table S3 General Characteristics of the studies included in the report
Publication / Phenotypic DST / No of isolates sequenced / Patients/strains characteristics / M. tuberculosis lineages / Originality of the data setsMethods used / Drugs tested and concentrations (μg/ml) / Countries where strains were collected / Settings / Risk groups / New/
retreatment cases
[7] / BACTEC MGIT 960 / [35] / 1000 / Russia, UK / High and low TB and DR TB / Yes / na / Beijing, CAS, Euro-American, EAI / Original
[22] / BACTEC MGIT 960 / [36] / 59 / Russia, UK / High and low TB and DR TB / na / na / Beijing, Euro-American / Original
[23] / Proportion method on solid media / INH 0.2, RIF 1.0, EMB 7.5, ETH 20.0, OFL 2.0, AMK 5.0, KAN 5.0, CAP 10.0 / 14 / South Africa / High TB and DR TB / na / na / Beijing / Original
[24] / Proportion method on solid media / 7H10: INH 0.2 and/or 1.0, RIF 1.0 , EMB 7.5, STR 2.0 and/or 10.0
LJ: INH 0.2 and 1.0 , RIF 40.0, EMB 2.0 , STR 4.0, ETH 20.0, KAN 20.0, CIP 5.0, OFL 2.5 / 11 / South Africa / High TB and DR TB / na / na / Haarlem, EAI, LAM, X / Original
[25] / Proportion method on solid media / INH 0.2, RIF 40.0, STR 4.0, EMB 2.0 / 97 / Panama / medium / na / na / LAM, X , T , H / Original
[26] / Na / na / 3 / UK / Low / na / 1 case R-emergence (one new and two retreatment) / Haarlem, LAM / Original
[27] / BACTEC MGIT 960 / [37, 38] / 4 / China / high TB / DR / na / na / Beijing / Original
[28] / BACTEC MGIT 960 (PZA), proportion method on solid media / RIF 1, INH 0.2, STR 2, EMB 5, PZA 100, CIP 2, CAP 10, AMK 5, KAN 6, ETH 5 / 42 / Pakistan / high TB / DR / na / na / CAS, EAI, X, T / Original
[14] / BACTEC MGIT 960 / STR 1.0, INH 0.1, RIF 1.0, EMB 5.0, PZA 100.0, OFL 2.0, AMK 1.0, CAP 2.5, KAN 5.0, ETH 5.0, PTH 2.5, PAS 4.0 / 58 / UK, Lithuania / High and low TB and DR TB / na / na / Nd / Original
[6] / na / [35, 36] / 792 / Canada, China, Malawi, Pakistan, Portugal and Russia / High and low TB and DR TB / na / na / Lineages: East African Indian, Beijing, Central Asian, Euro-American (LAM, X, T, S, H and other) / Downloaded from public domain
[5] / Proportion method on solid media, BACTEC460 / [35] / 92 / Sierra Leone / high TB/ DR TB / na / na / Beijing, East African Indian, Euro-American (Ghana, Haarlem, LAM, Cameroon, S), M. africanum (West African 1 and 2) / Original
[29] / Resistance ratio (1st line), biphasic (PZA), BACTEC MGIT960 (2nd line) / [36] / 28 / UK / Low TB/ DR / na / na / na / Original
[30] / BACTEC MGIT 960, proportion method on solid media / [36] / 2 / China / high TB / DR / na / retreatment / Beijing / Original
[31] / Proportion method on solid media / INH 0.2, RIF 1.0, EMB 5.0, STR 2.0, PZA 100.0, OFL 2.0, LVX 2.0, KAN 5.0, CAP 10.0, AMK 1.0, ETH 5.0, PAS 2.0 / 7 / China / high TB / DR / na / new and retreatment / Beijing / Original
[32] / Proportion method on solid media, MGIT960 / [35] / 652 / UK / Low TB/ DR / na / na / Nd / Original
[33] / BACTEC MGIT 960 / STR 1.0, INH 0.1, RIF 1.0, EMB 5.0, PZA 100.0
RIF MIC determined with Alamar Blue Broth Microdilution assay. / 8 / Haiti / high TB / DR / na / new and retreatment / SIT 53 / Original
[34] / Proportion method on solid media / INH 0.2, RIF 1.0, LNZ 1.0, EMB 5.0, ETH 5.0, AMK 6.0, KAN 6.0, STR 2.0, PAS 2.0, OFL 2.0, CIP 2.0, LVX 2.0, MOX 2.0, GFX 2.0 / 4 / Thailand / high TB / DR / na / new / Beijing / Original
[35] / BACTEC MGIT 960 / INH 0.1 / 366 / India, Moldova, Philippines, and South Africa / high TB / DR / na / new / East Asian, Beijing, indo-Oceanic, Euro-American, CAS / Original
[4] / Resistance ratio and proportion methods on solid media, BACTEC MGIT 960 / [36] / 3651 / UK, Sierra, Leone, S. Africa, Germany, Uzbekistan / High and low TB and DR TB / na / na / Euro-American, Central Asian, Beijing, Indo-Oceanic, Ethiopian / Original
[12] / Resistance ratio on solid media / [36] / 16 / UK / Low TB / DR / na / new and retreatment / Beijing / Original
1