Proficiency testing of virus diagnostics based on bioinformatics analysis of simulated in silico high-throughput sequencing datasets

Annika Brinkmann*, Andreas Andrusch, Ariane Belka, Claudia Wylezich, Dirk Höper, Anne Pohlmann, Thomas Nordahl Petersen, Pierrick Lucas, Yannick Blanchard, Anna Papa, Angeliki Melidou, Bas B Oude Munnink, Jelle Matthijnssens, Ward Deboutte, Richard J Ellis, Florian Hansmann, Wolfgang Baumgärtner, Erhard van der Vries, Albert Osterhaus, Cesare Camma & 16 others Iolanda Mangone, Alessio Lorusso, Maurilia Maracci, Alexandra Nunes, Miguel Pinto, Vítor Borges, Annelies Kroneman, Dennis Schmitz, Victor Max Corman, Christian Drosten, Terry C Jones, Rene S. Hendriksen, Frank Møller Aarestrup, Marion Koopmans, Martin Beer, Andreas Nitsche

*Corresponding author for this work

Research output: Contribution to journalJournal articleResearchpeer-review

59 Downloads (Pure)

Abstract

Quality management and independent assessment of high-throughput sequencing-based virus diagnostics have not yet been established as a mandatory approach for ensuring comparable results. Sensitivity and specificity of viral high-throughput sequence data analysis are highly affected by bioinformatics processing, using publicly available and custom tools and databases, and differ widely between individuals and institutions.Here, we present the results of the COMPARE (COllaborative Management Platform for detection and Analyses of [Re-] emerging and foodborne outbreaks in Europe) in silico virus proficiency test. An artificial, simulated in silico dataset of Illumina HiSeq sequences was provided to 13 different European institutes for bioinformatics analysis towards the identification of viral pathogens in high-throughput sequence data. Comparison of the participants' analyses shows that the use of different tools, programs, and databases for bioinformatics analyses can impact the correct identification of viral sequences from a simple dataset. The identification of slightly mutated and highly divergent virus genomes has been identified as being most challenging: Furthermore, the interpretation of the results together with a fictitious case report by the participants showed that in addition to the bioinformatics analysis, the virological evaluation of the results can be important in clinical settings.External quality assessment and proficiency testing should become an important part of validating high-throughput sequencing-based virus diagnostics and could improve harmonization, comparability, and reproducibility of results. Similar to what is established for conventional laboratory tests like PCR, there is a need for the establishment of international proficiency testing for bioinformatics pipelines and interpretation of such results.
Original languageEnglish
Article numbere00466-19
JournalJournal of Clinical Microbiology
Volume57
Issue number8
Number of pages12
ISSN0095-1137
DOIs
Publication statusPublished - 2019

Bibliographical note

This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International license.

Keywords

  • High-throughput sequencing
  • External quality assessment
  • next-generation sequencing
  • Proficiency testing
  • Virus diagnostics

Cite this

Brinkmann, Annika ; Andrusch, Andreas ; Belka, Ariane ; Wylezich, Claudia ; Höper, Dirk ; Pohlmann, Anne ; Petersen, Thomas Nordahl ; Lucas, Pierrick ; Blanchard, Yannick ; Papa, Anna ; Melidou, Angeliki ; Oude Munnink, Bas B ; Matthijnssens, Jelle ; Deboutte, Ward ; Ellis, Richard J ; Hansmann, Florian ; Baumgärtner, Wolfgang ; van der Vries, Erhard ; Osterhaus, Albert ; Camma, Cesare ; Mangone, Iolanda ; Lorusso, Alessio ; Maracci, Maurilia ; Nunes, Alexandra ; Pinto, Miguel ; Borges, Vítor ; Kroneman, Annelies ; Schmitz, Dennis ; Corman, Victor Max ; Drosten, Christian ; Jones, Terry C ; Hendriksen, Rene S. ; Aarestrup, Frank Møller ; Koopmans, Marion ; Beer, Martin ; Nitsche, Andreas. / Proficiency testing of virus diagnostics based on bioinformatics analysis of simulated in silico high-throughput sequencing datasets. In: Journal of Clinical Microbiology. 2019 ; Vol. 57, No. 8.
@article{bfd2bf26090f43b7b179ec735f5b618c,
title = "Proficiency testing of virus diagnostics based on bioinformatics analysis of simulated in silico high-throughput sequencing datasets",
abstract = "Quality management and independent assessment of high-throughput sequencing-based virus diagnostics have not yet been established as a mandatory approach for ensuring comparable results. Sensitivity and specificity of viral high-throughput sequence data analysis are highly affected by bioinformatics processing, using publicly available and custom tools and databases, and differ widely between individuals and institutions.Here, we present the results of the COMPARE (COllaborative Management Platform for detection and Analyses of [Re-] emerging and foodborne outbreaks in Europe) in silico virus proficiency test. An artificial, simulated in silico dataset of Illumina HiSeq sequences was provided to 13 different European institutes for bioinformatics analysis towards the identification of viral pathogens in high-throughput sequence data. Comparison of the participants' analyses shows that the use of different tools, programs, and databases for bioinformatics analyses can impact the correct identification of viral sequences from a simple dataset. The identification of slightly mutated and highly divergent virus genomes has been identified as being most challenging: Furthermore, the interpretation of the results together with a fictitious case report by the participants showed that in addition to the bioinformatics analysis, the virological evaluation of the results can be important in clinical settings.External quality assessment and proficiency testing should become an important part of validating high-throughput sequencing-based virus diagnostics and could improve harmonization, comparability, and reproducibility of results. Similar to what is established for conventional laboratory tests like PCR, there is a need for the establishment of international proficiency testing for bioinformatics pipelines and interpretation of such results.",
keywords = "High-throughput sequencing, External quality assessment, next-generation sequencing, Proficiency testing, Virus diagnostics",
author = "Annika Brinkmann and Andreas Andrusch and Ariane Belka and Claudia Wylezich and Dirk H{\"o}per and Anne Pohlmann and Petersen, {Thomas Nordahl} and Pierrick Lucas and Yannick Blanchard and Anna Papa and Angeliki Melidou and {Oude Munnink}, {Bas B} and Jelle Matthijnssens and Ward Deboutte and Ellis, {Richard J} and Florian Hansmann and Wolfgang Baumg{\"a}rtner and {van der Vries}, Erhard and Albert Osterhaus and Cesare Camma and Iolanda Mangone and Alessio Lorusso and Maurilia Maracci and Alexandra Nunes and Miguel Pinto and V{\'i}tor Borges and Annelies Kroneman and Dennis Schmitz and Corman, {Victor Max} and Christian Drosten and Jones, {Terry C} and Hendriksen, {Rene S.} and Aarestrup, {Frank M{\o}ller} and Marion Koopmans and Martin Beer and Andreas Nitsche",
note = "This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International license.",
year = "2019",
doi = "10.1128/JCM.00466-19",
language = "English",
volume = "57",
journal = "Journal of Clinical Microbiology",
issn = "0095-1137",
publisher = "American Society for Microbiology",
number = "8",

}

Brinkmann, A, Andrusch, A, Belka, A, Wylezich, C, Höper, D, Pohlmann, A, Petersen, TN, Lucas, P, Blanchard, Y, Papa, A, Melidou, A, Oude Munnink, BB, Matthijnssens, J, Deboutte, W, Ellis, RJ, Hansmann, F, Baumgärtner, W, van der Vries, E, Osterhaus, A, Camma, C, Mangone, I, Lorusso, A, Maracci, M, Nunes, A, Pinto, M, Borges, V, Kroneman, A, Schmitz, D, Corman, VM, Drosten, C, Jones, TC, Hendriksen, RS, Aarestrup, FM, Koopmans, M, Beer, M & Nitsche, A 2019, 'Proficiency testing of virus diagnostics based on bioinformatics analysis of simulated in silico high-throughput sequencing datasets', Journal of Clinical Microbiology, vol. 57, no. 8, e00466-19. https://doi.org/10.1128/JCM.00466-19

Proficiency testing of virus diagnostics based on bioinformatics analysis of simulated in silico high-throughput sequencing datasets. / Brinkmann, Annika; Andrusch, Andreas; Belka, Ariane; Wylezich, Claudia; Höper, Dirk; Pohlmann, Anne; Petersen, Thomas Nordahl; Lucas, Pierrick; Blanchard, Yannick; Papa, Anna; Melidou, Angeliki; Oude Munnink, Bas B; Matthijnssens, Jelle; Deboutte, Ward; Ellis, Richard J; Hansmann, Florian; Baumgärtner, Wolfgang; van der Vries, Erhard; Osterhaus, Albert; Camma, Cesare; Mangone, Iolanda; Lorusso, Alessio; Maracci, Maurilia; Nunes, Alexandra; Pinto, Miguel; Borges, Vítor; Kroneman, Annelies; Schmitz, Dennis; Corman, Victor Max; Drosten, Christian; Jones, Terry C; Hendriksen, Rene S.; Aarestrup, Frank Møller; Koopmans, Marion; Beer, Martin; Nitsche, Andreas.

In: Journal of Clinical Microbiology, Vol. 57, No. 8, e00466-19, 2019.

Research output: Contribution to journalJournal articleResearchpeer-review

TY - JOUR

T1 - Proficiency testing of virus diagnostics based on bioinformatics analysis of simulated in silico high-throughput sequencing datasets

AU - Brinkmann, Annika

AU - Andrusch, Andreas

AU - Belka, Ariane

AU - Wylezich, Claudia

AU - Höper, Dirk

AU - Pohlmann, Anne

AU - Petersen, Thomas Nordahl

AU - Lucas, Pierrick

AU - Blanchard, Yannick

AU - Papa, Anna

AU - Melidou, Angeliki

AU - Oude Munnink, Bas B

AU - Matthijnssens, Jelle

AU - Deboutte, Ward

AU - Ellis, Richard J

AU - Hansmann, Florian

AU - Baumgärtner, Wolfgang

AU - van der Vries, Erhard

AU - Osterhaus, Albert

AU - Camma, Cesare

AU - Mangone, Iolanda

AU - Lorusso, Alessio

AU - Maracci, Maurilia

AU - Nunes, Alexandra

AU - Pinto, Miguel

AU - Borges, Vítor

AU - Kroneman, Annelies

AU - Schmitz, Dennis

AU - Corman, Victor Max

AU - Drosten, Christian

AU - Jones, Terry C

AU - Hendriksen, Rene S.

AU - Aarestrup, Frank Møller

AU - Koopmans, Marion

AU - Beer, Martin

AU - Nitsche, Andreas

N1 - This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International license.

PY - 2019

Y1 - 2019

N2 - Quality management and independent assessment of high-throughput sequencing-based virus diagnostics have not yet been established as a mandatory approach for ensuring comparable results. Sensitivity and specificity of viral high-throughput sequence data analysis are highly affected by bioinformatics processing, using publicly available and custom tools and databases, and differ widely between individuals and institutions.Here, we present the results of the COMPARE (COllaborative Management Platform for detection and Analyses of [Re-] emerging and foodborne outbreaks in Europe) in silico virus proficiency test. An artificial, simulated in silico dataset of Illumina HiSeq sequences was provided to 13 different European institutes for bioinformatics analysis towards the identification of viral pathogens in high-throughput sequence data. Comparison of the participants' analyses shows that the use of different tools, programs, and databases for bioinformatics analyses can impact the correct identification of viral sequences from a simple dataset. The identification of slightly mutated and highly divergent virus genomes has been identified as being most challenging: Furthermore, the interpretation of the results together with a fictitious case report by the participants showed that in addition to the bioinformatics analysis, the virological evaluation of the results can be important in clinical settings.External quality assessment and proficiency testing should become an important part of validating high-throughput sequencing-based virus diagnostics and could improve harmonization, comparability, and reproducibility of results. Similar to what is established for conventional laboratory tests like PCR, there is a need for the establishment of international proficiency testing for bioinformatics pipelines and interpretation of such results.

AB - Quality management and independent assessment of high-throughput sequencing-based virus diagnostics have not yet been established as a mandatory approach for ensuring comparable results. Sensitivity and specificity of viral high-throughput sequence data analysis are highly affected by bioinformatics processing, using publicly available and custom tools and databases, and differ widely between individuals and institutions.Here, we present the results of the COMPARE (COllaborative Management Platform for detection and Analyses of [Re-] emerging and foodborne outbreaks in Europe) in silico virus proficiency test. An artificial, simulated in silico dataset of Illumina HiSeq sequences was provided to 13 different European institutes for bioinformatics analysis towards the identification of viral pathogens in high-throughput sequence data. Comparison of the participants' analyses shows that the use of different tools, programs, and databases for bioinformatics analyses can impact the correct identification of viral sequences from a simple dataset. The identification of slightly mutated and highly divergent virus genomes has been identified as being most challenging: Furthermore, the interpretation of the results together with a fictitious case report by the participants showed that in addition to the bioinformatics analysis, the virological evaluation of the results can be important in clinical settings.External quality assessment and proficiency testing should become an important part of validating high-throughput sequencing-based virus diagnostics and could improve harmonization, comparability, and reproducibility of results. Similar to what is established for conventional laboratory tests like PCR, there is a need for the establishment of international proficiency testing for bioinformatics pipelines and interpretation of such results.

KW - High-throughput sequencing

KW - External quality assessment

KW - next-generation sequencing

KW - Proficiency testing

KW - Virus diagnostics

U2 - 10.1128/JCM.00466-19

DO - 10.1128/JCM.00466-19

M3 - Journal article

VL - 57

JO - Journal of Clinical Microbiology

JF - Journal of Clinical Microbiology

SN - 0095-1137

IS - 8

M1 - e00466-19

ER -