Skip to main content
Download PDF

Predictive performance of automated surveillance algorithms for intravascular catheter bloodstream infections: a systematic review and meta-analysis

Antimicrobial Resistance & Infection Control volume 12, Article number: 87 (2023) Cite this article

Abstract

Background

Intravascular catheter infections are associated with adverse clinical outcomes. However, a significant proportion of these infections are preventable. Evaluations of the performance of automated surveillance systems for adequate monitoring of central-line associated bloodstream infection (CLABSI) or catheter-related bloodstream infection (CRBSI) are limited.

Objectives

We evaluated the predictive performance of automated algorithms for CLABSI/CRBSI detection, and investigated which parameters included in automated algorithms provide the greatest accuracy for CLABSI/CRBSI detection.

Methods

We performed a meta-analysis based on a systematic search of published studies in PubMed and EMBASE from 1 January 2000 to 31 December 2021. We included studies that evaluated predictive performance of automated surveillance algorithms for CLABSI/CRBSI detection and used manually collected surveillance data as reference. We estimated the pooled sensitivity and specificity of algorithms for accuracy and performed a univariable meta-regression of the different parameters used across algorithms.

Results

The search identified five full text studies and 32 different algorithms or study populations were included in the meta-analysis. All studies analysed central venous catheters and identified CLABSI or CRBSI as an outcome. Pooled sensitivity and specificity of automated surveillance algorithm were 0.88 [95%CI 0.84–0.91] and 0.86 [95%CI 0.79–0.92] with significant heterogeneity (I2 = 91.9, p < 0.001 and I2 = 99.2, p < 0.001, respectively). In meta-regression, algorithms that include results of microbiological cultures from specific specimens (respiratory, urine and wound) to exclude non-CRBSI had higher specificity estimates (0.92, 95%CI 0.88–0.96) than algorithms that include results of microbiological cultures from any other body sites (0.88, 95% CI 0.81–0.95). The addition of clinical signs as a predictor did not improve performance of these algorithms with similar specificity estimates (0.92, 95%CI 0.88–0.96).

Conclusions

Performance of automated algorithms for detection of intravascular catheter infections in comparison to manual surveillance seems encouraging. The development of automated algorithms should consider the inclusion of results of microbiological cultures from specific specimens to exclude non-CRBSI, while the inclusion of clinical data may not have an added-value.

Trail Registration Prospectively registered with International prospective register of systematic reviews (PROSPERO ID CRD42022299641; January 21, 2022). https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022299641

Background

Intravascular catheters (IVC) are indispensable and commonly used medical devices in hospitalized patients, and substantially predispose patients to develop healthcare-associated infections (HAIs) [1,2,3,4,5,6,7,8,9].

In 2016, the mean prevalence of HAIs in European countries was estimated to be 6.5% [1]. Hospital-acquired bloodstream infections (HA-BSI) account for 14.2% of HAI [2] and a large proportion of HA-BSI is attributable to IVC. In Europe, both central line-associated bloodstream infection (CLABSI) and catheter-related bloodstream infection (CRBSI) represent 36.5% of intensive care unit (ICU)-acquired bloodstream infections (BSIs) and the incidence has been found to vary between 1.7 and 4.8 episodes per 1000 catheter days [4]. CLABSI/CRBSI are associated with excess mortality rates, extended duration of hospitalization and greater healthcare expenditure [5,6,7,8,9]. More than 50% of CLABSI/CRBSI can be considered as preventable [3].

Surveillance of CLABSI/CRBSI among patients with IVC allows for the burden of disease to be quantified and for the effectiveness of interventions to prevent CLABSI/CRBSI to be assessed. Automated algorithms may offer approaches to improve the efficiency of CLABSI/CRBSI surveillance. Compared to manual surveillance, automated surveillance has been demonstrated to reduce time and workload for healthcare workers and infection control practitioners, and to provide data in real time that may allow for more timely clinical interventions [10,11,12]. However, little is known as to the predictive performance of automated algorithms for CLABSI/CRBSI surveillance, as well as the relative performance of different parameters that could be used within automated algorithms for the timely identification of CLABSI/CRBSI among patients with IVC.

The main objective of this study was to evaluate the predictive performance of automated surveillance systems for the detection of CLABSI/CRBSI among hospitalized patients, and to identify which parameters have a greater influence on the predictive performance, so as to inform future automated surveillance algorithms for CLABSI/CRBSI detection.

Methods

Design

We performed a systematic review and meta-analysis on the predictive performance of automated surveillance algorithms for the detection of CLABSI/CRBSI among hospitalized patients. This study was registered within the PROSPERO international prospective register of systematic reviews (CRD42022299641) on January 21, 2022, and was reported in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement [13, 14].

Search strategy

We conducted a systematic search using two electronic databases, PubMed and EMBASE, for relevant articles published between 1 January 2000 and 31 December 2021. We searched for original studies using the keyword algorithms described in the supplementary material. The search was limited to articles published in English. We searched for studies that reported on the predictive performance of automated algorithms for the detection of HAI (to increase the sensitivity of the search strategy) and of intravascular catheter infections (to increase the specificity of the search strategy). The records from the two databases search were merged and duplicates were removed using the EndNote program (Thomson Reuters, New-York, NY, USA).

Study selection

Two investigators (J.M.J. and N.L.) screened titles and abstracts and examined the full text of original articles selected for study inclusion independently and in duplicate; disagreements were resolved by consensus.

Inclusion criteria

Original studies were included if they assessed accuracy of automated algorithms for the surveillance of CLABSI and/or CRBSI. CLABSI was defined by one positive blood culture and clinical manifestation of infection in a patient with a catheter in place and with no other source of bacteremia except the catheter. CRBSI was defined as one positive blood culture obtained from peripheral vein and clinical manifestation of infection, and at least one of the following: (1) a positive CVC culture with the same micro-organism by qualitative or semi-quantitative (i.e., ≥ 15 CFU) methods or (2) a differential time to positivity of more than 120 min between central catheter blood culture and peripheral blood culture (blood samples drawn at the same time), or (3) a ratio of microorganism quantity from CVC blood sample on microorganism quantity from peripheral blood sample greater than 3 [15,16,17]. We selected studies that compared the predictive performance of automated algorithms with data from manual surveillance. Selected studies needed to include sensitivity and specificity estimates calculated using diagnostic test methods. Moreover, studies needed to directly or indirectly include all of the following: number of true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). Studies that did not provide all these data to determine the predictive performance of the automated algorithm were excluded.

Data extraction and quality assessment

Data were extracted from the selected studies according to predefined rules that were used to identify IVC infections. If multiple algorithms (i.e., algorithms with different definitions for identifying IVC infections) or multiple study populations were evaluated in a single study, we defined each single algorithm as a single observation in the meta-analysis. The total number of algorithms analyzed was therefore higher than the number included studies. For each algorithm, we extracted the data on the endpoint (CLABSI/CRBSI) and on the predictive performance of the automated algorithm: TP, FP, FN, and TN.

From these different algorithms, we identified individual and pooled parameters for CLABSI/CRBSI detection.

Using the revised tool for the Quality Assessment of Diagnostic Accuracy Studies (QUADAS 2) [18], we evaluated the quality of studies based on four items: patient selection, index test, reference standard, and flow and timing. We assessed intra-study risk of bias and concern for applicability using a three-level rating scale (high, low or unclear).

Statistical analysis

We first described characteristics of included studies (mono versus multicenter, type of catheter included, setting) and outcomes. We then estimated pooled sensitivity and specificity of automated surveillance algorithms for the identification of CLABSI/CRBSI with 95% confidence intervals (95% CI) for algorithms using bivariate random-effects models. We used I2 statistics to assess potential heterogeneity between algorithms [19, 20], with I2 > 75% representing considerable heterogeneity. We subsequently calculated areas under summary receiver operating characteristic curves (SAUROC), and we used plots observed data in ROC plane to assess threshold effect visually. In addition, we used graphical model tests to depict both the residual-based goodness-of-fit and the bivariate normal distribution, to check for how observations influenced analyses and to detect outliers.

Finally, we performed a meta-regression to explore how individual and pooled parameters included in the algorithms influenced the performance of automated algorithms for CLABSI/CRBSI detection as compared to manual surveillance.

We used the “midas” command (meta-analysis integration of diagnostic test accuracy studies), a comprehensive program of statistical and graphical routines for undertaking meta-analysis of diagnostic test performance in STATA developed by Dwamena [21]. Each individual parameter and combination of parameter were included in the meta-regression as an independent explanatory variable. We considered results as significant for P-values < 0.05. We used STATA/MP software (version 16.0). STATA codes are reported in the Additional file 1.

Results

Systematic literature search

We identified 586 non-redundant study records. Eighty (13.7%) full text articles were assessed for eligibility after abstract and title screening (Additional file 1: Fig. S1). Of these, 5 (1%) were included in the systematic review and meta-analysis [22,23,24,25,26]. Details on the search strategy are given in the Additional file 1.

Characteristics of included studies

Three of the studies included in the systematic review were monocentric [23,24,25] and the remaining two multicentric [22, 26] (Table 1). Four studies analyzed only central venous catheters (CVC) and used CLABSI as an outcome [22, 24,25,26]. One study included unspecified IVC and used CRBSI as an outcome [23]. Two studies (40%) were conducted in the ICU setting [24, 26]. All studies were observational and used manually collected surveillance data as reference [22,23,24,25,26].

Table 1 Characteristics of included studies
Full size table

Across the 5 studies, 32 different automated algorithms or population samples were identified and included in the meta-analysis. Among the 32 algorithms, we identified 7 individual parameters, and 9 combinations of parameters used for automated detection of CLABSI/CRBSI (Table 2 and Additional file 1: Table S1). These 16 single or pooled parameters were then tested in the univariable meta-regression.

Table 2 Individual and pooled parameters used in the meta-regression
Full size table

Quality of studies

Using the QUADAS-2 tool, the quality of the five studies included in the systematic review and meta-analysis was assessed to be high (Additional file 1: Table S2). Overall, the risk of bias was low: two studies were rated as low risk of potential bias, among all assessed categories [22, 24], and three studies were rated as high risk of bias for only one algorithm within each study [23, 25, 26]. The applicability was rated as high for four of the five studies [22, 24,25,26], and only one study had low applicability in two of three algorithms identified in the study [23]. The evaluation of publication bias is illustrated in Additional file (Additional file 1: Figs. S2 and S3).

Pooled sensitivity, specificity and SAUROC

The pooled sensitivity and specificity were 0.89 [95% CI 0.85–0.92] and 0.83 [95% CI 0.71–0.91] with significant heterogeneity between algorithms included in the meta-analysis (I2 = 91.36, p < 0.001 and I2 = 99.20, p < 0.001), respectively (Fig. 1).

Fig. 1
figure 1

Forest plot of diagnostic performance, including sensitivity and specificity. *Studies with the best performance based on both sensitivity/specificity (sensitivity ≥ 0.89 and specificity ≥ 0.83)

Full size image

The area under the SAUROC curve was 0.93 [95% CI 0.91–0.95] and this identified four algorithms with sensitivity greater than 0.89 and specificity greater than 0.83 (Figure S4A). These 4 best performing algorithms were all from the study of Woeltje et al. 2011 [26]. Algorithm 17 was defined by the combination of the four following parameters: “Hospital-acquired BSI with CVC”, “True BSI for common skin commensal (CSC)”, “Clinical data” and “Cultures from any other site” and has a sensitivity of 0.95 [95% CI 0.88–0.99] and a specificity of 0.98 [95% CI 0.95–0.99]. Three algorithms (19, 20 and 21) combined the same three parameters “Hospital-acquired BSI with CVC”, “True BSI for CSC” and “Cultures from any other site” and differed regarding the time window considered for the parameter “Cultures from other body sites” (from admission to the positive blood culture (algorithm 19; sensitivity 0.94 [95% CI 0.87–0.98] and specificity 0.94 [95% CI 0.91–0.96]) or from admission and within 7 days before or after the positive blood culture (algorithm 20; sensitivity 0.92 [95% CI 0.84–0.97] and specificity 0.98 [95% CI 0.96–0.99]) or from admission and within 14 days before to 7 days after the positive blood culture (algorithm 21; sensitivity 0.92 [95% CI 0.84–0.97] and specificity 0.98 [95% CI 0.96–0.99]).

Outliers regarding sensitivity and specificity are illustrated in the Additional file (Additional file 1: Fig. S2).

Meta-regression

Given the heterogeneity between algorithms included in the meta-analysis, the meta-regression sought to identify which parameters had the greatest influence on the measures of effect. Additional file 1: Fig. S5 shows the frequency of all individual and pooled parameters identified within the 32 algorithms. The individual and pooled parameters for CLABSI/CRBSI detection derived from univariable meta-regression is illustrated in Fig. 2.

Fig. 2
figure 2

Univariable meta-regression for intravascular catheter bloodstream infection criteria. Blue reference line shows the pooled sensitivity and specificity, respectively. Letters A to G refers to Individual and pooled parameters described in Table 2

Full size image

Figure 3 summarizes how each parameter affects the sensitivity and specificity of the algorithm. For two parameters only, “Hospital-acquired BSI with CVC” and “Administration of antibiotics”, the sensitivity increased, and the specificity decreased when the parameter was present. When “Hospital-acquired BSI with CVC” was the only parameter considered, the sensitivity was 0.98 (95% CI 0.96–1.00) as compared to 0.83 (95%CI 0.83- 0.90) when it was not considered (p = 0.25). When “administration of antibiotics” was the only parameter considered, the sensitivity was 0.92 (95% CI 0.85–0.99) as compared to 0.88 (95% CI 0.84–0.92) when it was not considered. For all other individual and pooled parameters, the sensitivity decreased, and the specificity increased when the parameter was present.

Fig. 3
figure 3

Decision tree based on different algorithm’s parameters

Full size image

The combination of parameters with the highest specificities always included the parameter “True BSI for common skin commensal”. In addition, combination with highest specificities included either “Culture from specific other sites” or “Culture from any other site” and either “New episode” or “Clinical data”. The combinations that included the parameter “Culture from specific other sites” had higher specificities (0.92; 95% CI 0.88–0.96) than the combinations that include “Culture from any other site” (0.88; 95% CI 0.81–0.95). The addition of the parameter “Clinical data” in the combination did not increase the specificity of the estimate: 0.92 (95% CI; 0.88–0.96) with the parameter included vs. 0.92 (95% CI; 0.88–0.96) without it.

Discussion

To our knowledge, this is the first meta-analysis that has reported pooled sensitivity and specificity as predictive performance of automated surveillance algorithms, using bivariate random-effects approach from 5 studies published in the last two decades [27,28,29,30]. This meta-analysis highlighted two main findings: (1) the pooled sensitivity was high but heterogeneous across all algorithms; and (2) the pooled specificity was also heterogeneous, but the meta-regression allowed to identify several individual and pooled parameters that had a greater influence on the measure of effect, and which could therefore inform the development of further automated surveillance algorithms for the detection of CLABSI/CRBSI.

Few studies have investigated the predictive accuracy of both CLABSI and CRBSI using automated surveillance systems. Frequently, authors have focused on CVC and CLABSI, thus disregarding other IVC and the more specific definition CRBSI. Indeed, the CRBSI definition frequently needs catheter removal and catheter tip culture, which are not commonly performed in most countries [31]. However, CRBSI allows a higher degree of certainty in the attribution of the catheter in the occurrence of the BSI [17], as compared to the CLABSI definition. To our knowledge, only one previous systematic review collated data on automated surveillance algorithms for the detection of IVC infections [27]. However, the authors of this systematic review focused on all HAIs, did not perform a meta-analysis and did not assess heterogeneity in the definitions used for automated surveillance of IVC infections.

In our study the pooled sensitivity was high (i.e., > 85%), and no individual or pooled parameter substantially influenced it. This finding is probably explained by the simplicity of detection of BSI compared to other HAIs (e.g. surgical site infections or ventilator-associated pneumonias) that require more complicated detection strategies and inclusion of clinical data or radiological findings.

Several factors influenced specificities and could improve the accuracy of automated monitoring algorithms. First, the addition of “clinical signs” did not substantially improve the accuracy of the specificity compared to similar algorithms that included microbiological cultures from other body sites. Second, higher specificities could be achieved by including only microbiological data from respiratory tract, urinary tract or wound samples instead of microbiological data from any other body sites. Clinical signs are frequently difficult to collect in hospital databases because of the lack of structure and standardization of the data, whereas microbiological cultures, which usually rely only on microbiological laboratories, are often more harmonized, more easily extractable and therefore integrated in an automated surveillance algorithm. Automated algorithms should firstly integrate microbiological data from other selected body sites to exclude non-catheter associated bloodstream infections.

Our results allow us to make some suggestions regarding which parameters are relevant for clinical decision making or hypothesis generation for further clinical studies investigating this issue, based on the balance between sensitivity and specificity and on clinical relevance (Fig. 3). We suggest that the following parameters should be relevant for inclusion in an automated CLABSI detection algorithm: (i) a parameter to differentiate a common skin contaminant from a true causative pathogen; (ii) a parameter to define two distinct infectious episodes; and (iii) a parameter to consider cultures from other specific body sites to exclude infection not associated with an IVC. That is, our results allow us to infer that any automated algorithm for IVC bloodstream surveillance needs to be able to: distinguish the causative pathogen from common skin commensal; consider the same microorganism isolated in a separate blood culture to be a new infectious episode if identified after at least between 3 and 7 days after the first episode; and differentiate any positive culture(s) from other body sites as being non-catheter related.

The use of a meta-regression versus predictive scores (e.g., Infection Probability Score (IPS) [32], or its modification for central venous catheter-related bloodstream infections [33]) remains debated. While the development and validation of predictive score are usually based one restricted and temporally defined sample (i.e., study), systematic reviews, meta-regression and meta-analysis include several samples (i.e., several clusters, representing different studies). Predictive scores can be validated externally in other settings, allowing model discrimination and calibration performance across several settings. However, random-effects meta-regression or meta-analysis provide the additional benefit of average performance and heterogeneity in performance across different studies. Accordingly, systematic reviews, meta-regression and meta-analysis of validation studies provide a summary of predictive performance from different settings and populations [34]. In other words, meta-regressions or meta-analysis provide a potentially more accurate and relevant measure of performance, taking into account context and heterogeneity factors. This makes meta-regressions and meta-analysis a more relevant tool for decision making purposes taking into account the external validation of an automated algorithm.

This study has some limitations. First, the low number of studies included suggests that the accuracy of the estimates of automated monitoring algorithms for CLABSI/CRBSI detection could be limited by the potential heterogeneity of data. Moreover, the inclusion of Bellini et al. [23] study, which relied on CRBSI definition only, could have increased the risk of heterogeneity. Second, classification of algorithms could be questionable, and the heterogeneity in definitions did not allow us to develop accurate algorithms (e.g., details of clinical signs were not specified, time-windows were simplified into our meta-analysis by regrouping time-windows from different studies). Third, the generalizability of our conclusions could be limited, because studies were frequently monocentric and were all performed in high income countries. Fourth, grey literature was not screened. Finally, mostly CVC were included; it is conceivable that our results are not applicable to other intravascular catheter types.

Conclusions

Our meta-regression examined the accuracy of automated algorithms developed to monitor CLABSI/CRBSI and provides valuable information while developing valid algorithms for automated monitoring of intravascular catheter infections. Microbiological cultures from selected other body sites could help to exclude BSI not related to IVC, whereas clinical signs did not substantially improve the accuracy of automated systems when microbiological cultures from selected other body sites were included.

Availability of data and materials

Data and materials are available in the text of the article and in Supplementary material. JMJ had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

References

  1. Suetens C, Latour K, Kärki T, Ricchizzi E, Kinross P, Moro ML, et al. Healthcare-Associated Infections Prevalence Study Group. Prevalence of healthcare-associated infections estimated incidence and composite antimicrobial resistance index in acute care hospitals and long-term care facilities: results from two European point prevalence surveys, 2016 to 2017. Euro Surveill. 2018;23(46):1800516. Erratum in: Euro Surveill. 2018;23(47).

  2. Zarb P, Coignard B, Griskeviciene J, Muller A, Vankerckhoven V, Weist K, et al. National Contact Points for the ECDC pilot point prevalence survey; Hospital Contact Points for the ECDC pilot point prevalence survey. The european centre for disease prevention and control (ECDC) pilot point prevalence survey of healthcare-associated infections and antimicrobial use. Euro Surveill. 2012;17(46):20316.

  3. Schreiber PW, Sax H, Wolfensberger A, Clack L, Kuster SP, Swissnoso. The preventable proportion of healthcare-associated infections 2005–2016: Systematic review and meta-analysis. Infect Control Hosp Epidemiol. 2018;39(11):1277–95.

  4. European Centre for Disease Prevention and Control. Healthcare-associated infections in intensive care units - Annual Epidemiological Report for 2017. ECDC; 2019. https://www.ecdc.europa.eu/sites/default/files/documents/AER_for_2017-HAI.pdf.

  5. Dimick JB, Pelz RK, Consunji R, Swoboda SM, Hendrix CW, Lipsett PA. Increased resource use associated with catheter-related bloodstream infection in the surgical intensive care unit. Arch Surg. 2001;136:229.

    Article  CAS  PubMed  Google Scholar 

  6. Olaechea PM, Palomar M, Álvarez-Lerma F, Otal JJ, Insausti J, López-Pueyo MJ, ENVIN-HELICS Group. Morbidity and mortality associated with primary and catheter-related bloodstream infections in critically ill patients. Rev Esp Quimioter. 2013;26:21–9.

  7. Ziegler MJ, Pellegrini DC, Safdar N. Attributable mortality of central line associated bloodstream infection: systematic review and metaanalysis. Infection. 2015;43:29–36.

    Article  PubMed  Google Scholar 

  8. Zimlichman E, Henderson D, Tamir O, Franz C, Song P, Yamin CK, et al. Health care-associated infections: a meta-analysis of costs and financial impact on the US health care system. JAMA Intern Med. 2013;173:2039–46.

    Article  PubMed  Google Scholar 

  9. Stevens V, Geiger K, Concannon C, Nelson RE, Brown J, Dumyati G. Inpatient costs, mortality and 30-day re-admission in patients with central-line-associated bloodstream infections. Clin Microbiol Infect. 2014;20(5):318–24.

    Article  Google Scholar 

  10. Bae S, Kim Y, Chang HH, Kim S, Kim HJ, Jeon H, et al. The effect of the multimodal intervention including an automatic notification of catheter days on reducing central line-related bloodstream infection: a retrospective, observational, quasi-experimental study. BMC Infect Dis. 2022;22(1):604.

    Article  PubMed  PubMed Central  Google Scholar 

  11. van Mourik MSM, van Rooden SM, Abbas M, Aspevall O, Astagneau P, Bonten MJM, et al. PRAISE network. PRAISE: providing a roadmap for automated infection surveillance in Europe. Clin Microbiol Infect. 2021;27 (Suppl 1):S3-S19.

  12. Verberk JDM, Aghdassi SJS, Abbas M, Nauclér P, Gubbels S, Maldonado N, et al. Automated surveillance systems for healthcare-associated infections: results from a European survey and experiences from real-life utilization. J Hosp Infect. 2022;122:35–43.

    Article  CAS  PubMed  Google Scholar 

  13. Moher D, Liberati A, Tetzlaff J, Altman DG. PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses. BMJ. 2009;339:b2535.

  14. https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022299641.

  15. Mermel LA, Allon M, Bouza E, Craven DE, Flynn P, O’Grady NP, et al. Clinical practice guidelines for the diagnosis and management of intravascular catheter-related infection: 2009 update by the infectious diseases society of America. Clin Infect Dis. 2009;49(1):1–45.

    Article  CAS  PubMed  Google Scholar 

  16. Timsit JF, Baleine J, Bernard L, Calvino-Gunther S, Darmon M, Dellamonica J, et al. Expert consensus-based clinical practice guidelines management of intravascular catheters in the intensive care unit. Ann Intensive Care. 2020;10(1):118.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Buetti N, Timsit JF. Management and prevention of central venous catheter-related infections in the ICU. Semin Respir Crit Care Med. 2019;40(4):508–23.

    Article  PubMed  Google Scholar 

  18. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2 Group. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529–36.

  19. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539–58.

    Article  PubMed  Google Scholar 

  20. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557–60.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Dwamena BA. MIDAS: Stata module for meta-analytical integration of diagnostic test accuracy studies. Statistical Software Components. 2007. https://econpapers.repec.org/RePEc:boc:bocode:s456880.

  22. Trick WE, Zagorski BM, Tokars JI, Vernon MO, Welbel SF, Wisniewski MF, et al. Computer algorithms to detect bloodstream infections. Emerg Infect Dis. 2004;10(9):1612–20.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Bellini C, Petignat C, Francioli P, Wenger A, Bille J, Klopotov A, et al. Comparison of automated strategies for surveillance of nosocomial bacteremia. Infect Control Hosp Epidemiol. 2007;28(9):1030–5.

    Article  PubMed  Google Scholar 

  24. Woeltje KF, Butler AM, Goris AJ, Tutlam NT, Doherty JA, Westover MB, et al. Automated surveillance for central line-associated bloodstream infection in intensive care units. Infect Control Hosp Epidemiol. 2008;29(9):842–6.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Woeltje KF, McMullen KM, Butler AM, Goris AJ, Doherty JA. Electronic surveillance for healthcare-associated central line-associated bloodstream infections outside the intensive care unit. Infect Control Hosp Epidemiol. 2011;32(11):1086–90.

    Article  PubMed  Google Scholar 

  26. Snyders RE, Goris AJ, Gase KA, Leone CL, Doherty JA, Woeltje KF. Increasing the reliability of fully automated surveillance for central line-associated bloodstream infections. Infect Control Hosp Epidemiol. 2015;36(12):1396–400.

    Article  PubMed  Google Scholar 

  27. Streefkerk H, Roel A, VerkooijenRoel PAJ, BramerWichor M, Verbrugh HA. Electronically assisted surveillance systems of healthcare-associated infections: a systematic review. Euro Surveill. 2020. https://doi.org/10.2807/1560-7917.ES.2020.25.2.1900321.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Cosentino CB, Mitchell BG, Brewster DJ, Russo PL. The utility of frailty indices in predicting the risk of health care associated infections: a systematic review. Am J Infect Control. 2021;49(8):1078–84.

    Article  PubMed  Google Scholar 

  29. van Mourik MS, van Duijn PJ, Moons KG, Bonten MJ, Lee GM. Accuracy of administrative data for surveillance of healthcare-associated infections: a systematic review. BMJ Open. 2015;5(8): e008424.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Ward MA, Schweizer ML, Polgreen PM, Gupta K, Reisinger HS, Perencevich EN. Automated and electronically assisted hand hygiene monitoring systems: a systematic review. Am J Infect Control. 2014;42(5):472–8.

    Article  PubMed  Google Scholar 

  31. Peterson LR, Smith BA. Nonutility of catheter tip culture for diagnosis of central line-associated bloodstream infection. Clin Infect Dis. 2015;60(3):492–3.

    Article  PubMed  Google Scholar 

  32. Peres Bota D, Mélot C, Lopes Ferreira F, Vincent JL. Infection Probability Score (IPS): a method to help assess the probability of infection in critically ill patients. Crit Care Med. 2003;31:2579–84.

    Article  PubMed  Google Scholar 

  33. Schalk E, Hanus L, Färber J, Fischer T, Heidel FH. Prediction of central venous catheter–related bloodstream infections (CRBSIs) in patients with haematologic malignancies using a modified Infection Probability Score (mIPS). Ann Hematol. 2015;94:1451–6.

    Article  PubMed  Google Scholar 

  34. Snell KI, Ensor J, Debray TP, Moons KG, Riley RD. Meta-analysis of prediction model performance across multiple studies: Which scale helps ensure between-study normality for the C-statistic and calibration measures? Stat Methods Med Res. 2018;27(11):3505–22.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank Daniel Teixeira and Loïc Fortchantre for their valuable inputs. We thank all members of Swissnoso.

Collaborators: For Swissnoso, the National Center for Infection Control: Carlo Balmelli, Niccolò Buetti, Delphine Berthod, Stephan Harbarth, Philipp Jent, Jonas Marschall, Hugo Sax, Matthias Schlegel, Alexander Schweiger, Laurence Senn, Rami Sommerstein, Sarah Tschudin-Sutter, Nicolas Troillet, Danielle Vuichard Gysin, Andreas F Widmer, Aline Wolfensberger, Walter Zingg

Funding

Open access funding provided by University of Geneva. This study was a part of the development module of the nationwide surveillance system of intravascular catheter infections in Switzerland financed by the NOSO strategy of the Federal Office of Public Health of Switzerland.

Author information

Author notes

  1. Gaud Catho and Niccolò Buetti have contributed equally to this work.

Authors and Affiliations

  1. Infection Control Program and WHO Collaborating Centre, Geneva University Hospitals and Faculty of Medicine, Service PCI, Rue Gabrielle-Perret-Gentil 4, 1205, Geneve, Switzerland

    Jean-Marie Januel, Nasim Lotfinejad, Rebecca Grant, Stephan Harbarth, Gaud Catho & Niccolò Buetti

  2. Division of Infectious Diseases & Hospital Epidemiology, University Hospital Basel and University of Basel, Basel, Switzerland

    Sarah Tschudin-Sutter

  3. Department of Infectious Diseases and Hospital Epidemiology, University Hospital Zurich and University of Zurich, Zurich, Switzerland

    Peter W. Schreiber

  4. Service of Hospital Preventive Medicine, Lausanne University Hospital, Lausanne, Switzerland

    Bruno Grandbastien

  5. Department of Infectious Diseases, Bern University Hospital, University of Bern, Bern, Switzerland

    Philipp Jent

  6. Department of Infectious Diseases and Hospital Epidemiology, EOC Regional Hospital of Lugano, Lugano, Switzerland

    Elia Lo Priore

  7. Swissnoso, National Center for Infection Control, Bern, Switzerland

    Alexandra Scherrer

  8. Division of Infectious Diseases, Central Institute, Valais Hospital, Sion, Switzerland

    Gaud Catho

  9. Université de Paris, INSERM, IAME UMR 1137, 75018, Paris, France

    Niccolò Buetti

Authors
  1. Jean-Marie Januel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nasim Lotfinejad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rebecca Grant
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sarah Tschudin-Sutter
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peter W. Schreiber
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bruno Grandbastien
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Philipp Jent
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Elia Lo Priore
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Alexandra Scherrer
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Stephan Harbarth
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Gaud Catho
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Niccolò Buetti
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

Swissnoso

  • Carlo Balmelli
  • , Niccolò Buetti
  • , Delphine Berthod
  • , Stephan Harbarth
  • , Philipp Jent
  • , Jonas Marschall
  • , Hugo Sax
  • , Matthias Schlegel
  • , Alexander Schweiger
  • , Laurence Senn
  • , Rami Sommerstein
  • , Sarah Tschudin-Sutter
  • , Nicolas Troillet
  • , Danielle Vuichard Gysin
  • , Andreas F Widmer
  • , Aline Wolfensberger
  •  & Walter Zingg

Contributions

Study concept and design: JMJ, NL, GG, RG, NB. Acquisition of data: JMJ, NL Statistical analysis: JMJ. Interpretation of data: JMJ, GC, RG, NB. Drafting of the manuscript: JMJ; GC, NB. Critical revision of the manuscript: JMJ, GC, RG, STS, PWS, BG, PJ, ELP, AS, SH.

Corresponding author

Correspondence to Jean-Marie Januel.

Ethics declarations

Ethical approval and consent to participate

Not applicable.

Competing interests

PWS received travel grants from Pfizer and Gilead, speaker’s honorary from Pfizer and fees for advisory board activity from Pfizer and Gilead outside of the submitted work. We declare that the authors have no competing interests as defined by BMC, or other interests that might be perceived to influence the results and/or discussion reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1

. Search strategy, Supplementray figures and tables.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Januel, JM., Lotfinejad, N., Grant, R. et al. Predictive performance of automated surveillance algorithms for intravascular catheter bloodstream infections: a systematic review and meta-analysis. Antimicrob Resist Infect Control 12, 87 (2023). https://doi.org/10.1186/s13756-023-01286-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13756-023-01286-0

Keywords

Antimicrobial Resistance & Infection Control

ISSN: 2047-2994

Contact us