Skip to main content
Download PDF

The accuracy of fully-automated algorithms for the surveillance of central venous catheter-related bloodstream infection in hospitalised patients

Antimicrobial Resistance & Infection Control volume 13, Article number: 15 (2024) Cite this article

Abstract

Background

Continuous surveillance for healthcare-associated infections such as central venous catheter-related bloodstream infections (CVC-BSI) is crucial for prevention. However, traditional surveillance methods are resource-intensive and prone to bias. This study aimed to develop and validate fully-automated surveillance algorithms for CVC-BSI.

Methods

Two algorithms were developed using electronic health record data from 1000 admissions with a positive blood culture (BCx) at Karolinska University Hospital from 2017: (1) Combining microbiological findings in BCx and CVC cultures with BSI symptoms; (2) Only using microbiological findings. These algorithms were validated in 5170 potential CVC-BSI-episodes from all admissions in 2018–2019, and results extrapolated to all potential CVC-BSI-episodes within this period (n = 181,354). The reference standard was manual record review according to ECDC’s definition of microbiologically confirmed CVC-BSI (CRI3-CVC).

Results

In the potential CVC-BSI-episodes, 51 fulfilled ECDC’s definition and the algorithms identified 47 and 49 episodes as CVC-BSI, respectively. Both algorithms performed well in assessing CVC-BSI. Overall, algorithm 2 performed slightly better with in the total period a sensitivity of 0.880 (95%-CI 0.783–0.959), specificity of 1.000 (95%-CI 0.999–1.000), PPV of 0.918 (95%-CI 0.833–0.981) and NPV of 1.000 (95%-CI 0.999–1.000). Incidence according to the reference and algorithm 2 was 0.33 and 0.31 per 1000 in-patient hospital-days, respectively.

Conclusions

Both fully-automated surveillance algorithms for CVC-BSI performed well and could effectively replace manual surveillance. The simpler algorithm, using only microbiology data, is suitable when BCx testing adheres to recommendations, otherwise the algorithm using symptom data might be required. Further validation in other settings is necessary to assess the algorithms’ generalisability.

Background

Healthcare-associated infections (HAIs) are common adverse events, affecting millions of patients annually, imposing a significant burden on the healthcare system and leading to extended hospital stays, increased morbidity, mortality, and higher costs [1,2,3]. Bloodstream infections (BSIs) constitute nearly 11% of all HAIs, affecting around 375,000 patients yearly in Europe [3]. Among these, central venous catheter-related BSIs (CVC-BSIs), associated with the use of these medical devices, remain a significant concern despite recent progress in their prevention [4, 5]. The incidence of CVC-BSI is often considered a key indicator of the effectiveness of infection prevention and control measures in healthcare settings [5].

A substantial proportion of HAIs are preventable [1, 4, 5]. Continuous surveillance with feedback to healthcare personnel and stakeholders is essential to effectively allocate resources and evaluate interventions [6, 7]. Still, most HAI surveillance relies on time-consuming and resource-intensive manual record review, prone to subjective interpretation and surveillance bias [8,9,10]. However, with the adoption of electronic health record (EHR) systems, detailed digital EHR data is accessible which is enabling the development of automated surveillance methods, thereby reducing workload, and providing standardised and continuous surveillance data [11, 12]. Nonetheless, these surveillance algorithms must undergo thorough validation before implementation.

Automated surveillance algorithms for CVC-BSI have been developed [13,14,15,16,17,18,19,20,21,22,23,24,25,26]. However, to our knowledge, the majority were based on an ICU population [13,14,15,16,17,18,19], were semi-automated [13,14,15,16, 20,21,22,23], and nearly all used the central line-associated BSI (CLABSI) criteria of the Centers for Disease Control and Prevention (CDC) as reference [13,14,15, 17,18,19,20,21,22,23,24,25,26] and none the microbiologically confirmed CVC-BSI (CRI3-CVC) definition of the European Centre for Disease Prevention and Control (ECDC).

In this study, the aim was to develop fully-automated rule-based surveillance algorithms using EHR data for the detection of CVC-BSI in hospitalised patients, and validate it against manual record review according to ECDC’s CRI3-CVC definition. The best performing algorithm was applied over a four-year period to visualise the incidence of CVC-BSI.

Methods

Study design and data source

This retrospective cohort study used prospectively entered EHR data from the Karolinska University Hospital (KUH) stored in the 2SPARE (2020 started Stockholm/Sweden Proactive Adverse Events REsearch) database. KUH is a tertiary care academic centre in Stockholm, Sweden with 1100 beds divided between two sites (Huddinge and Solna), which serves a population of 2.3 million inhabitants, i.e., the entire population of Region Stockholm. Data included demographics, hospital administrative data, disease and procedure codes, microbiological results, physiological parameters, medication, and medical notes. For this study hospital admissions between 2017 and 2019 were used (Fig. 1). Admissions which included obstetric wards were excluded due to lack of complete data. For the intensive care unit (ICU) no structured data on physiological parameters were available.

Fig. 1
figure 1

Flow chart of study. BCx: blood culture; CRI3-CVC: microbiologically confirmed central venous catheter-related bloodstream infection; CVC-BSI: central venous catheter-related bloodstream infection; ECDC: European Centre for Disease Prevention and Control; HA: healthcare-associated. Positive BCx: BCx with any growth of microorganisms (pathogens and/or contaminants). CVC-BSI-episode: all BCx taken during admission were regarded as potential CVC-BSI-episodes, and each admission without BCx taken counted as one potential CVC-BSI-episode

Full size image

The algorithms were functionally developed in a simple random sample of 1000 admissions with a positive blood culture (BCx) from 2017. Only (semi-)structured variables were used. Admissions from 2018–2019 (n = 132,850) were used as the validation period cohort (Fig. 1). These admissions were divided into three groups: (1) admissions with at least one positive BCx (n = 5145), (2) admissions with only negative BCx (n = 25,132), and (3) admissions without BCx performed (n = 102,573). From these three groups 1000, 250 and 250 admissions, respectively, were selected via simple random selection for the validation dataset. In this validation dataset the presence of CVC-BSI was assessed by three trained professionals by manual record review as gold standard. The reviewers were blinded for the algorithm results and forty cases were reviewed in overlap resulting in almost perfect agreement (95.8–100%) between them, with a Cohen’s kappa of 0.90–1.00. Complicated cases were discussed and classified using a consensus decision by all reviewers. All BCx during admission were regarded as potential CVC-BSI-episodes, and an admission with no BCx counted as one potential CVC-BSI-episode. The study was approved by the Regional Ethical Review Board in Stockholm under permission no. 2018/1030-31.

Case definition and reference standard of CVC-BSI

The assessment of CVC-BSI during manual record review was performed according to ECDC’s BSI and CRI3-CVC definitions at the time of this study [27]. Contaminant was defined according to the common commensal list of CDC [28]. All CVCs fulfilling the criteria of ECDC and CDC were assessed [27, 29]. All CVC-BSI were recorded, but the healthcare-associated (HA) classification was based on ECDC’s definition [27].

Algorithms

Two rule-based algorithms for CVC-BSI detection were developed in the development dataset (Fig. 2):

  1. 1.

    Relevant microbiological findings in BCx and CVC cultures combined with the presence of BSI symptoms.

  2. 2.

    Only relevant microbiological findings in BCx and CVC cultures.

Fig. 2
figure 2

Flow diagram of the two rule-based surveillance algorithms for central venous catheter-related bloodstream infection. BCx: blood culture; BSI: bloodstream infection; CFU: colony-forming unit; CVC: central venous catheter; HA: healthcare-associated. Positive BCx: any BCx with growth of microorganisms (pathogens and/or contaminants). BSI symptoms: fever (> 38 °C) and/or hypotension (systolic blood pressure < 90 mmHg or diastolic blood pressure ≤ 60 mmHg). CVC-BSI-episode: all BCx taken during admission were regarded as potential CVC-BSI-episodes, and each admission without BCx taken counted as one potential CVC-BSI-episode. RIT: repeat infection timeframe, i.e., within 14 days after the first positive BCx of a previous CVC-BSI-episode

Full size image

For BSI, a positive BCx was defined as any growth of microorganisms and the division between a recognised pathogen or contaminant was based on the organism classification list of the CDC [28]. For recognised pathogens one BCx was sufficient. For contaminants two different BCx taken within 48 h of each other should be present together with the presence of BSI symptoms. BSI symptoms comprised fever (> 38 °C) and hypotension, the latter defined as a systolic blood pressure < 90 mmHg or a diastolic blood pressure ≤ 60 mmHg [29]. BSI symptoms had to be present within three days before or after the BCx based on CDC’s infection window period [30]. To define a BSI as CVC-related either (1) a positive CVC-tip culture (quantitative ≥ 103 CFU/ml) with the same microorganism (species level) taken within 48 h after the BCx or (2) a peripheral and central BCx, taken max. 15 min apart from each other, with the same microorganism and ≥ 2 h differential time to positivity between both BCx (central BCx < peripheral BCx) had to be present. These two criteria also served as proxy for the BSI occurring before or after catheter removal. CDC’s 14-day repeat infection timeframe (RIT) criterion was used to assess when another CVC-BSI-episode could be registered when multiple CVC-BSI-episodes where present during one admission [30]. All CVC-BSI were assessed, but additionally they were defined as HA when the BCx was taken two days or more after admission (on day three or later where day of admission is day one) or within two days after admission (on day one or two of admission) if a previous admission within 48 h was present.

Statistical analysis

Data acquisition, management and analysis were performed using R (version 4.1.0) and Python (version 3.7), and in accordance with current regulations concerning privacy and ethics. Continuous variables are presented as median with interquartile ranges (IQR) and categorical variables as numbers with percentages. For algorithm performance, the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), area under the receiver operating characteristic curve (AUROC), and kappa statistics were assessed. The 95% confidence interval (95%CI) for estimates in the validation dataset were calculated using the asymptotic variance with Wilson score method. Results from the validation dataset were extrapolated to the validation period cohort of 2018–2019 to obtain performance estimates of the algorithms in the complete target population (Fig. 1). The CI for these estimates were calculated as the 2.5th and 97.5th percentiles of point estimates obtained from 10,000 bootstrap samples for each of the three groups. To account for uncertainty, the bootstrapping was performed before extrapolating the proportions from the validation dataset to the validation period cohort [31]. Finally, the best performing algorithm was applied to data of 2017–2020 to visualise the incidence of CVC-BSI per 1000 in-patient hospital-days by continuous automated surveillance.

Results

In the validation dataset 4326 BCx, of which 1766 positive (40.8%), were present among the 1000 admissions with at least one positive BCx, and 594 BCx in the 250 admissions with only negative BCx. Together with the 250 admissions with no BCx taken, this amounted to 5170 potential CVC-BSI-episodes used for the calculations of algorithm performance (Table 1 and Fig. 1). There were 181,354 potential CVC-BSI-episodes during admissions of the validation period cohort of 2018–2019 (n = 132,850) used for the calculation of extrapolated algorithm performance (Fig. 1). Of the 78,781 BCx present during this period, 9340 were positive (11.9%).

Table 1 Characteristics of three admission groups in validation dataset
Full size table

Overall, the patients within the validation dataset with positive BCx were older and consisted of more males than the patients with negative or no BCx during admission (Table 1). The length of hospital stay, number of ICU admissions and in-hospital mortality was highest in admissions with positive BCx and lowest in admissions with no BCx.

Algorithms performance

No CVC-BSI-episodes were identified during manual record review in admissions with only negative BCx or no BCx. In the 4326 potential CVC-BSI-episodes of admissions with at least one positive BCx, 51 episodes (1.2%) fulfilled ECDC’s CRI3-CVC definition, and 32 episodes (0.7%) the criteria for HA (Table 1). The two algorithms classified 47 (1.09%) and 49 (1.13%) as CVC-BSI-episodes, respectively, and 26 (0.60%) and 27 (0.62%) as HA, respectively.

In the validation dataset with positive BCx as well as the total validation period cohort (extrapolated data) for all CVC-BSI, the algorithm based on relevant microbiological findings in BCx and CVC cultures combined with the presence of BSI symptoms (algorithm 1) had a high sensitivity and PPV to determine CVC-BSI according to ECDC’s definition with a sensitivity of 0.861 (95%CI 0.764–0.959) and PPV of 0.935 (95%CI 0.850–1.000) in the validation period cohort (Table 2). The algorithm based on only relevant microbiological findings in BCx and CVC cultures (algorithm 2) had a sensitivity of 0.880 (95%CI 0.783–0.959) and PPV of 0.918 (95%CI 0.833–0.981) in the validation period cohort. The specificity and NPV was high (Table 2).

Table 2 Performance characteristics two rule-based algorithms for central venous catheter-related bloodstream infection according to ECDC definition
Full size table

In the HA CVC-BSI cases, for both algorithms the sensitivity was lower and PPV higher compared to all CVC-BSI cases (Table 2). In the validation period cohort, the sensitivity was 0.784 (95%CI 0.629–0.910) and the PVV was 0.956 (95%CI 0.868–1.000) for algorithm 1. For algorithm 2 the sensitivity was 0.814 (95%CI 0.659–0.940) and PPV was 0.958 (95%CI 0.873–1.000). Overall, the specificity and NPV was high and similar to those for all CVC-BSI cases (Table 2).

The false negative and false positive cases were mostly related to cases wrongly classified by the annotators or algorithm rules missing for specific clinical situations not present in the development dataset (Table 3). When correcting the misclassification in the reference standard, for algorithm 2 in the validation period cohort the sensitivity would increase to 0.925 (95%CI 0.816–1.000) and 0.939 (95%CI 0.859–1.000) for HA and all CVC-BSI cases, respectively. Theoretically, when also improving algorithm rules, it could even increase to 0.959 (95%CI 0.891–1.000) and 1.000 (95%CI 0.999–1.000) for HA and all CVC-BSI cases, respectively.

Table 3 Discrepancy analyses between algorithm results and annotation results (reference), for false-negative and false-positive cases
Full size table

CVC-BSI rates

The CVC-BSI rate was expressed against several possible denominators for the reference, i.e. manual record review, and the two algorithm results (Table 4). Both algorithms give slightly lower rates compared to the reference with algorithm 2 giving rates closest to the reference.

Table 4 Rates central venous catheter-related bloodstream infection based on annotation (reference) and algorithm results
Full size table

Algorithm 2 was applied over the period 2017–2020 to show the incidence of CVC-BSI over time with automated surveillance (Fig. 3). The hospital-wide incidence rate was overall 0.31 per 1000 in-patient hospital-days and ranged from 0.11 to 0.53 for all CVC-BSI, and it was overall 0.16 per 1000 in-patient hospital-days with a range of 0.03 to 0.42 for HA CVC-BSI. The incidence seemed to show a seasonal trend. In 2020, during the COVID-19 pandemic, the incidence increased and during some period the HA CVC-BSI rate seemed to diverge from the overall CVC-BSI rate.

Fig. 3
figure 3

Incidence rate of central venous catheter-related bloodstream infection per 1000 in-patient hospital-days during the period 2017–2020. BCx: blood culture; CVC-BSI: central venous catheter-related bloodstream infection; HA: healthcare-associated. Incidence rate based on application of algorithm which only used relevant microbiological findings in BCx and CVC cultures

Full size image

Discussion

We developed two fully-automated surveillance algorithms for the detection of CVC-BSI based on ECDC’s CRI3-CVC criteria. Both had a comparable and high performance for detecting CVC-BSI. To our knowledge, these are the first fully-automated surveillance algorithms developed for CVC-BSI based on ECDC’s definition.

Both algorithms had good performance and high agreement with the reference standard even though certain criteria of the surveillance definition were not part of the algorithm rules. Checking for pus cultures from the insertion site was excluded since it was assessed to be less common and more difficult to automate. Chills as symptom was excluded because integration would require free-text analysis. Also, no explicit check for the presence of a CVC was done as adequate structured data on insertion and extraction was lacking and it was assumed that the criteria for CVC culture and differential positivity between central and peripheral BCx could serve as proxy. These non-incorporated criteria of the definition were not relevant for the algorithm performance as none of the discrepant cases were related to these criteria. These cases were mostly related to misclassification of CVC-BSI cases during annotation or rules missing in the algorithm due to situations not present in the development data. As reported, when correcting the misclassification in the reference standard and additionally also improving the algorithm rules, the performance of the algorithms could be even higher. However, if this holds in other settings needs to be investigated.

No major difference in the performance between both algorithms was found, where the simpler algorithm (algorithm 2) was only using microbiology culture data and no symptom information. One could have expected a bigger difference considering that during the ICU period there was only access to data on BSI symptoms in medical notes, and algorithm 1 needed structured symptom data, and if in general many cultures were taken regardless of symptoms and positive for contaminants. This might indicate that in this setting peripheral/central blood and CVC cultures were primarily taken in patients with symptoms of infection, in line with the culture recommendations in this setting and Swedish recommendations [32] which also matches the surveillance definition [27]. Consequently, this simpler algorithm could be used hospital-wide in this setting, independently from the availability of structured symptom data, without any major effect on the quality of surveillance. An advantage is that it is less vulnerable to weaknesses in data availability. Also, it could further facilitate the implementation in other health care systems since the demand for quality data on symptoms is removed. However, it is not certain that this algorithm without symptoms would perform equally well in other settings where clinical practices for taking blood cultures on wider indications are present.

We developed algorithms to detect all CVC-BSI cases and additionally to detect the ones that are HA according to ECDC’s criteria. The CVC-BSI cases that did not fulfil these HA criteria (> 30%) were long-term CVCs inserted during a previous hospital admission with a discharge more than 48 h before, e.g., a peripherally inserted central catheter (PICC-line) or subcutaneous venous port (SVP), or CVCs that were inserted at another healthcare facility than a hospital and still present when admitted to the hospital. Currently, patients get outpatient care with PICC-lines or SVPs or get it at another facility than the hospital [33]. Consequently, these cases should also be considered HA CVC-BSI even though not all are preventable for the hospital where they are admitted with their infection. Thus, the surveillance algorithm for all CVC-BSI cases better assesses their true burden on the healthcare system while the surveillance algorithm for HA CVC-BSI cases mostly indicates the cases possibly preventable by the hospital itself. Furthermore, the performance of the algorithms for HA cases was slightly lower than for all cases which is related to a larger misclassification of HA cases by the algorithms as they missed information that was only available in free-text.

Previous studies have demonstrated that it is possible to develop both semi- [13,14,15,16, 20,21,22,23] and fully-automated [17,18,19, 24, 25] surveillance algorithms for CVC-BSI according to the CLABSI definition that all perform well in comparison to manual record review. The heterogeneity in settings, populations and methods complicates a good comparison. Still, the performance of the algorithms developed in this study, for CVC-BSI according to the CRI3-CVC definition, are comparable to the results of these previous studies. Yet, as expected, the semi-automated algorithms in general had higher sensitivity and NPV combined with lower specificity and PPV compared to fully-automated algorithms which is in line with their different purpose and characteristics [6]. Based on the kappa analysis the agreement between the reference and the algorithm results in our study was mostly higher than in other studies [19,20,21] and similar to one other [24]. One study used the HELICS definition, the predecessor of ECDC’s definition, instead of CLABSI, but as this was a semi-automated algorithm their sensitivity was higher while their specificity and PPV was much lower than in our study [16]. Only two fully-automated algorithms were developed in a non-ICU or general hospital population [24, 25], using the CLABSI definition. The algorithm in the study of Herson et al. [25] had much lower sensitivity compared to our algorithms, but that algorithm was mainly based on diagnosis codes which is known to have poor sensitivity [34]. In the study of Woeltje et al. [24], their best performing algorithm, using culture, central line and fever data, achieved a higher sensitivity and lower specificity compared to the algorithms in this study. This difference might be explained by the different definitions used. CDC’s CLABSI definition [30] is a more non-specific definition compared to ECDC’s CRI3-CVC definition [27] and primarily works by excluding other sources of infection instead of linking it to the CVC, which is reflected in the algorithm rules used. Weaknesses in data quality, lack of documentation or reduced testing could more easily lead to more non-infected patients being classified as having an infection, both by manual review and algorithms, which consequently boosts the sensitivity for CLABSI-based algorithms. In contrast, infection is more difficult to ascertain for CRI3-CVC, both by manual review and algorithms, which could boost the specificity for these algorithms.

For the fully-automated algorithms CVC-BSI rates were either close to the true rates [24, 25] or overestimated [17, 19]. One can argue that for surveillance purposes it is not crucial whether the infection rate is over- or understated since it should primarily, as emphasised by Kaiser et al. [14], demonstrate changes in infection rates over time. However, an estimated rate close to the true rate would provide a better estimation of the true burden. In this study, the rates generated by both algorithms were close to the true rates. Due to high accuracy of the algorithms, they could possibly even be used for identifying individual patients for targeted interventions.

The rates of HA CVC-BSI by number of positive BCx or number of admissions in this study, 1.8% and 0.13%, is lower than what has been presented in the other studies which developed algorithms where these rates ranged from 6% till 36% and 0.64% till 6.2%, respectively [14,15,16, 19,20,21,22, 24]. Partly this might be explained by the fact that most rates were from the ICU instead of the total hospital population. However, also here the difference in definition of CVC-BSI used might play a role, as the CLABSI definition used in these studies in general gives higher rates of CVC-BSI than the CRI3-CVC definition used in this study [35, 36]. Usually, the incidence of CVC-BSI is expressed by catheter-days. As catheter-days could not reliable be obtained from structured data in our study, in-patient hospital-days were used as alternative. Our rate of HA-CVC-BSI of 0.20 per 1000 in-patient hospital-days is similar to the 0.20 per 1000 patient-days in a recent study in Spain which used the CRBSI definition which is fairly similar to the ECDC definition, but also included BSI related to peripheral venous catheters (PVC) [37].

The results of the second algorithm were plotted over a four-year period as a use-case for automated surveillance. It shows that it potentially could be used to indicate failure in infection prevention and control during strained healthcare situations like the COVID-19 pandemic. It also demonstrates the importance of showing both all CVC-BSI and HA CVC-BSI cases as, as mentioned above, one measures the overall burden and the other the part possibly preventable by the hospital itself. Subsequently, a change in the portion of HA CVC-BSI cases can indicate a change in the composition of the hospital population.

This study has several strengths. The study used a large dataset representative for the clinical population for which the algorithms are designed. Also, implementation in real life is facilitated as comprehensive EHR data was used. Furthermore, the algorithms have been validated on a separate dataset than the dataset used for development, in contrast to most of the previous studies, which increases the generalisability of the results [34, 38]. In addition, annotators were blinded to the algorithm results to prevent this from affecting their assessment while annotating the validation dataset. Finally, the international recognised and established criteria of ECDC and CDC were used for algorithm development and validation, and manual annotation was performed according to ECDC’s definition which increases the comparability and generalisability with other studies. However, some limitations should also be considered. The algorithms have not been validated in the obstetric population. However, hardly any positive BCx were present during obstetric admissions, so this would probably not greatly influence the results. The developed algorithms focus only on microbiologically confirmed CVC-BSI based on the CRI3-CVC definition and not on local, general, or non-microbiology confirmed CVC-related infection (CRI1-CVC, CRI2-CVC, or C-CVC) nor on PVC-related infections which are also important infections. Consequently, the true burden of CVC-BSI is probably underestimated, and the burden of all CRI is not measured. Even though a different dataset was used for the development and validation, they were coming from a similar population and therefore, the algorithms still need to be validated in other settings and EHR systems to better assess their generalisability. As in this study setting, neither the semi-quantitative method for CVC cultures nor the quantitative blood culture ratio between central and peripheral blood samples is determined, these criteria were not incorporated in the algorithms and need to be added and validated in settings where these methods are used. Although comprehensive EHR data was used, implementation and evaluation in real life settings is still needed. While these developed fully-automated surveillance algorithms could replace manual surveillance, it remains important to consider that due to changes in methods and reporting over time they still need regular readjustment and revalidation. It is likely that the CVC-BSI rates, by both manual review and the algorithms, underestimated the ‘true’ rate as patients are possibly not cultured appropriately (e.g., no BCx taken while symptoms are present). Finally, wrong or missing information in the EHR system could have influenced algorithm performance, but this affects manual surveillance in a similar fashion.

Conclusions

The present study indicates that it is possible to develop an algorithm based on ECDC’s definition of CVC-BSI (CRI3-CVC) that is appropriate for continuous automated surveillance purposes and performs well in comparison to manual record review and previously developed algorithms using CDC’s CLABSI criteria. The simpler algorithm, only using microbiology data, can be used in settings with testing for CVC-BSI adhering to recommendations. However, in other settings the algorithm including symptom data might need to be applied and further validation of both algorithms in other settings is needed to assess the generalisability of the algorithms.

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available due to ethical limitations related to sharing patient information, but are available from the corresponding author on reasonable request.

Abbreviations

AUROC:

Area under the receiver operating characteristic curve

BCx:

Blood culture

BSI:

Bloodstream infection

CDC:

Centers for Disease Control and Prevention

CFU:

Colony-forming unit

CI:

Confidence interval

CLABSI:

Central line-associated bloodstream infection

CRI:

Catheter-related infection

CRI3-CVC:

Microbiologically confirmed central venous catheter-related bloodstream infection

CVC:

Central venous catheter

CVC-BSI:

Central venous catheter-related bloodstream infection

ECDC:

European Centre for Disease Prevention and Control

EHR:

Electronic health record

HA:

Healthcare-associated

HAI:

Healthcare-associated infection

ICU:

Intensive care unit

IQR:

Interquartile range

KUH:

Karolinska University Hospital

NPV:

Negative predictive value

PICC:

Peripherally inserted central catheter

PPV:

Positive predictive value

RIT:

Repeat infection timeframe

SVP:

Subcutaneous venous port

References

  1. Umscheid CA, Mitchell MD, Doshi JA, Agarwal R, Williams K, Brennan PJ. Estimating the proportion of healthcare-associated infections that are reasonably preventable and the related mortality and costs. Infect Control Hosp Epidemiol. 2011;32(2):101–14. https://doi.org/10.1086/657912.

    Article  PubMed  Google Scholar 

  2. Cassini A, Plachouras D, Eckmanns T, Abu Sin M, Blank HP, Ducomble T, et al. Burden of six healthcare-associated infections on European population health: Estimating incidence-based disability-adjusted life years through a population prevalence-based modelling study. PLoS Med. 2016;13(10): e1002150. https://doi.org/10.1371/journal.pmed.1002150.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Suetens C, Latour K, Karki T, Ricchizzi E, Kinross P, Moro ML, et al. Prevalence of healthcare-associated infections, estimated incidence and composite antimicrobial resistance index in acute care hospitals and long-term care facilities: Results from two European point prevalence surveys, 2016 to 2017. Euro Surveill. 2018;23(46):1800516. https://doi.org/10.2807/1560-7917.ES.2018.23.46.1800516.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Haque M, Sartelli M, McKimm J, Abu BM. Health care-associated infections: an overview. Infect Drug Resist. 2018;11:2321–33. https://doi.org/10.2147/IDR.S177247.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Gahlot R, Nigam C, Kumar V, Yadav G, Anupurba S. Catheter-related bloodstream infections. Int J Crit Illn Inj Sci. 2014;4(2):162–7. https://doi.org/10.4103/2229-5151.134184.

    Article  PubMed  PubMed Central  Google Scholar 

  6. van Mourik MSM, Perencevich EN, Gastmeier P, Bonten MJM. Designing surveillance of healthcare-associated infections in the era of automation and reporting mandates. Clin Infect Dis. 2018;66(6):970–6. https://doi.org/10.1093/cid/cix835.

    Article  PubMed  Google Scholar 

  7. Storr J, Twyman A, Zingg W, Damani N, Kilpatrick C, Reilly J, et al. Core components for effective infection prevention and control programmes: New WHO evidence-based recommendations. Antimicrob Resist Infect Control. 2017;6:6. https://doi.org/10.1186/s13756-016-0149-9.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Mitchell BG, Hall L, Halton K, MacBeth D, Gardner A. Time spent by infection control professionals undertaking healthcare associated infection surveillance: a multi-centred cross sectional study. Infect Dis Health. 2016;21(1):36–40. https://doi.org/10.1016/j.idh.2016.03.003.

    Article  Google Scholar 

  9. Russo PL, Barnett AG, Cheng AC, Richards M, Graves N, Hall L. Differences in identifying healthcare associated infections using clinical vignettes and the influence of respondent characteristics: a cross-sectional survey of Australian infection prevention staff. Antimicrob Resist Infect Control. 2015;4:29. https://doi.org/10.1186/s13756-015-0070-7.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Haut ER, Pronovost PJ. Surveillance bias in outcomes reporting. JAMA. 2011;305(23):2462–3. https://doi.org/10.1001/jama.2011.822.

    Article  CAS  PubMed  Google Scholar 

  11. Sips ME, Bonten MJM, van Mourik MSM. Automated surveillance of healthcare-associated infections: State of the art. Curr Opin Infect Dis. 2017;30(4):425–31. https://doi.org/10.1097/QCO.0000000000000376.

    Article  PubMed  Google Scholar 

  12. van Mourik MSM, van Rooden SM, Abbas M, Aspevall O, Astagneau P, Bonten MJM, et al. PRAISE: providing a roadmap for automated infection surveillance in Europe. Clin Microbiol Infect. 2021;27(Suppl 1):S3–19. https://doi.org/10.1016/j.cmi.2021.02.028.

    Article  PubMed  Google Scholar 

  13. Woeltje KF, Butler AM, Goris AJ, Tutlam NT, Doherty JA, Westover MB, et al. Automated surveillance for central line-associated bloodstream infection in intensive care units. Infect Control Hosp Epidemiol. 2008;29(9):842–6. https://doi.org/10.1086/590261.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Kaiser AM, de Jong E, Evelein-Brugman SF, Peppink JM, Vandenbroucke-Grauls CM, Girbes AR. Development of trigger-based semi-automated surveillance of ventilator-associated pneumonia and central line-associated bloodstream infections in a Dutch intensive care. Ann Intensive Care. 2014;4:40. https://doi.org/10.1186/s13613-014-0040-x.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Bond J, Issa M, Nasrallah A, Bahroloomi S, Blackwood RA. Comparing administrative and clinical data for central line associated blood stream infections in Pediatric Intensive Care Unit and Pediatric Cardiothoracic Intensive Care Unit. Infect Dis Rep. 2016;8(3):58–62. https://doi.org/10.4081/idr.2016.6275.

    Article  Google Scholar 

  16. Bouzbid S, Gicquel Q, Gerbier S, Chomarat M, Pradat E, Fabry J, et al. Automated detection of nosocomial infections: evaluation of different strategies in an intensive care unit 2000–2006. J Hosp Infect. 2011;79(1):38–43. https://doi.org/10.1016/j.jhin.2011.05.006.

    Article  CAS  PubMed  Google Scholar 

  17. Stamm AM, Bettacchi CJ. A comparison of 3 metrics to identify health care-associated infections. Am J Infect Control. 2012;40(8):688–91. https://doi.org/10.1016/j.ajic.2012.01.033.

    Article  PubMed  Google Scholar 

  18. Venable A, Dissanaike S. Is automated electronic surveillance for healthcare-associated infections accurate in the burn unit? J Burn Care Res. 2013;34(6):591–7. https://doi.org/10.1097/BCR.0b013e3182a2aa0f.

    Article  PubMed  Google Scholar 

  19. Snyders RE, Goris AJ, Gase KA, Leone CL, Doherty JA, Woeltje KF. Increasing the reliability of fully automated surveillance for central line-associated bloodstream infections. Infect Control Hosp Epidemiol. 2015;36(12):1396–400. https://doi.org/10.1017/ice.2015.199.

    Article  PubMed  Google Scholar 

  20. Trick WE, Zagorski BM, Tokars JI, Vernon MO, Welbel SF, Wisniewski MF, et al. Computer algorithms to detect bloodstream infections. Emerg Infect Dis. 2004;10(9):1612–20. https://doi.org/10.3201/eid1009.030978.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Bellini C, Petignat C, Francioli P, Wenger A, Bille J, Klopotov A, et al. Comparison of automated strategies for surveillance of nosocomial bacteremia. Infect Control Hosp Epidemiol. 2007;28(9):1030–5. https://doi.org/10.1086/519861.

    Article  PubMed  Google Scholar 

  22. Bearman GML, Oppenheim MI, Mendonca EA, Hupert N, Behta M, Christos PJ, et al. A clinical predictive model for catheter related bloodstream infections from the electronic medical record. Open Epidemiol J. 2010;3:24–8. https://doi.org/10.2174/1874297101003010024.

    Article  Google Scholar 

  23. Ridgway JP, Sun X, Tabak YP, Johannes RS, Robicsek A. Performance characteristics and associated outcomes for an automated surveillance tool for bloodstream infection. Am J Infect Control. 2016;44(5):567–71. https://doi.org/10.1016/j.ajic.2015.12.044.

    Article  PubMed  Google Scholar 

  24. Woeltje KF, McMullen KM, Butler AM, Goris AJ, Doherty JA. Electronic surveillance for healthcare-associated central line-associated bloodstream infections outside the intensive care unit. Infect Control Hosp Epidemiol. 2011;32(11):1086–90. https://doi.org/10.1086/662181.

    Article  PubMed  Google Scholar 

  25. Herson M, Curtis SJ, Land G, Stewardson AJ, Worth LJ. Performance of a hospital-acquired complication algorithm using administrative data for detection of central line-associated bloodstream infections: Experience at an Australian healthcare facility. J Hosp Infect. 2021;112:116–8. https://doi.org/10.1016/j.jhin.2021.03.029.

    Article  CAS  PubMed  Google Scholar 

  26. Januel JM, Lotfinejad N, Grant R, Tschudin-Sutter S, Schreiber PW, Grandbastien B, et al. Predictive performance of automated surveillance algorithms for intravascular catheter bloodstream infections: a systematic review and meta-analysis. Antimicrob Resist Infect Control. 2023;12(1):87. https://doi.org/10.1186/s13756-023-01286-0.

    Article  PubMed  PubMed Central  Google Scholar 

  27. European Centre for Disease Prevention and Control. Point prevalence survey of healthcare-associated infections and antimicrobial use in European acute care hospitals – Protocol version 5.3. Stockholm: ECDC; 2016.

  28. Centers for Disease Control and Prevention. NHSN Organism list. Atlanta: CDC; 2021.

  29. Lim WS, Baudouin SV, George RC, Hill AT, Jamieson C, Le Jeune I, et al. BTS guidelines for the management of community acquired pneumonia in adults: update 2009. Thorax. 2009;64 Suppl 3:1–55. https://doi.org/10.1136/thx.2009.121434.

    Article  Google Scholar 

  30. Centers for Disease Control and Prevention. National Healthcare Safety Network (NHSN): Patient safety component manual. Atlanta: CDC; 2021.

  31. Valik JK, Ward L, Tanushi H, Mullersdorf K, Ternhag A, Aufwerber E, et al. Validation of automated sepsis surveillance based on the Sepsis-3 clinical criteria against physician record review in a general hospital population: Observational study using electronic health records data. BMJ Qual Saf. 2020;29(9):735–45. https://doi.org/10.1136/bmjqs-2019-010123.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Acosta S, Frykholm P, Granath A, Hammarskjöld F, Lindgren S, Lindwall R, et al. SFAI:s riktlinjer för centrala venkateterisering. Sweden: Svensk Förening för Anestesi och Intensivvård; 2018.

    Google Scholar 

  33. Broadhurst D, Moureau N, Ullman AJ. Central venous access devices site care practices: an international survey of 34 countries. J Vasc Access. 2016;17(1):78–86. https://doi.org/10.5301/jva.5000450.

    Article  PubMed  Google Scholar 

  34. Streefkerk HRA, Verkooijen RP, Bramer WM, Verbrugh HA. Electronically assisted surveillance systems of healthcare-associated infections: a systematic review. Euro Surveill. 2020;25(2):1900321. https://doi.org/10.2807/1560-7917.ES.2020.25.2.1900321.

    Article  PubMed  PubMed Central  Google Scholar 

  35. O’Grady NP, Alexander M, Burns LA, Dellinger EP, Garland J, Heard SO, et al. Guidelines for the prevention of intravascular catheter-related infections. Clin Infect Dis. 2011;52(9):e162–93. https://doi.org/10.1093/cid/cir257.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Djuric O, Markovic-Denic L, Jovanovic B, Bumbasirevic V. Agreement between CDC/NHSN surveillance definitions and ECDC criteria in diagnosis of healthcare-associated infections in Serbian trauma patients. PLoS ONE. 2018;13(10): e0204893. https://doi.org/10.1371/journal.pone.0204893.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Badia-Cebada L, Penafiel J, Saliba P, Andres M, Camara J, Domenech D, et al. Trends in the epidemiology of catheter-related bloodstream infections; towards a paradigm shift, Spain, 2007 to 2019. Euro Surveill. 2022;27(19):2100610. https://doi.org/10.2807/1560-7917.ES.2022.27.19.2100610.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Govindan M, Van Citters AD, Nelson EC, Kelly-Cummings J, Suresh G. Automated detection of harm in healthcare with information technology: a systematic review. Qual Saf Health Care. 2010;19(5): e11. https://doi.org/10.1136/qshc.2009.033027.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank Anna Frej and Lina Abdel-Halim for their help with the manual annotation of medical records.

Funding

Open access funding provided by Karolinska Institute. The work was supported by Sweden´s Innovation Agency (Vinnova Grant 2018-03350) and Swedish Research Council (VR Grant 2021-02271). PN was supported by Region Stockholm (clinical research appointment; ALF Grant 2019-1054).

Author information

Authors and Affiliations

  1. Department of Medicine Solna, Division of Infectious Diseases, Karolinska Institutet, 171 77, Stockholm, Sweden

    Moa Karmefors Idvall, Hideyuki Tanushi, Andreas Berge, Pontus Nauclér & Suzanne Desirée van der Werff

  2. Department of Data Processing and Analysis, Karolinska University Hospital, Stockholm, Sweden

    Hideyuki Tanushi

  3. Department of Infectious Diseases, Karolinska University Hospital, Stockholm, Sweden

    Andreas Berge, Pontus Nauclér & Suzanne Desirée van der Werff

Authors
  1. Moa Karmefors Idvall
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hideyuki Tanushi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andreas Berge
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pontus Nauclér
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Suzanne Desirée van der Werff
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SvdW, AB and PN conceptualised this study. SvdW, MKI and HT developed the algorithms. SvdW and HT extracted the data. MKI annotated data. AB and PN gave necessary clinical input. SvdW and MKI analysed the data. All authors interpreted the data. SvdW supervised the study. SvdW and MKI prepared and drafted the manuscript with the input of all other authors. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Suzanne Desirée van der Werff.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the Regional Ethical Review Board in Stockholm (no. 2018/1030-31).

Consent for publication

Not applicable.

Competing interests

PN and SvdW are involved in a company that works on automated surveillance for adverse events. The other authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karmefors Idvall, M., Tanushi, H., Berge, A. et al. The accuracy of fully-automated algorithms for the surveillance of central venous catheter-related bloodstream infection in hospitalised patients. Antimicrob Resist Infect Control 13, 15 (2024). https://doi.org/10.1186/s13756-024-01373-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13756-024-01373-w

Keywords

Antimicrobial Resistance & Infection Control

ISSN: 2047-2994

Contact us