- Open Access
Predicting the occurrence of multidrug-resistant organism colonization or infection in ICU patients: development and validation of a novel multivariate prediction model
Antimicrobial Resistance & Infection Control volume 9, Article number: 66 (2020)
Multidrug-resistant organisms (MDROs) have emerged as an important cause of poor prognoses of patients in the intensive care unit (ICU). This study aimed to establish an easy-to-use nomogram for predicting the occurrence of MDRO colonization or infection in ICU patients.
In this study, we developed a nomogram based on predictors in patients admitted to the ICU in the First Affiliated Hospital of Xiamen University from 2016 to 2018 using univariate and multivariate logistic regression analysis. We externally validated this nomogram in patients from another hospital over a similar period, and assessed its performance by calculating the area under the receiver operating characteristic (ROC) curve (AUC) and performing a decision curve analysis.
331 patients in the primary cohort and 181 patients in the validation cohort were included in the statistical analysis. Independent factors derived from the primary cohort to predict MDRO colonization or infection were male sex, higher C-reactive protein (CRP) levels and higher Pitt bacteremia scores (Pitt scores), which were all assembled in the nomogram. The nomogram yielded good discrimination with an AUC of 0.77 (95% CI 0.70–0.84), and the range of threshold probabilities of decision curves was approximately 30–95%.
This easy-to-use nomogram is potentially useful for predicting the occurrence of MDRO colonization or infection in ICU patients.
Multidrug-resistant organisms (MDROs) have emerged as an important cause of morbidity and mortality in patients worldwide . Escalating rates of antibiotic resistance add substantially to the morbidity, mortality, and costs related to infections in hospitalized patients, especially those in the intensive care unit (ICU) setting [2, 3]. However, progress in treatment has been limited. Few antibiotics have been developed in the last 30 years . Fortunately, the relevant departments have formulated guidelines for the prevention and control of MDROs . Among them, the rational use of antibiotics is an important item. Predictors for the occurrence of MDRO colonization or infection would also be valuable, aiding in empirical treatment when infection occurs.
Previously reported factors, such as recent (within 90 days) treatment with three or more antimicrobial agents, a previous MDRO colonization or infection, a hospital stay > 15 days, an indwelling catheter for more than 20 days, and long-term mechanical ventilation, were all strongly correlated with MDRO colonization or infection [6, 7]. However, some of these factors are based on patient recall of their history, and ICU patients often experience disturbances of consciousness, which introduce recall bias, while other factors display limited predictive value. Furthermore, single predictors may not be reliable in individual cases due to the complexity and heterogeneity of diseases in the ICU. Therefore, a model for predicting the occurrence of MDRO colonization or infection is desirable.
A nomogram is a statistical instrument that accounts for numerous variables to predict an outcome of interest for an individual patient . Nomograms are routinely used to aid in decision making in cancer, trauma, neurocritical care, and other specialties [9,10,11]. The aim of this study was to construct an easy-to-use nomogram for predicting the occurrence of MDRO colonization or infection in ICU patients, with the aim of providing an aid for empirical treatment when infection occurs and for screening of high-risk patients for subsequent prospective randomized controlled studies.
Materials and methods
Study design and external validation
A retrospective observational study was conducted on a primary cohort of patients who presented to the ICU of the First Affiliated Hospital of Xiamen University between January 2016 and December 2018. We developed a nomogram from the primary cohort to predict MDRO colonization or infection in the ICU. For external validation, an independent cohort from another hospital, Xiamen Hospital of Traditional Chinese Medicine, was retrospectively enrolled over a similar period, using the same inclusion and exclusion criteria.
This study was approved by the Medical Ethics Committee of the First Affiliated Hospital of Xiamen University and the Medical Ethics Committee of Xiamen Hospital of Traditional Chinese Medicine. Informed consent was waived because the study was conducted retrospectively and no interventions were applied.
Study population and definitions
Patients were eligible for inclusion if they were aged ≥18 years and underwent at least one microbial culture during their stay in the ICU. The following exclusion criteria were used: MDROs were detected before the patient entered the ICU or within the first 48 h in the ICU.
Qualified samples were collected of cerebrospinal fluid, sputum, urine, faeces, secretions, serous cavity effusion, bile, blood, catheters, etc. The multidrug-resistant status of the cultured bacteria was defined according to the provisional standard definition of MDROs published in Clinical Microbiology and Infection in 2012. MDROs can be defined as those organisms that are simultaneously resistant to three or more antimicrobial agents . MDROs colonization was defined as the presence of MDROs cultured from microbiology specimens without evidence of tissue invasion or inflammation at that body site. MDROs infection was defined as the invasion of the body tissues by MDROs resulting in disease . MDRO colonization or infection mentioned in this study refers to the occurrence during the ICU stay. The neutrophil-to-lymphocyte ratio (NLR) was determined by dividing the absolute neutrophil count (N) by the absolute lymphocyte count (L) . The immunosuppressive status refers to acquired immune deficiency syndrome, advanced malignant tumors, transplantation, splenectomy, long-term oral immunosuppressant use, etc. Patients with MDRO colonization or infection constituted the MDRO group, and those without MDRO colonization or infection constituted the non-MDRO group.
Data collection and measurements
Demographic and clinical data were collected from the included patients by research coordinators and board-certified ICU physicians using a case report form. They reviewed the electronic medical records and verified the final data. The following data, which may be related to MDRO colonization or infection according to the literature and clinical experience, were extracted: general patient characteristics, including age, gender and comorbidities; previous hospitalization history; bacteriological and drug resistance test results; mechanical ventilation status, the use of pressure increasing drugs, tracheal intubation / incision, indwelling urethral catheter or central venous catheter, state of consciousness and the use of cardiopulmonary resuscitation, within the first 24 h in the ICU; the levels of albumin (ALB), creatinine (Cr), total bilirubin (TBil), platelets (PLTs), oxygenation index (PaO2/FiO2), leucocyte count (WBC), N, neutrophil percentage (N%), L, lymphocyte percentage (L%), procalcitionin (PCT) and C-reactive protein (CRP), within the first 48 h in the ICU; etc.. Laboratory indicators were collected within the first 48 h in the ICU because some data was not available within the first 24 h, and then circulating neutrophils and lymphocytes were counted to calculate the NLR. To predict the occurrence of MDRO colonization or infection, the Sequential Organ Failure Assessment (SOFA) score, Acute Physiology and Chronic Health Evaluation (APACHE) II score and Pitt bacteremia score (Pitt score) were calculated at the baseline from the information available in the ICU registry at admission [15, 16]. Pitt score is a severity of illness grading system, evaluating mental status, presence or absence of fever, hypotension, mechanical ventilation and cardiac status .
For the development of the nomogram, we searched for potential factors associated with MDRO colonization or infection reported in previous studies or based on clinical experience, that are able to be easily acquired in the early stages of ICU hospitalization. Randomly missing data were filled by using Monte Carlo multiple imputation methods . Continuous data were reported as the medians [interquartile ranges]. Categorical data were presented as frequencies or ratios. A univariate logistic regression model was used to analyze factors that may be related to MDRO colonization or infection. Variables with a P value of 0.2 or less in the univariate logistic regression analyses were subjected to multivariate logistic regression analysis. The final selection of the multivariate logistic regression model was based on backward stepwise regression with the Akaike Information Criterion (AIC) . A nomogram was formulated based on the results of the multivariate analysis of the primary cohort with the “rms” package, which allowed us to obtain occurrence probability estimates .
We tested the accuracy of the nomogram by determining its discrimination using an external validation cohort. The predictive ability of the nomogram was quantified by calculating the area under the receiver operating characteristic (ROC) curve (AUC). An AUC value of 1.0 indicates a perfect prediction, while 0.5 is equivalent to the toss of a coin. Furthermore, we used decision curve analysis to assess the benefits of nomogram-assisted decisions in a clinical context . A decision curve analysis incorporates the clinical consequences of using a prediction rule by applying a different weight to true and false positives. This weighting is varied to reflect different patient preferences or differences in opinion about the risks of a procedure. These preferences are expressed in terms of a threshold probability for an action . The present study was compliant with the standard guidelines for prediction models .
All analyses were performed using R software (version 3.4.2, http://www.r-project.org/). All statistical tests were two-sided, and P values of less than 0.05 were considered statistically significant.
The flow chart of the process used to screen eligible patients is presented in Fig. 1. Of 364 patients who met the inclusion criteria, 336 patients were eligible. Among them, 5 patients were lost to follow-up. Finally, 331 patients in the primary cohort were included in the statistical analysis.
The main characteristics of the study population are shown in Table 1. In the primary cohort, 178 patients (53.8%) comprised the MDRO group, and 153 patients (46.2%) constituted the non-MDRO group. In the validation cohort, 97 patients (53.6%) comprised the MDRO group, and 84 patients (46.4%) constituted the non-MDRO group.
Independent predictors of MDRO colonization or infection
Univariate analysis indicated that male sex, higher CRP levels and higher Pitt scores were associated with MDRO colonization or infection (Table 2). Multivariate analysis continued to demonstrate that the above three variables were independent predictors for MDRO colonization or infection, including female sex (odds ratio [OR] 0.56, 95% confidence interval [CI] 0.35–0.89, p = 0.014), CRP (OR 1.06, 95% CI 1.02–1.09, p = 0.002) and Pitt score (OR 1.13, 95% CI 1.03–1.22, p = 0.006) (Table 2).
The nomogram for predicting the occurrence of MDRO colonization or infection: performance and validation
We created a nomogram to predict the probability of the occurrence of MDRO colonization or infection in ICU patients (Fig. 2a). The nomogram integrated all three significant independent predictors determined from the multivariate analysis in the primary cohort. When using the nomogram, the patient’s gender was located, and a line was drawn straight up to the points on the axis to establish the score associated with that gender. This process was repeated for the other two covariates. The scores for each covariate were added, and the total score on the axis depicting the total points was located. A line was drawn straight down to the linear predictor axis to obtain the probability. An example of how to use the nomogram to predict the occurrence of MDRO colonization or infection in ICU patients is presented in Fig. 2b.
The predictive ability of the nomogram was externally confirmed in the validation cohort. The AUC for the nomogram was 0.77 (95% CI 0.70–0.84), indicating good discrimination (Fig. 3a). Figure 3b illustrates the decision curves for the nomogram. The gray line represents the net benefit of a strategy of predicting all patients. The black line indicates the net benefit of predicting no patients. The dotted line represents the net benefit of predicting patients according to the nomogram. The nomogram-based decisions were supported in the range of threshold probabilities of approximately 30–95%, indicating the clinical usefulness of the nomogram.
The distribution of MDROs in the primary cohort
In 178 cases of MDROs in the primary cohort, Acinetobacter baumannii (n = 62, 30.0%) was the most common among gram-negative bacteria, followed by Escherichia coli (n = 51, 24.6%); among gram-positive bacteria, the majority of cases involved Enterococcus faecium (n = 17, 37.8%), followed by methicillin-resistant Staphylococcus aureus (MRSA) (n = 9, 20.0%) (Table 3).
With the widespread use of antibiotics, bacterial drug resistance is becoming increasingly serious. However, progress in treatment has been limited . Rational use of antibiotics is of importance to prevent and control MDRO infections [4, 5, 23].
For empirical treatment decision assistance, we identified independent parameters from data available in the early stage of the ICU stay and constructed a novel nomogram for predicting the occurrence of MDRO colonization or infection.
Our nomogram has several strengths. A remarkable strength is its ease of use. The parameters obtained from the patient status in the early stages of the ICU stay are well defined, easily measured, and routinely available. The construction of our model using the Pitt score, CRP level and gender, which do not require information about the diagnosis and detailed medical history, may present an advantage in the ICU setting [15, 16]. Another advantage of our nomogram is that it was subjected to an independent external validation process and showed good discrimination. In addition, the use of the nomogram has another advantage over previous models. As long as the results of relevant factors are input into the nomogram according to the steps shown in Fig. 2b, the corresponding results are obtained easily. This helps in quantitatively calculating the probability of MDRO colonization or infection.
The nomogram established in the present study incorporated three factors, namely, the male sex, CRP level, and Pitt score. CRP is an inflammatory marker that has been used in clinical practice for decades . Many previous studies have suggested the utility of the CRP level as an outcome predictor in critically ill patients with sepsis [25, 26]. We determined for the first time that the CRP level also has predictive value for MDRO colonization or infection. Pitt scores have been widely used to evaluate the prognosis and infection potential of critically ill patients [27, 28]. To the best of our knowledge, the present study is the first to focus on assessing the Pitt score as an indicator of MDRO colonization or infection. In a study on nosocomial sepsis by Kavitha S et al., a high degree of multidrug resistance was observed among both gram-positive and gram-negative organisms in patients with nosocomial sepsis, and male gender was the independent predictor of mortality . These findings may confirm to some extent the relationship between male gender and MDROs infection. In other words, all the factors revealed as significantly predictive in our study are plausible.
Notably, the nomogram only applies to adult patients in the ICU. Its purpose is to predict the occurrence of MDRO colonization or infection during the ICU stay, excluding MDROs that are detected before entry into the ICU or within the first 48 h in the ICU.
Some limitations must also be recognized when interpreting the results. First, the nomogram was developed based on data obtained retrospectively at a single center, and only patients with microbial culture were included in the study. Further studies are warranted to explore whether this nomogram can be extended to all ICU patients. Second, other valuable predictors may have been ignored in our analysis. Our nomogram might be improved as additional predictive variables are incorporated; it is not a finished product that is perfectly able to predict MDRO colonization or infection in the ICU. Third, our nomogram only predicts the occurrence of MDRO colonization or infection, but does not implicate any causal relationship between gender, CRP, Pitt score and occurrence of MDRO colonization and/or infection.
Three independent predictors, male sex, higher CRP level and higher Pitt score, were identified in our study. These predictors are readily available during the early ICU stay and can be assembled to construct an easy-to-use nomogram predicting MDRO colonization or infection. Thus, the nomogram is potentially useful for predicting the occurrence of MDRO colonization or infection in ICU patients.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Intensive care unit
Receiver operating characteristic
Area under the receiver operating characteristic curve
- Pitt scores:
Pitt bacteremia scores
Absolute neutrophil count
Absolute lymphocyte count
- PaO2/FiO2 :
Sequential Organ Failure Assessment score
- APACHE II:
Acute Physiology and Chronic Health Evaluation II score
Akaike Information Criterion
Methicillin-resistant Staphylococcus aureus
Methicillin-sensitive Staphylococcus aureus
Rice LB. Federal funding for the study of antimicrobial resistance in nosocomial pathogens: no eskape. Infect Dis. 2008;197(8):1079–81.
Albiger B, Glasner C, Struelens MJ, Grundmann H, Monnet DL. Carbapenemase-producing Enterobacteriaceae in Europe: assessment by national experts from 38 countries. Euro Surveill. 2015;20(45):1–18.
Magiorakos AP, Srinivasan A, Carey RB, Carmeli Y, Falagas ME, Giske CG, et al. Multidrug-resistant, extensively drug-resistant and pandrug-resistant bacteria: an international expert proposal for interim standard deinitions for acquired resistance. Clin Microbiol Infect. 2012;18(3):268–81.
Organization W H. Antimicrobial resistance: global report on surveillance. Australasian Med J. 2014;7(4):237.
Ministry of Health PRC. Technical guidelines for the prevention and control of nosocomial infection with multidrug resistant bacteria (trial). Chin Crit Care Med. 2011;13(2):108–9.
Bai PF, Hu SQ, Wu XY, Yang WJ. Study on risk factors of infection or colonization caused by multiple drug-resistant organisms from ICU patients. World Latest Med Inform. 2018;18(09):4–8.
Huang X, Deng ZD, Ni YX, Deng M, Hu BJ, Li LY, et al. Consensus among Chinese experts on the prevention and control of multidrug resistant bacteria nosocomial infection. Chin J Infection Control. 2015;14(1):1–9.
Kattan MW. Nomograms. Intro Semin Urol Oncol. 2002;20:79–81.
Andrea N, Guru S, Salvatore LV, Daniele G, Aristotelis B, Crabb Simon J, et al. Nomogram-based Prediction of Overall Survival in Patients with Metastatic Urothelial Carcinoma Receiving First-line Platinum-based Chemotherapy: Retrospective International Study of Invasive/Advanced Cancer of the Urothelium (RISC). Eur. Urol. 2017;71:281–9.
Young LJ, Hyun CD, Gil LJ, Hyejung S, Lee Yeon J, Hwan LS. A nomogram predicting the need for abdominal and pelvic computed tomography in blunt trauma patients: a retrospective cohort study. Int J Surg. 2017;47:127–34.
Lara J, Ruta Y, Kevin C, Laura T, Lo RG, Gregory W, et al. Development and validation of nomograms to provide individualised predictions of seizure outcomes after epilepsy surgery: a retrospective analysis. Lancet Neurol. 2015;14:283–90.
Li CH, Wu AH. MDR, XDR and PDR provisional standard definition of multidrug resistance bacteria-international expert recommendation. Chin J Infect Cont. 2014;13(1):62–4.
Casadevall A, Pirofski L-A. Host–pathogen interactions: basic concepts of microbial commensalism, colonization, infection, and disease. Infect Immun. 2000;68:6511–8.
Zahorec R. Ratio of neutrophil to lymphocyte counts--rapid and simple parameter of systemic inflammation and stress in critically ill. Bratisl Lek Listy. 2001;102(1):5–14.
Hill PC, Birch M, Chambers S, Drinkovic D, Ellis-Pegler RB, Everts R. Et a1. Prospective study of 424 cases of Staphylococcus aureus bacteraemia:determination of factors affecting incidence and mortality. Int Med J. 2001;31(2):97–103.
Wu T. The incidence and prognosis of ventilator-associated pneumonia were evaluated by different scoring systems. Med Theory Practice. 2012;25(9):1020–9.
Van Buuren S, Groothuis-Oudshoorn K. Mice: multivariate imputation by chained equations in R (article). J Stat Softw. 2011;45(3):1–67.
Bozdogan H. Model selection and Akaike’s information criterion (AIC): the general theory and its analytical extensions. Psychometrika. 1987;52:345–70.
Frank E, Harrell Jr. Rms: Regression Modeling Strategies. R Package version 3.4–2. https://cran.r-project.org/web/packages/rms/index.html.
Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Mak. 2006;26(6):565–74.
Secin Fernando P, Bianco Fernando J, Angel C, Eastham James A, Scardino Peter T, Bertrand G, et al. Is it necessary to remove the seminal vesicles completely at radical prostatectomy? Decision curve analysis of European Society of Urologic Oncology criteria. J. Urol. 2009;181:609–13 discussion 614.
Collins GS, Reitsma JB, Altman DG, KGM M. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med. 2015;162:55–63.
Strich JR, Palmore TN. Preventing transmission of multidrug-resistant pathogens in the intensive care unit. Infect Dis Clin N Am. 2017;31(3):535–50.
Vincent JL, Donadello K, Schmit X. Biomarkers in the critically ill patient: C-reactive protein. Crit Care Clin. 2011;27:241–51.
Ryu JA, Yang JH, Lee D, Park CM, Suh GY, Jeon K, et al. Clinical usefulness of procalcitonin and C-reactive protein as outcome predictors in critically ill patients with severe sepsis and septic shock. PLoS One. 2015;10(9):e0138150.
Devran O, Karakurt Z, Adıgüzel N, Güngör G, Moçin OY, Balcı MK, et al. C-reactive protein as a predictor of mortality in patients affected with severe sepsis in intensive care unit. Multidiscip Respir Med. 2012;7(1):47.
Lee CC, Lee CH, Hong MY, Tang HJ, Ko WC. Timing of appropriate empirical antimicrobial administration and outcome of adults with community-onset bacteremia. Crit Care. 2017;21:119.
Vaquero-Herrero MP, Ragozzino S, Castaño-Romero F, Siller-Ruiz M, Sánchez González R, García-Sánchez JE, et al. The Pitt bacteremia score, Charlson comorbidity index and chronic disease score are useful tools for the prediction of mortality in patients with Candida bloodstream infection. Mycoses. 2017;60(10):676–85.
Kavitha S, Madhura P, Vandana Kalwaje E, Mukhopadhyay C. Clinico-microbiological profile and outcomes of nosocomial sepsis in an Indian tertiary care hospital - a prospective cohort study. Pathog Global Health. 2015;109(5):228.
The authors are indebted to Xiamen Municipal Bureau of Science and Technology, Fujian Provincial Health and Family Planning Commission, the medical records and bacteria department of our hospital for their help and thank all the anonymous reviewers for their valuable comments and suggestions on improving the quality of our study.
This study was supported by research grants from Xiamen Municipal Bureau of Science and Technology Project (No: 3502Z20199129 and 3502Z20154006) and Fujian Provincial key discipline Project (No: 2018. 145).
Ethics approval and consent to participate
Ethical approval was granted according to national requirements. This study was approved by the Medical Ethics Committee of the First Affiliated Hospital of Xiamen University and the Medical Ethics Committee of Xiamen Hospital of Traditional Chinese Medicine. Informed consent was waived because the study was conducted retrospectively and no interventions were applied.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Wang, L., Huang, X., Zhou, J. et al. Predicting the occurrence of multidrug-resistant organism colonization or infection in ICU patients: development and validation of a novel multivariate prediction model. Antimicrob Resist Infect Control 9, 66 (2020). https://doi.org/10.1186/s13756-020-00726-5
- Multidrug-resistant organisms
- Pitt score