- Open Access
Risk prediction models to guide antibiotic prescribing: a study on adult patients with uncomplicated upper respiratory tract infections in an emergency department
Antimicrobial Resistance & Infection Control volume 9, Article number: 171 (2020)
Appropriate antibiotic prescribing is key to combating antimicrobial resistance. Upper respiratory tract infections (URTIs) are common reasons for emergency department (ED) visits and antibiotic use. Differentiating between bacterial and viral infections is not straightforward. We aim to provide an evidence-based clinical decision support tool for antibiotic prescribing using prediction models developed from local data.
Seven hundred-fifteen patients with uncomplicated URTI were recruited and analysed from Singapore’s busiest ED, Tan Tock Seng Hospital, from June 2016 to November 2018. Confirmatory tests were performed using the multiplex polymerase chain reaction (PCR) test for respiratory viruses and point-of-care test for C-reactive protein. Demographic, clinical and laboratory data were extracted from the hospital electronic medical records. Seventy percent of the data was used for training and the remaining 30% was used for validation. Decision trees, LASSO and logistic regression models were built to predict when antibiotics were not needed.
The median age of the cohort was 36 years old, with 61.2% being male. Temperature and pulse rate were significant factors in all 3 models. The area under the receiver operating curve (AUC) on the validation set for the models were similar. (LASSO: 0.70 [95% CI: 0.62–0.77], logistic regression: 0.72 [95% CI: 0.65–0.79], decision tree: 0.67 [95% CI: 0.59–0.74]). Combining the results from all models, 58.3% of study participants would not need antibiotics.
The models can be easily deployed as a decision support tool to guide antibiotic prescribing in busy EDs.
Upper respiratory tract infection (URTI) is one of the most cited reasons for use of antibiotics . In the majority of URTIs, the routine use of antibiotics is not recommended [1,2,3,4,5]. In the United States (U.S.), it was estimated that antibiotics have been prescribed for over 60% of uncomplicated URTIs in adults and increasingly so for broad-spectrum antibiotics [6,7,8,9]. Between 2001 and 2010, 126 million (12.2%) emergency department (ED) visits in the U.S. were for acute respiratory tract infections, with almost half (47.9%) of patients with infections being administered antibiotics inappropriately . From 2009 to 2010, adults had the highest rate of inappropriate antibiotic use for acute respiratory tract infections (URTIs, influenza, and viral pneumonia), with 500 antibiotic prescriptions per 1000 ED visits for adults aged 20–64 years and 666 per 1000 visits for those aged > = 65 years .
In Singapore, while primary care clinics are highly accessible in the community, there are individuals who preferred to seek care at the ED for URTI, accounting for a substantial proportion of ED attendances . URTI accounted for 6–10% of ED visits by non-frequent attenders (1–4 ED visits in one year) and up to 25% of ED visits by frequent attenders (≥5 ED visits in one year) . A previous study at an adult general hospital has reported that 24% of adult patients attending at ED for URTI were inappropriately prescribed antibiotics, with the penicillin class of antibiotics being the most commonly prescribed .
Studies have shown a strong link between antibiotic prescribing and antimicrobial resistance [6, 14, 15]. In addition, a population-wide study on US pharmacy records showed that antibiotic use and resistance appears to be closely linked to broadly distributed low-intensity prescribing . As a consequence, antimicrobial resistance has risen to dangerously high levels globally. A global study estimated that Escherichia.coli and Klebsiella pneumoniae resistant to third-generation cephalosporin caused 6.4 million bloodstream infections and 50.1 million serious infections in 2014. Carbapenem-resistant strains were estimated to cause 0.5 million bloodstream infections and 3.1 million serious infections . Antimicrobial resistance is associated with higher medical cost, prolonged hospital stays, increased mortality and economic burden [18, 19]. Hence, there is an urgent need to ensure the prudent use of antibiotics for common illnesses predominantly of viral etiology such as URTIs.
In Singapore, considerable efforts have been made to address antibiotic resistance . Although computerized decision support systems have been developed to guide antibiotic prescribing, they are largely based on guidelines drawn by expert consensus and not on actual data derived from local patients . Furthermore, most studies on antibiotic prescribing focus on understanding behaviors and perceptions or finding associative factors for antibiotic prescribing decisions [22,23,24,25]. To date, prediction models to guide antibiotic prescribing has been confined largely to pediatric populations [26,27,28].
Differentiating bacterial and viral infections is not straightforward in adult URTIs. In uncertainty avoidance, physicians tend to over-prescribe antibiotics. In this study, we aim to develop prediction models based on local clinical and laboratory data to guide antibiotic prescribing for adult patients with uncomplicated URTI with the ultimate goal of deploying them as an evidence-based clinical decision support tool for routine practice.
Seven hundred-fifteen patients were recruited from the ED at Tan Tock Seng Hospital (TTSH), the second largest adult hospital in Singapore between June 2016 and November 2018. Eligible patients were 21 years and above attending at TTSH ED for the first time with a primary diagnosis of uncomplicated URTI (ICD10-AM J00-J06) within 30 days who provided informed consent. TTSH ED is the busiest ED in the country, attending to an average of 450 patients daily.
Selection of participants
At discharge from the emergency department, the patients were invited to participate in the study and consent was obtained. Patients who were subsequently admitted were excluded from the study. A nasopharyngeal swab was taken to determine the presence of respiratory viruses using multiplex PCR (Seeplex® RV15 ACE Detection). The panel detects 15 major respiratory viruses including adenovirus, bocavirus 1/2/3/4, coronavirus 229E/NL63 and OC43, enterovirus, influenza A and B, metapneumovirus, parainfluenza 1, 2, 3 and 4, respiratory syncytial virus A and B, and rhinovirus. We chose not to include the bacterial respiratory PCR panel in the study, as commensal bacteria are common in the upper respiratory tract and detection on PCR does not necessarily indicate a bacterial infection. A lower respiratory tract sample (such as sputum) was also not practicable for every participant. Instead, we performed a point-of-care C-reactive protein (CRP) test on a drop of capillary blood obtained from a finger prick (QuikRead go® CRP). CRP is widely used in clinical settings as a supportive test to diagnose bacterial infection .
Our main outcome of interest was to identify patients for whom antibiotics were clearly not recommended (1 = NABX) from those for whom the physician should review the need for antibiotics (0 = RABX). We defined NABX as patients with a respiratory virus detected via PCR and CRP < 20 mg/L or patients who did not have a respiratory virus detected via PCR and CRP ≤ 5 mg/L . Patients who did not fall into these 2 categories were assigned to the RABX group.
Demographic, clinical and laboratory data documented as part of the patients’ routine care were extracted from the hospital electronic medical records. These include age, gender, ethnicity, visit date, pre-existing comorbidities, respiratory symptoms, full blood count, kidney/liver panels, and biochemistry tests. According to comorbid status of the participants, Charlson’s Comorbidity Index was calculated . Additionally, epidemiologic data on smoking, influenza vaccination, travel history, and prior medical consultation and antibiotic consumption were obtained from an interviewer-administered questionnaire.
Descriptive statistics were performed and differences between the NABX and RABX groups compared using Mann-Whitney U-test for continuous variables and Chi-squared test for categorical variables. Where appropriate, Fisher’s exact tests were used to account for small cell sizes. Variables with more than 10% of data missing were excluded from the analysis. Categorical variables with data missing were recoded as 0 under the assumption that presence of any clinical covariates would have been recorded. Continuous variables were imputed according to their group medians.
With ease of use in mind, we decided to perform predictive modeling using 3 methods that could subsequently be easily deployed for implementation: logistic regression, LASSO regression and classification and regression trees (CART). The models were derived using 70% of the participants as training set. The optimal cutoffs for each model were decided by taking the predicted probability that achieved the highest sensitivity with specificity of at least 0.4. The final model performance was validated by calculating the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, positive predictive value and negative predicative values on the remaining 30% data.
Univariate analysis was performed on all 50 candidate variables. Demographic factors, clinically relevant variables and significant variables from univariate models were fitted into the final multivariable model via stepwise elimination using a cutoff of p < 0.1.
In stepwise regression, it is often difficult to tell the effect after removal of each variable. Model selection may also be difficult in datasets with a huge number of variables. LASSO regression addresses this by shrinking the coefficients of features that are less relevant or exhibit collinearity to zero. This reduces the problem of overfitting of prediction model and the variance without substantial increase in bias. We performed this by selecting a minimum optimal shrinking parameter of λ = 0.03971531 through a 10-fold cross validation of the training dataset, giving a set of coefficients governed by Eq. 1.
CART is a popular tool in supervised learning for classification as they are distribution-free and robust to outliers. Unlike generalized linear models, classification trees make an excellent tool for overcoming problems due to multicollinearity and skewed covariates. It uses the Gini index to iteratively split branches based on purity. This feature is an added benefit as important interactions can be easily detected. It also has the ability to identify patient subgroups that are more predictive than others. In our analysis, we created a maximum tree depth of 5 and a minimum of 10 subjects in a node before a split is attempted to prevent overfitting. The choice of the final tree size was decided by finding the number of splits that produce the smallest cross-validation error.
Characteristics of study subjects
The study participants were young, with a median age of 36 years (IQR: 28–51 years) and a slight preponderance of males (61.3%). (Table 1) Two-thirds (66.4%) had no pre-existing comorbidities and one-third (36.8%) had received influenza vaccination in the prior 12 months. Almost two-thirds (60%) of the patients presented with fever. While 50.3% of the patients had nasal problems like running and blocked nose, 45.6% of them had a sore throat. Almost half (47.8%) of the patients had a respiratory virus detected. Influenza (20.6%) and rhinovirus (14.4%) were common respiratory viruses detected. Influenza circulated year-round, with bimodal peaks observed in November and May–June, with rhinovirus dominating in the inter-influenza periods. (data not shown) The median CRP level was 6 mg/L (IQR 4–19 mg/L) and its natural logarithm had a correlation of 0.022 with the viral status. Patients with a detectable result on the PCR had higher CRP levels at 8 mg/L (IQR: 4–19 mg/L) as compared to those without at 4 mg/L (IQR: 4–19 mg/L) (p = 0.041). Among patients with low (< 5 mg/L), moderate (5–20 mg/L) and high (> 20 mg/L) levels of CRP, approximately 40, 60 and 45% of them respectively had a detectable result on PCR (Fig. 1). In total, 461 (64.5%) patients were classified as NABX (Fig. 1).
Baseline covariates were largely similar between patients in the RABX and NABX groups. Patients were less likely to have prior consultation 14 days before the ED visit in the NABX group compared with the RABX group (49.5% vs 57.5%, p = 0.04). Influenza vaccination uptake rates were similar in both groups (37.0% vs 36.7%, p = 0.675). There was no evidence of comorbidity being associated with antibiotic need, except those with steroid use and cancer. Median time from earliest symptom onset to ED visit was similar between both groups at 4 days (IQR: 3–7 days). RABX patients were more likely to present with symptoms of fever, body ache, sore throat and vomiting. NABX patients were likely to display symptoms of shortness of breath and giddiness. Patients in the RABX group had a higher median maximum body temperature and lower median systolic and diastolic blood pressures than those in the NABX group (Table 1).
Highest temperature and highest pulse rate were commonly identified to be important predictors in all logistic, LASSO and CART models (Table 2; Fig. 2). In addition, age, the presenting symptoms of fever, giddiness and shortness of breath were identified to be significant predictors in the final logistic regression model. Similarly, Indian ethnicity, fever, giddiness and cancer status were included in the LASSO model. (Table 2) The AUC on the validation set for all three models varied slightly with the highest value of 0.72 (95% CI: 0.65–0.79) for the logistic model, followed by 0.70 (95% CI: 0.62–0.77) for the LASSO model, then 0.67 (95% CI: 0.59–0.74) for the CART model. (Fig. 3) Using a predicted probability cutoff of 0.6, 0.625, and 0.675 for the logistic, LASSO, and CART models respectively, the logistic model produced a sensitivity (Sen) of 0.72 (95% CI: 0.64–0.79) and a specificity (Spe) of 0.65 (95% CI: 0.53–0.75) on the validation set. In contrast, the corresponding values in the LASSO model were 0.72 (95% CI: 0.64–0.79) and 0.62 (95% CI: 0.50–0.73) respectively. Among the three models, the CART model had the highest sensitivity of 0.77 (95% CI: 0.68–0.84) but was the least specific compared with the other 2 models at 0.49 (95% CI: 0.39–0.59). The positive predictive value (PPV) and negative predictive value (NPV) were respectively the smallest and largest for the CART (PPV = 0.62; NPV = 0.66) compared to the logistic and LASSO models (logistic: PPV = 0.78, NPV = 0.56; LASSO: PPV = 0.77, NPV = 0.55). (Table 3).
In addition, we looked at the corresponding metrics at a probability cut-off of 0.5. The models have marked improvement in sensitivity, but specificity fell below 0.5 (Logistic: Sen = 0.88, Spe = 0.34; LASSO: Sen = 0.94, Spe = 0.27; CART: Sen = 0.95, Spe = 0.29). Detailed documentation on different probability cutoffs can be found in Additional File 1.
A qualitative study previously conducted in our hospital revealed that ED physicians were confident with their clinical decisions. However, doctors had a lower threshold for prescribing antibiotics for older patients who were immunocompromised and suffering from chronic conditions. Junior physicians were observed to be uncomfortable not prescribing antibiotics for URTI patients . Patients with bacterial and viral infections present with similar symptoms and differentiation of patients requiring antibiotics from those who do not is problematic. Our algorithms developed using three rigorous statistical methods together with laboratory-based confirmatory tests served as a good guide for physicians in their decisions on antibiotic prescribing for URTI patients. A recent Cochrane Systematic Review provided evidence that patient satisfaction and clinical outcomes were similar between those for whom antibiotic prescribing was delayed and those not prescribed antibiotics at all. Delayed prescribing of antibiotics has been found to be associated with marked reduction in antibiotic use . Our results showed that the performance of all 3 prediction models were similarly modest. While we tried to be pragmatic with our algorithms, we also carried out similar analysis on more complex classification trees and random forests, both of which showed minimal or no improvement in performance (AUC ~ 0.7). A recent systematic review showed that there was minimal improvement using machine learning techniques over traditional regression models .
Relevant literature on prediction models for antibiotic prescribing in adults are limited and tended to focus on life-threatening infections. Several clinical prediction models were built for pneumonia and serious bacterial infections in children mostly using either logistic regression or decision trees [27, 37, 38]. The findings from this study add to the limited knowledge on clinical decision support tools for antibiotic prescribing in an adult ED setting.
We found that fever and pulse rate were significant factors in all 3 models. Most studies on viral respiratory infections have focused on influenza with a high temperature identified as a significant risk factor in both younger and older adults [39,40,41,42]. Heart rate was found to be significant in a group of patients presenting with influenza-like illness at a hospital emergency department . A significant proportion of patients with influenza infection present with tachycardia. This could be due to the physiologic response to fever although cardiac manifestations are not uncommon with complications of influenza . Shortness of breath and giddiness were also found to be significant predictors in two of our models. While these symptoms could be non-specific, we believe that it would have to be significant enough for adult patients to volunteer these symptoms to their physicians when they had them. As physicians often have to make antibiotic prescribing decisions based on subjective symptoms reported by their patients, we believe that our clinical decision support tool, developed from three different models, will provide physicians with a reliable tool when making antibiotic prescribing decisions for patients with URTI at the point-of-care.
There are a few limitations in our study. Firstly, the ability to predict well is dependent on the richness of the data. Our study is limited to the information obtained at the time that the patient medically attended at ED. Knowledge on baseline vital signs and trajectories prior to ED visit may be important information that could improve our models. A 2017 study by Stanford University on wearable devices detected that anomalies in skin temperature and heart rate corresponded to periods of high CRP levels . Secondly, we did not consider laboratory parameters like full blood count, renal and liver function panels in our model as 48% of patients did not receive a full blood count, and even fewer had renal and liver function tests. To address this, we created new variables simulating if full blood count or renal function panel were performed, postulating that doctors may have ordered these tests on patients who warranted further investigation. However, we could not detect any statistical differences. Nevertheless, our models represented real-life situations where such tests were not commonly ordered for uncomplicated URTI patients and the results would often not be factored into the antibiotic decision making by physicians. Thirdly, our study was based at a tertiary hospital and need to be validated in other settings to ensure generalizability. Patients might seek care in the ED after medical attendances at primary care clinics failed to alleviate their symptoms. Our data reflected this as 50% of patients had prior consultation although the time between the earliest symptom onset to ED visit was only 4 days on average. The local literature on vaccination uptake in the community is limited. To our knowledge, there is only one population health survey on influenza vaccination uptake in older adults done in 2013 . The authors found that the influenza vaccination uptake in this population was only 15.2%. Our patient cohort had a higher vaccination rate (37%) than in the community. However, this does not invalidate our findings and we believe that the impact on the generalizability of our models is minimal.
Nonetheless, our study had its strengths. We were able to take seasonality into account as the study spanned two years covering two influenza seasons each of Northern and Southern Hemispheres. The use of PCR together with appropriate CRP cutoffs were based on findings from several international studies and selected to be the most conservative estimates. The cutoff point for CRP was set lower to increase sensitivity of the RABX group [30, 47, 48]. We also note that the proportion of positive viral PCR among the CRP < 5 and CRP > 20 groups were quite similar (Fig. 1). Patients with high CRP and positive viral PCR represent patients with secondary bacterial infection. In a sub-group analysis of 229 patients with complete blood count performed, those with high CRP levels of > 20 and positive viral PCR were almost twice as likely to have leukocyte counts of > 9.6 × 109/L as those with CRP < 20 and positive viral PCR (39.3% vs. 20.2%, p = 0.003). This supports our exclusion of patients with high CRP and positive viral PCR from the NABX group. Comprehensive assessment of medical records was performed by two clinically trained individuals with standardization in data extraction methods and definitions to ensure data accuracy and consistency. Analysing the data with 3 different methods not only allowed us to compare models but also allowed us to triangulate the findings from all our models. Notably, maximum pulse rate and highest temperature were considered as important variables in all 3 models. Finally, our models were either coefficient or rule based. They can easily be entered into an excel sheet or the hospital electronic system without the need to integrate complicated programming codes.
Combining the results from the three models, 58.3% of study participants would not need antibiotics. Moving forward, physicians could use this tool as a useful complement to their clinical judgement in their practice to guide their decisions on antibiotic prescribing. Antibiotics should be prescribed with caution even during low influenza periods as there are still other viruses circulating throughout the year. At the time of writing, we have developed a mobile application (app) named the “Abx SteWARdS” to provide clinical decision support for busy physicians practicing in the ED on antibiotic prescribing for URTI (Fig. 4). ED physicians are required to fill in 9 parameters all on one screen. All fields are mandatory, and the app will provide a recommendation either to review the need for antibiotics or that antibiotics was not needed, based on the predicted outcomes of all 3 validated models. A validation study is underway. It is hoped that evidence-based clinical decision support tools accessible at the point-of-care can lead to better antibiotic prescribing decisions and the reduction of antibiotic resistance.
Availability of data and materials
All data generated or analysed during this study are included in this published article (and its Additional file 1).
Kenealy T, Arroll B. Antibiotics for the common cold and acute purulent rhinitis. Cochrane Database of Systematic Reviews. 2013. https://doi.org/10.1002/14651858.CD000247.pub3/abstract cited 2019 13 Dec.
Gonzales R, Malone DC, Maselli JH, Sande MA. Excessive antibiotic use for acute respiratory infections in the United States. Clin Infect Dis. 2001;33:757–62.
Gonzales R, Bartlett JG, Besser RE, Cooper RJ, Hickner JM, Hoffman JR, et al. Principles of appropriate antibiotic use for treatment of acute respiratory tract infections in adults: background, specific aims, and methods. Ann Emerg Med. 2001;37:690–7.
Smith SM, Fahey T, Smucny J, Becker LA. Antibiotics for acute bronchitis. Cochrane Database Syst Rev. 2017. https://doi.org/10.1002/14651858.CD000245.pub4/full cited 2019 13 Dec.
Aabenhus R, Jensen J-US, Jørgensen KJ, Hróbjartsson A, Bjerrum L. Biomarkers as point-of-care tests to guide prescription of antibiotics in patients with acute respiratory infections in primary care. Cochrane Database of Syst Rev. 2014. https://doi.org/10.1002/14651858.CD010130.pub2/full cited 2019 13 Dec.
Xu KT, Roberts D, Sulapas I, Martinez O, Berk J, Baldwin J. Over-prescribing of antibiotics and imaging in the management of uncomplicated URIs in emergency departments. BMC Emerg Med. 2013;13:7.
Thorpe JM, Smith SR, Trygstad TK. Trends in emergency department antibiotic prescribing for acute respiratory tract infections. Ann Pharmacother. 2004;38:928–35.
Mainous AG, Saxena S, Hueston WJ, Everett CJ, Majeed A. Ambulatory antibiotic prescribing for acute bronchitis and cough and hospital admissions for respiratory infections: time trends analysis. J R Soc Med. 2006;99:358–62.
Ebell MH, Radke T. Antibiotic use for viral acute respiratory tract infections remains common. Am J Manag Care. 2015;21:e567–75.
Donnelly JP, Baddley JW, Wang HE. Antibiotic utilization for acute respiratory tract infections in U.S. emergency departments. Antimicrob Agents Chemother. 2014;58:1451–7.
Paul P, Heng BH, Seow E, Molina J, Tay SY. Predictors of frequent attenders of emergency department at an acute general hospital in Singapore. Emerg Med J. 2010;27:843–8.
Chan JS-E, Tin AS, Chow WL, Tiah L, Tiru M, Lee CE. Frequent attenders at the emergency department: an analysis of characteristics and utilisation trends. Proceed Singapore Healthcare. SAGE Publications Ltd. 2018;27:12–9.
Lee W. Antibiotic prescribing for patients with upper respiratory tract infections by emergency physicians in a Singapore tertiary hospital. Hong Kong J Emerg Med. 2005;12:70–6.
Costelloe C, Metcalfe C, Lovering A, Mant D, Hay AD. Effect of antibiotic prescribing in primary care on antimicrobial resistance in individual patients: systematic review and meta-analysis. BMJ. 2010;340:c2096.
Bell BG, Schellevis F, Stobberingh E, Goossens H, Pringle M. A systematic review and meta-analysis of the effects of antibiotic consumption on antibiotic resistance. BMC Infect Dis. 2014;14:13.
Olesen SW, Barnett ML, MacFadden DR, Brownstein JS, Hernández-Díaz S, Lipsitch M, et al. The distribution of antibiotic use and its association with antibiotic resistance. eLife:7 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6307856/ [cited 2019 18 Oct].
Temkin E, Fallach N, Almagor J, Gladstone BP, Tacconelli E, Carmeli Y. Estimating the number of infections caused by antibiotic-resistant Escherichia coli and Klebsiella pneumoniae in 2014: a modelling study. Lancet Glob Health. 2018;6:e969–79.
Cassini A, Högberg LD, Plachouras D, Quattrocchi A, Hoxha A, Simonsen GS, et al. Attributable deaths and disability-adjusted life-years caused by infections with antibiotic-resistant bacteria in the EU and the European economic area in 2015: a population-level modelling analysis. Lancet Infect Dis. 2019;19:56–66.
Founou RC, Founou LL, Essack SY. Clinical and economic impact of antibiotic resistance in developing countries: A systematic review and meta-analysis. PLoS One. 2017;12 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5739407/ [cited 2019 18 Jul].
Teng CB, Lee W, Yeo CL, Lee SY, Ng TM, Yeoh SF, et al. Guidelines for antimicrobial stewardship training and practice. Ann Acad Med Singap. 2012;41:29–34.
Chow AL, Ang A, Chow CZ, Ng TM, Teng C, Ling LM, et al. Implementation hurdles of an interactive, integrated, point-of-care computerised decision support system for hospital antibiotic prescription. Int J Antimicrob Agents. 2016;47:132–9.
Chan YY, MAB I, Wong CM, Ooi CK, Chow A. Determinants of antibiotic prescribing for upper respiratory tract infections in an emergency department with good primary care access: a qualitative analysis. Epidemiol Infect. 2019;147 Available from: https://www.cambridge.org/core/journals/epidemiology-and-infection/article/determinants-of-antibiotic-prescribing-for-upper-respiratory-tract-infections-in-an-emergency-department-with-good-primary-care-access-a-qualitative-analysis/5471BF1F74EA361F7AA09655C13BF884/core-reader [cited 2019 15 Jul].
May L, Gudger G, Armstrong P, Brooks G, Hinds P, Bhat R, et al. Multisite exploration of clinical decision-making for antibiotic use by emergency medicine providers using quantitative and qualitative methods. Infect Control Hosp Epidemiol. 2014;35:1114–25.
Lee T-H, Wong JG, Lye DC, Chen MI, Loh VW, Leo Y-S, et al. Medical and psychosocial factors associated with antibiotic prescribing in primary care: survey questionnaire and factor analysis. Br J Gen Pract. 2017;67:e168–77.
Rebnord IK, Sandvik H, Mjelle AB, Hunskaar S. Factors predicting antibiotic prescription and referral to hospital for children with respiratory symptoms: secondary analysis of a randomised controlled study at out-of-hours services in primary care. BMJ Open. 2017;7:e012992.
Oonsivilai M, Mo Y, Luangasanatip N, Lubell Y, Miliya T, Tan P, et al. Using machine learning to guide targeted and locally-tailored empiric antibiotic prescribing in a children’s hospital in Cambodia. Wellcome Open Res. 2018;3 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6352926/ [cited 2019 15 Jul].
van de Maat J, Nieboer D, Thompson M, Lakhanpaul M, Moll H, Oostenbrink R. Can clinical prediction models assess antibiotic need in childhood pneumonia? A validation study in paediatric emergency care. PLoS One. 2019;14 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6563975/ [cited 2019 18 Jul].
Irwin AD, Grant A, Williams R, Kolamunnage-Dona R, Drew RJ, Paulus S, et al. Predicting risk of serious bacterial infections in febrile children in the emergency department. Pediatrics. 2017;28(2):140. https://doi.org/10.1542/peds.2016-2853.
Flanders SA, Stein J, Shochat G, Sellers K, Holland M, Maselli J, et al. Performance of a bedside c-reactive protein test in the diagnosis of community-acquired pneumonia in adults with acute cough. Am J Med. 2004;116:529–35.
Hopstaken RM, Muris JW, Knottnerus JA, Kester AD, Rinkens PE, Dinant GJ. Contributions of symptoms, signs, erythrocyte sedimentation rate, and C-reactive protein to a diagnosis of pneumonia in acute lower respiratory tract infection. Br J Gen Pract. 2003;53:358–64.
Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–83..
Therneau TM, Atkinson EJ, Foundation M. An introduction to recursive partitioning using the RPART routines. R Journal. 2019.
Williams GJ. Rattle: a data mining GUI for R. R Journal. 2009;1:45.
Hastie T, Qian J. An Introduction to glmnet. The R Journal. 2016; Available from: https://cran.r-project.org/web/packages/glmnet/vignettes/glmnet.pdf.
Spurling GK, Mar CBD, Dooley L, Foxlee R, Farley R. Delayed antibiotic prescriptions for respiratory infections. Cochrane Database Syst Rev. 2017. https://doi.org/10.1002/14651858.CD004417.pub5/full cited 2019 Jul 16.
Christodoulou E, Ma J, Collins GS, Steyerberg EW, Verbakel JY, Van Calster B. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol. 2019;110:12–22.
Nijman RG, Vergouwe Y, Thompson M, van Veen M, AHJ v M, van der Lei J, et al. Clinical prediction model to aid emergency doctors managing febrile children at risk of serious bacterial infections: diagnostic study. BMJ. 2013;346:f1706.
den Bruel AV, Hall H, Aertgeerts B, Bruyninckx R, Aerts M, Buntinx F. Signs and symptoms for diagnosis of serious infections in children: a prospective study in primary careCommentary. Br J Gen Pract. 2007;57:538–46.
Monto AS, Gravenstein S, Elliott M, Colopy M, Schweinle J. Clinical signs and symptoms predicting influenza infection. Arch Intern Med. 2000;160:3243–7.
Ho ZJM, Zhao X, Cook AR, Loh JP, Ng SH, Tan BH, et al. Clinical differences between respiratory viral and bacterial mono- and dual pathogen detected among Singapore military servicemen with febrile respiratory illness. Influenza Other Respir Viruses. 2015;9:200–8.
Lam P-P, Coleman BL, Green K, Powis J, Richardson D, Katz K, et al. Predictors of influenza among older adults in the emergency department. BMC Infect Dis. 2016;16:615.
Michiels B, Thomas I, Van Royen P, Coenen S. Clinical prediction rules combining signs, symptoms and epidemiological context to distinguish influenza from influenza-like illnesses in primary care: a cross sectional study. BMC Fam Pract. 2011;12:4.
Haran JP, Beaudoin FL, Suner S, Lu S. C-reactive protein as predictor of bacterial infection among patients with an influenza-like illness. Am J Emerg Med. 2013;31:137–44.
Estabragh ZR, Mamas MA. The cardiovascular manifestations of influenza: a systematic review. Int J Cardiol. 2013;167:2397–403.
Li X, Dunn J, Salins D, Zhou G, Zhou W, Rose SMS-F, et al. Digital health: tracking Physiomes and activity using wearable biosensors reveals useful health-related information. PLoS Biol. 2017;15:e2001402.
Ang LW, Cutter J, James L, Goh KT. Factors associated with influenza vaccine uptake in older adults living in the community in Singapore. Epidemiol Infect. 2017;145:775–86.
Melbye H, Hvidsten D, Holm A, Nordbø SA, Brox J. The course of C-reactive protein response in untreated upper respiratory tract infection. Br J Gen Pract. 2004;54:653–8.
Hu L, Shi Q, Shi M, Liu R, Wang C. Diagnostic Value of PCT and CRP for Detecting Serious Bacterial Infections in Patients With Fever of Unknown Origin: A Systematic Review and Meta-analysis. Appl Immunohistochem Mol Morphol. 2017:25 Available from: insights.ovid.com [cited 2020 10 Jan].
Ms. Ong Lay See and Grace Tin for collecting the data, staff of TTSH ED who helped with study recruitment, and patients who participated in the study.
This work was supported by the National Healthcare Group Singapore’s Clinician Scientist Career Scheme [NHG-CSCS/15005].
Ethics approval and consent to participate
Ethics approval was obtained from the National Healthcare Group Domain Specific Review Board (2015/00248). Written consent was obtained from each participant in current study.
All authors have no conflict of interest related to the submitted work.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Training diagnostic performance at different probability cutoffs. Appendix 1B. 2 x 2 tables for the actual vs predicted values on the training and validation set using the best probability cutoff
About this article
Cite this article
Wong, J.G., Aung, AH., Lian, W. et al. Risk prediction models to guide antibiotic prescribing: a study on adult patients with uncomplicated upper respiratory tract infections in an emergency department. Antimicrob Resist Infect Control 9, 171 (2020). https://doi.org/10.1186/s13756-020-00825-3
- Antibiotic prescribing
- Machine learning
- Prediction model