Skip to main content

Value of hospital antimicrobial stewardship programs [ASPs]: a systematic review

Abstract

Background

Hospital antimicrobial stewardship programs (ASPs) aim to promote judicious use of antimicrobials to combat antimicrobial resistance. For ASPs to be developed, adopted, and implemented, an economic value assessment is essential. Few studies demonstrate the cost-effectiveness of ASPs. This systematic review aimed to evaluate the economic and clinical impact of ASPs.

Methods

An update to the Dik et al. systematic review (2000–2014) was conducted on EMBASE and Medline using PRISMA guidelines. The updated search was limited to primary research studies in English (30 September 2014–31 December 2017) that evaluated patient and/or economic outcomes after implementation of hospital ASPs including length of stay (LOS), antimicrobial use, and total (including operational and implementation) costs.

Results

One hundred forty-six studies meeting inclusion criteria were included. The majority of these studies were conducted within the last 5 years in North America (49%), Europe (25%), and Asia (14%), with few studies conducted in Africa (3%), South America (3%), and Australia (3%). Most studies were conducted in hospitals with 500–1000 beds and evaluated LOS and change in antibiotic expenditure, the majority of which showed a decrease in LOS (85%) and antibiotic expenditure (92%). The mean cost-savings varied by hospital size and region after implementation of ASPs. Average cost savings in US studies were $732 per patient (range: $2.50 to $2640), with similar trends exhibited in European studies. The key driver of cost savings was from reduction in LOS. Savings were higher among hospitals with comprehensive ASPs which included therapy review and antibiotic restrictions.

Conclusions

Our data indicates that hospital ASPs have significant value with beneficial clinical and economic impacts. More robust published data is required in terms of implementation, LOS, and overall costs so that decision-makers can make a stronger case for investing in ASPs, considering competing priorities. Such data on ASPs in lower- and middle-income countries is limited and requires urgent attention.

Background

Antimicrobial resistance (AMR) is a global problem threatening not only public health but also economic development and security. Globally, AMR has the potential to cause 10 million deaths by 2050 based on high-level scenarios [1]. The World Bank estimates that there will be up to one trillion US dollars global increases in healthcare costs by 2050 due to AMR [2].

AMR can stem from inappropriate antibiotic use which can include overuse, misuse, underuse or abuse of antibiotics [3]. Rate of antimicrobial misuse in hospitals, including failure to de-escalate and overprescription of broad spectrum antibiotics, has remain unchanged at 50%. Antimicrobial stewardship programs (ASPs) are one way to address inappropriate antimicrobial use and AMR.

The goals of ASPs are to improve patient outcomes and safety and to reduce AMR and healthcare costs by promoting judicious use of antibiotics. Some core elements identified in successful ASPs include leadership commitment, prescriber accountability, drug expertise and education of clinicians and patients, among others [4, 5], ASPs may require additional resources, such as hospital personnel and equipment, in order to be effective and be sustainable. As such, the upfront costs associated with these additional resources can be a potential barrier to individuals who have not yet implemented an ASP. With the growing importance of measuring the impact of ASPs and health economic evaluations, there has been an increasing number of studies that have evaluated the clinical and economic impact of ASPs in the last few years.

A systematic review was conducted by Dik et al. to evaluate methods of published economic evaluations of hospital ASP studies from January 2000 to November 2014 [6]. The authors identified 99 studies, the majority of which were conducted in North America and Europe. Although distinct types of stewardship interventions were evaluated, “Therapy evaluation, review and/or feedback” was the most common. As the primary objective of this review was to evaluate only the quality of these studies, no synthesis of results was performed or aggregated results reported. Given the growing body of literature, an update of the Dik review including a complete summary of the outcomes from these studies was still needed. Therefore, our objective was to conduct a systematic review to update, evaluate, and broadly summarize the clinical and economic impact of ASPs using the Dik et al. framework.

Materials and methods

Literature search

This systematic review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) protocol to synthesize the results related to key outcomes from all the individual studies and define an overall value framework for ASPs. The previously identified systematic literature review by Dik et al. (January 2000–November 2014) was used to provide the framework for this review [6].

An updated search to the Dik et al. systematic review was performed within the EMBASE and MEDLINE databases to further include studies after 30 September 2014 up to 31 December 2017, using the following search strings: “antimicrobial stewardship,” “antimicrobial management,” “antimicrobial prescribing intervention,” and “antimicrobial program intervention.” All strings were in combination with the words “cost(s),” “financial,” “economic,” “dollar” or “euro,” or the respective symbols for the latter two. Two authors independently reviewed the retrieved abstracts for their eligibility followed by full-text screening of selected articles. Studies identified through a handsearching process were also included.

Study selection

Primary research studies in English that discussed a hospital intervention and key patient and/or economic outcomes, identified previously by Dik et al., were included in this analysis. Our review expanded the search to also include studies published through December 2017 (Fig. 1). Studies that did not contain an ASP intervention, measure any key outcomes, or that were conducted in an animal population were excluded. All inclusion and exclusion criteria were established prior to the review.

Fig. 1
figure 1

PRISMA chart of the search method

Data extraction

Information collected from each study included publication year, region and country, study objective, study design, setting, hospital size (number of beds), number of participants/patients included, inclusion and exclusion criteria for study data. Information collected about the intervention included type of intervention based on categories reported in Dik et al., description of the ASPs (stewards and their role, components, duration, primary outcomes), outcomes measured and their corresponding results, and the conclusion of the study findings.

The main outcomes of interest were antimicrobial, patient and economic outcomes. Antimicrobial outcomes included antibiotic usage and resistance. Length of stay (LOS), mortality rate, and readmission rates at varying timepoints were the main patient outcomes of interest. Key economic outcomes were costs associated with antimicrobials, LOS, and implementation and operation of ASPs. Outcomes of interest were separated into statistically significant results, non- significant results or results of unknown significance. The majority of the studies that measured significance reported p-values with only one study reporting odds ratios [7]. Confidence intervals were utilized in a few studies but also alongside p-values. In this review, we therefore only examined significance based on studies reporting p-values. Studies that measured outcomes of interest and found that the change in result values from pre- to post-intervention had a p-value of < 0.05 were classified as statistically significant. Studies with a change in result values that measured a p-value of ≥0.05 were classified as non-statistically significant. Studies that did not measure p-value were classified as no significance testing performed.

Results

The updated search from 2014 to 2017 identified a total of 487 potentially relevant citations, and of these, 411 abstracts were excluded based on previously defined exclusion criteria. A total of 76 papers were selected for full-review, of which 44 studies met the inclusion criteria. Furthermore, 8 articles were included from supplemental searching. In parallel to the updated search, the full-texts of all 99 studies included in Dik et al. review were evaluated, of which 94 studies met our inclusion/exclusion criteria. As the primary objective of the Dik et al. review was to evaluate only the quality of these studies, we worked to synthesize the quantitative results of all these studies bringing the total count to 146 primary research studies for final analysis from 2000 to 2017 (Fig. 1).

General characteristics

The majority of the studies analyzed were conducted in North America (49%) and Europe (25%) with approximately two-thirds of the included articles being published within the last 5 years. Most studies followed a quasi-experimental study design in medium-large hospitals with 500–1000 beds (Table 1). Most studies implemented an “audit of and/or feedback on the antimicrobial therapy provided” as an intervention strategy (57%), followed by “altered therapy guidelines” i.e., creation of hospital treatment guidelines specific to combating AMR or the alteration of previous antimicrobial therapy guidelines (25%) (Table 2). Even though we didn’t identify studies that exclusively assessed behavioral change therapy, studies with similar elements were included in “therapy feedback” and “giving education”.

Table 1 General characteristics of the reviewed studies
Table 2 Types of interventions and outcomes

Outcomes from the interventions generally fell into three categories: antibiotic use, patient outcomes, and economic outcomes. The majority of the studies primarily reported antibiotic outcomes, with 68% of studies reporting changes in antibiotic usage. The most commonly reported patient outcomes were LOS and mortality within the hospital. For economic outcomes, 62% reported changes in antibiotic expenditure and 37% studies reported overall cost-savings (Table 2).

Antimicrobial outcomes

Antimicrobial usage

Of the 100 studies that reported antibiotic usage, 80 studies reported 108 relevant outcomes related to antibiotic usage [8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87]. Statistical significance testing was performeded in 69% of these outcomes. Outcomes measured included changes in drug dosage, including defined daily dose (DDD), of certain antibiotics, changes in duration of antimicrobial therapy, including days of therapy (DOT), and proportion of patients on antimicrobial treatment. Most studies that measured antibiotic usage were conducted in medium-large sized hospitals with 500–1000 beds (26%).

Total usage of antibiotics decreased in majority of the studies, as measured by drug dose, duration of therapy, proportion of patients receiving antibiotic therapy, or other outcomes (Fig. 2). Usage of individual antibiotic classes (i.e., vancomycin, fluoroquinolones) demonstrated mixed trends. In most studies, use of some antimicrobials decreased, while use of other antimicrobials increased or did not change.

Fig. 2
figure 2

Effect of ASP on Total Antimicrobial Usage. SM = Significance measured; SNM = significance not measured. *Total usage of antibiotics decreased in a majority of studies, as measured by drug dose, duration of therapy, proportion of patients receiving antibiotic therapy, or other outcomes

Antimicrobial resistance

Eighteen of 22 studies measured relevant outcomes related to AMR [7, 10, 11, 17, 19, 21, 23, 33, 34, 43, 63,64,65, 68, 82, 88,89,90]. Of these studies, 11 (61%) found a statistically significant change in AMR following implementation of a hospital ASP. These studies measured resistance after a mean control period of 21.2 months (range: 6 months–36 months) and change in resistance after a mean intervention period of 24.5 months (range: 6 months–36 months). Half the studies demonstrated a decrease in resistance for at least one microbial strain against an antimicrobial; two (11%) studies demonstrated a decrease in resistance for at least one antimicrobial coupled with an increase in resistance towards a different antimicrobial. Such instances can occur in an intervention strategy where the favored use of a certain class of antibiotics may increase selection pressure towards resistance. There was no obvious correlation between primary intervention strategy utilized and resistance, although the use of therapy evaluation, review, and/or feedback was the strategy utilized in both studies that reported both significant increases and decreases in antimicrobial resistance between different bacterial strains.

Patient outcomes

LOS

Of the 79 studies reporting LOS, 68 measured 93 pertinent LOS outcomes [7, 8, 11, 17, 21, 24,25,26, 29, 31,32,33, 35,36,37,38, 40, 43,44,45,46,47,48, 50, 52, 56, 63, 66, 70, 72,73,74,75,76,77,78,79, 81,82,83,84, 89, 91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116]. Most of these studies occurred in medium-large sized hospitals with 500–1000 beds (32%).

The majority of the 68 studies (85%) reported either a reduction or no change in LOS that ranged from a decrease of 0 to 22 days (Table 3) after implementation of ASP. Only 10 studies reported an increase in LOS post-ASP implementation with a maximum increase of 5 days. More than half of the 93 measured outcomes (53%) did not show a statistically significant change; however, of the 33 outcomes that did reach statistical significance, approximately 88% showed a decrease in overall LOS. Studies of statistical significance showed an average decrease in LOS of 3.24 days or 20.6% per patient following ASP intervention.

Table 3 Literature synthesis of key outcomes: results and ranges

Mortality rate

Most studies that evaluated mortality rate occurred in medium-large sized hospitals with 500–1000 beds. Of 58 studies reporting mortality rate, 57 identified 73 relevant outcomes related to mortality rate. Among those studies, 54 studies reported 58 all-cause mortality rate outcomes [7, 11, 15, 19,20,21,22, 27, 28, 31,32,33, 35, 38, 40, 44,45,46,47,48, 50,51,52, 59, 62, 70, 72, 73, 75,76,77,78,79,80,81,82, 85, 91, 92, 95, 96, 98, 99, 102, 104, 105, 109, 110, 112, 113, 116,117,118,119]. Additionally, only 11 of the 57 relevant studies reported 15 infection-related mortality outcomes [7, 20, 48, 57, 74, 78, 79, 81, 83, 99, 109]. Statistical significance testing was performeded in 91% of these outcomes.

Of the 54 studies that reported changes in all-cause mortality, 77.2% reported reductions or no changes in mortality ranging from 0 to 18.1% decrease (Table 3). Similarly, the majority of the studies reporting infection-related mortality also showed reductions or no changes in mortality ranging from 0 to 12% decrease. Of the 58 all-cause mortality outcomes measured, 74% did not reach statistical significance. Of the 10 measurements that showed significance, 90% showed a decrease in mortality rate. Among studies that reported significant changes, there was an average decrease of 10.5% in all-cause mortality rates and 11.3% decrease in infection-related mortality rates following an ASP intervention.

Readmission rate

Twenty-three studies measured outcomes relevant to hospital readmission rate. 21 studies measured 24 relevant outcomes related to all-cause hospital readmission rate [15, 22, 29, 35, 38, 44, 48, 50, 51, 70, 72, 77,78,79, 83, 95, 105, 108, 109, 111, 118] and 10 studies measured 12 outcomes related to infection relapse readmission [38, 48, 77,78,79, 82, 105, 109, 111, 113]. Most of these studies occurred in medium-large sized hospitals with 500–1000 beds. Additionally, while there was variety in timepoint chosen for measuring readmission rate amongst the studies (range: 48 h to 90 days), most studies measured readmission rate at 28 or 30 days [15, 22, 35, 44, 50, 51, 77,78,79, 82, 95, 109, 111, 113, 118].

Of the 21 studies that evaluated all-cause readmission rates, 13 reported reduction or no changes ranging from 0 to 12%. Eight studies reported an increase in all-cause readmission with a maximum increase of 8.6% (Table 3). Of the 24 all-cause readmission outcomes, 86% did not reach statistical significance. Outcomes of statistical significance (n = 2) showed an average increase of 4.2% in the all-cause readmission rate following ASP intervention. Among the 12 infection-related readmission outcomes, 33% reached statistical significance and all showed a decrease in readmission rates (Table 3). Outcomes of statistical significance (n = 4) showed an average decrease of 1.2% in the infection-related readmission rate following ASP intervention.

Economic outcomes

Implementation costs

Only 9 studies measured the cost of implementing an ASP [9, 58, 68, 106, 114, 118, 120,121,122]. Overall implementation costs did not seem to be associated with the type of ASP intervention (Table 3).

Operational costs

Operational costs were defined as the total direct hospital costs associated with patient treatment for bacterial infection, typically including costs associated with LOS, diagnostics, and treatment. A few studies also measured costs associated with human resources. All included studies measured costs from the hospital perspective and reported either total hospital costs or per patient costs both pre- and post-intervention. Of 22 studies, 17 measured 20 relevant operational cost outcomes, however only 13 studies including 16 of those outcomes measured the change in operational costs from pre- to post-ASP intervention [7, 25, 56, 70, 86, 92, 93, 100,101,102,103, 112, 123]. Most of these studies were from large hospitals with > 1000 beds (40%). Rapid diagnostic tools were the most utilized intervention strategy.

Operational costs varied in if they increased or decreased following ASP implementation, with 69% of outcomes demonstrating a decrease in annual costs and 31% showing an increase. Studies varied in reported currency so studies were only averaged together based on currency type due to wide discrepancies between currency conversion rates and years that studies were conducted. Of the 12 USD ($) studies, average ASP operational costs per patient were $5580 per year and increased by 5.97% between pre- and post- intervention, according to the 11 studies that measured those changes. However, there was a wide range in this change, with studies showing decreases in annual operational costs down to 72% or increases up to 236% (Table 3). Of the 4 EUR (€) studies, average ASP operational costs per patient were €1974.47 per year. Similar trends of increasing operational costs were exhibited amongst the 2 of 4 EUR (€) studies that measured change in cost with no studies exhibiting decreases in operational costs following ASP implementation (range: 7.93 to 243%, Table 3).

Changes in operational costs seemed to have a correlation with the intervention strategy utilized. Use of altered therapy guidelines and antibiotic restriction lists of pre-authorized agents typically reduced annual operational costs, on average of 17.1 and 17.5%, respectfully. Therapy evaluation, review, and/or feedback increased operational costs by an average of 27.5%. Some interventions that utilized rapid diagnostic testing or new biomarkers as their primary intervention strategy more than tripled their operational costs at increases of 236 and 243%, respectfully.

Antimicrobial expenditures

Of 91 studies, 87 measured 94 pertinent antimicrobial expenditure outcomes but only 46% of outcomes were assessed for statistical significance [8, 9, 12,13,14,15,16,17,18,19,20,21,22, 26, 28, 30,31,32,33,34, 37, 39,40,41,42,43,44,45,46,47,48, 52, 53, 55,56,57,58, 60, 62,63,64,65,66,67,68, 70,71,72,73, 75, 76, 82,83,84, 87,88,89, 91, 93, 95, 97, 99, 100, 104, 106, 107, 117, 118, 120, 122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138]. Most studies were conducted in hospitals with 500–1000 patients (26%). Approximately 92% of the 87 studies showed a decrease in antimicrobial costs ranging from 0.06 to 80.1%. Of the 94 outcomes, only 15% did not reach statistical significance compared to 31% that demonstrated significant change. Of the outcomes that demonstrated significant changes, 97% showed a decrease in antimicrobial costs, averaging 35.6% decrease in costs following ASP implementation. Only one study showed a significant increase of antimicrobial costs (51.5%) (Table 3).

LOS costs

All 3 studies that reported changes in LOS costs demonstrated decreases in costs following ASP implementation [43, 66, 118]. This ranged from decreases of $18,305 for a small hospital to $1.95 M (Table 3) and 970,397 kr (Swedish Krona/SEK) for two large sized hospitals. All studies utilized therapy evaluation, review, and/or feedback as their primary intervention strategy.

Overall cost savings

Of 54 studies, 49 studies measured 58 relevant overall cost savings outcomes [9,10,11, 23, 26, 27, 35, 36, 38, 42, 46, 49, 50, 52, 54, 57, 59, 60, 64, 76, 81, 82, 90, 91, 95, 96, 98, 99, 105, 106, 108, 112, 114, 117, 119, 120, 127,128,129, 134, 137, 139,140,141,142,143,144,145,146]. Most of these studies were conducted in medium sized hospitals with 150–500 beds (26%). Studies varied widely in their reported currency as well as annual cost savings, although most of the studies measured costs in US Dollars (65%). Average cost savings in US Dollar (USD) studies were $435,000 (range: $9110–$2.06 M) per year for the hospital (Table 3), or $732 (range: $2.50 -$2640) per patient. Average cost savings in Euro (EUR) studies were €41,500 (range: €19,000–€66,200) per year for the hospital, or €198.00 (range: €40.40–€529.00) per patient. Average cost savings in Great Britain Pound (GBP) studies were £144,000 (range: £7120–£286,000) per year for the hospital, or £304.00 (range: £2.47–£1000) per patient.

Discussion

Through the implementation of a range of interventions, hospital ASPs aim to provide high value for patients, with value defined as the health outcomes achieved per dollar spent [147]. This goal unites the interests of all the stakeholders in the system, including patients. If value improves, patients, payers, providers, and suppliers can all benefit while the economic sustainability of the health care system increases. This approach to considering sustainable funding and prioritizing of ASP activity has been used successfully in a range of other disease areas such as cardiovascular disease, diabetes, musculoskeletal diseases, etc. [148].

Value in health care is measured by the outcomes achieved and not by the volume of services delivered thereby shifting focus from volume to value. Value is also not measured by the process of care used. Even though process measurement and improvement are important tactics they are not substitutes for measuring outcomes and costs. Cost reduction without regard to the outcomes achieved is dangerous and self-defeating, leading to false “savings” and potentially limiting effective care.

Hospital ASPs aim to promote the efficient and judicious use of antimicrobials to combat the rise in AMR. While most ASPs focus on changes to antimicrobial use practices, the effects of ASPs often yield downstream effects that extend from antimicrobial use into improved or maintained patient outcomes (numerator of the value equation) which drive down resource utilization and associated costs (denominator of the value equation). To reduce cost, the best approach is often to spend more on some services to reduce the need for others. Despite some significant limitations [described later] of the available data, the majority of the studies included here appear to produce similar results in support of our value framework (Fig. 3), particularly in relation to ASPs in North America and Europe. However, there were a number of studies which reported contradicting results, such as an increase in the use of certain antibiotics, worsening of some patient outcomes, and increased hospital costs. It is important to try to understand potential reasons for this.

Fig. 3
figure 3

Conceptual value framework for implementation

In relation to antibiotic use, a few studies demonstrated a significant increase in use of at least one antibiotic class [10, 31, 32, 72, 76]. This could be due to many reasons including an inappropriate metric for measuring antimicrobial use [76]. For example, an increase in use of narrow spectrum antibiotic use offsets a decrease in broad spectrum use yielding lower AMR [10, 31, 32, 72]. Additionally, if the consumption of only certain antibiotics is restricted, a global decrease in resistance cannot be expected [63]. It is possible that resistant strains are unrelated to changes made in hospitals, as there may be an inability to differentiate between community- or hospital-acquired isolates [17, 21].

Some studies found that patient LOS increased following ASP implementation [21, 36, 44,45,46, 70, 75, 82, 97, 102]. LOS can be influenced by factors beyond antibiotic use, including comorbidities and disease severity, yielding non-statistically significant increases in patient hospital days [21, 36, 44, 75, 82, 97, 102]. Targeted interventions (e.g., focus on reducing particular antibiotics) may not lead to overall changes in LOS [36, 97, 102]. Short study durations or changes in pre- and post-intervention populations may also compromise the direct results of the ASP on global LOS [45, 46, 75, 82, 97, 102]. Similarly, a few studies found that patient mortality increased following ASP implementation but majority of these studies did not show statistical significance [33, 45, 46, 70, 78, 79, 99, 102, 113]. Reasons could include short study durations [45, 82], presence of an existing ASP prior to the new intervention strategy that may have limited the ASP impact [82], delayed therapy due to unavailable diagnostic results [102, 113], or poor communication between ASP staff members regarding treatments [82, 113]. Differences in the pre- and post-ASP populations and poor communication between ASP staff members may be factors that led to an increase in readmission rates in some studies [72, 77].

The small number of economic outcome studies that do not show a significant reduction in antimicrobial costs following an ASP instead showed no significant changes or did not measure changes for significance [17, 28, 83, 97, 100]. Short study durations [97], increase in the use of higher cost antibiotics that are more effective in decreasing AMR [17, 41], and lack of adherence to ASP interventions by all hospital staff members [41] could lead to increase in antimicrobial costs.

Overall an increase in ASP costs may be offset by total cost savings for the hospital [70, 112]. In addition, at a per-patient level, the average per-patient cost savings represents a significant portion of a hospital bed-day cost. For example, in the EU and UK, the proportion of a bed day saved through ASP represents 60–80% of the cost of a bed day, while in the US the proportion of a bed day saved is lower (~ 32%). Reduced patient LOS is often the key driver of per patient cost savings for a hospital and may explain the significant difference in cost savings that were found between US, UK, and EU hospitals. As per Table 4, average higher per patient cost savings were realized in studies conducted in the US, largely due to the high cost of a hospital bed day in the country.

Table 4 Cost savings compared with bed day costs around the world

There were some limitations associated with the studies included in this review. First, there was heterogeneity among the studies, in terms of size of hospitals and patient populations included in the review, as previously described in Table 1. Some studies implemented ASPs across the entire hospital with patient pools of nearly 13,000 [43], whereas other hospitals only implemented their programs in specific wards or units with < 500 patients [75, 106]. Furthermore, numerous studies did not indicate the size of their hospital bed size, patient population, or both, making it challenging to compare their results to either smaller medical centers with < 100 beds [15] or large hospitals with > 600 beds [43]. The geographic distribution and provision of healthcare delivery is biased towards the North American and European hospital setting thereby rendering it difficult to apply these findings globally. This challenge is further driven by the wide range in years that the studies were conducted as well as their reported currency, preventing a comparison of the economic results on the same currency and same inflation rate. Homogenizing the currency and year of the studies would have allowed us to conduct more analysis into the effectiveness of different ASP strategies and program designs to draw conclusions on why certain programs drove improved outcomes and cost savings. Additionally, the studies varied in the type of ASPs, outcomes and follow-up periods. Despite this limitation, our review was not restrictive in the studies included. Second, the majority of the studies did not report start-up and implementation costs. The main economic outcome of interest was costs associated with antimicrobial use. More well-designed studies are needed that will evaluate the costs of implementing ASPs to truly capture the economic burden.

This is the first systematic review that provided a comprehensive summary of results of the economic impact of ASPs. Given our objective to conduct a broad review of the financial literature evaluating ASPs, the search was not restricted to only include certain study designs, interventions or study durations. A future step would be to conduct a meta-analysis of the results with restrictive inclusion/exclusion criteria to further delve into the relationships between specific ASP interventions and economic outcomes.

Conclusions

Overall this systematic review demonstrates that ASPs can offset or reduce costs while improving some patient outcomes, thereby suggesting high value for certain healthcare systems. The findings also suggest that costs associated with start-up and implementation of ASPs are potentially offset by subsequent cost-savings. Additionally, numerous systematic reviews and meta-analyses have demonstrated that such programs have beneficial effects on hospital LOS [149, 150], resistance patterns [63, 150], and infection incidence [151]. This data supports the value of ASPs in tandem with infection control measures [1]. However, for the findings to be globally relevant, more studies, particularly in real world settings across a diverse range of geographies and resource settings are required, so that a full critical appraisal of the true value of these programs can be made. This will not only allow our ability to develop high value bespoke models of ASP based on robust clinical and economic data but also consider creating benchmarks, an area fraught with challenges [152, 153].

Abbreviations

AMR:

Antimicrobial resistance

ASP:

Antimicrobial stewardship program

DDD:

Daily defined dose

DOT:

Days of therapy

EUR:

Euro

GBP:

Great Britain Pound

kr:

Swedish Krona

LOS:

Length of stay

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

USD:

United States Dollar

References

  1. O’Neill J. Antimicrobial resistance: tackling a crisis for the health and wealth of nations. Rev Antimicrob Resist. 2014;20:1–16.

    Google Scholar 

  2. KFF. Hospital Adjusted Expenses per Inpatient Day, 2015: Henry J Kaiser Family Foundation; 2016. https://www.kff.org/health-costs/state-indicator/expenses-per-inpatient-day/.

  3. Hand K. Antibiotic stewardship. Clinical Medicine. 2013;13(5):499–503.

    Article  PubMed  PubMed Central  Google Scholar 

  4. NICE. Antimicrobial stewardship: systems and processes for effective antimicrobial medicine use. United Kingdom: National Institute for Health and Care Excellence; 2015.

    Google Scholar 

  5. CDC. Get smart for healthcare. Checklist for Core Elements of Hospital Antibiotic Stewardship Programs. Atlanta: Centers for Disease Control and Prevention; 2013.

    Google Scholar 

  6. Dik J-WH, Vemer P, Friedrich AW, Hendrix R, Lo-Ten-Foe JR, Sinha B, et al. Financial evaluations of antibiotic stewardship programs—a systematic review. Front Microbiol. 2015;6:317.

    PubMed  PubMed Central  Google Scholar 

  7. Perez KK, Olsen RJ, Musick WL, Cernoch PL, Davis JR, Peterson LE, et al. Integrating rapid diagnostics and antimicrobial stewardship improves outcomes in patients with antibiotic-resistant gram-negative bacteremia. J Infect. 2014;69(3):216–25.

    Article  PubMed  Google Scholar 

  8. Agwu AL, Lee CK, Jain SK, Murray KL, Topolski J, Miller RE, et al. A world wide web–based antimicrobial stewardship program improves efficiency, communication, and user satisfaction and reduces cost in a tertiary care pediatric medical center. Clin Infect Dis. 2008;47(6):747–53.

    Article  PubMed  Google Scholar 

  9. Ansari F, Gray K, Nathwani D, Phillips G, Ogston S, Ramsay C, et al. Outcomes of an intervention to improve hospital antibiotic prescribing: interrupted time series with segmented regression analysis. J Antimicrob Chemother. 2003;52(5):842–8.

    Article  CAS  PubMed  Google Scholar 

  10. Apisarnthanarak A, Danchaivijitr S, Khawcharoenporn T, Limsrivilai J, Warachan B, Bailey TC, et al. Effectiveness of education and an antibiotic-control program in a tertiary care hospital in Thailand. Clin Infect Dis. 2006;42(6):768–75.

    Article  PubMed  Google Scholar 

  11. Bantar C, Sartori B, Vesco E, Heft C, Saúl M, Salamone F, et al. A hospitalwide intervention program to optimize the quality of antibiotic use: impact on prescribing practice, antibiotic consumption, cost savings, and bacterial resistance. Clin Infect Dis. 2003;37(2):180–6.

    Article  PubMed  Google Scholar 

  12. Bartlett JM, Siola PL. Implementation and first-year results of an antimicrobial stewardship program at a community hospital. Am J Health-Syst Pharm. 2014;71(11):943–9.

    Article  PubMed  Google Scholar 

  13. Bevilacqua S, Demoré B, Boschetti E, Doco-Lecompte T, May I, May T, et al. 15 years of antibiotic stewardship policy in the Nancy teaching hospital. Med Mal Infect. 2011;41(10):532–9.

    Article  CAS  PubMed  Google Scholar 

  14. Borde JP, Kaier K, Steib-Bauert M, Vach W, Geibel-Zehender A, Busch H, et al. Feasibility and impact of an intensified antibiotic stewardship programme targeting cephalosporin and fluoroquinolone use in a tertiary care university medical center. BMC Infect Dis. 2014;14(1):201.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Boyles TH, Whitelaw A, Bamford C, Moodley M, Bonorchis K, Morris V, et al. Antibiotic stewardship ward rounds and a dedicated prescription chart reduce antibiotic consumption and pharmacy costs without affecting inpatient mortality or re-admission rates. PLoS One. 2013;8(12):e79747.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Bozkurt F, Kaya S, Gulsun S, Tekin R, Deveci Ö, Dayan S, et al. Assessment of perioperative antimicrobial prophylaxis using ATC/DDD methodology. Int J Infect Dis. 2013;17(12):e1212–e7.

    Article  CAS  PubMed  Google Scholar 

  17. Bruno-Murtha LA, Brusch J, Bor D, Li W, Zucker D. A pilot study of antibiotic cycling in the community hospital setting. Infect Control Hosp Epidemiol. 2005;26(1):81–7.

    Article  PubMed  Google Scholar 

  18. Carling P, Fung T, Killion A, Terrin N, Barza M. Favorable impact of a multidisciplinary antibiotic management program conducted during 7 years. Infect Control Hosp Epidemiol. 2003;24(9):699–706.

    Article  PubMed  Google Scholar 

  19. Cheng V, To K, Li I, Tang B, Chan J, Kwan S, et al. Antimicrobial stewardship program directed at broad-spectrum intravenous antibiotics prescription in a tertiary hospital. Eur J Clin Microbiol Infect Dis. 2009;28(12):1447.

    Article  CAS  PubMed  Google Scholar 

  20. Christ-Crain M, Stolz D, Bingisser R, Muller C, Miedinger D, Huber PR, et al. Procalcitonin guidance of antibiotic therapy in community-acquired pneumonia: a randomized trial. Am J Respir Crit Care Med. 2006;174(1):84–93.

    Article  CAS  PubMed  Google Scholar 

  21. Cook PP, Catrou PG, Christie JD, Young PD, Polk RE. Reduction in broad-spectrum antimicrobial use associated with no improvement in hospital antibiogram. J Antimicrob Chemother. 2004;53(5):853–9.

    Article  CAS  PubMed  Google Scholar 

  22. Danaher PJ, Milazzo NA, Kerr KJ, Lagasse CA, Lane JW. The antibiotic support team--a successful educational approach to antibiotic stewardship. Mil Med. 2009;174(2):201.

    Article  PubMed  Google Scholar 

  23. Del Arco A, Tortajada B, de la Torre J, Olalla J, Prada J, Fernández F, et al. The impact of an antimicrobial stewardship programme on the use of antimicrobials and the evolution of drug resistance. Eur J Clin Microbiol Infect Dis. 2015;34(2):247–51.

    Article  CAS  PubMed  Google Scholar 

  24. Dunn K, O’Reilly A, Silke B, Rogers T, Bergin C. Implementing a pharmacist-led sequential antimicrobial therapy strategy: a controlled before-and-after study. Int J Clin Pharm. 2011;33(2):208–14.

    Article  PubMed  Google Scholar 

  25. Forrest GN, Mehta S, Weekes E, Lincalis DP, Johnson JK, Venezia RA. Impact of rapid in situ hybridization testing on coagulase-negative staphylococci positive blood cultures. J Antimicrob Chemother. 2006;58(1):154–8.

    Article  CAS  PubMed  Google Scholar 

  26. Hagert BL, Williams C, Wieser CM, Rohrich MA, Lo TS, Newman WP, et al. Implementation and outcome assessment of an inpatient antimicrobial stewardship program. Hosp Pharm. 2012;47(12):939–45.

    Article  Google Scholar 

  27. Katsios CM, Burry L, Nelson S, Jivraj T, Lapinsky SE, Wax RS, et al. An antimicrobial stewardship program improves antimicrobial treatment by culture site and the quality of antimicrobial prescribing in critically ill patients. Crit Care. 2012;16(6):R216.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Krivoy N, Abed El-Ahal W, Bar-Lavie Y, Haddad S. Antibiotic prescription and cost patterns in a general intensive care unit. Pharm Pract (Granada). 2007;5(2):67–73.

    Article  Google Scholar 

  29. Kullar R, Davis SL, Kaye KS, Levine DP, Pogue JM, Rybak MJ. Implementation of an antimicrobial stewardship pathway with Daptomycin for optimal treatment of methicillin-resistant Staphylococcus aureus bacteremia. Pharmacotherapy: The Journal of Human Pharmacology and Drug Therapy. 2013;33(1):3–10.

    Article  CAS  Google Scholar 

  30. Leung V, Gill S, Sauve J, Walker K, Stumpo C, Powis J. Growing a “positive culture” of antimicrobial stewardship in a community hospital. Can J Hosp Pharm. 2011;64(5):314.

    PubMed  PubMed Central  Google Scholar 

  31. Lin Y-S, Lin I-F, Yen Y-F, Lin P-C, Shiu Y-c HH-Y, et al. Impact of an antimicrobial stewardship program with multidisciplinary cooperation in a community public teaching hospital in Taiwan. Am J Infect Control. 2013;41(11):1069–72.

    Article  PubMed  Google Scholar 

  32. Lutters M, Harbarth S, Janssens JP, Freudiger H, Herrmann F, Michel JP, et al. Effect of a comprehensive, multidisciplinary, educational program on the use of antibiotics in a geriatric university hospital. J Am Geriatr Soc. 2004;52(1):112–6.

    Article  PubMed  Google Scholar 

  33. Mach R, Vlcek J, Prusova M, Batka P, Rysavy V, Kubena A. Impact of a multidisciplinary approach on antibiotic consumption, cost and microbial resistance in a Czech hospital. Pharm World Sci. 2007;29(5):565–72.

    Article  PubMed  Google Scholar 

  34. Magedanz L, Silliprandi EM, Dos Santos RP. Impact of the pharmacist on a multidisciplinary team in an antimicrobial stewardship program: a quasi-experimental study. Int J Clin Pharm. 2012;34(2):290–4.

    Article  PubMed  Google Scholar 

  35. Malani AN, Richards PG, Kapila S, Otto MH, Czerwinski J, Singal B. Clinical and economic outcomes from a community hospital's antimicrobial stewardship program. Am J Infect Control. 2013;41(2):145–8.

    Article  PubMed  Google Scholar 

  36. Martínez MJ, Freire A, Rodriguez I, Bardán B, Castro I, Inaraja MT, et al. Clinical and economic impact of a pharmacist-intervention to promote sequential intravenous to oral clindamycin conversion. Pharm World Sci. 2000;22(2):53–8.

    Article  PubMed  Google Scholar 

  37. McLaughlin C, Bodasing N, Boyter A, Fenelon C, Fox J, Seaton R. Pharmacy-implemented guidelines on switching from intravenous to oral antibiotics: an intervention study. QJM. 2005;98(10):745–52.

    Article  CAS  PubMed  Google Scholar 

  38. Mertz D, Koller M, Haller P, Lampert ML, Plagge H, Hug B, et al. Outcomes of early switching from intravenous to oral antibiotics on medical wards. J Antimicrob Chemother. 2009;64(1):188–99.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Michaels K, Mahdavi M, Krug A, Kuper K. Implementation of an antimicrobial stewardship program in a community hospital: results of a three-year analysis. Hosp Pharm. 2012;47(8):608–16.

    Article  Google Scholar 

  40. Miyawaki K, Miwa Y, Tomono K, Kurokawa N. The impact of antimicrobial stewardship by infection control team in a Japanese teaching hospital. Yakugaku Zasshi. 2010;130(8):1105–11.

    Article  CAS  PubMed  Google Scholar 

  41. Mondain V, Lieutier F, Hasseine L, Gari-Toussaint M, Poiree M, Lions C, et al. A 6-year antifungal stewardship programme in a teaching hospital. Infection. 2013;41(3):621–8.

    Article  CAS  PubMed  Google Scholar 

  42. Ng C, Wu T, Chan W, Leung Y, Li C, Tsang D, et al. Clinical and economic impact of an antibiotics stewardship programme in a regional hospital in Hong Kong. Qual Saf Health Care. 2008;17(5):387–92.

    Article  CAS  PubMed  Google Scholar 

  43. Niwa T, Shinoda Y, Suzuki A, Ohmori T, Yasuda M, Ohta H, et al. Outcome measurement of extensive implementation of antimicrobial stewardship in patients receiving intravenous antibiotics in a Japanese university hospital. Int J Clin Pract. 2012;66(10):999–1008.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Nowak MA, Nelson RE, Breidenbach JL, Thompson PA, Carson PJ. Clinical and economic outcomes of a prospective antimicrobial stewardship program. Am J Health-Syst Pharm. 2012;69(17):1500–8.

    Article  PubMed  Google Scholar 

  45. Palmay L, Elligsen M, Walker SA, Pinto R, Walker S, Einarson T, et al. Hospital-wide rollout of antimicrobial stewardship: a stepped-wedge randomized trial. Clin Infect Dis. 2014;59(6):867–74.

    Article  PubMed  Google Scholar 

  46. Pate PG, Storey DF, Baum DL. Implementation of an antimicrobial stewardship program at a 60-bed long-term acute care hospital. Infect Control Hosp Epidemiol. 2012;33(4):405–8.

    Article  PubMed  Google Scholar 

  47. Rimawi RH, Mazer MA, Siraj DS, Gooch M, Cook PP. Impact of regular collaboration between infectious diseases and critical care practitioners on antimicrobial utilization and patient outcome. Crit Care Med. 2013;41(9):2099–107.

    Article  PubMed  Google Scholar 

  48. Rüttimann S, Keck B, Hartmeier C, Maetzel A, Bucher HC. Long-term antibiotic cost savings from a comprehensive intervention program in a medical department of a university-affiliated teaching hospital. Clin Infect Dis. 2004;38(3):348–56.

    Article  PubMed  Google Scholar 

  49. Schwartzberg E, Rubinovich S, Hassin D, Haspel J, Ben-Moshe A, Oren M, et al. Developing and implementing a model for changing physicians’ prescribing habits–the role of clinical pharmacy in leading the change. J Clin Pharm Ther. 2006;31(2):179–85.

    Article  CAS  PubMed  Google Scholar 

  50. Seah XFV, Ong YLR, Tan SW, Krishnaswamy G, Chong CY, Tan NWH, et al. Impact of an antimicrobial stewardship program on the use of carbapenems in a tertiary women's and children's hospital, Singapore. Pharmacotherapy: The Journal of Human Pharmacology and Drug Therapy. 2014;34(11):1141–50.

    Article  CAS  Google Scholar 

  51. Solomon DH, Van Houten L, Glynn RJ, Baden L, Curtis K, Schrager H, et al. Academic detailing to improve use of broad-spectrum antibiotics at an academic medical center. Arch Intern Med. 2001;161(15):1897–902.

    Article  CAS  PubMed  Google Scholar 

  52. Standiford HC, Chan S, Tripoli M, Weekes E, Forrest GN. Antimicrobial stewardship at a large tertiary care academic medical center: cost analysis before, during, and after a 7-year program. Infect Control Hosp Epidemiol. 2012;33(4):338–45.

    Article  PubMed  Google Scholar 

  53. Storey DF, Pate PG, Nguyen AT, Chang F. Implementation of an antimicrobial stewardship program on the medical-surgical service of a 100-bed community hospital. Antimicrob Resist Infect Control. 2012;1(1):32.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Teo J, Kwa A, Loh J, Chlebicki M, Lee W. The effect of a whole-system approach in an antimicrobial stewardship programme at the Singapore General Hospital. Eur J Clin Microbiol Infect Dis. 2012;31(6):947–55.

    Article  CAS  PubMed  Google Scholar 

  55. Tsang J, Tsang O, Yao R, Lai S. The Antimicrobal Stewardship Programme: where have we been... where are we going? World Hosp Health Serv: the official journal of the International Hospital Federation. 2013;49(2):18–21.

    Google Scholar 

  56. Yen Y-H, Chen H-Y, Wuan-Jin L, Lin Y-M, Shen WC, Cheng K-J. Clinical and economic impact of a pharmacist-managed iv-to-po conversion service for levofloxacin in Taiwan. Int J Clin Pharmacol Ther. 2012;50(2):136–41.

    Article  CAS  PubMed  Google Scholar 

  57. Yu K, Rho J, Morcos M, Nomura J, Kaplan D, Sakamoto K, et al. Evaluation of dedicated infectious diseases pharmacists on antimicrobial stewardship teams. Am J Health-Syst Pharm. 2014;71(12):1019–28.

    Article  PubMed  Google Scholar 

  58. Zhang H-X, Li X, Huo H-Q, Liang P, Zhang J-P, Ge W-H. Pharmacist interventions for prophylactic antibiotic use in urological inpatients undergoing clean or clean-contaminated operations in a Chinese hospital. PLoS One. 2014;9(2):e88971.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  59. Alawi MM, Darwesh BM. A stepwise introduction of a successful antimicrobial stewardship program: experience from a tertiary care university hospital in Western, Saudi Arabia. Saudi Med J. 2016;37(12):1341–9.

    Article  Google Scholar 

  60. Borde JP, Nussbaum S, Hauser S, Hehn P, Hübner J, Sitaru G, et al. Implementing an intensified antibiotic stewardship programme targeting daptomycin use in orthopaedic surgery: a cost-benefit analysis from the hospital perspective. Infection. 2016;44(3):301–7.

    Article  CAS  PubMed  Google Scholar 

  61. Brink AJ, Messina AP, Feldman C, Richards GA, Becker PJ, Goff DA, et al. Antimicrobial stewardship across 47 South African hospitals: an implementation study. Lancet Infect Dis. 2016;16(9):1017–25.

    Article  PubMed  Google Scholar 

  62. Bunsow E, González-Del Vecchio M, Sanchez C, Muñoz P, Burillo A, Bouza E. Improved Sepsis Alert With a Telephone Call From the Clinical Microbiology Laboratory: A Clinical Trial. Medicine. 2015;94(39):e1454.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Cai T, Verze P, Brugnolli A, Tiscione D, Luciani LG, Eccher C, et al. Adherence to European Association of Urology guidelines on prophylactic antibiotics: an important step in antimicrobial stewardship. Eur Urol. 2016;69(2):276–83.

    Article  PubMed  Google Scholar 

  64. Day SR, Smith D, Harris K, Cox HL, Mathers AJ. An Infectious diseases physician-led antimicrobial stewardship program at a small community hospital associated with improved susceptibility patterns and cost-savings after the first year. Open Forum Infectious Diseases. 2015;2(2):ofv064.

    Article  PubMed  PubMed Central  Google Scholar 

  65. Shridhar DP, Anitha K, Rai M, Fernandes A. Reserve drug indent form and its impact on antimicrobial consumption and sensitivity pattern in the medical intensive care unit of a tertiary care hospital. J Clin Diagn Res. 2015;9(2):FC05.

    PubMed  PubMed Central  Google Scholar 

  66. Estep PM, Ferreira JA, Dupree LH, Aldridge PJ, Jankowski CA. Impact of an antimicrobial stewardship initiative to evaluate β-lactam allergy in patients ordered aztreonam. Am J Health-Syst Pharm : AJHP : official journal of the American Society of Health-System Pharmacists. 2016;73(5 Suppl 1):S8–13.

    Article  Google Scholar 

  67. Ferrat E, Le Breton J, Guéry E, Adeline F, Audureau E, Montagne O, et al. Effects 4.5 years after an interactive GP educational seminar on antibiotic therapy for respiratory tract infections: a randomized controlled trial. Fam Pract. 2016;33(2):192–9.

    Article  CAS  PubMed  Google Scholar 

  68. Fürst J, Čižman M, Mrak J, Kos D, Campbell S, Coenen S, et al. The influence of a sustained multifaceted approach to improve antibiotic prescribing in Slovenia during the past decade: findings and implications. Expert Rev Anti-Infect Ther. 2015;13(2):279–89.

    Article  CAS  PubMed  Google Scholar 

  69. Hamilton KW, Gerber JS, Moehring R, Anderson DJ, Calderwood MS, Han JH, et al. Point-of-prescription interventions to improve antimicrobial stewardship. Clin Infect Dis. 2015;60(8):1252–8.

    Article  PubMed  PubMed Central  Google Scholar 

  70. Hohn A, Heising B, Hertel S, Baumgarten G, Hochreiter M, Schroeder S. Antibiotic consumption after implementation of a procalcitonin-guided antimicrobial stewardship programme in surgical patients admitted to an intensive care unit: a retrospective before-and-after analysis. Infection. 2015;43(4):405–12.

    Article  CAS  PubMed  Google Scholar 

  71. Hunter R. Cost-effectiveness of point-of-care C-reactive protein tests for respiratory tract infection in primary care in England. Adv Ther. 2015;32(1):69–85.

    Article  PubMed  PubMed Central  Google Scholar 

  72. Hurst AL, Child J, Pearce K, Palmer C, Todd JK, Parker SK. Handshake stewardship: a highly effective rounding-based antimicrobial optimization service. Pediatr Infect Dis J. 2016;35(10):1104–10.

    Article  PubMed  Google Scholar 

  73. Hussein AK, Ibrahim MM, Tammam TF, Sarhan HA, Gad GF. Impact on use of antimicrobial prescribing order form in ICU daily practice in adjustment of antimicrobial prescribing and in enhancement of patients clinical outcomes. Int J Pharm Bio Sci. 2017;8(2):P214–P20.

    Article  Google Scholar 

  74. Lambert M-L, Bruyndonckx R, Goossens H, Hens N, Aerts M, Catry B, et al. The Belgian policy of funding antimicrobial stewardship in hospitals and trends of selected quality indicators for antimicrobial use, 1999-2010: a longitudinal study. BMJ Open. 2015;5(2):e006916.

    Article  PubMed  PubMed Central  Google Scholar 

  75. Lee KR, Bagga B, Arnold SR. Reduction of broad-Spectrum antimicrobial use in a tertiary Children’s hospital post antimicrobial stewardship program guideline implementation. Pediatr Crit Care Med : a journal of the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies. 2016;17(3):187–93.

    Article  Google Scholar 

  76. Lee TC, Frenette C, Jayaraman D, Green L, Pilote L. Antibiotic self-stewardship: trainee-led structured antibiotic time-outs to improve antimicrobial use. Ann Intern Med. 2014;161(10 Suppl):S53–8.

    Article  PubMed  Google Scholar 

  77. Liew YX, Lee W, Kwa AL-H, Chlebicki MP. Cost effectiveness of an antimicrobial stewardship programme. Int J Antimicrob Agents. 2015;46(5):594–5.

    Article  CAS  PubMed  Google Scholar 

  78. MacVane SH, Nolte FS. Benefits of adding a rapid PCR-based blood culture identification panel to an established antimicrobial stewardship program. J Clin Microbiol. 2016;54(10):2455–63.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  79. Nilholm H, Holmstrand L, Ahl J, Månsson F, Odenholt I, Tham J, et al. An audit-based, infectious disease specialist-guided antimicrobial stewardship program profoundly reduced antibiotic use without negatively affecting patient outcomes. Open Forum Infect Dis. 2015;2(2):ofv042 Oxford University Press.

    Article  PubMed  PubMed Central  Google Scholar 

  80. Okumura LM, da Silva MMG, Veroneze I. Effects of a bundled antimicrobial stewardship program on mortality: a cohort study. Braz J Infect Dis. 2015;19(3):246–52.

    Article  PubMed  Google Scholar 

  81. Pardo J, Klinker KP, Borgert SJ, Butler BM, Giglio PG, Rand KH. Clinical and economic impact of antimicrobial stewardship interventions with the FilmArray blood culture identification panel. Diagn Microbiol Infect Dis. 2016;84(2):159–64.

    Article  PubMed  Google Scholar 

  82. Smith T, Philmon C, Johnson G, Ward W, Rivers L, Williamson S, et al. Original article: antimicrobial stewardship in a community hospital: attacking the more difficult problems. Hosp Pharm. 2014;49(9):839–46.

    Article  PubMed  PubMed Central  Google Scholar 

  83. Taggart LR, Leung E, Muller MP, Matukas LM, Daneman N. Differential outcome of an antimicrobial stewardship audit and feedback program in two intensive care units: a controlled interrupted time series study. BMC Infect Dis. 2015;15:480.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  84. Zhou L, Ma J, Gao J, Chen S, Bao J. Optimizing prophylactic antibiotic practice for cardiothoracic surgery by Pharmacists’ effects. Medicine. 2016;95(9):e2753.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  85. Matsuda S, Ogasawara T, Sugimoto S, Kato S, Umezawa H, Yano T, et al. Prospective open-label randomized comparative, non-inferiority study of two initial antibiotic strategies for patients with nursing-and healthcare-associated pneumonia: guideline-concordant therapy versus empiric therapy. J Infect Chemother. 2016;22(6):400–6.

    Article  CAS  PubMed  Google Scholar 

  86. Rutman L, Wright DR, O'Callaghan J, Spencer S, Lion KC, Kronman MP, et al. A comprehensive approach to pediatric pneumonia: relationship between standardization, antimicrobial stewardship, clinical testing, and cost. J Healthc Qual. 2017;39(4):e59–69.

    Article  PubMed  Google Scholar 

  87. Taniguchi T, Tsuha S, Shiiki S, Narita M. Gram-stain-based antimicrobial selection reduces cost and overuse compared with Japanese guidelines. BMC Infect Dis. 2015;15:458.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  88. Arnold FW, McDonald LC, Smith RS, Newman D, Ramirez JA. Improving antimicrobial use in the hospital setting by providing usage feedback to prescribing physicians. Infect Control Hosp Epidemiol. 2006;27(4):378–82.

    Article  PubMed  Google Scholar 

  89. Ding H, Yang Y, Wei J, Fan S, Yu S, Yao K, et al. Influencing the use of antibiotics in a Chinese pediatric intensive care unit. Pharm World Sci. 2008;30(6):787–93.

    Article  PubMed  Google Scholar 

  90. Philmon C, Smith T, Williamson S, Goodman E. Controlling use of antimicrobials in a community teaching hospital. Infect Control Hosp Epidemiol. 2006;27(3):239–44.

    Article  PubMed  Google Scholar 

  91. Al-Eidan FA, McElnay JC, Scott M, Kearney M, Corrigan J, McConnell J. Use of a treatment protocol in the management of community-acquired lower respiratory tract infection. J Antimicrob Chemother. 2000;45(3):387–94.

    Article  CAS  PubMed  Google Scholar 

  92. Barenfanger J, Drake C, Leon N, Mueller T, Troutt T. Clinical and financial benefits of rapid detection of respiratory viruses: an outcomes study. J Clin Microbiol. 2000;38(8):2824–8.

    CAS  PubMed  PubMed Central  Google Scholar 

  93. Bauer KA, West JE, Balada-Llasat J-M, Pancholi P, Stevenson KB, Goff DA. An antimicrobial stewardship program's impact. Clin Infect Dis. 2010;51(9):1074–80.

    Article  PubMed  Google Scholar 

  94. Frighetto L, Marra CA, Stiver HG, Bryce EA, Jewesson PJ. Economic impact of standardized orders for antimicrobial prophylaxis program. Ann Pharmacother. 2000;34(2):154–60.

    Article  CAS  PubMed  Google Scholar 

  95. Gentry CA, Greenfield RA, Slater LN, Wack M, Huycke MM. Outcomes of an antimicrobial control program in a teaching hospital. Am J Health Syst Pharm. 2000;57(3):268–74.

    Article  CAS  PubMed  Google Scholar 

  96. Heil EL, Daniels LM, Long DM, Rodino KG, Weber DJ, Miller MB. Impact of a rapid peptide nucleic acid fluorescence in situ hybridization assay on treatment of Candida infections. Am J Health Syst Pharm. 2012;69(21):1910–4.

    Article  CAS  PubMed  Google Scholar 

  97. Ho BP, Lau TT, Balen RM, Naumann TL, Jewesson PJ. The impact of a pharmacist-managed dosage form conversion service on ciprofloxacin usage at a major Canadian teaching hospital: a pre-and post-intervention study. BMC Health Serv Res. 2005;5(1):48.

    Article  PubMed  PubMed Central  Google Scholar 

  98. Maddox ML, DeBoer EC, Hammerquist RJ. Administration of Extended Infusion Piperacillin-TazobactamWith the use of smart pump technology. Hosp Pharm. 2014;49(5):444–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  99. McGregor JC, Weekes E, Forrest GN, Standiford HC, Perencevich EN, Furuno JP, et al. Impact of a computerized clinical decision support system on reducing inappropriate antimicrobial use: a randomized controlled trial. J Am Med Inform Assoc. 2006;13(4):378–84.

    Article  PubMed  PubMed Central  Google Scholar 

  100. Oosterheert JJ, Van Loon AM, Schuurman R, Hoepelman AI, Hak E, Thijsen S, et al. Impact of rapid detection of viral and atypical bacterial pathogens by real-time polymerase chain reaction for patients with lower respiratory tract infection. Clin Infect Dis. 2005;41(10):1438–44.

    Article  PubMed  Google Scholar 

  101. Perez KK, Olsen RJ, Musick WL, Cernoch PL, Davis JR, Land GA, et al. Integrating rapid pathogen identification and antimicrobial stewardship significantly decreases hospital costs. Arch Pathol Lab Med. 2012;137(9):1247–54.

    Article  PubMed  Google Scholar 

  102. Reed EE, West JE, Keating EA, Pancholi P, Balada-Llasat J-M, Mangino JE, et al. Improving the management of candidemia through antimicrobial stewardship interventions. Diagn Microbiol Infect Dis. 2014;78(2):157–61.

    Article  PubMed  Google Scholar 

  103. Wong JR, Bauer KA, Mangino JE, Goff DA. Antimicrobial stewardship pharmacist interventions for coagulase-negative staphylococci positive blood cultures using rapid polymerase chain reaction. Ann Pharmacother. 2012;46(11):1484–90.

    Article  CAS  PubMed  Google Scholar 

  104. Barber KE, Bell AM, Travis King S, Parham JJ, Stover KR. Impact of piperacillin-tazobactam shortage on meropenem use: implications for antimicrobial stewardship programs. Braz J Infect Dis : an official publication of the Brazilian Society of Infectious Diseases. 2016;20(6):631–4.

    Article  Google Scholar 

  105. Cai Y, Shek PY, Teo I, Tang SSL, Lee W, Liew YX, et al. A multidisciplinary antimicrobial stewardship programme safely decreases the duration of broad-spectrum antibiotic prescription in Singaporean adult renal patients. Int J Antimicrob Agents. 2016;47(1):91–6.

    Article  CAS  PubMed  Google Scholar 

  106. Dik J-WH, Hendrix R, Friedrich AW, Luttjeboer J, Panday PN, Wilting KR, et al. Cost-minimization model of a multidisciplinary antibiotic stewardship team based on a successful implementation on a urology ward of an academic hospital. PLoS One. 2015;10(5):e0126106.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  107. Egger ME, Myers JA, Arnold FW, Pass LA, Ramirez JA, Brock GN. Cost effectiveness of adherence to IDSA/ATS guidelines in elderly patients hospitalized for community-Aquired pneumonia. BMC Med Inform Decis Mak. 2016;16:34.

    Article  PubMed  PubMed Central  Google Scholar 

  108. Fanning M, McKean M, Seymour K, Pillans P, Scott I. Adherence to guideline-based antibiotic treatment for acute exacerbations of chronic obstructive pulmonary disease in an Australian tertiary hospital. Intern Med J. 2014;44(9):903–10.

    Article  CAS  PubMed  Google Scholar 

  109. MacVane SH, Hurst JM, Boger MS, Gnann JW Jr. Impact of a rapid multiplex polymerase chain reaction blood culture identification technology on outcomes in patients with vancomycin-resistant Enterococcal bacteremia. Infect Dis. 2016;48(10):732–7.

    Article  CAS  Google Scholar 

  110. Lockwood AM, Perez KK, Musick WL, Ikwuagwu JO, Attia E, Fasoranti OO, et al. Integrating rapid diagnostics and antimicrobial stewardship in two community hospitals improved process measures and antibiotic adjustment time. Infect Control Hosp Epidemiol. 2016;37(4):425–32.

    Article  PubMed  Google Scholar 

  111. Pasquale TR, Trienski TL, Olexia DE, Myers JP, Tan MJ, Leung AK, et al. Impact of an antimicrobial stewardship program on patients with acute bacterial skin and skin structure infections. Am J Health Syst Pharm. 2014;71(13):1136–9.

    Article  PubMed  Google Scholar 

  112. Patel TS, Kaakeh R, Nagel JL, Newton DW, Stevenson JG. Cost analysis of implementing matrix- assisted laser desorption ionization-time of flight mass spectrometry plus real-time antimicrobial stewardship intervention for bloodstream infections. J Clin Microbiol. 2017;55(1):60–7.

    Article  PubMed  Google Scholar 

  113. Wenzler E, Goff DA, Mangino JE, Reed EE, Wehr A, Bauer KA. Impact of rapid identification of Acinetobacter Baumannii via matrix-assisted laser desorption ionization time-of-flight mass spectrometry combined with antimicrobial stewardship in patients with pneumonia and/or bacteremia. Diagn Microbiol Infect Dis. 2016;84(1):63–8.

    Article  CAS  PubMed  Google Scholar 

  114. van Daalen FV, Opmeer BC, Prins JM, Geerlings SE, Hulscher MEL. The economic evaluation of an antibiotic checklist as antimicrobial stewardship intervention. J Antimicrob Chemother. 2017;72(11):3213–21.

    Article  CAS  PubMed  Google Scholar 

  115. Gibson S, Rofe O. Development of a Successful Business Case for Significant Expansion of an Antimicrobial Stewardship Program [poster]: Eastern Health. [Poster]. Melbourne: Department of Pharmacy, Eastern Health; 2015.

  116. Trupka T, Fisher K, Micek ST, Juang P, Kollef MH. Enhanced antimicrobial de-escalation for pneumonia in mechanically ventilated patients: a cross-over study. Crit Care. 2017;21(1):180.

    Article  PubMed  PubMed Central  Google Scholar 

  117. Zahar J-R, Rioux C, Girou E, Hulin A, Sauve C, Bernier-Combes A, et al. Inappropriate prescribing of aminoglycosides: risk factors and impact of an antibiotic control team. J Antimicrob Chemother. 2006;58(3):651–6.

    Article  CAS  PubMed  Google Scholar 

  118. Lanbeck P, Tennvall GR, Resman F. A cost analysis of introducing an infectious disease specialist-guided antimicrobial stewardship in an area with relatively low prevalence of antimicrobial resistance. BMC Health Serv Res. 2016;16(1):311.

    Article  PubMed  PubMed Central  Google Scholar 

  119. Slayton RB, Scott RD, Baggs J, Lessa FC, McDonald LC, Jernigan JA. The cost-benefit of federal investment in preventing Clostridium difficile infections through the use of a multifaceted infection control and antimicrobial stewardship program. Infect Control Hosp Epidemiol. 2015;36(6):681–7.

    Article  PubMed  PubMed Central  Google Scholar 

  120. Fu RZ, Anwar DR, Laban JT, Maratos EC, Minhas PS, Martin AJ. Pre-emptive intrathecal vancomycin therapy reduces external ventricular drain infection: a single Centre retrospective case-control study. Br J Neurosurg. 2017;31(1):16–20.

    Article  PubMed  Google Scholar 

  121. Okumura LM, Riveros BS, Gomes-da-Silva MM, Veroneze I. A cost-effectiveness analysis of two different antimicrobial stewardship programs. Braz J Infect Dis : an official publication of the Brazilian Society of Infectious Diseases. 2016;20(3):255–61.

    Article  Google Scholar 

  122. Sick AC, Lehmann CU, Tamma PD, Lee CK, Agwu AL. Sustained savings from a longitudinal cost analysis of an internet-based preapproval antimicrobial stewardship program. Infect Control Hosp Epidemiol. 2013;34(6):573–80.

    Article  PubMed  Google Scholar 

  123. Gross R, Morgan AS, Kinky DE, Weiner M, Gibson GA, Fishman NO. Impact of a hospital-based antimicrobial management program on clinical and economic outcomes. Clin Infect Dis. 2001;33(3):289–95.

    Article  CAS  PubMed  Google Scholar 

  124. Benson JM. Incorporating pharmacy student activities into an antimicrobial stewardship program in a long-term acute care hospital. Am J Health-Syst Pharm. 2014;71(3):227–30.

    Article  PubMed  Google Scholar 

  125. Grant EM, Kuti JL, Nicolau DP, Nightingale C, Quintiliani R. Clinical efficacy and Pharmacoeconomics of a continuous-infusion piperacillin-Tazobactam program in a large community teaching hospital. Pharmacotherapy: The Journal of Human Pharmacology and Drug Therapy. 2002;22(4):471–83.

    Article  CAS  Google Scholar 

  126. Hermsen ED, Shull SS, Puumala SE, Rupp ME. Improvement in prescribing habits and economic outcomes associated with the introduction of a standardized approach for surgical antimicrobial prophylaxis. Infect Control Hosp Epidemiol. 2008;29(5):457–61.

    Article  PubMed  Google Scholar 

  127. LaRocco A Jr. Concurrent antibiotic review programs—a role for infectious diseases specialists at small community hospitals. Clin Infect Dis. 2003;37(5):742–3.

    Article  PubMed  Google Scholar 

  128. López-Medrano F, San Juan R, Lizasoain M, Catalán M, Ferrari J, Chaves F, et al. A non-compulsory stewardship programme for the management of antifungals in a university-affiliated hospital. Clin Microbiol Infect. 2013;19(1):56–61.

    Article  PubMed  Google Scholar 

  129. Melo D, Cassettari V, Ribeiro E. Educational intervention program to optimize the use of antibiotics: impact on prescribing habits, consumption of antimicrobial agents and cost savings. Revista de Ciências Farmacêuticas Básica e Aplicada. 2009;29(3):251–6.

    Google Scholar 

  130. Saizy-Callaert S, Causse R, Furhman C, Le Paih M, Thébault A, Chouaıd C. Impact of a multidisciplinary approach to the control of antibiotic prescription in a general hospital. J Hosp Infect. 2003;53(3):177–82.

    Article  CAS  PubMed  Google Scholar 

  131. South M, Royle J, Starr M. A simple intervention to improve hospital antibiotic prescribing. Med J Aust. 2003;178(5):207–9.

    PubMed  Google Scholar 

  132. Suryadevara M, Steidl KE, Probst LA, Shaw J. Inappropriate vancomycin therapeutic drug monitoring in hospitalized pediatric patients increases pediatric trauma and hospital costs. J Pediatr Pharmacol Ther. 2012;17(2):159–65.

    PubMed  PubMed Central  Google Scholar 

  133. Ullman MA, Parlier GL, Warren JB, Mateo N, Harvey C, Sullivan CJ, et al. The economic impact of starting, stopping, and restarting an antibiotic stewardship program: a 14-year experience. Antibiotics. 2013;2(2):256–64.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  134. Weeks C, Jones G, Wyllie S. Cost and health care benefits of an antimicrobial management programme. HOSPITAL PHARMACIST-LONDON. 2006;13(5):179.

    Google Scholar 

  135. Yam P, Fales D, Jemison J, Gillum M, Bernstein M. Implementation of an antimicrobial stewardship program in a rural hospital. Am J Health Syst Pharm. 2012;69(13):1142.

    Article  PubMed  Google Scholar 

  136. Dela-Pena J, Kerstenetzky L, Schulz L, Kendall R, Lepak A, Fox B. Top 1% of inpatients administered antimicrobial agents comprising 50% of expenditures: a descriptive study and opportunities for stewardship intervention. Infect Control Hosp Epidemiol. 2017;38(3):259–65.

    Article  PubMed  Google Scholar 

  137. Franklin M, Wailoo A, Dayer MJ, Jones S, Prendergast B, Baddour LM, et al. The cost-effectiveness of antibiotic prophylaxis for patients at risk of infective endocarditis. Circulation. 2016;134(20):1568–78.

    Article  PubMed  PubMed Central  Google Scholar 

  138. Lemaire X, Robineau O, Blanc AL, Vialle B, Béguec J, Hendricx S, et al. Distribution of antimicrobial guidelines: useful but not enough for an antibiotic stewardship. Presse Med. 2014;43(11):e365–e8.

    Article  PubMed  Google Scholar 

  139. Aiken AM, Wanyoro AK, Mwangi J, Juma F, Mugoya IK, Scott JAG. Changing use of surgical antibiotic prophylaxis in Thika hospital, Kenya: a quality improvement intervention with an interrupted time series design. PLoS One. 2013;8(11):e78942.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  140. Apisarnthanarak A, Yatrasert A, Mundy LM. Impact of education and an antifungal stewardship program for candidiasis at a Thai tertiary care center. Infect Control Hosp Epidemiol. 2010;31(7):722–7.

    Article  PubMed  Google Scholar 

  141. Beardsley JR, Williamson JC, Johnson JW, Luther VP, Wrenn RH, Ohl CC. Show me the money: long-term financial impact of an antimicrobial stewardship program. Infect Control Hosp Epidemiol. 2012;33(4):398–400.

    Article  PubMed  Google Scholar 

  142. Calloway S, Akilo HA, Bierman K. Impact of a clinical decision support system on pharmacy clinical interventions, documentation efforts, and costs. Hosp Pharm. 2013;48(9):744–52.

    Article  PubMed  PubMed Central  Google Scholar 

  143. Hamblin S, Rumbaugh K, Miller R. Prevention of adverse drug events and cost savings associated with PharmD interventions in an academic level I trauma center: an evidence-based approach. J Trauma Acute Care Surg. 2012;73(6):1484–90.

    Article  PubMed  Google Scholar 

  144. Rimawi RH, Cook PP, Gooch M, Kabchi B, Ashraf MS, Rimawi BH, et al. The impact of penicillin skin testing on clinical practice and antimicrobial stewardship. J Hosp Med. 2013;8(6):341–5.

    Article  PubMed  Google Scholar 

  145. Sutherland T, Beloff J, Lightowler M, Liu X, Nascimben L, Kaye AD, et al. Description of a multidisciplinary initiative to improve SCIP measures related to pre-operative antibiotic prophylaxis compliance: A single-center success story. Patient Saf Surgery. 2014;8(1):37.

    Article  Google Scholar 

  146. Saeed K, Dryden M, Bourne S, Paget C, Proud A. Reduction in antibiotic use through procalcitonin testing in patients in the medical admission unit or intensive care unit with suspicion of infection. J Hosp Infect. 2011;78(4):289–92.

    Article  CAS  PubMed  Google Scholar 

  147. Porter ME. What is value in health care? N Engl J Med. 2010;363(26):2477–81.

    Article  CAS  PubMed  Google Scholar 

  148. Catalá-López F, García-Altés A, Álvarez-Martín E, Gènova-Maleras R, Morant-Ginestar C, Parada A. Burden of disease and economic evaluation of healthcare interventions: are we investigating what really matters? BMC Health Serv Res. 2011;11:75.

    Article  PubMed  PubMed Central  Google Scholar 

  149. Davey P, Marwick CA, Scott CL, Charani E, McNeil K, Brown E, et al. Interventions to improve antibiotic prescribing practices for hospital inpatients. The Cochrane Library. 2017.

    Google Scholar 

  150. Karanika S, Paudel S, Grigoras C, Kalbasi A, Mylonakis E. Systematic review and meta-analysis of clinical and economic outcomes from the implementation of hospital-based antimicrobial stewardship programs. Antimicrob Agents Chemother. 2016;60(8):4840–52.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  151. Baur D, Gladstone BP, Burkert F, Carrara E, Foschi F, Döbele S, et al. Effect of antibiotic stewardship on the incidence of infection and colonisation with antibiotic-resistant bacteria and Clostridium difficile infection: a systematic review and meta-analysis. Lancet Infect Dis. 2017;17(9):990–1001.

    Article  PubMed  Google Scholar 

  152. Ibrahim OM, Polk RE. Benchmarking antimicrobial drug use in hospitals. Expert Rev Anti-Infect Ther. 2012;10(4):445–57.

    Article  CAS  PubMed  Google Scholar 

  153. Momattin H, Al-Ali AY, Mohammed K, Al-Tawfiq JA. Benchmarking of antibiotic usage: an adjustment to reflect antibiotic stewardship program outcome in a hospital in Saudi Arabia. J Infect Public Health. 2018;11(3):310–3.

    Article  PubMed  Google Scholar 

  154. Nathwani D, Eckmann C, Lawson W, Stephens J, Macahilig C, Solem C, et al. Pan-European early switch/early discharge opportunities exist for hospitalized patients with methicillin-resistant Staphylococcus aureus complicated skin and soft tissue infections. Clin Microbiol Infect. 2014;20(10):993–1000.

    Article  CAS  PubMed  Google Scholar 

  155. WHO. Health service delivery costs 2008 [cited 2018 March 30]. Available from: http://www.who.int/choice/cost-effectiveness/inputs/health_service/en/.

Download references

Acknowledgements

Stephens J, Varghese D, and Martin S are employees of Pharmerit International, who were paid consultants to Pfizer in connection with the study and development of this manuscript.

Funding

This study was sponsored and supported by Pfizer, Inc.

Availability of data and materials

The literature data used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to the concept and design of the study. DN, DV, JS and SM contributed to the preparation of the initial manuscript. WA and CC and other authors contributed to the critical review, revision and approval of the final content of the manuscript.

Corresponding author

Correspondence to Jennifer Stephens.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

At the time this manuscript was completed Ansari W and Charbonneau C were employees of Pfizer, Inc. Nathwani D is a key opinion leader on the topic of antimicrobial stewardship and a paid consultant for Pfizer Inc. Stephens J is a health economic consultant and is a managing partner and co-owner of Pharmerit International. Varghese D and Martin S were employees of Pharmerit International who were paid consultants in connection with this study.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nathwani, D., Varghese, D., Stephens, J. et al. Value of hospital antimicrobial stewardship programs [ASPs]: a systematic review. Antimicrob Resist Infect Control 8, 35 (2019). https://doi.org/10.1186/s13756-019-0471-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13756-019-0471-0

Keywords