Table 3 Performance characteristics of automated surveillance systems vs reference standards

Stern [22]

Patients who met surveillance criteria

250

nvHAP

1

1

CDC-NHS PNU1

both reviewers

one reviewer

–

–

42% (39–46)

67% (64–70)

–

Stern [22]

Patients who met surveillance criteria

250

nvHAP

1

2

Clinical nvHAP (treating physician)

both reviewers

one reviewer

–

–

42% (39–46)

60% (57–63)

–

Stern [22]

Patients who met surveillance criteria

250

nvHAP

1

6

Presence of clinical deterioration

both reviewers

one reviewer

–

–

87% (85–89)

98% (98–99)

–

Stern [22]

Patients who met surveillance criteria

250

nvHAP

1

4

Discharge summary nvHAP

both reviewers

one reviewer

–

–

35% (32–38)

49% (46–52)

–

Stern [22]

Patients who met surveillance criteria

250

nvHAP

1

5

Expert reviewer nvHAP

both reviewers

one reviewer

–

–

50% (46–53)

71% (68–74)

–

Stern [22]

Patients who met surveillance criteria

250

nvHAP

1

7

VA claims based definition of nvHAP

–

–

7.9% (4–12)

–

Ji [23]

Patients with impaired oxygenation

120

nvHAP

8

2

Clinical nvHAP (treating physician)

56% (40–70)

77% (68–86)

60% (47–71)

–

Jones [25]

Patients who met surveillance criteria

250

nvHAP

8

2 or 4 or 5

Clinical nvHAP (treating physician) OR discharge summary nvHAP OR expert reviewer

–

–

81%

–

Ramirez-Batlle [24]

Patients with impaired oxygenation

120

nvHAP

10

1

CDC-NHSN PNEU

59% (39–77)

73% (62–81)

41% (30–52)

85% (78–90)

Ramirez-Batlle [24]

Patients with impaired oxygenation

120

nvHAP

10

2

Clinical nvHAP (treating physician)

56% (40–70)

77% (66–86)

60% (47–71)

74% (67–80

Ramirez-Batlle [24]

Patients with impaired oxygenation

120

nvHAP

10

3

“True pneumonia” (clinical plus AB-treatment)

71% (51–87)

76% (66–84)

48% (37–58)

90% (83–94)

Valentine [28]

Patients with hemato-oncolic disease

151

HAP post-EMR:34% pre-EMR:13%

12

1

CDC-NHSN PNU3

–

–

18% (12–25)

Wolfensberger [21]

Random sample plus patients with ICD10

2091

nvHAP

15

1

x

ECDC nvHAP

98% (93–99) or higher

(100% not mentioned in the paper, but per definition)

(100% not mentioned in the paper, but per definition)

99% (98–100) or higher

Wolfensberger [27]

Patients from 3 surgical and 1 medical department

747 (590 random sample, 157 with ICD10)

HAP

16

1

HELICS/ECDC nvHAP

59% (48–69) (extrapolated)

98% (98–99) (extrapolated)

35% (27–42)

99% (98–100)

FitzHenry [31]

Procedures with surgical procedures

7743 patients with 8186 procedures (4098 in development and 4088 in validation cohort)

HAP (postoperative pneumonia)

17

1

VASQIP

79% (no CI)

–

–

–

Bouzbid [29]

Patients > 48 h on ICU

1499

HAP (on ICU)

18

1

HELICS

99% (98–100)

58% (56–61)

22% (19–25)

100% (99–100)

Bouzbid [29]

Patients > 48 h on ICU

1499

HAP (on ICU)

19

1

HELICS

86% (81–91)

89% (88–91)

49% (43–55)

98% (97–99)

Bouzbid [29]

Patients > 48 h on ICU

1499

HAP (on ICU)

20

1

HELICS

93% (89–97)

64% (61–66)

23% (20–27)

99% (98–99)

Bouzbid [29]

Patients > 48 h on ICU

1499

HAP (on ICU)

21

1

HELICS

40% (33–48)

92% (90–93)

36% (29–43)

93% (91–94)

Bouzbid [29]

Patients > 48 h on ICU

1499

HAP (on ICU)

22

1

HELICS

99% (97–100)

61% (58–63)

23% (20–26)

100% (99–100)

Bouzbid [29]

Patients > 48 h on ICU

1499

HAP (on ICU)

23

1

HELICS

81% (74–86)

92% (91–94)

55% (48–61)

98% (97–98)

Haas [19] and Mendonca [20]

NICU

1688 or 1692

HAP (on Neo)

24

1

NNIS-criteria

71% (no CI)

95% (no CI)

8% (no CI)

99.8% (no CI)