Skip to main content

Table 3 Performance measures for developed machine learning algorithms for the detection of SSIs

From: Development of machine learning models for the detection of surgical site infections following total hip and knee arthroplasty: a multicenter cohort study

Algorithms

F1 score

ROC AUC (95% CI)

PR AUC (95% CI)

Sensitivity% (95% CI)

Specificity% (95% CI)

PPV % (95% CI)

NPV% (95% CI)

Admin data algorithms

 Deep SSIs

0.421 (0.214–0.605)

0.836 (0.687–0.984)

0.224 (0.114–0.334)

72.73 (39.03–93.98)

94.41 (91.41–96.60)

29.63 (19.27–42.61)

99.07 (97.60–99.65)

 Organ SSIs

0.552 (0.385–0.693)

0.868 (0.765–0.97)

0.348 (0.237–0.459)

80.00 (56.34–94.27)

93.51 (90.34–95.89)

42.11 (31.47–53.52)

98.75 (97.06–99.48)

 Complex SSIs

0.699 (0.575–0.805)

0.934 (0.873–0.995)

0.527 (0.419–0.635)

93.55 (78.58–99.21)

93.24 (90.02–95.66)

55.77(45.67–65.41)

99.37 (97.65–99.84)

Text data algorithms

 Deep SSIs

0.414 (0.174–0.640)

0.755 (0.587–0.923)

0.196 (0.967–0.296)

54.55 (23.38–83.25)

96.47 (93.92–98.16)

33.33 (18.73–52.03)

98.50 (97.17–99.21)

 Organ SSIs

0.638 (0.439–0.794)

0.857 (0.752–0.962)

0.431 (0.307–0.554)

75.00 (50.90–91.34)

96.46 (93.90–98.16)

55.56 (40.43–69.71)

98.49 (96.84–99.29)

 Complex SSIs

0.735 (0.607–0.848)

0.886 (0.808–0.963)

0.561 (0.452–0.67)

80.65 (62.53–92.55)

96.47 (93.92–98.16)

67.57 (53.80–78.85)

98.20 (96.38–99.12)

Admin& Text algorithms

 Deep SSIs

0.500 (0.222–0.714)

0.762 (0.596–0.929)

0.266 (0.142–0.39)

54.55 (23.38–83.25)

97.94 (95.80–99.17)

46.15 (25.65–68.05)

98.52 (97.21–99.22)

 Organ SSIs

0.714 (0.522–0.857)

0.865 (0.762–0.968)

0.525 (0.393–0.657)

75.00 (50.90–91.34)

97.94 (95.79–99.17)

68.18 (49.66–82.31)

98.52 (96.88–99.30)

 Complex SSIs

0.788 (0.667–0.896)

0.906 (0.835–0.978)

0.637 (0.528–0.746)

83.87 (66.27–94.55)

97.35 (95.03–98.78)

74.29 (59.82–84.86)

98.51 (96.74–99.33)

  1. 95% CI 95% confidence interval, NPV negative predictive value, PPV positive predictive value, PR AUC the area under the precision–recall curve, ROC AUC the area under the receiver operating characteristic curve, SSIs surgical site infections