Standardisation of disk diffusion results for antibiotic susceptibility testing using the sirscan automated zone reader

Abstract

Background

Standardisation of disk diffusion readings could improve reproducibility and accuracy of antibiotic susceptibility testing (AST). This study evaluated accuracy, reproducibility, and precision of automated inhibition zone reading using the “Sirscan automatic” zone reader (i2a, Perols Cedex, France).

Results

In a first step we compared Sirscan results with manual calliper measurements for comparability and accuracy. Sirscan readings were checked and adjusted on-screen as recommended by the manufacturer. One hundred clinical bacterial isolates representing a broad spectrum of organisms routinely isolated in a clinical laboratory were tested, and zone diameter values and interpretation according to EUCAST guidelines were compared. In a second step we analysed, whether fully automated zone reading can decrease standard deviation of diameter measurements and, thus, improve reproducibility and precision of the disk diffusion method. Standard deviations of manual measurements, on-screen adjusted Sirscan measurements, and fully automated Sirscan readings were compared for 19 repeat independent measurements of inhibition zones of S. aureus ATCC 29213, E. coli ATCC 25922, and P. aeruginosa ATCC 27853 (EUCAST quality control strains).

On-screen adjusted Sirscan and calliper measurements displayed high comparability. No significant differences were detected comparing the results of both reading methods. Standard deviations of inhibition zone diameters were significantly lower for fully automated Sirscan measurements compared with both adjusted Sirscan readings and the manual method, resulting in better reproducibility and precision of the automated readings.

Conclusions

Our results indicate that fully automated zone reading can further improve standardisation of AST by decreasing standard deviation and, thus, improve precision of inhibition zone diameter results.

Background

Disk diffusion has been the mainstay for antimicrobial susceptibility testing (AST) in most clinical microbiological laboratories since Bauer, Kirby et al. first described this technique in the 1960s [1]. During the past decade automated AST microdilution systems based on determination or extrapolation of minimal inhibitory concentrations have been introduced in the diagnostic market, e.g. systems like the Vitek 2 (BioMérieux), Phoenix (Becton-Dickinson), or Microscan (Siemens Healthcare Diagnostics).

The main advantages of commercial microdilution systems including automated reading and rapidity are compromised by the still lower sensitivities in the detection of important resistance mechanisms compared with the disk diffusion method, e.g. inducible macrolide-lincosamide-streptogramin resistance (MLS_B-Type), extended spectrum beta-lactamases (ESBL), and AmpC beta-lactamases [2–5]. In addition, some combinations of resistance mechanisms are not reliably detected by automated microdilution systems e.g. ESBL in Enterobacteriaceae isolates co-producing chromosomally- or plasmid-encoded AmpC beta-lactamases [6]. The sensitivity for detection of resistance mechanisms largely depends on the composition of the antibiotic drug panel used in the automated microdilution systems, which cannot be changed or modified by the user [2, 7].

The disk diffusion method readily permits detection of inducible phenotypes and most combinations of resistance mechanisms including ESBL and AmpC co-production. The antibiotic panel composition is flexible and enables the clinical laboratory to readily adjust the composition of panels to its needs [8, 9]. Disadvantages of the disk diffusion method are its labour cost due to manual measurements and manual data documentation, and the investigator dependence and variation of results [10].

During the past decade several systems have been developed to automate disk diffusion readings. Systems like Sirscan (i2a, Montpellier, France), OSIRIS and ADAGIO (both BIO-RAD, Marne La Coquotte, France), Oxoid Aura (Oxoid Ltd., Basingstoke, UK), or BIOMIC (Giles Scientific Inc., Santa Barbara, California, USA) are able to automatically read inhibition zone diameters and incorporate expert systems for AST interpretation. These systems allow fully automated (Sirscan) or semi-automated reading (ADAGIO, Aura, BIOMIC), documentation and data interpretation using expert systems. The few studies available investigating the performance of automated zone reading systems indicate a high agreement with standard manual calliper (correlation coefficients ranging from 0.91 to 0.96) resulting in only few susceptibility categorisation errors [10–15]. However, some systems are no longer available (OSIRIS, Oxoid Aura), or have reported practical problems for routine use (BIOMIC) [16].

No studies are available investigating, if and to which extent fully automated zone reading is able to facilitate standardisation of inhibition zone diameter measurements. High reproducibility and low variation of results become even more important in the light of the new CLSI and EUCAST AST guidelines that contain smaller intermediate susceptibility categories or, in case of EUCAST, have even partially abandoned the use of the intermediate category. Directly adjacent susceptible and resistant categories lead to a higher frequency of major and very major errors (i.e. susceptible to resistant, resistant to susceptible) simply due to technical reasons, i.e. variation of individual measurements [17–19].

This study aimed at comparing the fully automated Sirscan with standard calliper measurements assessing: i) The agreement of inhibition zone diameter results (comparability), ii) The frequency of discrepancies in susceptibility categorisation (accuracy), and iii) Variation of repeat diameter measurements (reproducibility and precision).

Methods

Clinical isolates

One hundred clinical bacterial isolates were selected as a representative sample of organisms routinely isolated in the clinical microbiological laboratory. Bacterial strains comprised 21 Escherichia coli, 17 Staphylococcus aureus, 17 Enterococcus spp., 16 coagulase-negative staphylococci, 9 Pseudomonas aeruginosa, 7 Klebsiella pneumoniae, 4 Enterobacter spp., 2 Serratia spp., 2 Stenotrophomonas maltophilia, 2 Acinetobacter spp., 2 Proteus spp., and 1 Citrobacter spp. For reproducibility testing, Staphylococcus aureus ATCC 29213, Escherichia coli ATCC 25922, and Pseudomonas aeruginosa ATCC 27853 (EUCAST quality control strains) were used. The following non-duplicate clinical isolates with confirmed resistance mechanisms were included to test for adequate detection of individual resistance mechanisms by the Sirscan instrument: 117 Extended-spectrum beta-lactamase (ESBL) producing Enterobacteriaceae isolates (105 CTX-M type, 10 SHV-ESBL-type, and 2 TEM-ESBL type), 38 AmpC producing Enterobacteriaceae isolates (24 plasmid-encoded CIT-type AmpC, 2 plasmid-encoded DHA-type AmpC, and 12 E. coli isolates harboring ampC promoter mutations leading to overexpression of AmpC), 13 carbapenemase producing Enterobacteriaceae isolates (6 KPC type, 3 VIM type, 2 OXA-48 type, 1 NDM-1 type, 1 GIM-1 type), 17 vancomycin-resistant enterococci (VRE) isolates, and 50 methicillin-resistent S. aureus (MRSA) isolates [5, 9].

Susceptibility testing

Disk diffusion testing was done according to the 2011 guidelines of the European Committee of Antimicrobial Susceptibility Testing (EUCAST) using standard antibiotic disks (i2a, Perols Cedex, France) and Mueller-Hinton agar plates (BD, Franklin Lakes, NJ). All measurements except those for investigator dependence were done by the same experienced laboratory technician to eliminate inter-person bias. In parallel, the disk diffusion Mueller-Hinton agar plates were measured with the Sirscan instrument (i2a, Perols Cedex, France) and manually using a standard calliper. Sirscan measurements were checked and corrected on-screen by the laboratory technician as recommended by the manufacturer. Standard deviations of zone diameter measurements were calculated from 19 independent and blinded readings by 19 experienced persons using antibiotic disk diffusion inhibition zones of S. aureus ATCC 29213, E. coli ATCC 25922, and P. aeruginosa ATCC 27853 (EUCAST quality control strains). Discrepancies of manual and Sirscan readings were categorised as follows: Discrepancies resulting in erratic assignment of bacterial isolates to adjacent interpretative categories (susceptible to intermediate, intermediate to susceptible, intermediate to resistant, resistant to intermediate) were referred to as “minor discrepancies”. Erroneous categorisation of true-susceptible isolates as resistant (considering the manual method as the gold standard) were referred to as “major discrepancies”. Categorisation of true-resistant isolates as susceptible (considering the manual method as the gold standard) were referred to as “very major discrepancies”.

The following parameters were used to test for the presence of individual resistance mechanisms using Sirscan readings: ESBL-screening was done using EUCAST clinical breakpoints for non-susceptibility to cefpodoxime, and/or ceftazidime, and/or cefotaxime, ceftriaxone, and/or cefepime, AmpC and MRSA-screening was done using EUCAST clinical breakpoints for non-susceptibility to cefoxitin, carbapenemase-screening was done using EUCAST clinical breakpoints for non-susceptibility to ertapenem, and/or meropenem, and/or imipenem, and VRE-screening was done using EUCAST clinical breakpoints for non-susceptibility to vancomycin [18].

All inhibition zone diameter results were recorded by the Sirweb software (i2a, Perols Cedex, France) and statistical parameters were calculated with the Microsoft Excel 2010 Software (Microsoft Corp., Redmond, WA).

Antibiotic drugs

Different antibiotic drug panels were tested for Gram-negative rods, Staphylococcus spp., and Enterococcus spp. Antibiotic drugs tested for Gram-negative rods comprised ampicillin, amoxicillin/clavulanic acid, piperacillin/tazobactam, cefuroxime, cefpodoxime, ceftriaxone, ceftazidime, cefotaxime, cefepime, cefoxitin, ertapenem, imipenem, meropenem, amikacin, gentamicin, tobramycin, nalidixic acid, ciprofloxacin, levofloxacin, nitrofurantoin, and trimethoprim-sulfamethoxazole. Antibiotic drugs tested for Staphylococcus spp. comprised penicillin, cefoxitin, amikacin, gentamicin, tobramycin, ciprofloxacin, levofloxacin, rifampicin, erythromycin, clindamycin, and trimethoprim-sulfamethoxazole. Antibiotic drugs tested for Enterococcus spp. comprised ampicillin and vancomycin.

Results

Mean differences of inhibition zone diameter measurements were less than 2 mm for all antibiotic classes and bacterial groups comparing on-screen adjusted Sirscan readings (manufacturer recommended) and manual readings for the 100 clinical strains (Table 1), with the exception of ampicillin and Enterococcus spp. that showed a mean difference of 2.5 mm. On average, mean differences of all antibiotic drug classes were higher for Staphylococcus spp. and Enterococcus spp. than for Gram-negative rods (1.2 mm, 1.7 mm, and 0.9 mm, respectively, see Table 1). For Gram-negative rods the carbapenems showed mean differences of inhibition zone diameters above average, for staphylococci clindamycin, penicillins, and quinolones showed mean differences of inhibition zone diameters higher than the average (Table 1).

Table 1 Mean differences of zone diameters measurements as determined by calliper and Sirscan on-screen adjusted

The relative deviations of inhibition zone diameter measurements (higher or lower inhibition zone diameter values of one method compared to the other) were almost equally distributed between on-screen adjusted Sirscan and manual measurements (Table 2). Enterococcus spp. constituted an exception as lower zone diameters with the Sirscan were observed in 53% of the cases. However, no major or very major discrepancies resulted from these deviations comparing on-screen adjusted Sirscan with manual calliper measurements that were considered as the gold standard (using EUCAST 2011 AST guidelines) [18]. Reported AST results with the on-screen adjusted Sirscan system were as accurate as the currently recommended manual method.

Table 2 Relative deviation of zone diameter values and resulting discrepancies of the Sirscan (on-screen adjusted) and manual calliper measurements

All isolates with confirmed resistance mechanisms, i.e. ESBL-, AmpC, and carbapenemase producing Enterobacteriaceae isolates, VRE, and MRSA were adequately detected using Sirscan readings with two exceptions: One CIT-type AmpC producing isolate, and one MRSA isolate showing cefoxitin inhibition zone diameters of 21 mm (corresponding non-susceptible EUCAST breakpoint <19 mm), and 22 mm (corresponding non-susceptible EUCAST breakpoint <22 mm), respectively. Inhibition zone diameters could subsequently be confirmed by manual reading.

The reproducibility and precision of repeat readings by 19 experienced persons were significantly higher with fully automated Sirscan readings compared with the manufacturer recommended on-screen adjusted Sirscan readings and manual calliper measurements (Table 3). The average standard deviations for S. aureus ATCC 29213, E. coli ATCC 25922, and P. aeruginosa ATCC 27853 were 0.8 mm, 0.7 mm, and 0.6 mm (Sirscan fully automated), and 1.6 mm, 1.4 mm, and 0.8 mm (manual readings). Standard deviations of on-screen adjusted Sirscan readings were comparable to the manual method (1.3 mm, 1.4 mm, and 1.0 mm, for S. aureus ATCC 29213, E. coli ATCC 25922, and P. aeruginosa ATCC 27853, respectively). The lower standard deviation of fully automated Sirscan readings was pronounced for certain antibiotics (Table 3): E.g. for trimethoprim-sulfamethoxazole and S. aureus ATCC 29213 (0.9 mm versus 4.7 mm for fully automated Sirscan and manual readings, respectively) or for trimethoprim-sulfamethoxazole and nitrofurantoin in E. coli ATCC 25922 (0.4 and 0.5 mm versus 0.9 and 1.6 mm for fully automated Sirscan and manual readings, respectively).

Table 3 Comparison of standard deviations of measurements with calliper, the Sirscan system adaped on-screen by the human eye and the Sirscan fully automated mode

Examples of measurement variations are shown in Table 4 as scattergram illustrations: 6 / 19 manual calliper measurements for nitrofurantoin in E. coli ATCC 25922 were lower than the EUCAST recommended quality control range. Adjusted Sirscan readings showed slightly lower variation, but 6 / 19 nitrofurantoin measurements were still out of the quality control range. Sirscan measurements for nitrofurantoin in the fully automated mode showed significantly lower variation and all were in the quality control range. A comparable pattern was seen with ertapenem for E. coli ATCC 25922 and amikacin for S. aureus ATCC 29213. The most prominent effect of fully automated readings on standard deviation of zone diameter measurements was observed for trimethoprim-sulfamethoxazole and S. aureus ATCC 29213: 14 / 19 calliper measurements were not in the EUCAST quality control range and inter-person variation was high (lowest value 18 mm, highest value 34 mm, i.e. a range of 18 mm). In contrast, fully automated Sirscan readings had a range of 4 mm and only 4 out of 19 values were 1 mm out of the quality control range.

Table 4 Examples of scattergrams of inhibition zone measurements with calliper, the Sirscan system adaped on-screen by the human eye and the Sirscan fully automated mode

Discussion

Automation of inhibition zone readings was developed to avoid disadvantages of disk diffusion AST such as high manual workload, laborious data documentation, and low speed of manual readings. Our results show excellent comparability of on-screen adjusted automated measurements using the Sirscan instrument compared with the manual calliper method for a broad range of species representing the most common isolates in a routine clinical microbiological laboratory (Table 1). The present results are in agreement with other studies that found a high correlation of Sirscan and manual measurements [12, 13]. Relative deviations of Sirscan and manual measurements were almost equally distributed pointing to random deviations rather than systematical errors (Table 2). Neither method tended to systematically higher or lower diameter measurements compared with the other with the exception of Enterococcus spp., which tended to produce lower zone diameters with the Sirscan (53% of the cases, Table 2) - most probably because the faint bacterial growth was better visible on the computer screen than by the unaided eye. However, this did not result in interpretation discrepancies (Table 2).

Most important, on-screen adjusted automation of disk diffusion readings did not result in an increased frequency of susceptibility categorisation errors. The results of this study showed no major and very major discrepancies occurring with on-screen adjusted Sirscan readings when compared to manual measurements serving as the gold standard. Other authors found low numbers of major and very major errors with the Sirscan system as well [12, 13]. Isolates with confirmed resistance mechanisms such as ESBL, AmpC, carbapenemases, VRE, or MRSA were reliably detected except for two isolates showing inhibition zone diameters close to the EUCAST breakpoint. However, both isolates would have been missed by manual reading, too.

Reproducibility and precision of diameter measurements are critical for AST interpretation and antimicrobial therapy. Previous investigations have focused on the correlation of manual and automated measurements using systems like Sirscan, OSIRIS, BIOMIC, or Oxoid Aura [12–16, 20]. While correlation of manual and automated systems is well established, we here used a fully automated system to assess, if automated reading is principally able to decrease standard deviation of measurements and, thus, can increase precision. This is of particular importance given the changes in recent EUCAST and, in part, CLSI AST guidelines to decrease or even abandon the intermediate AST zone [19].

Investigator dependence of manual measurements with the disk diffusion method is partly due to non-standardised conditions such as ambient light, angle of vision, reading plates from top or bottom, or physical and mental condition of the investigator. The Sirscan analysis software reads under standardised light, positioning and background conditions. The lack or downsizing of the intermediate category by CLSI and/or EUCAST 2011/12 guidelines enhances the probability of major and very major errors of repeat measurements since susceptible and resistant categories lie directly adjacent to each other [17–19]. Standardisation of measurements with concomitant lower standard deviations will facilitate consistent AST reports for repeatedly tested strains, or for ASTs of one strain isolated from multiple patient samples. The reproducibility of fully automated Sirscan readings without human interaction (on-screen adjustments) was significantly higher compared with manual calliper measurements. The average standard deviation for repeat measurements of E. coli ATCC 25922 and S. aureus ATCC 29213 inhibition zones was reduced by half using the fully automated reading mode. If, however, Sirscan readings were adjusted on-screen, standard deviations were not significantly lower (Table 3). For P. aeruginosa ATCC 27853 the reduction of standard deviations was less pronounced, probably because calliper measurements already showed a low standard deviation of 0.8 mm. This may originate from the pyoverdin-pigmented growth of P. aeruginosa ATCC 27853 that allows a more precise measurement of zone edges by the unaided human eye. In contrast, compounds forming fuzzy zone edges showed high standard deviations with manual readings, e.g. trimethoprim-sulfamethoxazole, ertapenem, or cefpodoxime (Table 3). Particularly trimethoprim-sulfamethoxazole forms fuzzy zone edges resulting in a broad variation of manual measurements (Tables 3, and 4). For trimethoprim-sulfamethoxazole the EUCAST reading guide for disk diffusion testing recommends to “ignore faint or haze growth up to the disk within a zone with otherwise clear zone edge” [21]. The definition of the zone edge and “faint or haze growth” is strongly dependent on factors like positioning of the plate, ambient light, or even the visual acuity of the investigator. Reading inhibition zones by a camera under standardised conditions and defining the zone edge by picture analysis with a well-defined software algorithm can help to standardise readings and enhance reproducibility and precision of AST reports. Other examples for reading difficulties are chromogenic compounds such as nitrofurantoin that appears as a yellow coloring of the agar hampering precise inhibition zone measurements. The size of the nitrofurantoin inhibition zone tends to be underestimated by the unaided eye and measurement variations are comparably high, frequently resulting in non-fulfilled quality control criteria (Table 4). Fully automated Sirscan readings solved these problems and resulted in low measurement variation along with zone diameters that were in agreement with EUCAST quality control criteria. Manual measurements of amikacin diameters in S. aureus ATCC 29213 and ertapenem diameters in E. coli ATCC 25922 tended to be higher than the quality control range. With fully automated Sirscan readings all measurements were in agreement with EUCAST quality control criteria. These examples illustrate the utility of fully automated zone diameter readings to enhance reproducibility and precision of the Kirby-Bauer method.

Conclusions

Fully automated readings proved to be a useful tool to automate and standardise disk diffusion measurements improving the quality and reproducibility of AST reports. This is of particular interest in the light of decreasing and/or abandoning intermediate zones by EUCAST or CLSI and the associated need of more precise measurements to avoid interpretation errors.