EVALUATION OF TEST-RETEST RELIABILITY OF AUTONRT MEASUREMENTS IN COCHLEAR IMPLANT RECIPIENTS
2Department of Audiology, Faculty of Health Sciences, Istanbul Aydin University, Istanbul, Türkiye
3Department of Audiology, Faculty of Medicine, Marmara University, Istanbul, Türkiye
Summary
Objective: Establishing the test-retest measurement reliability of Evoked Compound Action Potential (ECAP) thresholds using the AutoNRT system.Materials and methods: 50 CI users, aged 3 to 41 years (Mean: 7 years), with normal cochlear anatomy and at least six months of implant use included in the study. Except for one participant, others were unilateral CI users. ECAP thresholds were measured using the AutoNRT system, conducted twice with a 10-min. interval between measurements. Reliability was assessed by comparing AutoNRT threshold measurements across two consecutive sessions on an electrode basis. Consistencies between electrode measurements were calculated using Spearman Rho correlation coefficients and Intraclass Correlation Coefficients (ICC). Average deviations between electrodes were examined by calculating Bland–Altman Consistency Limits (Upper/Lower Consistency Limits — LOA) with a 95% confidence interval.
Results: Nearly all electrodes demonstrated a high level of measurement reliability (ICC > 0.80). Electrodes E19 (ICC = 0.96), E20 (ICC = 0.97), and E8 (ICC = 0.94) showed the highest consistency. The small Mean Bias values and narrow LOA ranges for these electrodes indicate strong agreement between measurements. General trend indicates that most systematic differences were small, and their confidence intervals typically included zero.
Conclusion: AutoNRT can be utilized effectively and reliably in clinical settings. These findings suggest that AutoNRT may provide clinicians with a rapid and objective tool for cochlear implant programming, particularly in populations where behavioral responses are limited or unreliable.
Introduction
The cochlear implant system has revolutionized the treatment of severe to profound hearing loss by providing electrical stimulation to the auditory nerve, effectively bypassing the non-functional inner ear. This technology has been widely adopted, especially for individuals who do not benefit from traditional hearing aids, making it an essential tool for restoring hearing in both children and adults.[1] The cochlear implant system has the ability to measure the Evoked Compound Action Potential (ECAP) resulting from electrical stimulation of the spiral ganglion neurons. This capability is essential for monitoring the implant's functionality and the auditory nerve's integrity, both intraoperatively and postoperatively. It provides valuable information about the electrode-nerve interface and the status of the auditory nerve, which can be used to guide the programming of speech processors and ensure optimal outcomes for cochlear implant recipients.[2-4]The ECAP is recorded as a negative peak (N1) at about 0.2-0.4 ms following stimulus onset, followed by a much smaller positive peak or plateau (P2) occurring at about 0.6-0.8 ms.[5,6] The ECAP is recorded via the intracochlear electrodes of the implant. The electrical pulse is delivered to an intracochlear electrode and the neural response is recorded at a neighbouring electrode. The measured responses were transmitted via radio frequency back to the system's programming interface for clinical analysis. NRT (Neural Response Telemetry) is a technology specifically developed for Cochlear brand cochlear implants to perform ECAP measurements. It enables the rapid and non-invasive measurement of auditory nerve responses during or after surgery.[7]
The cochlear implant system also has an automated system to determine this threshold level, known as AutoNRT. This system has been designed to automatically detect and measure ECAP thresholds, thereby reducing the time required for manual measurements and minimizing potential inconsistencies associated with clinician subjectivity.[8,9] The AutoNRT algorithm increases stimulation intensity from the low current level using regular step sizes until a reliable ECAP response is found on two consecutive traces. After detecting two consecutive NRT responses, the direction of the step size is reversed and decreased to step down until a reliable nonresponse is found. The arithmetic mean of the current levels of the lowest reliable response and the highest reliable nonresponse is used as the threshold estimator.[8,9] AutoNRT provides a completely automated method for measuring ECAP thresholds in both intraoperative and postoperative settings.[8] The detailed information about the AutoNRT algorithm can be found in Botros et al., 2007.[8] The accuracy of measurements including test-retest measurement reliability has been partially reported in the literature.[10-12] In addition, there is an incomplete understanding of whether etiology or electrode type affects this measurement reliability. This investigation therefore aims to establish the test-retest measurement reliability of the ECAP threshold and whether the reliability is correlated to demographic or device factors.
Several studies have demonstrated the clinical utility of AutoNRT in both intraoperative and postoperative settings. For instance, Botros et al.[8] showed that AutoNRT could accurately measure ECAP thresholds, making it a reliable tool for monitoring cochlear implant function. In a separate study, van Dijk et al.[9] reported that AutoNRT has a high level of agreement with manually obtained thresholds, further validating its use as an objective and time-saving method for ECAP threshold measurement. Moreover, the automated nature of AutoNRT has proven advantageous in clinical settings, where it has streamlined the process of cochlear implant programming and facilitated the objective fitting of speech processors.[2]
Despite its broad applicability, the accuracy of test-retest measurements using AutoNRT has only been partially reported in the literature. A study by Brown et al.[2] found that ECAP measurements exhibit a certain degree of variability, but further investigation into the factors affecting this variability is needed. Understanding the factors that contribute to test-retest reliability is essential for ensuring consistent and accurate programming of cochlear implants, which ultimately affects patient outcomes.[13] However, there is still limited evidence regarding electrode-based reliability patterns and the potential influence of demographic and device-related factors. Therefore, this study aims to establish the test–retest reliability of ECAP thresholds using the AutoNRT system, with a specific focus on electrode-level analysis to enhance its clinical applicability.
Methods
50 patients who had previously received a Nucleus CI24RE, CI422, or CI512 cochlear implant for the treatment of severe to profound hearing loss were included in the study. One subject was bilateral and 49 subjects were unilateral CI users. The age range of the patients was 3 to 41 years. They had all been using their cochlear implants for at least 6 months. The demographic factors were summarized in Table 1.){kind=link}
Table 1: Summary of Demographic Factors.
Ethical Approval
Ethical approval for this study was obtained from the Clinical Research Ethics Committee of the Faculty of Medicine at X University on November 04, 2016 (Protocol No: 09.2016.576). All procedures were conducted in accordance with the ethical standards of the institutional and/or national research committee and the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.
Procedure
Nucleus® Cochlear Implant system (Cochlear Ltd., Sydney, Australia) was used for this study. This cochlear implant system provides both auditory nerve stimulation and the ability to measure Evoked Compound Action Potentials (ECAPs) through its integrated electrode array and programming interface. The AutoNRT™ system (Cochlear Ltd., Sydney, Australia), an automated ECAP measurement tool, was used for this purpose. This system is integrated into the Custom Sound EP v3.2 clinical programming software, specifically designed to detect and measure ECAP thresholds efficiently. AutoNRT thresholds are measured postoperatively on all operational electrodes twice, with 10 minutes between measurements. All subjects were tested with CP810 speech processor while the subjects attended the clinic for their routine speech processor fitting. Additionally, all measurements were obtained under controlled clinical conditions using the same device and protocol, which may limit the generalizability of the findings to different clinical settings or implant systems.
Statistical Analysis
Before beginning the statistical analysis of the study, the dataset was examined for outliers and missing values. Missing data analysis was performed for the missing values in the dataset, and since the losses were determined to be random (p>.05), imputations were made based on the mean values. IBM SPSS 25 software and MedCalc 23.3.7 software were used in the analysis of the data. Consistencies between electrode measurements were calculated using Spearman Rho correlation coefficients and Intraclass Correlation Coefficients (ICC). The ICC was calculated using a two-way mixed-effects model with absolute agreement definition, which is appropriate for test–retest reliability analysis. Average deviations between electrodes were examined by calculating Bland–Altman Consistency Limits (Upper/Lower Consistency Limits — LOA) with a 95% confidence interval.
Results
Table 2 presents the mean differences (Mean Bias), Bland–Altman Limits of Agreement (Upper/Lower LOA), and Intraclass Correlation Coefficients (ICC) with their corresponding 95% confidence intervals (CI) for each electrode (E1–E22). The table summarizes the systematic bias between two repeated measurements and their relative consistency.){kind=link}
The Mean Bias column indicates the average difference between the two measurements (positive values indicate that the second measurement was higher; negative values indicate that the first measurement was higher). The Upper and Lower LOA (and their 95% CIs), calculated using the Bland–Altman method, provide the expected range of random differences between the two measurements. These LOA values can be compared to clinically acceptable limits to evaluate the practical significance of agreement. The ICC column represents the relative reliability of repeated measures, ranging from 0 to 1. A high ICC value (e.g., ≥ 0.75–0.80) indicates good/strong reliability; values between 0.50–0.75 reflect moderate reliability; and values below 0.50 suggest poor reliability. Each ICC is presented with its 95% CI; narrow intervals indicate more precise estimates, whereas wider intervals reflect greater uncertainty.
Overall, nearly all electrodes demonstrated a high level of measurement reliability (ICC > 0.80). Electrodes E19 (ICC = 0.96), E20 (ICC = 0.97), and E8 (ICC = 0.94) showed the highest consistency. The small Mean Bias values and narrow LOA ranges indicate strong agreement between repeated measurements. Although some electrodes (e.g., E3 and E14) showed relatively larger negative differences, the overall pattern demonstrates minimal systematic variation, supporting the high consistency and reliability of the measurements. In conclusion, electrode-based analyses confirmed a generally high level of reliability across measurements.
Additionally, correlation coefficients between the repeated measurements were calculated and are illustrated in Figure A. The distribution of mean values across both measurements for all electrodes is presented in Figure B, and the electrodes are ranked according to their ICC values in Figure C.
 Büyütmek İçin Tıklayın |
Figure A: Correlation Coefficients Between Two Measurements. |
Figure A displays the correlation coefficients between the two measurements obtained for each electrode (E1–E22). The correlation values ranged from 0.518 to 0.945, all positive and statistically significant at p < .001. This indicates a strong and consistent relationship between the two measurements at each electrode position, demonstrating high repeatability and reliability.
 Büyütmek İçin Tıklayın |
Figure B: Mean Values Obtained from the Two Measurement Sessions. |
Figure B shows scatterplots of the mean values obtained from the two measurement sessions for each electrode. The regression analysis yielded the equation y = 0.9 + 0.99x with a coefficient of determination R² = 0.93, indicating a strong linear relationship and high consistency between measurements. The close clustering of points around the regression line suggests no systematic bias and supports the high repeatability of the measurement system.
 Büyütmek İçin Tıklayın |
Figure C: Intraclass Correlation Coefficients (ICC). |
Figure C presents the ranking of electrodes according to their ICC values. Electrodes such as E20, E19, and E8 showed the highest reliability, while lower-ranking electrodes (e.g., E10, E9) demonstrated relatively lower consistency. However, all ICC values were above 0.75, indicating that most electrodes exhibited good to excellent reliability.
Discussion
AutoNRT automatically measures ECAP thresholds both intraoperatively and postoperatively. The comparison of two AutoNRT measurements with a 10-minute discrepancy did not indicate significant differences. The substantial correlations between the two metrics yielded significant results. AutoNRT can be utilized effectively and reliably in clinical settings. These findings suggest that AutoNRT may provide clinicians with a rapid and objective tool for cochlear implant programming, particularly in populations where behavioral responses are limited or unreliable.These findings were consistent with earlier research, such as Botros et al.[8], who first introduced the AutoNRT system and demonstrated its accuracy in measuring ECAP thresholds, reporting that the system performed on par with human experts in terms of accuracy.
Furthermore, our study aligned with the work of van Dijk et al.[9], who showed that AutoNRT offered a high level of reliability and could be effectively used in clinical practice to streamline cochlear implant fitting and programming. Similarly, the study by Abbas et al.[5] emphasized the importance of ECAP measurements in providing objective information about neural response, supporting the utility of AutoNRT in facilitating such measurements. This consistency across studies underscored AutoNRT's role as a valuable tool in objectively fitting cochlear implant systems, especially for populations that could not reliably participate in behavioral testing.
One of the critical benefits of AutoNRT was its applicability to populations that could not perform behavioral testing, such as young children or individuals with cognitive impairments. As demonstrated in earlier studies[6,14], the ability to obtain objective measures of neural response was crucial in programming cochlear implants for these groups. AutoNRT provided an effective and efficient means of achieving this, which led to more accurate fitting and potentially improved auditory outcomes.
While AutoNRT demonstrated high reliability, certain limitations and areas for future research were noted. Relatively small sample size may have limited the generalizability of these findings. Studies with larger and more diverse populations would help validate these results and provide further insight into the factors influencing measurement reliability. Moreover, other studies, such as Botros[8], emphasized that different cochlear implant systems exhibit variability in ECAP measurement reliability and sensitivity, reinforcing the need for comparative studies to assess the performance of AutoNRT against other automated systems. Similarly, Abbas et al.[5] reported differences in ECAP outcomes across implant devices, and Brown et al.[15] highlighted the necessity of validating the efficacy of automated ECAP measurement systems in clinical practice. Last limitation for this study, is the relatively short test–retest interval (10 minutes). Although this design minimizes physiological variability, it may overestimate reliability compared to longer intervals typically encountered in clinical follow-up.
Conclusion
In conclusion, AutoNRT can be utilised effectively and reliably in clinical settings. Unlike previous studies, the present study provides a detailed electrode-based reliability analysis, highlighting that reliability may vary across electrodes. This information may contribute to more individualized and precise mapping strategies in clinical practice.Reference
1) Clark G. Cochlear Implants: Fundamentals and Applications. Springer-Verlag; 2003.
2) Brown CJ, Hughes ML, Luk B, Abbas PJ, Gervais JP, Wolaver AA. The relationship between EAP and EABR thresholds and levels used to program the nucleus 24 speech processor: data from adults. Ear Hear 2000;21:151–9. doi:10.1097/00003446-200004000-00009. [ Özet ]
3) Smoorenburg GF. Predicting the perceptual threshold of electrical stimulation in cochlear implants. Acta Otolaryngol 2002;122(4):10–4.
4) Thai-Van H, Chanal JM, Coudert C, Veuillet E, Berger-Vachon C. Relationship between NRT measurements and behavioral levels in children with the Nucleus 24 cochlear implant array over time: preliminary report. Int J Pediatr Otorhinolaryngol 2001;58:153–62. [ Özet ]
5) Abbas PJ, Brown CJ, Shallop JK, Firszt JB, Hughes ML, Hong SH, et al. Summary of results using the Nucleus CI24M implant to record the electrically evoked compound action potential. Ear Hear 1999;20(1):45–59. [ Özet ]
6) Brown CJ, Abbas PJ, Gantz BJ. Preliminary experience with neural response telemetry in the Nucleus CI24M cochlear implant. Am J Otol 1998;19(3):320–7. [ Özet ]
7) Brown CJ, Abbas PJ, Gantz BJ. Electrically evoked whole-nerve action potentials: data from human cochlear implant users. J Acoust Soc Am 1990;88(3):1385-1391. [ Özet ]
8) Botros A, van Dijk B, Killian M. AutoNRT: An automated system that measures ECAP thresholds with the Nucleus Freedom™ cochlear implant via machine intelligence. Artif Intell Med 2007;40(1):15–28. [ Özet ]
9) van Dijk B, Botros AM, Battmer RD, Backus P, Topham C. Clinical results of AutoNRT: A completely automatic ECAP recording system for cochlear implants. Ear Hear 2007;28(5):558–70. [ Özet ]
10) Kiss JZ, Cristobal R, Kawano A, Shepherd RK. Neural response telemetry in cochlear implant users: Its correlation with behavioral thresholds. Tinnitus J 2003;9(2):94-99. [ Özet ]
11) Unsal S, Gunduz B. Comparison of neural response telemetry (NRT) results of cochlear-implanted children in view of preoperative auditory brainstem response (ABR) tests. Tinnitus J 2019;24(2):85-92. [ Özet ]
12) Deniz E, Polat H, Kırkım G. The effect of cochlear implant electrode position on neural response telemetry thresholds. Ann Med Res 2019;26(11):2557-2561.
13) Undurraga JA, van Wieringen A, Carlyon RP, Macherey O, Wouters J. Polarity effects on neural responses of the electrically stimulated auditory nerve at different cochlear sites. Hear Res 2010;269(1–2):146–61. doi:10.1016/j.heares.2010.06.017. [ Özet ]
14) Franck KH. A model of a Nucleus 24 cochlear implant fitting protocol based on the electrically evoked whole nerve action potential. Ear Hear 2002;23:67S–71S. [ Özet ]
15) Brown CJ, Abbas PJ, Etler CP, O'Brien S, Oleson JJ. Effects of long-term use of a cochlear implant on the electrically evoked compound action potential. J Am Acad Audiol 2010;21(1):5–15. doi:10.3766/jaaa.21.1.2. [ Özet ]