6

Test Comparisons and Recommendations

This chapter addresses the third item in the Statement of Task:

Among the hearing tests discussed in task 2 of the Statement of Task, identify those with characteristics most similar to the Hearing in Noise Test (HINT), determine which tests, performed in the sound field, either binaurally or monaurally, in either quiet or noise, produce measurements most closely analogous to the word recognition score of the HINT (given HINT testing parameters of properly functioning cochlear implants set at normal settings, with no visual testing cues, in a quiet sound field, at 60 dB HL), and describe to the degree possible:

1. What differences exist between the identified tests and the HINT in terms of the specific elements of hearing ability they measure;
2. The committee’s recommendations as to how scores from the identified tests can be compared or converted to equivalent scores on the HINT; and
3. The committee’s recommendations for the scores on hearing tests that correspond to a level of functional hearing ability that causes marked and severe functional limitation; whether those scores or outcome measures can be expressed in a form comparable between hearing tests such as percentile or standard deviation from the norm.

Nearly 50 years ago, speech in noise measurement was recommended by Carhart and Tillman (1970) as an ecologically valid measure of

communication. Since then, speech-in-noise measures have become the norm for several aspects of audiologic evaluation including the assessment of cochlear implantation. However, there remains a paucity of speech-in-noise assessments and accompanying information on the validity and reliability of materials and comparisons across materials. Given that complete information is lacking, the committee will address the third task in the Statement of Task to the degree possible, and the committee will use its expertise and judgment to make its recommendations to the U.S. Social Security Administration (SSA).

There are relatively few speech-in-noise assessments in the literature. Some of the most common sentence-in-noise tests are the HINT (Nilsson et al., 1994), the Quick Speech-in-Noise Test (QuickSIN; Killion et al., 2004), and the Bamford-Kowal-Bench Speech-in-Noise Test (BKB-SIN; Etymōtic Research, 2005; Niquette et al., 2003). Chapter 4 provides a discussion of each of those tests.

TEST COMPARISONS

McArdle et al. (2005) compared speech tests that use digits (the digit triplet test), words (the Words in Noise test, also known as the WIN), and sentences (the QuickSIN) and found the same separation of about 8 decibels (dB) between mean recognition performances by listeners with normal hearing and those with hearing loss across tests. Additionally, the authors found that the QuickSIN and the WIN produced recognition performances by listeners with hearing loss that were equivalent (about 12 dB signal-to-noise ratio [SNR]). The data from the McArdle and colleagues study suggest that the WIN and the QuickSIN provide comparable speech recognition performance in terms of the 50 percent point.

The HINT employs a different procedure to measure the 50 percent point, using an adaptive protocol in which the 50 percent point is measured by averaging the peaks and valleys of the individual psychometric tracks. Psychometric tracks refer to the data as plotted with the level of the signal on the y-axis and the presentation number on the x-axis (Wilson et al., 2007). The HINT is also scored by averaging all data points from the 4th through the 11th data point. Furthermore, the HINT uses noise that mimics the speech spectrum, whereas the other speech tests in noise use babble as background noise (Wilson et al., 2007). More normative data exist for speech recognition test materials used in quiet than for speech recognition testing in noise (Wilson et al., 2007).

Wilson et al. (2007) concluded that for both listeners with normal hearing and listeners with hearing loss, better performance was obtained on the BKB-SIN and HINT materials than on the QuickSIN and the WIN materials, although the QuickSIN and the WIN materials provided more

separation in terms of a difference in performance between listeners with normal hearing and listeners with hearing loss. Wilson et al. (2007) recommend QuickSIN as the sentence test of choice and WIN for monosyllable tests. They also note that because BKB-SIN and HINT materials are easier, they should be used for protocols with young children and for individuals with severe hearing loss, such as cochlear implant candidates.

Cullington and Aidi (2017) evaluated the digit triplet test as a way to assess speech in adults with cochlear implants. They note that although sentence tests in quiet have high face validity, they may not be appropriate for all people for the following reasons:

• They can be more a test of an individual’s ability to parse out context cues in a sentence;
• Language barriers can be a problem, resulting in people obtaining floor scores; and
• Performance with a cochlear implant has improved over the years, resulting in 49–71 percent of those taking the test approaching or at the ceiling on sentence testing in quiet (Firszt et al., 2004; Gifford et al., 2008), with 28 percent scoring 100 percent (Gifford et al., 2008). Near-ceiling scores can be achieved by adults with post-lingual deafness after as little as 3 months of implant use (Litovsky et al., 2006). Ceiling scores are a challenge because improvements in performance over time or differences between two conditions cannot be evaluated.

A recent survey of cochlear implant clinical practice found that 100 percent of responding clinics used AzBio sentences testing and 68 percent used speech-in-noise testing to determine cochlear implant candidacy. There were no consistent SNRs¹ used for the assessment; 26 percent of clinics reported using +10 dB, 16 percent used +5 dB, and the remaining 55 percent reported using some combination of +5 and +10 dB SNR (Carlson et al., 2017).

Holder et al. (2018) tested 81 adults with normal hearing and cognitive function in a cochlear implant program, stratified by age (20–49, 50–59, 60–69, and 70–79 years). The study’s purpose was to characterize speech recognition in noise for adults with normal hearing and cognitive function across the four age groups for the Arizona Biomedical (AzBio) Sentences Test, BKB-SIN, and QuickSIN. The authors hypothesized that many clinicians who are evaluating older adults for cochlear implantation prefer to not use the speech-in-noise testing for that evaluation because they believe

___________________

¹ The signal-to-noise ratio compares the level of a desired signal to the level of background noise.

that even older adults with normal hearing do not score well, so a poor score on that test for older adults does not necessarily indicate the need for cochlear implantation. One purpose of their study was to test that theory. Subjects completed AzBio Sentences Tests in quiet and in five SNRs (+10, +5, 0, –5, and –10 dB), as well as BKB-SIN and QuickSIN tasks. The study yielded four main findings:

1. Older adults with normal hearing exhibited ceiling-level performance for AzBio sentence recognition for all SNRs tested except for –5 and –10 dB SNR;
2. BKB-SIN data obtained in the sound field at 60 A-weighted dB (dB A) are in agreement with previously published insert earphone norms, but only for the youngest age group; all older age groups (50+ years) performed more than one standard deviation outside the published insert earphone normative data;
3. QuickSIN data obtained in the sound field were not significantly different for 60 and 70 dB A presentation levels, and the study’s data replicated previously published normative data for this task; and
4. There was a significant effect of age on performance for all tests.

Based on the premise that ceiling effects are more common in the HINT than in AzBio in adults with cochlear implants, Massa and Ruckenstein (2014) conducted a retrospective review to compare 130 cochlear implant users assessed with the HINT against 125 cochlear implants users assessed with AzBio. Their results did not support the hypothesis that HINT testing would reach a performance plateau sooner. The authors found that 34 devices reached a plateau on the HINT and 30 devices reached a plateau on AzBio. Patients reached a plateau in hearing performance at similar median times using AzBio and the HINT (18.8 weeks post-operatively for the HINT, and 16.5 weeks post-operatively with AzBio) (Massa and Ruckenstein, 2014).

ALTERNATIVE MEASURES FOR THE HEARING IN NOISE TEST EQUIVALENCE

Given the historical use of the HINT sentences for measuring cochlear implant candidacy, post-operative outcomes, and SSA disability determination, it is reasonable to ask which speech recognition measures and conversion factors might be equivalent to the HINT. However, to identify validated speech recognition measures and associated conversion for HINT equivalence, one would require within- and between-subjects comparison data on a number of speech recognition measures for hundreds if not

thousands of listeners. Those studies would need to demonstrate a direct comparison between HINT sentence recognition in quiet and speech recognition performance on other measures for large sample sizes in a prospective manner. Unfortunately, a body of literature meeting those criteria does not currently exist. The committee did identify one retrospective review that can inform the question, although it does not meet the aforementioned criteria required to generate a conversion value for the HINT. Gifford et al. (2008) reported speech recognition performance for 206 adult listeners with cochlear implants (n = 156) or hearing aids (n = 50) on a number of measures, including HINT sentences in quiet, consonant–nucleus–consonant (CNC) monosyllables, AzBio sentences in quiet, and BKB-SIN. When individual speech recognition performance was considered for each measure independently, the skewness of the data—an estimate of distribution asymmetry—was –1.44 for HINT sentence scores, an estimate that was more than double the skewness for all other measures tested. That is a direct result of the large proportion of listeners (28 percent) who achieved a perfect score for HINT sentences recognition in quiet as compared with the other three measures, for which < 1 percent achieved ceiling-level performance (Gifford et al., 2008).

The authors also reported the relationship between HINT sentence scores in quiet and CNC word recognition (n = 135) and BKB-SIN (n = 169) using a within-subjects, repeated-measures design. They reported that while there was a statistically significant correlation between scores obtained for HINT sentences in quiet and CNC words (r = 0.79) and BKB-SIN (r = –0.77), there was no reliable relationship between the measures that offered clinical or diagnostic utility. For example, considering the current SSA cut-off of 60 percent HINT sentences in quiet, individuals scoring within this range achieved scores between 8 and 53 percent for CNC words and between 9 and 21 dB SNR for BKB-SIN (Gifford et al., 2008). Thus, the committee is unable to accurately identify a conversion factor between HINT sentence recognition in quiet and other validated measures of speech recognition performance for at least two primary reasons: (1) there are no published datasets with large samples collected prospectively describing speech recognition scores across a number of measures, including HINT in quiet, and (2) the characteristics of the HINT sentences and the resultant non-normal, skewed distribution of scores in quiet limits the committee’s ability to generate a reliable conversion value.

SUMMARY AND RECOMMENDATIONS

As described in Chapter 2, any measure of sentence recognition may be affected by talker rate, talker gender, enunciation, language complexity, a listener’s native language, and various neurocognitive processes. Indeed, each

measure of speech perception has its own set of strengths and weaknesses as it tests a diverse set of auditory, cognitive, and neurolinguistic abilities. It is for this reason that most professionals believe in the use of multiple measures of speech perception to fully capture the auditory and communication profile of the listener. The fields of audiology and otology have been moving toward the use of monosyllabic word recognition both for measuring cochlear implant candidacy and for tracking post-operative outcomes. In fact, a number of recent U.S. Food and Drug Administration (FDA)-approved clinical trials of new cochlear implant systems have used CNC monosyllabic word recognition as the primary measure for determining candidacy, with preoperative criteria ranging from 40 percent (Sladen et al., 2017; Wick et al., 2020; National Clinical Trial [NCT] 01337076; NCT 03236909) up to 60 percent correct (Roland et al., 2016; NCT 00678899; NCT 00747435). The use of monosyllabic words is already considered standard of care for speech audiometry in audiology clinics and is readily available. Additionally, within the context of determining candidacy for an intervention or for disability status, word recognition tasks will not penalize a listener for being able to “fill in the gaps” using higher level communication repair strategies. That is, word recognition tasks more accurately reflect the transmission of acoustic speech cues through the impaired auditory system.

The committee was tasked with recommending how scores from the identified tests can be compared with or converted to equivalent scores on the HINT. However, given the committee’s concerns with the utility of the HINT and the limitations expressed in Chapter 3, such as ceiling effects and lack of availability, deriving equivalent scores on the HINT would produce scores with limited interpretation. Additionally, while it may be of value to have a common metric or a conversion equivalent in the presence of newer tests, the task is complicated by a lack of large research studies with head-to-head comparisons of the HINT with other tests. Methods developed in the context of psychologic testing, which can be generalized to any type of measurement, indicate the need for rigorous study designs to measure test scores on the same individuals in a single group (Dorans, 2007). It is also possible to link instruments that have not been administered to the same set of individuals, but the methods produce weaker data and require stronger assumptions. In summary, the lack of rigorous research studies prohibits completion of this task.

As noted in previous chapters, the current use of the HINT sentences as criteria for cochlear implantation may suffer from ceiling effects of the test materials, given modern technology and the lack of availability of test materials. Research may support an update of assessment of cochlear implementation via different materials. Speech assessment via sentences fundamentally differs from assessment via words because sentences offer context to the information, and context may result in improved scores in

speech understanding. In Chapter 4 the committee made the following recommendation based on those factors and others, including the difficulty of obtaining the HINT, the shift in the cochlear implant community toward using word tests, and the fact that SSA already uses word tests for individuals with hearing loss who do not have a cochlear implant:

Examples of such tests, as of this writing, include the CNC words or the Northwestern University Test No. 6 for adults and the Phonetically Balanced Kindergarten or Lexical Neighborhood Test for children.

The Statement of Task also requests the committee “to identify and recommend generalized testing procedures and criteria for evaluating the level of functional hearing ability needed to make a disability determination in adults and children after cochlear implantation.” Based on the information detailed in Chapters 2 through 5 and the committee’s professional judgment:

Finally, the Statement of Task asks the committee

The committee suggests that SSA use the same cut-off criteria for evaluating hearing loss in individuals with cochlear implants as the current Listing for hearing loss in individuals without cochlear implants. That cut-off aligns with the criteria used in the most recent FDA clinical trials for cochlear implants. Specifically, the FDA trials use a cut-off score of 40 percent correct or less in the ear to be implanted and 50 percent correct or less in the contralateral ear on a recorded monosyllabic word test presented at 60 dB sound pressure level, with A-weighting.

The committee believes that should an individual with a cochlear implant continue to meet the criteria for cochlear implantation after they have been implanted with their device, they clearly have demonstrated that the cochlear implant has not provided significant benefit. As such, it is highly likely that the cochlear implant recipient has a disability related to hearing loss.

As mentioned, no single test can fully capture the broad neurological faculties that allow for speech and language understanding. Given that speech tests do not capture all of hearing function (see the section titled Considerations Beyond Auditory Testing in Chapter 4), the committee also notes that additional information from self-report or parent-report questionnaires may be useful for better characterizing an individual’s real-world communicative functioning. Examples of such questionnaires, as of this writing, include the Cochlear Implant Quality of Life; the Cochlear Implant Function Index; the Nijmegen Cochlear Implant Questionnaire; the Hearing Handicap Inventory for Adults/Elderly; the Communication Profile for the Hearing Impaired; the Abbreviated Profile of Hearing Aid Benefit; the Speech, Spatial, and Qualities of Hearing Scale; the LittlEARS Auditory Questionnaire; the Meaningful Auditory Integration Scale (standard and the infant-toddler version); the Parents’ Evaluation of Aural/Oral Performance of Children; and the Auditory Skills Checklist.

The recommendations of this committee were made based on the state of knowledge available to committee members at the time of writing. As

advances in clinical practice, assessment measures, and hearing technology emerge, it is possible that a better measure for assessing significant disability will become available. Therefore, should more information become known in the future, it may be necessary to revisit the above recommendations.

REFERENCES

Sladen, D. P., R. H. Gifford, D. Haynes, D. Kelsall, A. Benson, K. Lewis, T. Zwolan, Q. J. Fu, B. Gantz, J. Gilden, B. Westerberg, C. Gustin, L. O’Neil, and C. L. Driscoll. 2017. Evaluation of a revised indication for determining adult cochlear implant candidacy. Laryngoscope 127(10):2368–2374.

Wick, C. C., M. J. Butler, L. H. Yeager, D. Kallogjeri, N. Durakovic, J. L. McJunkin, M. A. Shew, J. A. Herzog, and C. A. Buchman. 2020. Cochlear implant outcomes following vestibular schwannoma resection: Systematic review. Otology & Neurotology 41(9):1190–1197.

Wilson, R. H., R. A. McArdle, and S. L. Smith. 2007. An evaluation of the BKB-SIN, HINT, QuickSIN, and WIN materials on listeners with normal hearing and listeners with hearing loss. Journal of Speech, Language, and Hearing Research 50(4):844–856.