PDF
Citation
Print
INTRODUCTION
Hearing impairments were suffered by 360 million people worldwide in 2012, comprising 328 million adults and 32 million children [1]. These numbers have been steadily increasing as the life expectancy increased with advancements in medicine and technology [2]. However, despite the overwhelming number of hearing-impaired people, market surveillance data show that only 1 in 5 adults have adopted hearing aids, and many hearing-impaired persons still hesitate to purchase hearing aids due to the high-cost and low satisfaction in various noisy situations [3].
When purchasing hearing aids, many people are unsatisfied with the settings prescribed by their audiologists because adjustments made in a quiet clinic environment are often not optimal for real-life use [4-6]. People new to hearing aids may not be aware of what an optimal setting is for them, which would affect the effectiveness of the initial adjustment at the hospital [7]. In such cases, the device user has no choice but to revisit the hospital to readjust the device settings as many times as needed until satisfied with the tuning. However, if there is no relevant hospital or tuning clinic near the user’s home or office, repeated visits can incur large time and financial burdens [8]. In addition, due to differences in personal preferences, a setting suitable for one environmental situation may not be suitable for others [5]. Some people prefer to have the directional microphone function enabled when listening to speech sounds in noisy environments, but disabled when listening to music in a music hall or speech sounds in a quiet room. Others prefer to leave the function enabled all the time, regardless of the ambient situation. Additionally, many prefer to adjust the level of noise suppression depending on the variations in the ambient noise, whereas others do not [4]. Further, it has been reported that audiologist-driven settings and patient-driven settings can be quite different [9,10]. For these reasons, a need exists for a hearing-support device that can auto-tune its internal parameters according to the user’s personal preferences under various ambient situations, and to meet this need, a trainable or self-fitting hearing aid was introduced [7].
In the initial stage of research on trainable hearing aids, the main focus of training was only the relationship between the intensity of ambient sounds (input sound level [ISL]) and the output volume of the device (amplification constant [AMP]) [7,9,11,12]. Later research revealed that many other factors besides the ISL affects human perception, such as listening situation (LS), type of ambient sound, and internal device factors such as the degree of noise suppression (DNS) [7]. For example, Chalupper et al. [4] demonstrated that the settings of a hearing aid, such as overall gain, compression parameters, noise-reduction strength, and activation/deactivation of the directional microphone, may be sensitive to personal preferences. However, even the recent studies on trainable hearing aids have reflected only one or at most two of these environmental factors into the device training [4,6,13,14], which limited the user satisfaction under various acoustic conditions. In other words, personal satisfaction of the user would be greatly improved if the device is able to reflect the owner’s personal preference in a more expanded environmental acoustic conditions, since the auditory perception is affected by surrounding complex acoustic conditions.
In this study, we propose a trainable hearing aid algorithm that reflects the user’s personal preferences in three environmental acoustic conditions (ISL, LS, and DNS), and evaluate the performance of the proposed algorithm using 10 normal-hearing subjects.
MATERIALS AND METHODS
Algorithm implementation scenario
The scenario of the implementation of the proposed algorithm is as follows. Normally, for everyday or first-time use, the user sets the device to normal-operation mode, in which the device adjusts the internal parameters automatically while reflecting the user’s preference. Upon encountering an uncomfortable or unoptimized sound in a new environment, the user can train the device by selecting one or both of the two training modes, volume-training and β-training. In volume-training mode, the user can make manual adjustments to the overall output volume via the user interface to add or subtract a certain offset (in 1 dB steps) from the current volume to select an amplification (AMP, dB) value. In β-training mode, the user can select a β value of the beamforming algorithm that fits his or her personal preference. Once training is complete for a specific environment, the user can store the preferred internal parameters selected during training–the outputs of the LS classifier and the ISL detector, which describes the specific environment training was performed in, and the user-selected AMP and β values which are the values the user selected during training–in the database. The algorithm utilizes data gathered during training to better auto-tune to the user’s preference. If training is performed in an environment that was trained before (i.e., for the same LS and ISL), the most recent adjustment overrides the previous data. The algorithm in normal-operation mode uses the user-trained database and predetermined interpolation and extrapolation protocols to automatically adjust the AMP, β, and activation or deactivation of beamforming in real-time.
Structure and operation of the implemented algorithm
Fig. 1 shows the overall schematic of the proposed algorithm. It consists of three sub-blocks: a signal interface block, a user interface block, and a database manager block. The following assumptions were made for simplification: (1) The device contains two user-adjustable algorithms—beamforming, which adjusts the DNS using the β value, and an output volume adjustment, which adjusts the total output volume of the device using AMP; (2) The wide dynamic-range compression is always active and its channel gains are fixed at purchase; (3) Beamforming is enabled only in speech-in-noise situations and disabled in speech-only and noise-only situations; and (4) The ISL is limited to a 45–90 dB of sound pressure level (dB SPL) range, considering the normal, everyday use presented by Chalupper et al. [4] and Walden et al. [15]. In this study, we used the differential microphone array algorithm suggested by Teutsch and Elko [16] as the beamforming, and the neural network-based LS classifier suggested by Yook et al. [17] that was slightly modified as follows: (1) Additional amplification in front of the classifier to ensure the proper ISL for stable classification performance and (2) Among the four situations classified in the original LS classifier, only three situations—speech-only, noise-only, and speech-in-noise—were enabled (i.e., music-only was disabled).
When external sounds enter the embedded microphones, the average ISL of the most recent 2.5 seconds (which was determined to be the most optimal with respect to accuracy and ability to operate in real-time through repetitive trials) is measured by the ISL detector, and at the same time, the type of ambient situation is determined by the listening situation classifier. When the device is turned on (i.e., proposed algorithm is initiated), initial operation mode is set to normal-operation mode. When the user wants to further train the device, the user needs to change the operation mode to either volume-training or β-training mode. (In this study, we assumed that the user has the means to freely adjust the operation mode, the AMP, and the β; e.g., a three-step slider and a clickable dial on the surface of the device that enables the user to freely adjust the device setting manually during the training phase, a dedicated remote controller connected to the hearing aid wirelessly that allows manual adjustment, or a dedicated computer or smartphone application for this purpose). In volume-training mode, users can adjust the AMP to their preferences in different ambient listening situations in their normal daily lives. When an adjustment is made, the device records the value of the new AMP, the ambient situation detected by the listening situation classifier, and the ambient sound level detected by the ISL detector into the database. In β-training mode, users can select the DNS to their liking by adjusting the β among the three preset values (1.76, 2.65, and 3.47) [18]. As in volume-training mode, the β value is recorded into the database along with the current LS and ISL values. When reverted back to normal-operation mode after training, the overall training-related functions are disabled, and the device automatically adjusts its internal parameters (AMP, beamforming on/off, and β) in accordance with the training data. In order to perform auto-tuning in normal-operation mode, the proposed algorithm monitors the outputs of the ISL detector and LS classifier in real time to search the database for a matching condition. If an exactly-matched condition exists, the stored value of the parameters are used to auto-tune; however, if one does not exist, (1) the AMP is determined by linear interpolation and extrapolation using the neighboring values in the database, and (2) β is determined by selecting the closest value between the two adjacent values in the database. Since parameter outputs are very sensitive to ISL, preventative measures are taken in normal-operation mode to ensure correct parameter outputs are implemented when the listening environment changes abruptly. Specifically, an ISL change of more than 5 dB is considered abrupt, and for such instants, implementation of the auto-tune parameter output is skipped one interval.
Subject recruitment and test sound preparation for evaluation
Ten normal-hearing subjects (7 males and 3 females; age range, 21 to 30 years; median age, 25 years; mean age, 25.5 years) were recruited in accordance with the method approved by the Institutional Review Board of Hanyang University (HYI-15-011-3). Written consents to participate were acquired from all participants before the experiments, and they were compensated for their participation.
The sound files for evaluation were recorded using a combination of 40 clean speech signals, 30 babble noise signals, and 30 car noise signals [19]. Three types of sound files were recorded to simulate the three listening situations: speech-only, noise-only, and speech-in-noise. The recording took place in a nonreverberant chamber located at Hanyang University using a KEMAR mannequin (Type 45BA; G.R.A.S. Sound & Vibration, Holte, Denmark) with two artificial ears (Type KB0060; G.R.A.S. Sound & Vibration) placed in the center of the chamber. Two behind-the-ear type hearing aid mock-ups, shaped identically to the commercialized device (Canta7; GNReSound A/S, Ballerup, Denmark), were mounted on the two artificial ears to record the sound in separate files. Four identical speakers (HS50M; Yamaha Co., Hamamatsu, Japan) were each positioned 120 cm from the mannequin, surrounding it, in the 0°, 90°, 180°, and 270° directions. The four speakers were calibrated before the recording using a commercial sound-level meter (2250 Light; Brüel & Kjær Co., Naerum, Denmark) to make sure the outputs were the same for all the speakers. The originating direction of sound during the recording for each listening situation were as follows: (1) speech-only: clean speech from 0°; (2) noise-only: identical noises from 0°, 90°, 180°, and 270°; and (3) speech-in-noise: clean speech from 0° and identical noises from 90°, 180°, and 270°. The input signal-to-noise ratio of the mixed signals was fixed at 0 dB. For the recordings of each of the three situations, the overall ISL was adjusted from 45 to 90 dB in 5 dB steps. A total of 12 test sets were recorded for algorithm verification (PS_1 and PS_2), subjective training (TS_1), and subjective evaluation (TS_2) as listed below (10 sound clips per set; no duplication or overlap between CS_1–CS_5, BN_1–BN_3, and CN_1–CN_2).
- PS_1 contained one test set consisting of seven 1,000-Hz pure tone signals with 60, 65, 70, 75, 80, 85, and 90 dB SPL.
- PS_2 contained one test set consisting of six 1,000-Hz pure tone signals with 63, 67, 73, 77, 83, and 87 dB SPL.
- TS_1 contained three test sets, each consisting of one clean speech set (CS_1) and one babble noise set (BN_1): one speech-only set (ES_1), one noise-only set (BN_1), and one speech-in-noise set (CS_1+BN_1).
- TS_2 contained seven test sets, each consisting of three clean speech sets (CS_2, CS_3, and CS_4), two babble noise sets (BN_2 and BN_3), and two car noise sets (CN_1 and CN_2): one speech-only set (CS_2), two noise-only sets (BN_2 and CN_1), and two speech-in-noise sets (CS_3+BN_3 and CS_4+CN_2).
Evaluation of the implemented algorithm
Before evaluating the perceptual benefit of the proposed algorithm, we verified the proper operation of the implemented blocks of the algorithm: first, the performance of the ISL detector was evaluated with PS_1; second, the interpolation performance of the database manager was evaluated by training the algorithm with PS_1 in volume-training mode and evaluating the algorithm using PS_2 in normal-operation mode; and lastly, the performance of the LS classifier was evaluated using both TS_1 and TS_2.
Next, we evaluated the performance of the overall algorithm: first, an examiner trained the algorithm with TS_1 and adjusted the AMP and β values to his preferred setting for each test sound (each sound was repeatedly played until the examiner completed training). Then, the algorithm was switched to normal-operation mode, and TS_1 and TS_2 signals were sequentially entered into the signal interface block in random order, and the automatically-adjusted AMP and β values were recorded. Each sound clip was repeated until five parameter outputs were recorded, and then the next clip was played.
After the overall verification process of the algorithm was completed, the Comparison Mean Opinion Score (CMOS) test [20] was performed with 10 subjects to compare the differences in perceptual benefits among the three different parameter settings.
- Fixed setting (T1 setting; representing a traditional hearing aid): fixed amplification constant and β value regardless of the variations in input signals (both the ISL detector and LS classifier were disabled).
- User-trained setting without LS classifier (T2 setting; representing a traditional trainable hearing aid): automatically adapting the user-defined amplification constants and β values to the variations in the input sound level (only the ISL detector was enabled, and the LS classifier was disabled).
- User-trained setting with LS classifier (T3 setting; representing the proposed algorithm): automatically adapting the user-defined amplification constants and β values to the variations in both the input sound level and the listening situation (both the ISL detector and the LS classifier were enabled).
TS_2 sounds were utilized to record test sounds for the CMOS test, and wide dynamic-range compression in Fig. 1 was disabled (bypassed) during recording because the test was performed using normal-hearing subjects only.
To record test sounds for the CMOS test, the algorithm must be trained prior to the recording as the recording would be done with the auto-tuned internal parameters based on the trained data. Each subject was given thorough instructions regarding the experiment before being positioned in the middle of the nonreverberant chamber wearing a headset. Training for T1, T2, and T3 setting differed from each other: for the T1 setting: (1) Training-related algorithms were all disabled; (2) The β value was preset to 3.1 [18]; (3) Each subject listened to a 1,000-Hz pure tone signal through the headset (70 dB SPL) and was asked to manually adjust the AMP to his or her preferred sound level (comfortable level); and (4) The subject-selected AMPs were recorded for all subjects. For the T2 setting: (1) The LS classifier was disabled (and the database manager was also adjusted to accommodate the T2 setting); (2) Each subject was asked to train (i.e., manually adjust the AMPs and β values to his or her preferred setting for each sound) the algorithm using TS_1; and (3) The subject-selected parameters for each sound were recorded for all subjects. For the T3 setting: (1) All algorithms in Fig. 1 were enabled; (2) Each subject was asked to train the algorithm using TS_1; and (3) The subject-selected parameters for each sound were recorded for all subjects. During training, each sound clip was repeated until the subject selected his or her preferred parameter setting. After the training phase, the subject was given a break while the examiner recorded test sounds (output signals of the algorithm in normal-operation mode when TS_2 signals were entered into the algorithm with the recorded parameter settings for each subject) for the CMOS test.
After recording the test sounds, the subject returned to the original position in the chamber and wore the headset again to proceed with the CMOS test. During the CMOS test, identical test sounds with two different settings (either T1 and T3, or T2 and T3) were played sequentially, and the subject was asked to score the relative quality of the second sound (comparison audio) compared to the first sound (reference audio), in the range of –3 to +3. For instance, if the comparison audio sounded much better than the reference audio, a score of 3 would be given or a –2 if the comparison audio was worse than the reference audio. Objective preference between two settings can be measured by comparing identical audio in different settings. The order of test sounds (from TS_2), and the selection of reference and comparison audio (e.g., T1 after T3 or T3 after T1), were all randomized (the total scores of the three parameter settings represent the total quality score of each setting). To minimize the expected learning effects from the duplicated use of test sounds (although it may not be as significant for a quality test compared to an intelligibility test), the order was carefully arranged to prevent same test sound from repeating within 5 trials. In addition, to reduce physical and psychological fatigue of the subject, resting intervals of 5 seconds were given between tests, and an additional resting interval of 2 minutes every 25 trials.
Statistical analysis
In this study, repeated-measures analysis of variance (RM-ANOVA) test was performed to determine whether there was a significant differences in each subject’s training result among the three listening situations: speech-only, noise-only, and speech-in-noise. Post-hoc Tukey honest significant difference test was further applied only when significant difference (P<0.05) was observed.
RESULTS
Table 1 provides the results of the ISL detection test and ISL interpolation test. The implemented ISL detector could detect the intensity of the actual input signals with a 0.2%–4.6% error rate, and in addition, it could also interpolate the volumes of the input signals as intended with a 0.3%–4.2% error rate.
Fig. 2 shows the AMP variation patterns when both training and verification were performed using TS_1. The AMPs in normal-operation mode followed the trained values in all situations with some amount of time delay, and parameter interpolation also operated properly when required. Errors between the trained and adjusted AMPs were 0.01–2.71 dB, 0–4.23 dB, and 0–3.33 dB for speech-only, noise-only, and speech-in-noise respectively.
Fig. 3 shows the AMP variation patterns when training was performed using TS_1 and verification using TS_2. The AMPs in normal-operation mode also followed the trained values in all situations, and interpolation operated properly. Errors between the trained and adjusted AMPs were 0.06–4.37 dB, 0.02–4.99 dB, and 0–11.20 dB for speech-only, noise-only, and speech-in-noise respectively.
Table 2 shows the user-selected AMP and β values during the training phase of the subjective test. RM-ANOVA results showed significant difference existed among the three listening situations for all ISL (in the order from 45 dB to 90 dB, F-value is 8.00, 15.81, 19.99, 14.75, 17.67, 11.58, 9.86, 9.06, 6.14, and 4.70 respectively and P<0.05), and further, post-hoc analysis indicated that, in speech-in-noise conditions, subjects select significantly higher AMP values compared to speech-only and noise-only in all ISL levels except 90 dB where this was the case only in noise-only (P=0.008, Bonferroni adjustment).
Tables 3 and 4 give the results of the CMOS test for T1–T3 and T2–T3, respectively. In both tables, higher score indicate preference of T3 setting over the other setting being compared to. The T3 setting showed higher CMOS scores than the T1 setting for all subjects and listening situations. In contrast, the T2 setting and the T3 setting showed almost equivalent CMOS scores; that is, the preferences were different among the subjects.
DISCUSSION
In the subjective test, all subjects showed significant differences in their AMP settings (P<0.05) in accordance with the variations in the LS condition under the same ISL (usually between noise-only and speech-in-noise). Also, the user-selected AMP values in speech-in-noise were generally higher than those in noise-only and speech-only situations under the same ISL, which implies that the listener prefers to reduce the volume of the device when there is no interesting sound (noise-only) or there is no acoustic interference (speech-only) to reduce the perceptual discomfort from the noise. In contrast, when an interesting sound is mixed with environmental interference, the listener usually prefers to increase the volume of the device to increase the intelligibility of the interesting sound while enduring the perceptual annoyance from the loud interference. This preference, though it was derived from normal-hearing subjects, demonstrates the high potential for LS classification to benefit hearing-impaired subjects. Furthermore, when considering the user-selected β values in speech-in-noise situations under different ISL condition, the participants all preferred different β settings (e.g., subjects 1, 2, 6, 8, 9, and 10 selected 2.65 and 3.47, subject 5 selected 1.76 and 2.65, and subjects 3, 4, and 7 selected 1.76, 2.65, and 3.47) with no apparent pattern. Additionally, participants sometimes preferred different β values as the ISL condition changed, which further demonstrates the necessity of user-adjustable noise-reduction algorithm (adjusting the DNS) for the trainable hearing aid. These results demonstrate that all ISL, LS, and DNS conditions can affect the results of training and as a result, affect the performance (personal satisfaction) of the device in real-life situations, which justifies the incorporation of all of those factors in the device operation.
In the CMOS test, the T3 setting showed significantly improved sound quality over the T1 setting [21]; however, the T3 setting showed no significant difference compared to the T2 setting despite the difference in parameter setting between T2 and T3 settings. These results may be associated with the fact that the LS classifier used in the current study occasionally misclassified noise-only (babble) situations as speech-only situations (Table 2), which may have introduced error to the results. In addition, the output of the LS classifier varied depending on when the user selection was made during the training phase (e.g., an identical babble noise could be classified as noise-only or speech-only, depending on the timing of the user selection), which greatly affected the classification accuracy among the participants. The CMOS test results between T2 and T3 would be improved by enhancing the performance of the LS classifier [5]. For example, looking at only the speech-only situation (which was least affected by the misclassification) in Table 4, T3 showed improved CMOS score (on average) in a majority of the subjects, which indicates the potential benefit of the proposed algorithm. In the current study, the proposed T3 setting resulted in CMOS scores 1.42, 0.60, and 0.51 (on average) higher than the T1 setting, in speech-only, noise-only, and speech-in-noise situations, respectively. The proposed T3 setting also resulted in scores 0.45 and 0.35 higher than the T2 setting, in speech-only and noise-only situations, respectively (scores were lower only in the speech-in-noise situation), which is consistent with the report by Chalupper et al. [4], which suggests that the classifier control training achieved more comfortable loudness ratings (in about 72%–88% of situations) than did the gain control training (in about 68%–82% of situations) on average.
There are three scenarios that may cause abnormal operation of the proposed algorithm in normal-operation mode. First, when the ambient sound level changes abruptly (by more than 5 dB), the output of the ISL detector may not detect the change as quickly because it uses the most recent 2.5 seconds of data to calculate (i.e., the input sound data from the previous track is temporarily dominant over the new one). As a result, the ISL detector may display a sound level with less than 5 dB difference (even though the change in environment was permanent, not temporary), and hence the algorithm assumes the environment did not change and proceed to implement incorrect internal parameters instead of refraining from an update for one interval as intended. For example, note the errors in the 1st recording of 65 dB shown in Fig. 2A, as well as the errors in the 1st recording of 65 dB and the 1st recording of 50 dB shown in Fig. 3A. Second, the LS classifier may misclassify the category of the input sounds during training or during normal-operation mode, which would negatively influence the database, thereby negatively affecting the auto-tuned internal parameters. For example, as shown in Figs. 2B, 3B, the 55 dB babble noise sound was misclassified as speech-only during the training phase (in other words, the parameters for 55 dB noise-only were recorded as 55 dB speech-only, which resulted in an absence of data for 55 dB in the noise-only database). As a result, the parameters for 55 dB babble noise was applied to 55 dB speech-only in normal-operation mode, and the parameters for the 55 dB babble noise-only situation were determined by interpolation of the surrounding values, 50 dB and 60 dB noise-only parameters. The errors in 55 dB (babble noise) and 50 dB (car noise) shown in Fig. 3B were also caused by this scenario. Third, the first and second error-inducing scenarios may occur simultaneously, as shown in Fig. 3C: the 1st recording of 80 dB (babble noise) and the 1st recording of 75 dB (car noise).
There are several limitations in the current study. First, during the subjective evaluations, we considered only one user-controllable noise reduction algorithm: β-adjustable beamforming. However, in commercial devices, additional algorithms may be required to achieve environment-adaptive adjustment, such as single-microphone-based noise-reduction algorithms using spectral subtraction and Wiener filtering [17,22]. To modify the currently implemented algorithm to have a more conventional hearing aid algorithm structure, it is necessary to include additional user-controllable algorithms, such as single-microphone-based noise-reduction, to better reflect users’ personal preferences for various degrees of ambient noise suppression and speech distortion. Second, wide dynamic-range compression was bypassed during the verification process because only normal-hearing subjects participated in the subjective test. However, in actual hearing aids, the use of wide dynamic-range compression is indispensable because it (with an additional fitting formula [23]) compensates for the threshold shift in the audiogram caused by damage to the inner and outer hair cells, which may also affect the personal parameter preferences set during the training phase [4,13]. For an implemented algorithm to be suitable for hearing-impaired persons, wide dynamic-range compression with appropriate band gains for individual hearing-impaired subjects should be added before the output volume adjustment, and then, the procedures described in the current study should be performed. In fact, compression learning achieved the largest comfortable loudness ratings, in about 76%–91% of situations, in the report of Chalupper et al. [4]. Third, the amount of time spent in the training phase was relatively short; continuous training of weeks or months after the initial training would be ideal to more accurately verify the effect of user training [24]. In addition, according to previous reports, the final parameter settings can be affected by the starting point of the training [22]. The effect of such initial parameter settings on the proposed algorithm should also be investigated in future works. Furthermore, in the current study, each subject performed the training phase with a one-signal-for-each-condition protocol. In real circumstances, the personal preferences in a specific condition can differ from time to time depending on the variations in emotional and circumstantial states of the device user. To deal with such ambiguous situations, in the current study, the algorithm was designed to override the previous parameter settings with the most recent parameter setting for the same exact environment. We based this on the assumption that the device user would adapt and become more familiar with the device, and as this happens, preference settings would also adapt. However, this overriding protocol may not always be appropriate in all situations; therefore, for example, it may be more effective to save all duplicated user selections for the same condition in the database, and then determine the output parameters by one of the following methods: (1) averaging the saved values or (2) giving different weights to the saved values in accordance with the recorded time, and averaging the weighted values. These approaches should also be considered in future studies. Finally, due to limited number of hearing-impaired patients available to participate in this study, normal-hearing persons participated in this study instead, which is the limitation of this study. Though this is less than ideal, the positive results observed with normal-hearing participants are highly likely to affect hearing-impaired persons in a similar way.
In conclusion, as the experimental results showed, the preferred settings for each of the ISL, LS, and DNS conditions were different among subjects, which justifies the incorporation of more environmental factors during the training phase for more dedicated device individualization. Based on the result of this study, it is very likely for greater personal satisfaction to be achieved under various ambient situations by reflecting individual preference to a more extensive acoustic conditions to further personalize hearing aid control.
XML Download