ROC Analysis Cut-off Points of Hellenic Voice Handicap Index for Neurogenic Voice Disorders Patients: an Exploratory Study

Published:October 01, 2020DOI:



      The inclusion of subjective methods for evaluating Voice Disorders is proven an essential factor for diagnosis as these methods include self-reported questionnaires (eg, Voice Handicap Index-VHI) for everyday clinical practice. In turn, by obtaining cut-off scores of self-perceived questionnaires intended for assessment procedures of different voice disorders (eg, patients with neurological problems), the clinicians might be helped toward finding their patients’ needs leading to better monitoring, and treatment suggestions. Consequently, the purpose of this study was to estimate the cut-off scores for the Greek VHI relevant to patients with neurogenic voice disorders.


      Ninety subjects participated in this research. Sixty-six of them served as the control group while the remaining 24 patients exhibited Neurogenic Voice Disorders (eg, spasmodic dysphonia or vocal fold paralysis). They filled in the VHI and the Voice Evaluation Template. All participants were examined with the use of video laryngeal endoscopy and stroboscopy.


      The analysis revealed higher medians in all domains (of the VHI) for the patients compared to the control group. The cut-off points were estimated at the values of 24.50 (Total Score–AUC 0.932, P = 0.000), 9.00 (Functional Domain–AUC 0.917, P = 0.000), 10.00 (Physical Domain–AUC 0.948, P = 0.000), and 9.00 (Emotional Domain–AUC 0.830, P = 0.000).


      The estimated cut-off scores are in agreement with previous studies. These scores could probably used to enhance therapeutic monitoring of patients who suffer from neurogenic voice disorders. This study underlines the importance of considering different cutoff points for individuals with voice disorders due to diverse neurogenic etiologies.

      Key Words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Voice
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Dresel C.
        • Mergell P.
        • Hoppe U.
        • et al.
        An asymmetric smooth contour two-mass model for recurrent laryngeal nerve paralysis.
        Logoped Phoniatr Vocol. 2006; 3: 61-75
        • Fleischer S.
        • Schade G.
        • Hess M.M.
        Office-based laryngoscopic observations of recurrent laryngeal nerve paresis and paralysis.
        Ann Otol Rhinol Laryngol. 2005; 114: 488-493
        • Pinho C.M.
        • Jesus L.M.
        • Barney A.
        Aerodynamic measures of speech in unilateral vocal fold paralysis (UVFP) patients.
        Logoped Phoniatr Vocol. 2013; 38: 19-34
        • Sulica L.
        • Blitzer A.
        Vocal fold paresis: evidence and controversies.
        Curr Opin Otolaryngol Head Neck Surg. 2007; 15: 159-162
        • Dursun G.
        • Sataloff R.T.
        • Spiegel J.R.
        • et al.
        Superior laryngeal nerve paresis and paralysis.
        J Voice. 1996; 10: 206-211
        • Lindestad P.Å.
        • Hertegård S.
        • Ylitalo R.
        Effects of botulinum toxin treatment for adductor spasmodic dysphonia on voice and social function.
        Logoped Phoniatr Vocol. 1997; 22: 129-134
        • Loven J.Ø.
        • Brøndbo K.
        • Ganes T.
        • et al.
        Botulinum toxin treatment for spasmodic dysphonia (adductor laryngeal dystonia) Experience with 31 patients.
        Scand J Logoped Phoniatr. 1994; 19: 107-111
        • Moerman M.
        • Martens J.P.
        • Dejonckere P.
        Multidimensional assessment of strongly irregular voices such as in substitution voicing and spasmodic dysphonia: a compilation of own research.
        Logoped Phoniatr Vocol. 2015; 40: 24-29
        • Blumin J.H.
        • Pcolinsky D.E.
        • Atkins J.P.
        Laryngeal findings in advanced Parkinson's disease.
        Ann Otol Rhinol Laryngol. 2004; 113: 253-258
        • Hartelius L.
        • Svensson P.
        • Bubach A.
        Clinical assessment of dysarthria: Performance on a dysarthria test by normal adult subjects, and by individuals with Parkinson's disease or with multiple sclerosis.
        Scand J Logoped Phoniatr. 1993; 18: 131-141
        • Rusz J.
        • Čmejla R.
        • Růžičková H.
        Acoustic assessment of voice and speech disorders in Parkinson's disease through quick vocal test.
        Mov Disord. 2011; 26: 1951-1952
        • Rusz J.
        • Čmejla R.
        • Růžičková H.
        Quantitative acoustic measurements for characterization of speech and voice disorders in early untreated Parkinson's disease.
        J Acoust Soc Am. 2011; 129: 350-367
        • Sodersten M.
        • Hammarberg B.
        Effects of voice training in normal-speaking women: Videostroboscopic, perceptual, and acoustic characteristics.
        Scand J Logoped Phoniatr. 1993; 18: 33-42
        • Van Lierde K.M.
        • De Bodt M.
        • Dhaeseleer E.
        • et al.
        The treatment of muscle tension dysphonia: a comparison of two treatment techniques by means of an objective multiparameter approach.
        J Voice. 2011; 24: 294-301
        • Ulis J.M.
        • Yanagisawa E.
        What's new in differential diagnosis and treatment of hoarseness?.
        Curr Opin Otolaryngol Head Neck Surg. 2009; 17: 209-215
        • Dejonckere P.H.
        • Bradley P.
        • Clemente P.
        • et al.
        A basic protocol for functional assessment of voice pathology, especially for investigating the efficacy of (phonosurgical) treatments and evaluating new assessment techniques.
        Eur Arch Otorhinolaryngol. 2001; 258: 77-82
        • DeJonckere P.H.
        • Crevier-Buchman L.
        • Marie J.P.
        • et al.
        Implementation of the European Laryngological Society (ELS) basic protocol for assessing voice treatment effect.
        Rev Laryngol Otol Rhinol. 2003; 124: 279-283
        • Patel R.R.
        • Awan S.N.
        • Barkmeier-Kraemer J.
        • et al.
        Recommended Protocols for Instrumental Assessment of Voice: American Speech-Language-Hearing Association Expert Panel to Develop a Protocol for Instrumental Assessment of Vocal Function.
        Am J Speech Lang Pathol. 2018; 27: 887-905
        • Carding P.
        • Carlson E.
        • Epstein R.
        • et al.
        Formal perceptual evaluation of voice quality in the United Kingdom.
        Logoped Phoniatr Vocol. 2000; 25: 133-138
        • Oates J.
        Auditory-perceptual evaluation of disordered voice quality.
        Folia Phoniatr Logop. 2009; 61: 49-56
        • Tveterås G.
        Perceptual analysis of voice quality.
        Scand J Logoped Phoniatr. 1992; 17: 145-149
        • Undestad P.Å.
        • Hertegård S.
        • Hammarberg B.
        • et al.
        An audioperceptual, videostroboscopic and electromyographic study of spindle-shaped glottal insufficiency with and without sulcus vocalis.
        Scand J Logoped Phoniatr. 1995; 20: 91-101
        • Hess M.M.
        • Ludwigs M.
        • Kobler J.B.
        • et al.
        Imaging of the larynx–extending the use of stroboscopy-related techniques.
        Logoped Phoniatr Vocol. 2002; 27: 50-58
        • Ramig L.A.
        • Titze I.R.
        • Scherer R.C.
        • et al.
        Acoustic analysis of voices of patients with neurologic disease: rationale and preliminary data.
        Ann Otol Rhinol Laryngol. 1988; 97: 164-172
        • Schönweiler R.
        • Hess M.
        • Wübbelt P.
        • et al.
        Novel approach to acoustical voice analysis using artificial neural networks.
        J Assoc Res Otolaryngol. 2000; 1: 270-282
        • Titze I.R.
        Workshop on acoustic voice analysis: Summary statement.
        National Center for Voice and Speech. 1995; : 6-30
        • Aboras Y.
        • El-Banna M.
        • El-Magraby R.
        • et al.
        The relationship between subjective self-rating and objective voice assessment measures.
        Logoped Phoniatr Vocol. 2010; 25: 34-38
        • Hallin A.E.
        • Fröst K.
        • Holmberg E.B.
        • et al.
        Voice and speech range profiles and Voice Handicap Index for males—methodological issues and data.
        Logoped Phoniatr Vocol. 2012; 37: 47-61
        • Rasch T.
        • Günther S.
        • Hoppe U.
        • et al.
        Voice-related quality of life in organic and functional voice disorders.
        Logoped Phoniatr Vocol. 2015; 30: 9-13
        • Rodríguez-Parra M.J.
        • Adrián J.A.
        • Casado J.C.
        Comparing voice-therapy and vocal-hygiene treatments in dysphonia using a limited multidimensional evaluation protocol.
        J Commun Disord. 2011; 44: 615-630
        • Tafiadis D.
        • Tatsis G.
        • Ziavra N.
        • et al.
        Voice data on female smokers: coherence between the voice handicap index and acoustic voice parameters.
        AIMS Med Sci. 2017; 4: 151-163
        • Burn D.
        • Emre M.
        • McKeith I.
        • et al.
        Effects of rivastigmine in patients with and without visual hallucinations in dementia associated with Parkinson's disease.
        Mov Disord. 2006; 21: 1899-1907
        • Hauser R.A.
        • Rascol O.
        • Korczyn A.D.
        • et al.
        A Ten‐year follow‐up of Parkinson's disease patients randomized to initial therapy with ropinirole or levodopa.
        Mov Disord. 2007; 22: 2409-2417
        • Goetz C.G.
        • Tilley B.C.
        • Shaftman S.R.
        • et al.
        Movement Disorder Society‐sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS‐UPDRS): Scale presentation and clinimetric testing results.
        Mov Disord. 2008; 23: 2129-2170
        • Amezquita-Sanchez J.P.
        • Adeli A.
        • Adeli H.
        A new methodology for automated diagnosis of mild cognitive impairment (MCI) using magnetoencephalography (MEG).
        Behav Brain Res. 2016; 305: 174-180
        • Bronnick K.
        • Ehrt U.
        • Emre M.
        • et al.
        Attentional deficits affect activities of daily living in dementia-associated with Parkinson's disease.
        J Neurol Neurosurg Psychiatry. 2006; 77: 1136-1142
        • Gronchi‐Perrin A.
        • Viollier S.
        • Ghika J.
        • et al.
        Does subthalamic nucleus deep brain stimulation really improve quality of life in Parkinson's disease?.
        Mov Disord. 2006; 21: 1465-1468
        • Orozco-Arroyave J.R.
        • Belalcazar-Bolanos E.A.
        • Arias-Londoño J.D.
        • et al.
        Characterization methods for the detection of multiple voice disorders: Neurological, functional, and laryngeal diseases.
        IEEE J Biomed Health Inform. 2015; 19: 1820-1828
        • Sukanen O.
        • Sihvo M.
        • Rorarius E.
        • et al.
        Voice Activity and Participation Profile (VAPP) in assessing the effects of voice disorders on patients’ quality of life: Validity and reliability of the Finnish version of VAPP.
        Logoped Phoniatr Vocol. 2007; 32: 3-8
        • Hogikyan N.D.
        • Sethuraman G.
        Validation of an instrument to measure voice-related quality of life (V-RQOL).
        J Voice. 1999; 13: 557-569
        • Wuyts F.L.
        • De Bodt M.S.
        • Molenberghs G.
        • et al.
        The dysphonia severity index: an objective measure of vocal quality based on a multiparameter approach.
        J Speech Lang Hear Res. 2000; 43: 796-809
        • Dreary I.J.
        • Wilson J.A.
        • Carding P.N.
        • et al.
        VoiSS: a patient-derived voice symptom scale.
        J Psychosom Res. 2003; 54: 483-489
        • Jacobson B.H.
        • Johnson A.
        • Grywalski C.
        • et al.
        The voice handicap index (VHI): development and validation.
        Am J Speech Lang Pathol. 1997; 6: 66-70
        • Amir O.
        • Ashkenazi O.
        • Leibovitzh T.
        • et al.
        Applying the voice handicap index (VHI) to dysphonic and nondysphonic Hebrew speakers.
        J Voice. 2006; 20: 318-324
        • Behlau M.
        • dos Santos L.D.M.A.
        • Oliveira G.
        Cross-cultural adaptation and validation of the voice handicap index into Brazilian Portuguese.
        J Voice. 2011; 25: 354-359
        • Bonetti A.
        • Bonetti L.
        Cross-cultural adaptation and validation of the voice handicap index into Croatian.
        J Voice. 2013; 27: 30-e7
        • Helidoni M.E.
        • Murry T.
        • Moschandreas J.
        • et al.
        Cross-cultural adaptation and validation of the voice handicap index into Greek.
        J Voice. 2010; 24: 221-227
        • Karlsen T.
        • Grieg A.R.H.
        • Heimdal J.H.
        • et al.
        Cross-cultural adaption and translation of the voice handicap index into Norwegian.
        Folia Phoniatr Logop. 2012; 64: 234-240
        • Malki K.H.
        • Mesallam T.A.
        • Farahat M.
        • et al.
        Validation and cultural modification of Arabic voice handicap index.
        Eur Arch Otorhinolaryngol. 2010; 267: 1743-1751
        • Moradi N.
        • Pourshahbaz A.
        • Soltani M.
        • et al.
        Cross-cultural equivalence and evaluation of psychometric properties of voice handicap index into Persian.
        J Voice. 2013; 27: 258-e15
        • Ohlsson A.C.
        • Dotevall H.
        Voice handicap index in Swedish.
        Logop Phoniat Vocol. 2009; 34: 60-66
        • Schindler A.
        • Ottaviani F.
        • Mozzanica F.
        • et al.
        Cross-cultural adaptation and validation of the voice handicap index into Italian.
        J Voice. 2010; 24: 708-714
        • Taguchi A.
        • Mise K.
        • Nishikubo K.
        • et al.
        Japanese version of voice handicap index for subjective evaluation of voice disorder.
        J Voice. 2012; 26: 668-e15
        • Trinite B.
        • Sokolovs J.
        Adaptation and validation of the voice handicap index in Latvian.
        J Voice. 2014; 28: 452-457
        • Hakkesteegt M.M.
        • Wieringa M.H.
        • Gerritsma E.J.
        • et al.
        Reproducibility of the Dutch version of the voice handicap index.
        Folia Phoniatr Logop. 2006; 58: 132-138
        • Behlau M.
        • Madazio G.
        • Moreti F.
        • et al.
        Efficiency and cutoff values of self-assessment instruments on the impact of a voice problem.
        J Voice. 2016; 30: 506-5e9
        • Moradi N.
        • Pourshahbaz A.
        • Soltani M.
        • et al.
        Cutoff point at voice handicap index used to screen voice disorders among Persian speakers.
        J Voice. 2013; 27: 130-1e1
        • Solomon N.P.
        • Helou L.B.
        • Henry L.R.
        • et al.
        Utility of the voice handicap index as an indicator of postthyroidectomy voice dysfunction.
        J Voice. 2013; 27: 348-354
        • Tafiadis D.
        • Kosma E.I.
        • Chronopoulos S.K.
        • et al.
        Voice handicap index and interpretation of the cutoff points using receiver operating characteristic curve as screening for young adult female smokers.
        J Voice. 2018; 32: 64-69
        • Tafiadis D.
        • Chronopoulos S.K.
        • Kosma E.I.
        • et al.
        Using receiver operating characteristic curve to define the cutoff points of voice handicap index applied to young adult male smokers.
        J Voice. 2018; 32: 443-448
        • Tafiadis D.
        • Kosma E.I.
        • Chronopoulos S.K.
        • et al.
        A preliminary receiver operating characteristic analysis on voice handicap index results of the greek voice-disordered patients.
        Inter J Otolaryngol Head Neck Surg. 2018; 7: 98
        • Tafiadis D.
        • Helidoni M.E.
        • Chronopoulos S.K.
        • et al.
        Preliminary receiver operating characteristic analysis on voice handicap index of laryngeal inflammation in greek patients.
        Inter J Otolaryngol Head Neck Surg. 2018; 7: 115
        • Krzanowski W.J.
        • Hand D.J.
        ROC curves for continuous data.
        Chapman and Hall/CRC, 2009
        • Schindler A.
        • Bottero A.
        • Capaccio P.
        • et al.
        Vocal improvement after voice therapy in unilateral vocal fold paralysis.
        J Voice. 2008; 22: 113-118
        • Rosen C.A.
        • Murry T.
        • Zinn A.
        • et al.
        Voice handicap index change following treatment of voice disorders.
        J Voice. 2001; 14: 619-623
        • Majdinasab F.
        • Karkheiran S.
        • Moradi N.
        • et al.
        Relation between voice handicap index (VHI) and disease severity in Iranian patients with Parkinson's disease.
        Med J Islam Repub Iran. 2012; 26: 157-163
        • Letanneux A.
        • Walshe M.
        • Viallet F.
        • et al.
        The Dysarthria impact profile: a preliminary French experience with Parkinson's disease.
        Parkinson's disease. 2013; (In press)
        • Louis E.D.
        • Gerbin M.
        Voice handicap in essential tremor: a comparison with normal controls and Parkinson's disease.
        Tremor Other Hyperkinet Mov (NY). 2013; 3 (tre-03-114-970-1)
        • Postuma R.B.
        • Berg D.
        • Stern M.
        • et al.
        MDS clinical diagnostic criteria for Parkinson's disease.
        Mov Disord. 2015; 30: 1591-1601
      1. American Speech Hearing Association. Voice Evaluation Template. 14.02.2019).

        • Bauer V.
        • Alerić Z.
        • Jančić E.
        • et al.
        Voice quality in Parkinson's disease in the Croatian language speakers.
        Coll. Antropol. 2011; 35: 209-212
        • Wingate J.M.
        • Ruddy B.H.
        • Lundy D.S.
        • et al.
        Voice handicap index results for older patients with adductor spasmodic dysphonia.
        J Voice. 2005; 19: 124-131
        • Pawlukowska W.
        • Szylińska A.
        • Kotlęga D.
        • et al.
        Differences between subjective and objective assessment of speech deficiency in Parkinson disease.
        J Voice. 2018; 32: 715-722
        • Hallin A.E.
        • Fröst K.
        • Holmberg E.B.
        • et al.
        Voice and speech range profiles and voice handicap index for males—methodological issues and data.
        Logoped Phoniatr Vocol. 2012; 37: 47-61
        • Carmichael J.N.
        Introducing objective acoustic metrics for the Frenchay Dysarthria Assessment procedure [dissertation].
        University of Sheffield, SheffieldUK2018
        • Carmichael J.N.
        Introducing microphone calibration techniques for reliable loudness estimation in the CFDA tests.
        in: Proceedings of the 7th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), Vancouver, Canada IEEE, 2016: 1-7 (2016 Oct13-15)
        • Herbst C.T.
        • Oh J.
        • Vydrová J.
        • et al.
        DigitalVHI—a freeware open-source software application to capture the voice handicap index and other questionnaire data in various languages.
        Logoped Phoniatr Vocol. 2015; 40: 72-76
        • Godall P.
        • Gassull C.
        • Godoy A.
        • et al.
        Epidemiological voice health map of the teaching population of Granollers (Barcelona) developed from the EVES questionnaire and the VHI.
        Logoped Phoniatr Vocol. 2015; 40: 171-178
        • Chronopoulos S.K
        • Tatsis G.
        • Kostarakis P.
        Turbo coded OFDM with large number of subcarriers.
        J Signal Inform Process. 2012; 3: 161-168
        • Tatsis G.
        • Votis C.
        • Raptis V.
        • et al.
        Performance of UWB‐impulse radio receiver based on matched filter implementation with imperfect channel estimation.
        Proceedings of the AIP Conference. 2010; 1203: 573-578
        • Chronopoulos S.K.
        • Kosma E.I.
        • Tafiadis D.
        • et al.
        Reduced ecological footprints of modern facilities introducing the implementation of advanced wireless technologies, and human resources’ benefits.
        Commun Network. 2018; 10: 11-29