Voice-Vibratory Assessment With Laryngeal Imaging (VALI) Form: Reliability of Rating Stroboscopy and High-speed Videoendoscopy

Published:December 28, 2016DOI:



      The purpose of the study was to evaluate the inter-judge and intra-judge reliability of raters using the Voice-Vibratory Assessment with Laryngeal Imaging (VALI) rating form that was developed for assessing videostroboscopy and high-speed videoendoscopic (HSV) recordings.

      Subjects and Methods

      Nine speech-language pathologists with an average of 12.8 years of experience with laryngeal imaging were trained to use the VALI form for rating 66 de-identified and randomized samples with voice disorders. Inter-judge reliability for parameters with scale data (amplitude, mucosal wave, nonvibratory portion, supraglottic activity, phase closure, symmetry, and regularity or periodicity) was assessed with intraclass correlations, and parameters with nominal data (glottal closure, vertical level, and free edge contour) were assessed with Fleiss' kappa. Intra-judge reliability was assessed using the Spearman rho statistic for scale data and percentage of concordant pairs for nominal data.


      Inter-judge reliability for parameters with scale data ranged from 0.57 to 0.96 for stroboscopy and from 0.81 to 0.94 for HSV. For nominal parameters, correlations ranged from 0.18 to 0.35 for stroboscopy and from 0.13 to 0.33 for HSV. Intra-judge reliability correlations for parameters with scale data ranged from 0.19 to 0.87 for stroboscopy and from 0.28 to 0.85 for HSV. For parameters with nominal data, percentage of concordance ranged from 44% to 78% for stroboscopy and from 52% to 89% for HSV.


      The VALI rating form and the training protocol is a first, a priori developed rating form that includes visual-perceptual ratings of both stroboscopy and HSV. The current form can be used to make reliable visual-perceptual judgments for selected features of vibratory motion from stroboscopy and HSV.

      Key Words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Voice
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Hirano M.
        • Bless D.
        Videostroboscopic Examination of the Larynx.
        Singular Publishing Group, INC., San Diego, CA1993
        • Zanartu M.
        • Mehta D.D.
        • Ho J.C.
        • et al.
        Observation and analysis of in vivo vocal fold tissue instabilities produced by nonlinear source-filter coupling: a case study.
        J Acoust Soc Am. 2011; 129: 326-339
        • Woo P.
        Quantification of videostrobolaryngoscopic findings—measurements of the normal glottal cycle.
        Laryngoscope. 1996; 106: 1-27
        • Elidan G.
        • Elidan J.
        Vocal folds analysis using global energy tracking.
        J Voice. 2012; 26: 760-768
        • Hanson D.G.
        • Jiang J.
        • D'Agostino M.
        • et al.
        Clinical measurement of mucosal wave velocity using simultaneous photoglottography and laryngostroboscopy.
        Ann Otol Rhinol Laryngol. 1995; 104: 340-349
        • Saadah A.K.
        • Galatsanos N.P.
        • Bless D.
        • et al.
        Deformation analysis of the vocal folds from videostroboscopic image sequences of the larynx.
        J Acoust Soc Am. 1998; 103: 3627-3641
        • Sercarz J.A.
        • Berke G.S.
        • Arnstein D.
        • et al.
        A new technique for quantitative measurement of laryngeal videostroboscopic images.
        Arch Otolaryngol Head Neck Surg. 1991; 117: 871-875
        • Yan Y.
        • Chen X.
        • Bless D.
        Automatic tracing of vocal-fold motion from high-speed digital images.
        IEEE Trans Biomed Eng. 2006; 53: 1394-1400
        • Lohscheller J.
        • Toy H.
        • Rosanowski F.
        • et al.
        Clinically evaluated procedure for the reconstruction of vocal fold vibrations from endoscopic digital high-speed videos.
        Med Image Anal. 2007; 11: 400-413
        • Patel R.
        • Donohue K.D.
        • Unnikrishnan H.
        • et al.
        Kinematic measurements of the vocal-fold displacement waveform in typical children and adult populations: quantification of high-speed endoscopic videos.
        J Speech Lang Hear Res. 2015; 58: 227-240
        • Mehta D.D.
        • Deliyski D.D.
        • Quatieri T.F.
        • et al.
        Automated measurement of vocal fold vibratory asymmetry from high-speed videoendoscopy recordings.
        J Speech Lang Hear Res. 2011; 54: 47-54
        • Ikuma T.
        • Kunduk M.
        • McWhorter A.J.
        Advanced waveform decomposition for high-speed videoendoscopy analysis.
        J Voice. 2013; 27: 369-375
        • Yamauchi A.
        • Imagawa H.
        • Yokonishi H.
        • et al.
        Evaluation of vocal fold vibration with an assessment form for high-speed digital imaging: comparative study between healthy young and elderly subjects.
        J Voice. 2012; 26: 742-750
        • Krausert C.R.
        • Liang Y.
        • Zhang Y.
        • et al.
        Spatiotemporal analysis of normal and pathological human vocal fold vibrations.
        Am J Otolaryngol. 2012; 33: 641-649
        • Yiu E.M.
        • Wang G.
        • Lo A.C.
        • et al.
        Quantitative high-speed laryngoscopic analysis of vocal fold vibration in fatigued voice of young karaoke singers.
        J Voice. 2013; 27: 753-761
        • Woo P.
        Objective measures of laryngeal imaging: what have we learned since Dr. Paul Moore.
        J Voice. 2014; 28: 69-81
        • Bless D.M.
        • Hirano M.
        • Feder R.J.
        Videostroboscopic evaluation of the larynx.
        Ear Nose Throat J. 1987; 66: 289-296
        • Poburka B.J.
        • Bless D.M.
        A multi-media, computer-based method for stroboscopy rating training.
        J Voice. 1998; 12: 513-526
        • Poburka B.J.
        A new stroboscopy rating form.
        J Voice. 1999; 13: 403-413
        • Rosen C.A.
        Stroboscopy as a research instrument: development of a perceptual evaluation tool.
        Laryngoscope. 2005; 115: 423-428
        • Olthoff A.
        • Woywod C.
        • Kruse E.
        Stroboscopy versus high-speed glottography: a comparative study.
        Laryngoscope. 2007; 117: 1123-1126
        • Mehta D.D.
        • Deliyski D.D.
        • Zeitels S.M.
        • et al.
        Voice production mechanisms following phonosurgical treatment of early glottic cancer.
        Ann Otol Rhinol Laryngol. 2010; 119: 1-9
        • Patel R.
        • Dailey S.
        • Bless D.
        Comparison of high-speed digital imaging with stroboscopy for laryngeal imaging of glottal disorders.
        Ann Otol Rhinol Laryngol. 2008; 117: 413-424
        • Patel R.R.
        • Liu L.
        • Galatsanos N.P.
        • et al.
        Differential vibratory characteristics of adductor spasmodic dysphonia and muscle tension dysphonia on high-speed digital imaging.
        Ann Otol Rhinol Laryngol. 2011; 120: 21-32
        • Patel R.R.
        • Pickering J.
        • Stemple J.
        • et al.
        A case report in changes in phonatory physiology following voice therapy: application of high-speed imaging.
        J Voice. 2012; 26: 734-741
        • Deliyski D.D.
        • Hillman R.E.
        State of the art laryngeal imaging: research and clinical implications.
        Curr Opin Otolaryngol Head Neck Surg. 2010; 18: 147-152
        • Mendelsohn A.H.
        • Remacle M.
        • Courey M.S.
        • et al.
        The diagnostic role of high-speed vocal fold vibratory imaging.
        J Voice. 2013; 27: 627-631
        • Yamauchi A.
        • Yokonishi H.
        • Imagawa H.
        • et al.
        Quantification of vocal fold vibration in various laryngeal disorders using high-speed digital imaging.
        J Voice. 2016; 30: 205-214
        • Yamauchi A.
        • Yokonishi H.
        • Imagawa H.
        • et al.
        Visualization and estimation of vibratory disturbance in vocal fold scar using high-speed digital imaging.
        J Voice. 2016; 30: 493-500
        • Peppard R.C.
        • Bless D.
        A method for improving measurement reliability in laryngeal videostroboscopy.
        J Voice. 1990; 4: 280-285
        • Uloza V.
        • Vegiene A.
        • Pribuisiene R.
        • et al.
        Quantitative evaluation of video laryngostroboscopy: reliability of the basic parameters.
        J Voice. 2013; 27: 361-368
        • Otto K.J.
        • Hapner E.R.
        • Baker M.
        • et al.
        Blinded evaluation of the effects of high definition and magnification on perceived image quality in laryngeal imaging.
        Ann Otol Rhinol Laryngol. 2006; 115: 110-113
        • Kendall K.A.
        High-speed laryngeal imaging compared with videostroboscopy in healthy subjects.
        Arch Otolaryngol Head Neck Surg. 2009; 135: 274-281
        • Inwald E.C.
        • Döllinger M.
        • Schuster M.
        • et al.
        Multiparametric analysis of vocal fold vibrations in healthy and disordered voices in high-speed imaging.
        J Voice. 2011; 25: 576-590
      1. Hirano M. Clinical Examination of Voice. Springer-Verlag, New York1981
        • Association A.S.-L.-H.
        The roles of otolaryngologists and speech-language pathologists in the performance and interpretation of strobovideolaryngoscopy (Relevant Paper).
        (Available at:) (Accessed November 11, 2016)
        • Deliyski D.D.
        • Powell M.E.G.
        • Zacharias S.R.C.
        • et al.
        Experimental investigation on minimum frame rate requirements of high-speed videoendoscopy for clinical voice assessment.
        Biomed Signal Proces. 2015; 17: 21-28
        • Bonilha H.S.
        • Deliyski D.D.
        • Gerlach T.T.
        Phase asymmetries in normophonic speakers: visual judgments and objective findings.
        Am J Speech Lang Pathol. 2008; 17: 367-376