Summary
Objectives
The primary aim was to compare two methods for perceptual evaluation of voice – paired
comparison (PC) and visual analog scale (VAS) ratings. Secondary aims were to assess
the correspondence between two dimensions of voice– overall severity of voice quality
and resonant voice, and to investigate the influence of rater experience on perceptual
rating scores and rating confidence scores.
Study Design
Experimental design.
Methods
Voice samples from six children (pre and post therapy) were rated by 15 Speech-Language
Pathologists specialized in voice. Raters completed four tasks corresponding to the
two rating methods and voice qualities: PC-severity, PC-resonance, VAS-severity, and
VAS-resonance. For PC tasks, raters chose the better of two voice samples (better
voice quality or better resonance, depending on the task) and indicated the degree
of confidence in each choice. Rating and confidence score were combined to produce
a number on a 1-10 scale (PC-confidence adjusted). VAS ratings involved rating voices
on a scale for degree of severity and resonance, respectively.
Results
PC-confidence adjusted and VAS ratings were moderately correlated for overall severity
and also vocal resonance. VAS ratings were normally distributed and had greater rater
consistency than PC-confidence adjusted ratings. VAS scores reliably predicted binary
PC choices (choice of voice sample only). Overall severity and vocal resonance were
weakly correlated and rater experience was not linearly related to rating scores or
confidence.
Conclusions
Results suggest that the VAS rating method holds advantages over PC, including normally
distributed ratings, superior consistency of ratings, and the ability to provide more
finely grained detail regarding the auditory perception of voice. Overall severity
and vocal resonance were not redundant in the current data set, suggesting that resonant
voice and overall severity are not isomorphic. Finally, the number of years of clinical
experience was not linearly related to perceptual ratings or rating confidence.
Key Words
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of VoiceAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
REFERENCES
- Laryngeal Function and Voice Disorders: Basic Science to Clinical Practice.Thieme Publishers, New York, NY2019
- Defining and studying voice across disciplinary boundaries.in: Eidsheim N Meizel K The Oxford Handbook of Voice Studies. Oxford Handbooks, New York, NY2019: 493-514
- Foundations of Voice Studies: An Interdisciplinary Approach to Voice Production and Perception.John Wiley & Sons, Chichester, UK2011
- Perceptual evaluation of voice quality: review, tutorial, and a framework for future research.J Speech Hear Res. 1993; 36: 21-40https://doi.org/10.1044/jshr.3601.21
- The relationship between auditory-perceptual rating scales and objective voice measures in children with voice disorders.Am J Speech Lang Pathol. 2021; 30: 228-238https://doi.org/10.1044/2020_AJSLP-20-00188
- Perceptual evaluation of severe pediatric voice disorders: rater reliability using the consensus auditory perceptual evaluation of voice.J Voice. 2010; 24: 441-449https://doi.org/10.1016/j.jvoice.2008.09.004
- Development of the human true vocal fold: depth of cell layers and quantifying cell types within the lamina propria.Ann Otol Rhinol Laryngol. 2006; 115: 784-788https://doi.org/10.1177/000348940611501012
Döllinger M, Dubrovskiy D, Patel R. Spatiotemporal analysis of vocal fold vibrations between children and adults. the Laryngoscope. 2012;122:2511-2518. https://doi.org/10.1002/lary.23568
- Morphology of the human larynx during the first five years of life studied on whole organ serial sections.Ann Otol Rhinol Laryngol. 1999; 108: 232-238https://doi.org/10.1177/000348949910800303
- A morphological study of the human prepubertal and pubertal larynx.Am J Anat. 1978; 151: 11-19https://doi.org/10.1002/aja.1001510103
- Science of voice production from infancy through adolescence.in: Hartnick CJ Boseley ME Pediatric Voice Disorders. Plural Publishing, San Diego, CA2008: 23-29
- Developmental changes of laryngeal dimensions in unparalyzed.Sedated Child: Anesthesiol. 2003; 98: 41-45https://doi.org/10.1097/00000542-200301000-00010
- Vibratory onset and offset times in children: A laryngeal imaging study.Int J Pediatr Otorhinolaryngol. 2016; 87: 11-17https://doi.org/10.1016/j.ijporl.2016.05.019
- Measurement of glottal cycle characteristics between children and adults: physiological variations.J Voice Off J Voice Found. 2014; 28: 476-486https://doi.org/10.1016/j.jvoice.2013.12.010
- Kinematic measurements of the vocal-fold displacement waveform in typical children and adult populations: quantification of high-speed endoscopic videos.J Speech Lang Hear Res. 2015; 58: 227-240https://doi.org/10.1044/2015_JSLHR-S-13-0056
- Fundamental frequency changes of persian speakers across the life span.J Voice. 2014; 28: 274-281https://doi.org/10.1016/j.jvoice.2013.10.012
- Acoustic characteristics of voice in different cycles of life: an integrative literature review.Rev CEFAC. 2019; 21: e15018https://doi.org/10.1590/1982-0216/201921315018
- Acoustic characteristics of the voice for Brazilian Portuguese speakers across the life span.J Voice. 2020; (Published online October)S0892199720303635https://doi.org/10.1016/j.jvoice.2020.09.019
- Changes in acoustic characteristics of the voice across the life span: measures from individuals 4–93 years of age.J Speech Lang Hear Res. 2011; 54: 1011-1021https://doi.org/10.1044/1092-4388(2010/10-0036
- Comparison of rater's reliability on perceptual evaluation of different types of voice sample.J Voice. 2012; 26 (666.e13-666.e21)https://doi.org/10.1016/j.jvoice.2011.08.003
- The effect of speaking task on perceptual judgment of the severity of dysphonic voice.J Voice. 2005; 19: 574-581https://doi.org/10.1016/j.jvoice.2004.08.009
- Individual differences in voice quality perception.J Speech Lang Hear Res. 1992; 35: 512-520https://doi.org/10.1044/jshr.3503.512
- Validity of rating scale measures of voice quality.J Acoust Soc Am. 1998; 104: 1598-1608https://doi.org/10.1121/1.424372
- Auditory-perceptual evaluation of the degree of vocal deviation: correlation between the visual analogue scale and numerical scale.CoDAS. 2015; 27: 279-284https://doi.org/10.1590/2317-1782/20152014167
- The reliability of three perceptual evaluation scales for dysphonia.Eur Arch Otorhinolaryngol. 2004; 261: 429-434https://doi.org/10.1007/s00405-003-0707-7
- Is the reliability of a visual analog scale higher than an ordinal scale? An experiment with the GRBAS scale for the perceptual evaluation of dysphonia.J Voice. 1999; 13: 508-517https://doi.org/10.1016/S0892-1997(99)80006-X
- Auditory-perceptual evaluation of normal and dysphonic voices using the voice deviation scale.J Voice. 2017; 31: 67-71https://doi.org/10.1016/j.jvoice.2016.01.004
- The multidimensional nature of pathologic vocal quality.J Acoust Soc Am. 1994; 96: 1291-1302https://doi.org/10.1121/1.410277
- The perceptual structure of pathologic voice quality.J Acoust Soc Am. 1996; 100: 1787-1795https://doi.org/10.1121/1.416074
- Mok RS -M. Reliability and confidence in using a paired comparison paradigm in perceptual voice quality evaluation.Clin Linguist Phon. 2007; 21: 129-145https://doi.org/10.1080/02699200600756355
- Comparing internal and external standards in voice quality judgments.J Speech Lang Hear Res. 1993; 36: 14-20https://doi.org/10.1044/jshr.3601.14
- Measuring vocal quality with speech synthesis.J Acoust Soc Am. 2001; 110: 7https://doi.org/10.1121/1.1409969
- Comparing reliability of perceptual ratings of roughness and acoustic measures of jitter.J Speech Lang Hear Res. 1995; 38: 26-32https://doi.org/10.1044/jshr.3801.26
- Quantifying dysphonia severity using a spectral/cepstral-based acoustic index: Comparisons with auditory-perceptual judgements from the CAPE-V.Clin Linguist Phon. 2010; 24: 742-758https://doi.org/10.3109/02699206.2010.492446
- The effect of anchor modality on the reliability of vocal severity ratings.J Voice. 2009; 23: 341-352https://doi.org/10.1016/j.jvoice.2007.10.006
- Acoustic and perceptual classification of within-sample normal, intermittently dysphonic, and consistently dysphonic voice types.J Voice. 2017; 31: 218-228https://doi.org/10.1016/j.jvoice.2016.04.016
- Sources of listener disagreement in voice quality assessment.J Acoust Soc Am. 2000; 108: 1867-1876https://doi.org/10.1121/1.1289362
- Reliability and correlations between overall severity, roughness and breathiness in the perception of dysphonic voices: investigating cognitive aspects.J Voice. 2021; (Published online August)S0892199721002460https://doi.org/10.1016/j.jvoice.2021.07.010
- Individual voice dimensions’ prediction of overall dysphonia severity on two auditory-perceptual scales.J Speech Lang Hear Res. 2022; 65: 2759-2777https://doi.org/10.1044/2022_jslhr-21-00689
- A quantitative output-cost ratio in voice production.J Speech Lang Hear Res. 2001; 44: 29-37https://doi.org/10.1044/1092-4388(2001/003
- Comparison of aerodynamic and electroglottographic parameters in evaluating clinically relevant voicing patterns.Ann Otol Rhinol Laryngol. 1994; 103: 335-346https://doi.org/10.1177/000348949410300501
- Preliminary study of two methods of treatment for laryngeal nodules.J Voice. 1995; 9: 74-85https://doi.org/10.1016/S0892-1997(05)80225-5
- Test-retest study of the GRBAS scale: Influence of experience and professional background on perceptual rating of voice quality.J Voice. 1997; 11: 74-80https://doi.org/10.1016/S0892-1997(97)80026-4
- Effect of experience on judgments of adductor spasmodic dysphonia.Ann Otol Rhinol Laryngol. 2007; 116: 695-701https://doi.org/10.1177/000348940711600912
- The role of experience on judgments of dysphonia.J Voice. 2010; 24: 564-573https://doi.org/10.1016/j.jvoice.2008.12.005
- The impact of long-term botulinum toxin injections on symptom severity in patients with spasmodic dysphonia.J Voice. 2004; 18: 415-422https://doi.org/10.1016/j.jvoice.2000.11.001
- Listener experience and perception of voice quality.J Speech Hear Res. 1990; 33: 103-115https://doi.org/10.1044/jshr.3301.103
- The effect of listener experience and anchors on judgments of dysphonia.J Speech Lang Hear Res. 2011; 54: 430-447https://doi.org/10.1044/1092-4388(2010/09-0205
- Adventures in Voice: A Whole New Way of Doing Things for Kids Clinician Manual.3rd ed. Visions in Voice, Lawrence, KS2015
- Cognitive mechanisms in pediatric voice therapy – an initial examination.J Voice. 2021; (Published online)S0892199721003246https://doi.org/10.1016/j.jvoice.2021.09.014
- Consensus auditory-perceptual evaluation of voice: development of a standardized clinical protocol.Am J Speech Lang Pathol. 2009; 18: 124-132https://doi.org/10.1044/1058-0360(2008/08-0017
- The REDCap consortium: building an international community of software platform partners.J Biomed Inform. 2019; 95103208https://doi.org/10.1016/j.jbi.2019.103208
- Correlation coefficients: appropriate use and interpretation.Anesth Analg. 2018; 126: 1763-1768https://doi.org/10.1213/ANE.0000000000002864
- A guideline of selecting and reporting intraclass correlation coefficients for reliability research.J Chiropr Med. 2016; 15: 155-163https://doi.org/10.1016/j.jcm.2016.02.012
- Applied Logistic Regression.Wiley-Interscience, Chichester, UK2001
- Some guiding principles in emerging models of voice therapy for children.Semin Speech Lang. 2013; 34: 080-093https://doi.org/10.1055/s-0033-1342979
- Treatment of laryngeal hyperfunction with flow phonation: a pilot study.J Laryngol Voice. 2012; 2: 64https://doi.org/10.4103/2230-9748.106980
- A randomized controlled trial of stretch-and-flow voice therapy for muscle tension dysphonia.The Laryngoscope. 2015; 125: 1420-1425https://doi.org/10.1002/lary.25155
- Intervention outcomes of two treatments for muscle tension dysphonia: a randomized controlled trial.J Speech Lang Hear Res. 2019; 62: 272-282https://doi.org/10.1044/2018_JSLHR-S-18-0118
- 2017 SLP Health Care Survey: Caseload Characteristics.2017 (Accessed June 30, 2020)
- 2018 Schools Survey: SLP Caseload and Workload Characteristics.2018 (Accessed June 30, 2020)
- 2019 SLP Health Care Survey: Caseload Characteristics.2019 (Accessed June 30, 2020)
- 2020 Schools Survey: SLP Caseload and Workload Characteristics.2020 (Accessed August 24, 2022)
- 2022 Schools Survey: SLP Caseload and Workload Characteristics.2022 (Accessed August 24, 2022)
- A comparison of two perceptual voice evaluation training programs for naive listeners.J Voice. 2006; 20: 229-241https://doi.org/10.1016/j.jvoice.2005.03.007
- Developing clinically relevant scales of breathy and rough voice quality.J Voice. 2021; 35 (663.e9-663.e16)https://doi.org/10.1016/j.jvoice.2019.12.021
Article info
Publication history
Published online: March 11, 2023
Accepted:
January 5,
2023
Publication stage
In Press Corrected ProofFootnotes
The research was partially supported by the University of Delaware Graduate School Dissertation Fellowship award to H.F. Support was also received from NSF SES 2150615 to J.A. (Purdue University), NIDCD 2R01DC009029-13A1 to S.A. (University of South Florida), and from NIDCD 2R01DC017923 to K.V.A (PI, University of Delaware) and S.A. (consultant, University of South Florida). We wish to thank the Speech-Language Pathologists who participated in the study for their time and efforts.
Identification
Copyright
© 2023 The Voice Foundation. Published by Elsevier Inc. All rights reserved.
