Summary
Transgender individuals may seek a variety of gender-affirming health and educational services, including voice modification from speech-language pathologists. Measuring the client's self-perception of their communication experiences is crucial for providing client-centered services and measuring outcomes. However, there is currently no validated assessment tool for the nonbinary population, a part of the transgender population. This study explores the voice-related concerns and experiences among the nonbinary population to create a valid measure of their self-perception of voice. Ten nonbinary individuals were surveyed about their voice-related concerns and experiences. A thematic analysis of the responses led to the development of the questionnaire, titled the Voice-related Experiences of Nonbinary Individuals. The questionnaire was systematically evaluated for its content validity by a panel of speech-language pathologist experts in transgender voice services. Outcomes of this analysis supported the measure's content validity and motivated further revisions. This is the first assessment tool that measures self-perception of voice and voice-related experiences for nonbinary individuals. Initial psychometric testing supported its content validity and further research is needed for large-scale testing of validity and reliability.
Key Words
INTRODUCTION
Transgender individuals may seek a variety of healthcare services for gender-affirming care. A transgender person is someone whose gender differs from the gender associated with their sex assigned at birth. Some transgender individuals identify exclusively with a binary gender (eg, men or women), whereas others do not (ie, nonbinary individuals). Some transgender individuals receive services to modify characteristics of their voice and communication style from speech-language pathologists (SLPs). To-date, the majority of research on gender-affirming voice services has focused on transgender men and women,
2
with less attention paid to nonbinary people, yet they constitute up to 35% of the transgender population.- Davies S.
- Papp V.G.
- Antoni C.
Voice and communication change for gender nonconforming individuals: giving voice to the person inside.
Int J Transgenderism. 2015; https://doi.org/10.1080/15532739.2015.1075931
3
While most frameworks related to gender posit nonbinary individuals as transgender, some nonbinary people do not identify with the term transgender because they feel their experiences are incongruous with culturally dominant assumptions about the transgender experience.4
In this article, the authors consider nonbinary individuals as part of the transgender population.- Garrison S
On the Limits of “Trans Enough”: authenticating trans identity narratives.
Gend Soc. 2018; https://doi.org/10.1177/0891243218780299
The available yet limited research suggests that some nonbinary people are interested in gender-affirming voice services. In a 2015 survey of over 27,000 transgender adults in the United States, among nonbinary respondents who were assigned male at birth, 2% reported receiving voice therapy, 34% reported wanting it someday, 29% reported feeling unsure if they wanted it, and 35% reported not wanting voice therapy.
3
In the same survey, nonbinary participants assigned female at birth were not presented with questions related to voice therapy. As a result, there was no information collected about whether they had received, or had interest in receiving, voice therapy. Some nonbinary individuals have received or desire gender-affirming voice therapy;3
however, it is unclear how much of this population is interested in the service.Speech-language pathologists providing voice and communication services to transgender clients often collect clients' perceptions of their voice through self-report measures. However, current measures to evaluate perception of voice and voice-related experiences were not developed for, nor validated on, the nonbinary population. The Transgender Self-Evaluation Questionnaire (TSEQ)
5
was developed for the transgender population based on the Voice Handicap Index.6
The TSEQ is composed of statements to which the client responds on a five-point Likert-scale from “never” to “always.” These questions are designed to address the functional, physical, and emotional aspects of the client's voice. The SLP uses the overall score to ascertain the extent to which the client's voice impacts their activity restrictions and life participation, with a higher score signifying greater impact.- Jacobson B.H.
- et al.
The Voice Handicap Index (VHI): development and validation.
Am J Speech Language Pathol. 1997; https://doi.org/10.1044/1058-0360.0603.66
5
,7
Hancock et al- Dacakis G.
- Davies S.
- Oates J.M.
- et al.
Development and preliminary evaluation of the transsexual voice questionnaire for male-to-female transsexuals.
J. Voice. 2013; https://doi.org/10.1016/j.jvoice.2012.11.005
8
examined the relationship between quality of life (QoL) and voice femininity. The authors administered the TSEQ to 20 transgender women who recorded samples of their speaking voices. The TSEQ scores were used to reflect QoL. The speakers and unfamiliar listeners rated the femininity of the voice samples. The authors found a correlation between the transgender womens' QoL scores and their ratings of the femininity of their voice sample. Additionally, their QoL scores correlated with unfamiliar listeners’ femininity ratings of the samples. A client's subjective measurement of their voice can provide insightful information for clinical decision making.- Hancock A.B.
- Krissinger J.
- Owen K.
Voice perceptions and quality of life of transgender people.
J Voice. 2011; https://doi.org/10.1016/j.jvoice.2010.07.013
To the best of the authors’ knowledge, there is currently one peer-reviewed, published voice-related measure that underwent psychometric evaluation. Dacakis et al
7
collected feedback on the TSEQ from SLPs and transgender women. The researchers also completed a thematic analysis of data from Byrne's dissertation- Dacakis G.
- Davies S.
- Oates J.M.
- et al.
Development and preliminary evaluation of the transsexual voice questionnaire for male-to-female transsexuals.
J. Voice. 2013; https://doi.org/10.1016/j.jvoice.2012.11.005
9
featuring interviews with transgender women about the role of communication in their lives. The analysis identified additional and more nuanced voice-related experiences, warranting the creation of a new measure: the Trans Woman Voice Questionnaire (TWVQ), formerly known as the Transsexual Voice Questionnaire (TVQMtF). The TWVQ includes 30 voice-related statements. The client responds to each of these statements on a 4-point Likert scale ranging from “never or rarely” to “usually or always.” After development of the TWVQ, Dacakis et al7
conducted further psychometric testing of this questionnaire. One resulting analysis found that the TWVQ has strong internal consistency and test-retest reliability.- Dacakis G.
- Davies S.
- Oates J.M.
- et al.
Development and preliminary evaluation of the transsexual voice questionnaire for male-to-female transsexuals.
J. Voice. 2013; https://doi.org/10.1016/j.jvoice.2012.11.005
In a related study, Dacakis et al
10
explored the TWVQ's construct validity and further evaluated its reliability through principal components analysis. Researchers found that the questionnaire had strong reliability through its high internal consistency. The principal components analysis revealed a high construct validity. Similarly, in 2015, Davies and Johnston- Dacakis G.
- Oates J.M.
- Douglas J.M.
Further evidence of the construct validity of the transsexual voice questionnaire (TVQMtF) using principal components Analysis.
J Voice. 2017; https://doi.org/10.1016/j.jvoice.2016.07.001
11
completed an analysis that explored the content validity of the TWVQ. They grouped the TWVQ items into categories based on the questions’ content; example categories included “pitch” and “effect of voice on social interaction.” The researchers interviewed five transgender women who were asked to comment on their voices and voice-related experiences. They used the predetermined themes to classify topics that were spontaneously addressed during these interviews. A total of 29 of the 30 items in the TWVQ were spontaneously addressed. Researchers also found a significant correlation between the frequency that a given theme was spontaneously addressed and the average rating of that theme on the TWVQ. Overall, the TWVQ provides clinicians a valuable measure to use when providing voice services to transgender women. However, there is no validated measure to assess nonbinary individuals’ voice-related concerns and experiences.Current practice in gender-affirming voice and communication services primarily consists of voice masculinization or feminization.
2
This is reflected in the existing voice-related questionnaires for the transgender community. For instance, the TWVQ was not developed for the nonbinary population. The psychometric properties of the TWVQ apply only to transgender women. The unpublished TSEQ contains items based on the binary concept of gender, assuming the client is a transgender man or woman (eg, “I feel my voice gets in the way of me living as a woman/man”). Furthermore, items in both questionnaires assume the client desires a masculine or feminine voice, which may not represent the target voice(s) of the nonbinary community. For example, one item reads, “I am envious of other women/men who have more feminine/masculine voices than mine.” This item assumes that the client is a woman or man and desires a feminine or masculine voice, respectively. Nonbinary individuals’ target voices may be more or less feminine or masculine relative to their current voices. They may also desire a voice that is gender-neutral or gender-expansive (able to alternate between masculine, gender-neutral, and feminine voices). Development of a new questionnaire is needed for assessing self-perception of voice and voice-related experiences among the nonbinary population. Availability of such a questionnaire in clinical practice would affirm their unique experiences as nonbinary individuals and facilitate their participation in voice and communication services.- Davies S.
- Papp V.G.
- Antoni C.
Voice and communication change for gender nonconforming individuals: giving voice to the person inside.
Int J Transgenderism. 2015; https://doi.org/10.1080/15532739.2015.1075931
Nonbinary individuals pursuing voice therapy would benefit from an apt assessment of the impact of their voice on their lives. In this study, the authors aimed to gain insight into nonbinary individuals' voice-related experiences, concerns, and their desired voices. This data led to the development of the first validated assessment tool that measures self-perception of voice and voice-related experiences among nonbinary individuals. In the first phase of this study, nonbinary individuals were surveyed about their self-perception of voice and voice-related experiences. A thematic analysis was performed on the survey responses, which resulted in the creation of a subjective voice questionnaire for the nonbinary population. In the second phase, preliminary content validity of the measure was evaluated through expert review.
PHASE ONE: QUESTIONNAIRE DEVELOPMENT
Method
Participants
Ten participants were recruited to participate in this study. Recruitment consisted of postings in online groups and forums created for the lesbian, gay, bisexual, and transgender (LGBTQ+) population, as well as distribution of flyers and emails among LGBTQ+ resource and health centers. To be eligible, participants had to be over the age of 18, identify as nonbinary, and have a desire to modify their voice. There were 13 eligible respondents in total, among which three were excluded because they did not progress to the final submission page to complete the survey. No compensation was provided for this study. All participants in this study are nonbinary adults who reside in a diverse metropolitan area within the United States. All participants received a consent notice for this study. This study received San José State University's Human Subjects Research Institutional Review Board approval (#S19101).
Materials
An anonymous online survey was developed to gather qualitative data for the development of the questionnaire (see Appendix A). The term “survey” will be used to reference this initial step, whereas “questionnaire” will be used exclusively to refer to the assessment tool developed from this study. Previous research suggests that participants experience less anxiety, higher self-esteem, and less pressure to respond in a socially desirable way when using an anonymous web-based survey compared to nonanonymous web-based surveys.
12
Participants may be more inclined to respond to personal items and share sensitive data when responding to web-based surveys compared to phone surveys, and they may be more willing to share sensitive information when responding to a self-administered survey compared to an interviewer-administered one.- Joinson A.
Social desirability, anonymity, and Internet-based questionnaires.
Behav Res Methods Instruments Comput. 1999; https://doi.org/10.3758/BF03200723
13
,- Parks K.
- Pardi A.M.
- Bradizza C.M.
Collecting data on alcohol use and alcohol-related victimization: a comparison of telephone and web-based survey methods.
J Stud Alcohol. 2006; https://doi.org/10.15288/jsa.2006.67.318
14
- Rogers S.M.
- Miller H.G.
- Turner C.F.
Effects of interview mode on bias in survey measurements of drug use: do respondent characteristics make a difference?.
Subst U5se Misuse. 1998; https://doi.org/10.3109/10826089809069820
This survey was developed based on the International Classification of Functioning, Disability and Health (ICF) created by the World Health Organization.
15
To reflect this model, the survey included three sections, which corresponded to components of the ICF: (1) activities and participation, (2) body functions and structures, and (3) environmental and personal factors. Each of these three survey sections began with one general open-ended question. These initial questions were designed to elicit the participants’ voice-related experiences and self-perceptions of voice as they related to the section topic. Participants were encouraged to share experiences in all three of these areas in order to capture a more comprehensive understanding of their voice-related experiences (positive, neutral and/or negative) related to the three ICF components. While participants could proceed through the survey without answering the main question, if any of the three main questions were not answered, the participant's response was deleted and excluded from analysis since the response did not provide sufficient information for assessing the overall experiences in all three ICF domains.After each main question, three follow-up questions pertinent to that topic were presented. These were designed to elicit more detailed responses about certain components of the general question. Participants were notified in each section that the follow-up questions were optional and could be left blank if they had already addressed them in response to the main question. One follow-up question asked participants to list up to five situations in which they had concerns about their voice. They were asked to then rank them from most to least comfortable and briefly share their concerns in these situations. An additional question asked participants to describe their current voice as well as their ideal voice, in order to confirm they met eligibility criteria for this study, and to provide insight into the personal perception of voices among nonbinary individuals and their interest in modifying their voices.
Following the initial development of the survey questions, a pilot study took place with three nonbinary participants, all of whom were personal contacts of the first author. Modifications to the survey were made as a result of the pilot study responses to improve clarity and to encourage participants to write continuously for five minutes in response to the main questions, in order to elicit more lengthy responses.
Procedure
Interested participants were directed to the anonymous online survey through a link provided on the recruitment material. Prospective participants first completed a screening to determine eligibility for the study. Participants were asked to confirm they were over the age of 18 and were nonbinary. In order to confirm that participants desired to modify their voice, they were asked to describe their voice as either “very masculine,” “somewhat masculine,” “gender neutral or androgynous,” “somewhat feminine,” or “very feminine.” They were then asked to identify which of these descriptors represented their ideal voice. They were also provided the additional options of “more than one” and “I like my voice as it is/ I don't have a desire to modify my voice.” If “more than one” was selected, participants were asked to briefly describe their ideal voice. If they selected “I like my voice as is...” they did not meet the inclusion criteria. Individuals who met the inclusion criteria proceeded to complete the survey. Individuals who did not meet the criteria did not receive access to the survey and their responses were deleted. This was the case for one prospective participant.
Eligible participants were then directed to answer the three survey sections in the following order: (1) activities and participation, (2) body function and structure, and (3) environmental and personal factors. Although participants could progress through the survey without responding to the first main question in each section, all participants responded to these three main questions. Upon completion of all three sections, participants were presented with the option to provide their email address for potential follow-up contact. These email addresses were collected through a second survey to ensure their responses remained anonymous.
Thematic coding and analysis
Individual participant responses were unitized. The data were then imported into NVivo (version 12.4) for thematic coding and analysis. There were two rounds of thematic coding: coding of the data into three main themes, then identification and coding of subthemes.
Data unitization
The respondents’ free-response data were separated into units following recommendations from Campbell et al.
16
A unit can be considered any part of a participant's response that is judged to represent one idea, action, or event. It could be one or multiple sentences long. A standardized method of unitization is needed to establish inter-coder agreement. The first author and a second coder, a graduate student with experience providing transgender voice services, discussed the principles of data unitization and, together, completed the unitization of data from one pilot study participant. Then, the first author unitized data from a second pilot study participant. The second coder reviewed the unitization and had the opportunity to disagree with it, which led to further discussion and revision. Thirty percent of the responses were unitized this way. The remaining responses were independently unitized by the first author.- Campbell J.L.
- Quincy C.
- Osserman J.
- et al.
Coding in-depth semistructured interviews: problems of unitization and intercoder reliability and agreement.
Sociol Methods Res. 2013; https://doi.org/10.1177/0049124113500475
Coding
Both coders discussed the criteria needed to associate the following predetermined themes with the units: functional, physical, and emotional. These codes were selected in accordance with the ICF. Together, the coders created a codebook that specified the criteria needed to apply each code to a unit following the recommendations of Campbell et al.
16
The now-unitized participant data were imported into NVivo for coding. Together, both coders reviewed the units from the first pilot study participant. Utilizing the codebook, they determined which code(s) to apply to each unit. One, two, or all three codes could be applied to each unit. The coders then independently coded the unitized data from the second pilot study to evaluate inter-coder agreement. Next, the two coders discussed areas of disagreement to reach negotiated agreement. The coders made updates to the codebook as needed to improve future agreements. This process was completed again for the second pilot study participant's data and 30% of the participants’ data, with a percentage agreement of 94.3% (range: 90.7% - 98.5%), and a newly-created theme, “positive,” for units that included positivity, such as moments of voice satisfaction. The inter-coder reliability was examined by calculating percent agreement, following the procedure recommended by Campbell et al for qualitative research, and the agreement rate was considered acceptable.- Campbell J.L.
- Quincy C.
- Osserman J.
- et al.
Coding in-depth semistructured interviews: problems of unitization and intercoder reliability and agreement.
Sociol Methods Res. 2013; https://doi.org/10.1177/0049124113500475
16
The remaining participants’ data were coded individually by the author.- Campbell J.L.
- Quincy C.
- Osserman J.
- et al.
Coding in-depth semistructured interviews: problems of unitization and intercoder reliability and agreement.
Sociol Methods Res. 2013; https://doi.org/10.1177/0049124113500475
Once all of the data were coded, each unit was reviewed for emerging subthemes. Identification of subthemes was done using adapted principles of grounded theory analysis.
17
This approach allows for thoughtful, exploratory, and iterative analysis of the data within the context of the research, which aimed to understand voice-related experiences and concerns among the nonbinary population. Through this approach, the first author repeatedly analyzed the participants’ data to code for subthemes. The coding of subthemes was completed by the author with regular discussion with the second author. This process involved identifying the subtheme(s) represented by each existing unit for all participants. After that was completed, the units and subthemes were reviewed for either further consolidation or expansion of the subthemes in order to reflect a growing understanding of them. For example, two participants' data units were coded as “pitch increase due to stress” and “pitch influenced by negative emotion,” respectively. After revisiting these subthemes and their associated units during the second round of coding, these subthemes were consolidated to “inability to control voice/unintended fluctuations.” After repeated exposure and coding of the data, additional subthemes for “pitch” and “volume” were created to further specify units. The resulting unique subthemes were further evaluated for creation of questionnaire items.- Charmaz K.
- Belgrave L.L.
Qualitative interviewing and grounded theory analysis.
in: The SAGE Handbook of Interview Research: The Complexity of the Craft. 2012https://doi.org/10.4135/9781452218403.n25
Subthemes that did not fit within the context of this research (did not contribute to the research goals or answer the research questions) were disregarded. For example, subthemes such as, “asked by mother not to change voice pitch due to her low-frequency hearing loss,” were judged as highly-specific experiences that could not be grouped with other subthemes and/or did not contribute to the research questions.
Creation of questionnaire items
For each unique subtheme, the researchers determined the number of unique participant responses present. If a subtheme contained data units from at least 30% of participants, an item was created for the questionnaire to address that subtheme. The 30% criterion was chosen in an effort to capture a broad range of shared individual experiences and yet not too conservative or liberal, considering the potentially heterogenous nature of the population.
RESULTS
Desired voices
Of the 10 participants, four desired a “gender neutral or androgynous” voice and three desired a “somewhat masculine” voice. The remaining three participants each desired a “very masculine,” “very feminine,” and “somewhat feminine” voice, respectively. Half of participants desired a voice that is considered more gender neutral than how they perceived their voice. Among the other half of participants, four desired a voice that was more masculine than their current voice and one participant desired a more feminine voice (see Table 1).
Table 1Participants’ Self-described Current Voice vs Desired Voice
| Participant | Current Voice | Desired Voice |
|---|---|---|
| 1 | Somewhat feminine | Gender neutral or androgynous |
| 2 | Somewhat masculine | Gender neutral or androgynous |
| 3 | Very feminine | Gender neutral or androgynous |
| 4 | Somewhat masculine | Very masculine |
| 5 | Somewhat feminine | Somewhat masculine |
| 6 | Somewhat masculine | Very feminine |
| 7 | Very feminine | Somewhat feminine |
| 8 | Very feminine | Somewhat masculine |
| 9 | Gender neutral or androgynous | Somewhat masculine |
| 10 | Somewhat feminine | Gender neutral or androgynous |
Challenging voice-related situations
All participants responded to this optional follow-up question. When asked to list, rank, and elaborate on situations in which the participants experienced voice-related concerns, the most frequently mentioned situation was talking on the phone (60% of participants), followed by talking to strangers and at work (40%), public speaking or presentations and talking in public (30%), speaking at the drive-through or ordering food at a restaurant, engaging in legal situations, talking to self, talking to familiar partners, and continuing a conversation after being misgendered (20%), talking to other transgender people, talking when presenting highly feminine or masculine, and continuing a conversation after being outed (10%). Common reasons for voice-related concerns across situations included fear of being misgendered, self-consciousness or lack of confidence, fear that others think negatively of their voice, fear that others will trivialize what they are saying because of their voice, and worry based on previous negative experiences in the specified situation. These situations and concerns were also reflected and represented in the free-response sections throughout the survey.
Codes and emerging subthemes
Data unitization resulted in a total of 175 voice-related response units across participants (M = 17.5 per participant; range 9 - 28). All units were first coded based on the predetermined themes (functional, physical, emotional, positive) and then subthemes, followed by consolidation and/or expansion for identified unique subthemes. Coding resulted in a total of 666 codes across participants (M = 66.6; range: 40 - 110), including 351 theme codes (130 functional, 129 physical, 57 emotional, and 35 positive) and 315 subtheme codes. Following subtheme consolidation and expansion, a total of 28 unique subthemes were identified (10 under functional, 14 under physical and four under emotional).
Creation of questionnaire items
Items were developed based on unique subthemes shared by at least 30% of participants. A total of 18 subthemes met the criteria for creation of a questionnaire item. For example, the subtheme, “inability to control voice and unintended fluctuations” contained data units from 90% of respondents. Thus, an item was created on the questionnaire to reflect this theme and its corresponding units. The resulting item was, “My voice changes unexpectedly depending on the situation.” This process was completed for all subthemes, resulting in the first draft of the measure, which contained 18 items.
PHASE TWO: EXPERT REVIEW
To evaluate content validity of the questionnaire, a panel of experts was recruited to review the measure. Content validity measures the extent to which a scale contains suitable and representative items of a construct. A popular way to measure content validity of a multi-item scale is through expert review.
18
In this phase, experts were asked to rate each item based on its relevance to the nonbinary population, given their clinical knowledge and experience with this community.- Polit D.F.
- Tatano Beck C.
- Owen S.F
Is the CVI an acceptable indicator of content validity? Appraisal and recommendations.
Res Nurs Health. 2007; https://doi.org/10.1002/nur.20199
Method
Participants
Four experts participated in this phase of study. This sample size was determined based on the criteria established in 1986 by Lynn,
19
which requires a minimum of three participants in order to establish content validity via expert review. While having at least five participations is optimal, three is acceptable if the field has relatively few experts. Because this is a specialized practice within the scope of speech-language pathologists, and because the field of transgender voice modification is relatively new, a minimum of three experts was determined to be appropriate for this phase of the study.- Lynn M.R.
Determination and quantification of content validity.
Nurs Res. 1986; https://doi.org/10.1097/00006199-198611000-00017
Clinicians who were thought to meet the expert criteria were identified. They were invited over email to participate in the study. Expert criteria were determined by adopting the augmentative and alternative communication personnel framework developed in 2008 by Beukelman et al.
20
Experts needed to have at least five years of experience providing transgender voice and communication therapy, including services to the nonbinary population, and at least 50% of their current or most recent caseload dedicated to this service, or comparable experience. Examples of comparable experience include three years of experience with 100% of the caseload dedicated to the service, or 15 years of experience with current efforts dedicated to teaching and supervising graduate students in this practice area. Interested participants who met the inclusion criteria were directed to the online survey. Participants were asked to share information about their experience in the area of transgender voice and communication therapy.- Beukelman D.R.
- Ball L.J.
- Fager S.
An AAC personnel framework: adults with acquired complex communication needs.
AAC Augment Altern Commun. 2008; https://doi.org/10.1080/07434610802388477
The four experts had an average of nearly 15 years of experience providing clinical services to gender nonconforming and nonbinary individuals (range: 3–31 years). Full-time clinicians, university professors, and researchers were represented within the expert panel. No compensation was provided for participation in this study. All participants received a consent notice for this study. This study received San José State University's Human Subjects Research Institutional Review Board approval (#S19101).
Materials
An online survey was constructed in Qualtrics. This survey followed the judgement-quantification stage of establishing content validity.
19
This stage requires experts to assess the content relevance of each individual item in the measure, as well as the measure as a whole. The survey listed each item individually, and experts provided a relevance rating for each. Additionally, a text box was created for participants to provide feedback on each individual item. The survey also featured questions regarding the clarity and relevance of the measure as a whole.- Lynn M.R.
Determination and quantification of content validity.
Nurs Res. 1986; https://doi.org/10.1097/00006199-198611000-00017
Procedure
Participants were provided the instructions that a nonbinary client taking the questionnaire would use. The instructions read as follows: “For each of the following statements, please circle the rating that fits best based on your experience as a nonbinary individual.” Then, they were asked to review each item based on their clinical knowledge of, and experience with, the nonbinary population. Experts were asked to rate items individually on a scale of 1-4 (1 is “not relevant,” 2 is “somewhat relevant,” 3 is “quite relevant,” and 4 is “highly relevant”). In addition to rating, they were invited to provide written feedback on each item. Then, they indicated the extent to which they agreed with a series of statements, including: “the instructions for this assessment are easy to follow,” “this assessment is easy to follow,” and “this assessment is useful.” Experts were also asked to provide written feedback on the overall measure.
Data analysis
Content validity was determined for each individual item as well as the measure overall. Content validity indices (CVIs) are considered an index of inter-rater agreement that measures the extent to which a group of raters agree on an interpretation of the measure.
18
Researchers have critiqued the CVI for its lack of calculation for chance agreement on relevance. As a result, chance agreement was also calculated for each item.- Polit D.F.
- Tatano Beck C.
- Owen S.F
Is the CVI an acceptable indicator of content validity? Appraisal and recommendations.
Res Nurs Health. 2007; https://doi.org/10.1002/nur.20199
18
- Polit D.F.
- Tatano Beck C.
- Owen S.F
Is the CVI an acceptable indicator of content validity? Appraisal and recommendations.
Res Nurs Health. 2007; https://doi.org/10.1002/nur.20199
Item validity was measured by the item content validity index (I-CVI), which is the percentage of experts rating an item as “relevant” (a rating of “3” or “4”). When there are only four expert raters, all experts must agree an item is relevant for it to be considered valid.
19
Additionally, probability of chance agreement (pc) was calculated alongside a modified kappa statistic (κ*). This statistic is an index of agreement on relevance; the result of κ* can be described as fair, good, or excellent.- Lynn M.R.
Determination and quantification of content validity.
Nurs Res. 1986; https://doi.org/10.1097/00006199-198611000-00017
18
- Polit D.F.
- Tatano Beck C.
- Owen S.F
Is the CVI an acceptable indicator of content validity? Appraisal and recommendations.
Res Nurs Health. 2007; https://doi.org/10.1002/nur.20199
The validity of the entire measure (measure validity) was measured by the scale content validity index (S-CVI). Two calculations of S-CVI were completed. One calculates the average of the I-CVIs (S-CVI/ Ave), and the other more rigorous measure calculates the proportion of items rated as relevant by all experts, known as universal agreement (S-CVI/ UA).
18
- Polit D.F.
- Tatano Beck C.
- Owen S.F
Is the CVI an acceptable indicator of content validity? Appraisal and recommendations.
Res Nurs Health. 2007; https://doi.org/10.1002/nur.20199
Measure revisions
Based on the I-CVIs, the authors decided which items to retain, discard, or revise. Items that have an I-CVI of 1 are considered valid and are retained. Items with an I-CVI of “somewhat lower” of 0.78 are eligible for revision, while items with an I-CVI of 0.5 or lower are unacceptable and should be deleted.
18
- Polit D.F.
- Tatano Beck C.
- Owen S.F
Is the CVI an acceptable indicator of content validity? Appraisal and recommendations.
Res Nurs Health. 2007; https://doi.org/10.1002/nur.20199
RESULTS
Item validity
A total of 16 of the 18 items were rated as relevant by all four experts, meeting the criterion for them to be considered valid. The calculated κ* classified these items as “excellent” (see Table 2).
Table 2Item Validity Analyses
| Item | I-CVI | pc | κ* |
|---|---|---|---|
| 1 | 1 | 0.063 | 1 = excellent |
| 2 | 1 | 0.063 | 1 = excellent |
| 3 | 1 | 0.063 | 1 = excellent |
| 4 | 1 | 0.063 | 1 = excellent |
| 5 | 0.5 | N/A | N/A |
| 6 | 1 | 0.063 | 1 = excellent |
| 7 | 1 | 0.063 | 1 = excellent |
| 8 | 1 | 0.063 | 1 = excellent |
| 9 | 1 | 0.063 | 1 = excellent |
| 10 | 1 | 0.063 | 1 = excellent |
| 11 | 1 | 0.063 | 1 = excellent |
| 12 | 0.75 | 0.25 | 0.67 = good |
| 13 | 1 | 0.063 | 1 = excellent |
| 14 | 1 | 0.063 | 1 = excellent |
| 15 | 1 | 0.063 | 1 = excellent |
| 16 | 1 | 0.063 | 1 = excellent |
| 17 | 1 | 0.063 | 1 = excellent |
| 18 | 1 | 0.063 | 1 = excellent |
Measure validity
The S-CVI/ Ave was .96, and the S-CVI/ UA was .83, suggesting the measure's good to excellent content validity, following interpretation guidelines suggested by prior research.
18
,- Polit D.F.
- Tatano Beck C.
- Owen S.F
Is the CVI an acceptable indicator of content validity? Appraisal and recommendations.
Res Nurs Health. 2007; https://doi.org/10.1002/nur.20199
21
Feedback on the measure as a whole also suggests that this measure is valid. All experts indicated that they “agreed” or “strongly agreed” that the measure's instructions were easy to follow, the measure itself was easy to follow, and the measure was useful.Measure revisions
All experts rated 16 items as relevant; these items each had an I-CVI of 1, indicating they are valid. These items also had a κ* of 1, which evaluates them as excellent. Minor revisions were made to some items based on written expert feedback in order to improve clarity and minimize bias. The two items that had an I-CVI of less than 1, thus considered invalid, underwent further review.
18
,- Polit D.F.
- Tatano Beck C.
- Owen S.F
Is the CVI an acceptable indicator of content validity? Appraisal and recommendations.
Res Nurs Health. 2007; https://doi.org/10.1002/nur.20199
19
One of these items, “People react negatively to my voice,” had an I-CVI of 0.75 and was eligible for revision. The expert who did not rate this item as relevant was consulted over the phone to elaborate on her written feedback for this item, which led to its modification. The expert expressed concerns with the idea that individuals react negatively to voice alone; while one may feel that someone's negative reaction to their voice is due to voice alone, that may not be true. As a result, the item was modified: “I suspect that people react negatively to my voice.”- Lynn M.R.
Determination and quantification of content validity.
Nurs Res. 1986; https://doi.org/10.1097/00006199-198611000-00017
The other item that did not receive an I-CVI of 1 was, “My voice is quieter in the evening.” This item received an I-CVI of 0.5, which warrants deletion from the measure.
18
After further review from experts and upon revisiting the data from phase one, it was determined that this question was not specific enough to the nonbinary population. The participants who discussed this theme were in professions that involved high voice-use. As a result, their experiences may have reflected fatigue due to voice use unrelated to any attempts to modify their voice. After these edits were made, the second draft of the measure was completed, titled the Voice-related Experiences of Nonbinary Individuals (VENI), with a total of 17 items (seven associated with the theme physical, eight with functional, and two with emotional). See Appendix B for the questionnaire.- Polit D.F.
- Tatano Beck C.
- Owen S.F
Is the CVI an acceptable indicator of content validity? Appraisal and recommendations.
Res Nurs Health. 2007; https://doi.org/10.1002/nur.20199
DISCUSSION
Previous research related to nonbinary voice, though scarce, suggests that some nonbinary individuals experience voice-related concerns and have a desire to modify their voices;
3
however, no published assessment tool is available for this population. This motivated the development of a self-report measure for clinical use when supporting this population. This study aimed to explore the voice-related concerns and experiences of the nonbinary population to develop a subjective voice assessment for nonbinary individuals seeking gender-affirming communication services. Responses from the survey indicated that among nonbinary people who want to modify their voice, some may desire feminine, masculine, gender-neutral, or gender-expansive voices. Given the varied target voices of this population, SLPs should provide clinical measures and services that do not assume a singular desired voice for nonbinary individuals.The benefits of the creation of a valid and reliable questionnaire for nonbinary individuals who pursue voice modification are manifold. This questionnaire can provide SLPs with insightful information about their nonbinary clients’ voice-related concerns, experiences, and target voice(s). It can also help SLPs track their clients’ self-perception changes over time. This, in turn, will assist SLPs in developing appropriate individualized, client-centered interventions for therapy. By doing so, SLPs will support their nonbinary clients as they learn to modify their communication styles to meet their daily communication needs. As a result of gender-affirming care, nonbinary clients may have improved self-confidence, resilience, social and work connections, and an overall increased QoL.
22
This study provides initial development and validation of a questionnaire that could ultimately serve as a clinical tool for improving the quality of therapeutic decision-making and intervention.- Bockting W
- Coleman E
- Deutsch MB
- et al.
Adult development and quality of life of transgender and gender nonconforming people.
Curr Opin Endocrinol Diabetes Obes. 2016; https://doi.org/10.1097/MED.0000000000000232
Resulting themes from the phase one analysis, alongside expert feedback, led to the creation of the Voice-related Experiences of Nonbinary Individuals (VENI). A few of the VENI's items share similar themes to those in the TWVQ, suggesting there are shared voice-related experiences between nonbinary individuals and transgender women. However, many items reflect inherently different experiences and concerns. For example, one item in the VENI is, “My voice changes unexpectedly depending on the situation;” no similar item exists in the TWVQ. Additionally, the TWVQ contains an item that reads, “My voice makes me feel less feminine than I would like,” and no item in the VENI presumes the client desires a more feminine voice. While the VENI can provide valuable information to clinicians, it should not replace medically-oriented voice questionnaires, such as the Voice Handicap Index, because the VENI does not assess experiences related to vocal pathology symptoms.
One limitation of this study was that researchers were not able to ask follow-up questions to participants in phase one to gain further insight into their specific responses. While it may have been useful to follow up with participants over phone or video calls, our participants’ initial responses indicated high discomfort with talking on the phone and speaking with unfamiliar interlocutors. Thus, it is possible that follow-up interviews may have imposed high risk of bias in participant self-selection or that participants may have been less comfortable, open, and honest. Instead, this study examined content validity through an expert panel in phase two. Future research could include a nonanonymous survey, which could include follow-up interviews.
Further research is needed to examine the questionnaire's construct and criterion validity and test-retest reliability. Additionally, this study only explored voice-related experiences. More research is needed to examine the overall communication experiences in this population, considering both verbal and nonverbal communication.
CONCLUSION
Client self-report measurements of voice are clinically useful for SLPs when assessing the wide range of potential needs among transgender individuals seeking voice and communication modification services. Existing self-reported measures of voice for the transgender population are not appropriate to use for nonbinary clients. This study developed the measure Voice-related Experiences of Nonbinary Individuals (VENI) to meet the assessment need for serving nonbinary individuals. While there is currently no normative data for this measure, the quantitative scores can be used to identify client needs, track progress, and document changes in voice-related experiences. Initial evaluation supported its content validity, and further psychometric evaluation is needed. Overall, the VENI provides a questionnaire specifically designed for assessing the diverse voice-related experiences among nonbinary individuals, and contributes to developing client-centered, gender-affirming communication services.
Acknowledgments
This research was supported in part by the San José State University Alumni Association Dean's Scholarship. The authors would like to thank all the participants of this study.
They would also like to thank Jessica Buckley for assisting in data analysis.
Appendix A: Questions used in Phase One Survey
- I.Screening questions to determine eligibility:
- •Do you identify as nonbinary, genderqueer, genderfluid, agender, or as another gender nonconforming identity?
- •Are you over the age of 18?
- •What country do you reside in?
- •From your perspective, how would you best describe your voice?
- ○Very masculine
- ○Somewhat masculine
- ○Gender neutral or androgynous
- ○Somewhat feminine
- ○Very feminine
- ○
- •Which of the following best represents your ideal voice?
- ○I like my voice as it is / I do not desire to modify my voice
- ○Very masculine
- ○Somewhat masculine
- ○Gender neutral or androgynous
- ○Somewhat feminine
- ○Very feminine
- ○More than one of the above
- ○
- •
- II.Survey questions (primary and follow-up questions) corresponding to the three ICF components:
Activity and Participation
- 1.Describe your concerns with your voice and how it affects your day-to-day activities. For example, are there things you avoid doing because of your concerns with your voice? Please try to write continuously for five minutes.1a. Think about your daily activities and daily routine. Please list five situations or contexts where you have concerns about your voice. Then, rank them from most to least comfortable. Ranking a situation as a “5” indicates the most amount of discomfort.1b. Please briefly describe your concerns related to your voice in these situations (text entry for each situation).1c. Please share how, if at all, your voice influences your interactions with interpersonal relationships. Examples include, but are not limited to, relationships with romantic partners, family, strangers, formal relationships (work setting), and/or informal relationships (neighbors, friends, peers).
- 1.
Body Function and Structure:
- 2.How, if at all, do you experience physical discomfort related to your voice? (possible examples include, but are not limited to, vocal strain or hoarseness, vocal fatigue, voice “cracking.”)2a. Describe any physical injury, disorder, or structural abnormality that influences your vocal quality (examples include head or neck trauma, infections, impaired respiration, etc.).2b. Describe how, if at all, aspects of your voice (loudness, speed, pitch, etc.) change throughout the day.2c. Describe the ways (if any) you have attempted to modify aspects of your voice (such as pitch). What were the results?
- 2.
Environmental and Personal Factors
- 3.Please describe how, if at all, your environment influences your voice. Examples include, but are not limited to, the attitudes and thoughts of your family, friends, strangers, and/or societal attitudes. Please try to write continuously for five minutes.3a. Please share your thoughts about how people you know perceive your voice.3b. Please share your thoughts about how people you don't know perceive your voice.3c. Please share how, if at all, societal attitudes and norms influence your voice.
- 3.
Appendix B: Voice-related Experiences of Nonbinary Individuals (VENI)
Rating scale:
- 1 = never or rarely
- 2 = sometimes
- 3 = often
- 4 = usually or always
For each of the following statements, please circle the rating that fits best based on your experience as a nonbinary individual.
Tabled
1
| 1. The quality of my voice varies throughout the day. | 1 | 2 | 3 | 4 |
| 2. It is difficult to control the pitch of my voice. | 1 | 2 | 3 | 4 |
| 3. Some emotions cause my pitch to change beyond my control. | 1 | 2 | 3 | 4 |
| 4. My voice changes unexpectedly depending on the situation. | 1 | 2 | 3 | 4 |
| 5. My pitch becomes less desirable by the end of the day. | 1 | 2 | 3 | 4 |
| 6. I experience strain when trying to make my voice sound like I want it to. | 1 | 2 | 3 | 4 |
| 7. It takes a lot of effort and focus to sound the way I want to. | 1 | 2 | 3 | 4 |
| 8. I speak in public less often than I would like to because of my voice. | 1 | 2 | 3 | 4 |
| 9. I suspect that people misgender me because of my voice. | 1 | 2 | 3 | 4 |
| 10. I speak to people close to me less often than I would like because of my voice. | 1 | 2 | 3 | 4 |
| 11. I suspect that people react negatively to my voice. | 1 | 2 | 3 | 4 |
| 12. My voice gets in the way of me living as myself. | 1 | 2 | 3 | 4 |
| 13. I dislike the sound of my voice. | 1 | 2 | 3 | 4 |
| 14. I feel that others take me less seriously because of my voice. | 1 | 2 | 3 | 4 |
| 15. I feel that others think poorly of me because of my voice. | 1 | 2 | 3 | 4 |
| 16. I'm uncomfortable talking on the phone because I might be misgendered. | 1 | 2 | 3 | 4 |
| 17. I worry about how strangers perceive my voice. | 1 | 2 | 3 | 4 |
Note: The three primary themes and their corresponding question items are listed below.
Physical Items: 1, 2, 3, 4, 5, 6, 7.
Functional Items: 8, 9, 10, 11, 12, 14, 15, 16.
Emotional Items: 13, 17.
References
- GLAAD Media Reference Guide - 10th Edition.2016 (Available at:) (Accessed September 4, 2020)
- Voice and communication change for gender nonconforming individuals: giving voice to the person inside.Int J Transgenderism. 2015; https://doi.org/10.1080/15532739.2015.1075931
- The report of the 2015 US transgender survey.Natl Cent Transgend Equal. 2016;
- On the Limits of “Trans Enough”: authenticating trans identity narratives.Gend Soc. 2018; https://doi.org/10.1177/0891243218780299
- Transgender Self-Evaluation of Voice Questionnaire.2nd ed. Voice and Communication Therapy for the Transgender/Transsexual Client: A Comprehensive Clinical Guide. 2012. Plural Publishing, San Diego2006
- The Voice Handicap Index (VHI): development and validation.Am J Speech Language Pathol. 1997; https://doi.org/10.1044/1058-0360.0603.66
- Development and preliminary evaluation of the transsexual voice questionnaire for male-to-female transsexuals.J. Voice. 2013; https://doi.org/10.1016/j.jvoice.2012.11.005
- Voice perceptions and quality of life of transgender people.J Voice. 2011; https://doi.org/10.1016/j.jvoice.2010.07.013
- My Life as a Woman: Placing Communication Within the Social Context of Life for the Transsexual Woman [Doctoral Thesis].La Trobe University, Melbourne, Australia2007
- Further evidence of the construct validity of the transsexual voice questionnaire (TVQMtF) using principal components Analysis.J Voice. 2017; https://doi.org/10.1016/j.jvoice.2016.07.001
- Exploring the validity of the transsexual voice questionnaire for male-to-female transsexuals.Can J Speech Language Pathol Audiol. 2015; 39: 40-51
- Social desirability, anonymity, and Internet-based questionnaires.Behav Res Methods Instruments Comput. 1999; https://doi.org/10.3758/BF03200723
- Collecting data on alcohol use and alcohol-related victimization: a comparison of telephone and web-based survey methods.J Stud Alcohol. 2006; https://doi.org/10.15288/jsa.2006.67.318
- Effects of interview mode on bias in survey measurements of drug use: do respondent characteristics make a difference?.Subst U5se Misuse. 1998; https://doi.org/10.3109/10826089809069820
- Towards a common language for functioning, disability and health ICF.International Classification. 2002 (Available at:) (Accessed Janaury 22, 2021)
- Coding in-depth semistructured interviews: problems of unitization and intercoder reliability and agreement.Sociol Methods Res. 2013; https://doi.org/10.1177/0049124113500475
- Qualitative interviewing and grounded theory analysis.in: The SAGE Handbook of Interview Research: The Complexity of the Craft. 2012https://doi.org/10.4135/9781452218403.n25
- Is the CVI an acceptable indicator of content validity? Appraisal and recommendations.Res Nurs Health. 2007; https://doi.org/10.1002/nur.20199
- Determination and quantification of content validity.Nurs Res. 1986; https://doi.org/10.1097/00006199-198611000-00017
- An AAC personnel framework: adults with acquired complex communication needs.AAC Augment Altern Commun. 2008; https://doi.org/10.1080/07434610802388477
- Instrument review: getting the most from your panel of experts.Appl Nurs Res. 1992; 5: 194-197
- Adult development and quality of life of transgender and gender nonconforming people.Curr Opin Endocrinol Diabetes Obes. 2016; https://doi.org/10.1097/MED.0000000000000232
Article info
Publication history
Published online: January 28, 2021
Footnotes
Grace Shefcik received partial support for this study from the San José State University Alumni Association Dean's Scholarship. Pei-Tzu Tsai is a paid full-time employee at San José State University.
Identification
Copyright
© 2021 The Authors. Published by Elsevier Inc. on behalf of The Voice Foundation.
User license
Creative Commons Attribution – NonCommercial – NoDerivs (CC BY-NC-ND 4.0) | How you can reuse 
Elsevier's open access license policy
Creative Commons Attribution – NonCommercial – NoDerivs (CC BY-NC-ND 4.0)
Permitted
For non-commercial purposes:
- Read, print & download
- Redistribute or republish the final article
- Text & data mine
- Translate the article (private use only, not for distribution)
- Reuse portions or extracts from the article in other works
Not Permitted
- Sell or re-use for commercial purposes
- Distribute translations or adaptations of the article
Elsevier's open access license policy
