Introduction
In qualitative research, the development and validation of data collection instruments are of utmost importance for obtaining results that are valid and appropriate to the research objectives (Ganesha & Aithal, 2022).This process is not limited to constructing the necessary tools but also includes validating them so that the information obtained is faithful to the phenomenon under study. When designing research tools, especially in complex fields such as language acquisition, a broad understanding of the theoretical and methodological context of the phenomenon under investigation is essential so that the instrument aligns with the prevailing research paradigms. Thus, the construction and validation of the instruments become the foundation upon which the legitimacy of the research rests (Aithal & Aithal, 2020).
Developing an instrument for qualitative research is an arduous process with multiple stages (Coleman, 2021)Characterize the initial stage of the construction process as the design stage, which requires meticulous consideration, since each item of the instrument must target the central elements that the researcher intends to investigate. In the case of the research study on the acquisition of Mexican Sign Language (LSM), the task of designing the instrument requires considering the specific linguistic, cultural, and pedagogical issues related to teaching and learning with individuals who are deaf or hard of hearing (Hamed, 2021). The validation process for a qualitative research instrument, therefore, goes far beyond traditional validation metrics and includes the integration of a robust qualitative evaluation alongside the quantitative one (Tavakol & Wetzel, 2020)
The typical explanation of the document validation process focuses on two stages. The first focuses on testing the relevance and internal logical consistency of the content. This involves an exhaustive review of the literature to identify the most pertinent questions or items for capturing the phenomenon of interest. Subsequently, quantitative validation determines the evaluative criteria for the questions, such as measures of univocality and relevance. These aspects are critical for a robust interpretation of the data (Mousavinasab et al., 2021)This two-pronged approach to validation not only reinforces the instrument’s integrity but also ensures its responsiveness to the target behaviors and attitudes, thereby enhancing the clarity of the analysis and the interpretation of the LSL acquisition process (Alam, 2021)
Methodology
In designing the inquiry instrument on the acquisition of Mexican Sign Language as a target language by hearing individuals who use the language and are graduates of the Faculty of Physical Culture Sciences at the Autonomous University of Chihuahua, the two stages mentioned above serve as the basis: the qualitative stage for content design and the quantitative stage for evaluating the content’s metric criteria.
In research on the acquisition of Mexican Sign Language as a target language by hearing individuals, the design of the instrument is crucial, as it will identify the acquisition process from their experience. Development of stages:
Content validity
Measurement instruments allow for the quantitative assignment of expressions of a construct whose measurement is indirect (Bernal-García et al., 2020)They are considered tools that carry out a data collection process with a defined epistemology (Sandin, 2003)The above implies that the instrument has a paradigmatic basis for research, a theoretical framework, a methodological process, and various techniques for collecting and analyzing data, all grounded in a clear understanding of measurement, reliability, validity, and scoring norms (Escobar-Pérez & Cuervo-Martínez, 2008)
The construction and validation process of the instrument went thru two stages.
First stage:
The construction of a semi-structured interview is determined as the instrument (Lopezosa et al., 2022) defines it as “the research technique for exploring the participant’s perspective from their experience.” One of its main characteristics is that carefully crafted questions are designed to explore reality according to the object of study under investigation(Babativa et al., 2024) In his definition and typology of interviews, also refers to it as an open-ended standardized interview, meaning a list of ordered questions worded identically for each participant, with free or open-ended responses. This interview offers a degree of flexibility; (Klimenko et al., 2025) notes that the instrument “is associated with a high degree of freedom for interviewees to express their viewpoints.” (Tarazona-Mirabal, 2020)state that the semi-structured interview can be referred to as an “ethnographic interview.”
In the instrument, the researcher has the opportunity to introduce additional questions, since the objective is to gather information about the reality being studied (Ponce-López et al., 2020).The collection of information will facilitate the researcher’s analysis process; to this end, the importance of the question guide is paramount, and, as the author says, when asking the questions, they can be modified or supplemented (Kholis et al., 2020).
In constructing the theoretical framework for the analytical instrument of this article, substantial elements of the proposal presented by the theoretical positioning were identified, establishing five guiding axs for the topic. Although Stephen Krashen’s Monitor Theory (Setiawan et al., 2024) posits five hypotheses, the study of the author’s work outlined each hypothesis with key elements for its investigation. Among them, and as already mentioned in the second chapter of this document, the instrument represents a) the acquisition vs. learning hypothesis; b) the monitor hypothesis; c) the natural order hypothesis; d) the affective filter hypothesis; and e) the comprehensible input hypothesis. Based on the content proposed in each hypothesis, open-ended questions were chosen, resulting in 38 items distributed across five axes, which are established in 1. Language exposure; 2. Acquisition; 3. Language awareness; 4. Linguistic rules; the facilitator’s role and materials; and 6. Emotional process (Alibakhshi et al., 2013).
Second stage: Once the questions have been designed, this stage evaluates whether the questions or items cover the content that the research objective seeks. At this stage, experts are consulted, as they are the ones who know the subject matter; this is ideal, since it will enhance the instrument’s rational validity (Farimani & Shahri, 2020)Thus, expert judgment is established as the foundational process of this stage.
According to stage one, the content design was established using a sample of questions or items that will form the core of the semi-structured interview, which implies that each question is theoretically representative of the design’s dimensions. Because it is a qualitative design, there is no direct way to assess validity; therefore, it will be estimated using indirect methods, with expert judgment being the most efficient (El-Den et al., 2020).This expert judgment process will review the instrument from the perspective and viewpoint of specialists who have extensive experience in the dimensions of the object of study and in instrument validation (Ferrando et al., 2022) they will determine whether the content items belong to a representative sample.
For the present analysis, the instrument validation process by this expert judgment method convenes ten doctoral-level experts with experience in conducting semi-structured interviews and in the linguistic aspects of language acquisition. Based on a call issued by the researcher, they analyze each question and rate two central elements: univocality and relevance. For this, the methodology proposed by Mialhe et al., (2022)is used, drawing on theoretical elements referenced in the questions that serve as a benchmark for instrument validation. The conceptualization of the terms was sent to each expert by email or digitally to contextualize what was being sought and to enable them to respond according to the following scale.
Levels of univocity.
| Levels of univocity | Meaning | Assigned value |
|---|---|---|
| Optimal univocity (OU) | The item is susceptible to being understood or interpreted unequivocally in only one way.. | 4 |
| High univocity (UE) | The item is open to interpretation, but it can be understood primarily or generally in only one way.. | 3 |
| Low univocity (UB) | The item is susceptible to being understood in various ways and is closer to ambiguity. | 2 |
| Null univocity (UN) | The item is susceptible to not being understood or to being interpreted with very different meanings, falling into equivocation. | 1 |
| (Carrera et al., 2011) | ||
Levels of relevance
| Levels of Relevance | Meaning | Assigned Value |
|---|---|---|
| Optimal relevance (OR) | The item is unequivocally suitable for the target demographic and age group.. | 4 |
| High relevance (HE) | The item is likely to be primarily relevant to the target demographic and age group. | 3 |
| Low relevance (LB) | The item is likely to be primarily relevant to the target demographic and age group. The item is likely to be relevant to multiple demographics and age groups, or it is of little relevance to the target demographic and age group. | 2 |
| Zero relevance (UN) | The item may not belong to the collective. And whether or not the age group is relevant to the target demographic or age group. | 1 |
| (Carrera et al., 2011) | ||
The evaluation of the instrument’s items using a validation questionnaire based on a conceptual scale allowed them to assess the level of univocality (U) and the level of relevance (P) (Carrera et al., 2011). The observation table for each item helps create cohesion in what the response seeks.
In the responses analyzed by each judge and, thru a scale review process, the univocality index (UI) and relevance index (RI) are obtained. Based on a value of 1, the indices and precision are calculated using the following formula:
The representation of the elements contained in the formula is based on the following:
Index of univocity and relevance.
| Univocality Index (iU) | Univocality Index (iU) | Relevance Index (iP) | Relevance Index (iP) |
|---|---|---|---|
| nUO | Number of responses obtained at the optimal level of univocality. | nPO | Number of responses obtained from Optimal relevance level. |
| nUE | Number of responses obtained at the high-unanimity level. | nPE | Number of responses obtained at the high relevance level. |
| nUB | Number of responses obtained from the low-unanimity level. | nPB | Number of responses obtained at the low relevance level. |
| nUN | Number of responses obtained at the low relevance level | nPN | Number of responses obtained from No relevance level. |
| nTU | Number of total responses obtained for the level of univocality. | nTP | Number of total responses obtained for the relevance level. |
| VUO | Value assigned to the level of univocacy. | VPO | Value assigned to the relevance level. |
| Optimal | |||
| VUE | Value assigned to the high level of univocality. | VPE | Value assigned to the high relevance level. |
| VUB | Value assigned to the low level of univocality. | VPB | Value assigned to the low relevance level. |
| VUN | Value assigned to the null univocality level. | VPN | Value assigned to the level of no relevance. |
| VMU | Maximum value of the univocality scale levels. | VMP | Maximum value of the relevance scale levels |
| (Carrera et al., 2011) | |||
With the maximum indicator reference set to 4 and the minimum to 1 in the formula application, the results and suggestions, the validation is established according to the following table:
Criteria of univocity and relevance.
| Criteria according to iU | Criteria according to iU | Criteria according to iP | Criteria according to iP |
|---|---|---|---|
| The items remain in their original form. | The items are kept in their original form. | ||
| The items are modified in their wording. | The items are modified in their wording or placement. | ||
| The items are removed. | The items are removed. |
Results of the instrument validation
The validation instrument was a 38-item questionnaire, based on data obtained from judges who evaluated the document, from working with the procedure to applying the equation. The results of the instrument validation are as follows:
Validation of research instrument items.
| item | Indicator | Uniqueness | Relevance |
|---|---|---|---|
| 1 | When learning sign language, what was your experience like at the beginning of the process? | 0.847 | 0.961 |
| 2 | In the early stages of acquiring LSM, did you receive comprehensible information, or did you get a general idea of what was happening or the main topic? | 0.833 | 0.897 |
| 3 | How would you describe the impact of this specific environment on your performance? Did it facilitate your acquisition process or was it ineffective? | 0.786 | 0.886 |
| 4 | What is your opinion on exposure to sign language while learning? Based on your experience, what do you think about this? | 0.850 | 0.847 |
| 5 | What role does understanding the message play in the acquisition process? | 0.802 | 0.871 |
| 6 | What is your opinion on LSM fluency when watching videos and interacting with native speakers? | 0.893 | 0.981 |
| 7 | What is your opinion on practicing sign language at home by yourself in front of a mirror? | 0.882 | 0.931 |
| 8 | In your experience, is it better to develop a second language in an academic setting by rethinking all the rules related to the target language? Or is it preferable to acquire it thru direct interaction and communication with fluent speakers in a natural way? | 0.917 | 0.981 |
| 9 | How do you think a subject appropriates language? | 0.939 | 0.961 |
| 10 | What steps or procedures can you define for the acquisition process? | 0.931 | |
| 11 | What approach have you taken to any formal method in the acquisition process? | 0.939 | 0.974 |
| 12 | Do you think that acquiring this second language has similarities to acquiring the first language? | 0.867 | 0.971 |
| 13 | Each language has its own grammatical order; in the process of acquiring Mexican Sign Language, how was your experience adapting to its order? | 0.931 | 0.974 |
| 14 | ¿How similar is what has been described to your experience with learning sign language? | 0.953 | 0.974 |
| 15 | What was the input format/source I received when acquiring the LSM? | 0.939 | 0.981 |
| 16 | Are there any stages it went thru that aren’t mentioned in the description above? | 0.921 | 1.000 |
| 17 | What do you think about error awareness? Is it important in language acquisition? | 0.882 | 0.961 |
| 18 | From your experience, at what point did you realize you had mastered sign language? | 0.961 | 1.000 |
| 19 | From your experience, what is the difference between language acquisition and language learning? | 0.981 | 1.000 |
| 20 | In your opinion, when appropriating LSM, are you aware of the process, or are there unconscious aspects? | 0.953 | |
| 21 | From your experience, when was the first language (Spanish) required to learn sign language? | 0.859 | 0.961 |
| 22 | Can you argue what role the first language (Spanish) plays in acquiring the second language (LSM)? | 1.000 | |
| 23 | In the process of acquiring LSM, at what point does a normal conversation with a deaf person take place? | 0.917 | 0.961 |
| 24 | What role does the use of grammar play in language acquisition? | 0.961 | 0.981 |
| 25 | At what point does one acquire the notion of grammatical rules in sign language? | 0.939 | 0.981 |
| 26 | Comment on your experience with the grammatical notion in the acquisition of LSM? | 0.931 | 0.974 |
| 27 | How important is it for a facilitator to have specialized training in LSM to teach a course? What if the facilitator isn’t specialized and is just a native speaker of the language? | 0.961 | 0.981 |
| 28 | What is the facilitator’s role in helping the learner acquire sign language? | 0.931 | 0.981 |
| 29 | From your experience, what materials did you use to learn LSM? | 0.939 | 0.961 |
| 30 | Currently, as a language user and based on your experience using it, what other resources do you think are needed for language acquisition? | 0.981 | |
| 31 | From your experience, what role does the learner’s emotional state play in learning sign language? | 0.961 | 0.974 |
| 32 | What emotional characteristics are important in the user’s language acquisition process? | 0.961 | 0.953 |
| 33 | What moment during your process of learning LSM motivated you? | 1.000 | 0.981 |
| 34 | What moment do you remember in your process of learning LSM when a situation caused you anxiety? | 0.981 | |
| 35 | What moments did you encounter with a comprehension block while they were talking to you? | 0.961 | 0.981 |
| 36 | How does the comprehension process occur when acquiring sign language? | 0.953 | 0.981 |
| 37 | What would you recommend to a future Mexican Sign Language user who is just starting this process? | 0.974 | |
| 38 | We’ve reached the end of our questions. Is there anything else you’d like to add? | 0.981 | 1.000 |
| Based on the results obtained, the univocality score exceeded the maximum threshold of iU ≥ .80, and the relevance score exceeded the threshold of iP ≥ .80. | |||
In light of this, only item 3 in the univocality criterion falls below the 80th percentile, indicating a recommendation to modify the established wording, and the correction is implemented based on the suggestions made in the expert judgment.
The modification to the previous wording refers to the following: “P3. How would you describe the outcome of this specific environment on your performance?” How would you describe the outcome of this specific environment on your performance? Did it facilitate your acquisition process or was it ineffective? The modification is as follows: P3: “Based on the previous question, how would you describe the outcome of this initial process on your performance?” Did it facilitate your acquisition process?
Discussion
The process of constructing and validating instruments with a qualitative component can be interpreted as a technical procedure necessary to legitimize the interpretation of the data. In this regard, (Carrera et al., 2011) propose the well-known term “concept elicitation,” or the extraction of meanings derived from real-world experience thru qualitative methods that ensure the items faithfully reflect the participant’s experience and the construct’s domain.
Validation from the perspective of scientific rigor not only transcends the technical aspects but also involves linking the theoretical framework with empirical realities. This validation process identifies representative points of the domain they intend to measure, considering both the relevance and the clarity of their wording (Mialhe et al., 2022).This approach reinforces the need for an iterative process, in which experts’ suggestions not only correct formal aspects but also ensure conceptual consistency between theory and practice. In this regard, the designed instrument complies with this principle by integrating the theoretical dimensions of LSM acquisition and the perceptions derived from the educational context in which learning takes place (Ferrando et al., 2022).
The literature holds that the validity of qualitative instruments cannot be understood as a quantitative process, but rather as an interdisciplinary integration of theoretical, empirical, and experiential criteria. The construction of an instrument in qualitative research involves a process of consolidating the instrument itself within epistemological scientific rigor, where methodological decisions must reflect the interpretive nature of the phenomenon (Sandi, 2003). Validity is not a static characteristic; it is related to the gradual accumulation of evidence that legitimizes the instrument in different contexts (Kholis et al., 2020).
Regarding the present instrument, the initial design of the 38 items, based on an exhaustive literature review and a theoretical framework grounded in Krashen’s work, aligns with this recommendation by not starting with prefabricated items but rather with the domain’s own theoretical and empirical assumptions. The instrument seeks to ensure coherence with this approach, comprising 38 items distributed across six thematic axs to understand the complexity of the LSM acquisition process from a sociolinguistic and affective perspective.
The validation of this instrument aims to ensure that the items capture the full essence of participants’ experiences at their various levels of linguistic competence. The process of using expert judgment to validate this content according to the criteria of univocality (iU) and relevance (iP), following (Carrera et al., 2011).This process allows for quantifying the degree of clarity and relevance of each item. Of the 37 items, 38 exceeded the threshold of iU ≥ .80 and iP ≥ .80, a very positive result that suggests excellent alignment with the criteria of clarity and relevance(Kholis et al., 2020).
The suggestion to adjust a specific item confirms that validation is an iterative process, since the judges’ qualitative observations allow for refining the wording without altering the conceptual meaning, highlighting that the judges’ critical review contributes to purifying the instrument and reinforcing its internal coherence (Tarazona-Mirabal, 2020).
The construction and validation of this instrument represent a significant methodological contribution to research in the disciplinary fields of education and applied linguistics. A rigorous procedure was implemented that legitimizes the data obtained and ensures the relevance of the results (Babativa et al., 2024; Tarazona-Mirabal, 2020). This process, although demanding, enabled the establishment of a reliable and adaptable tool for future research on LSM acquisition by hearing individuals, strengthening systematicity and methodological rigor in a field that has historically lacked validated instruments. Consequently, it is recommended to continue the empirical validation of the instrument and to explore its applicability in other sign language learning communities, in order to consolidate its scientific and practical value (Escobar-Pérez & Cuervo-Martínez, 2008)
The applied methodology provides a rigorous methodological foundation for future studies in the field of LSM, but it should not be considered definitive without further empirical calibration. Likewise, the instrument’s validation is envisioned as a metascientific reflection on how to validate tools in emerging linguistic contexts, encouraging other researchers to continue the validation process and empirical application of the instrument. The development of analyzes based on expert judgment enabled consensus among the judges, confirming the relevance of the items and their alignment with the theoretical dimensions proposed.
Conclusions
Research that carefully handles the data collected with a validated instrument will allow for the evaluation of variable relationships without bias or any unfavorable skew toward the study’s objective; this is why the systematic process from design development to the reliability every researcher requires is of significant value. In designing the questions, a construct validation process will help prevent biases, resulting in an appropriate instrument thru the quantitative process.
In conclusion, this instrument, since its validation, is a useful tool for future studies that investigate the acquisition of Mexican Sign Language as a target language by hearing individuals; it will also allow us to understand the importance of the sign language courses offered by the Autonomous University of Chihuahua’s Faculty of Physical Culture Sciences.
Conflicts of interest
Clear demonstration of financial and material support, or of any dispute that arises during the process of investigating or disclosing information.