Development and Validation of Social Anxiety Disorder Instrument in Students: Testing Content Validity, Empirical Validity, and Reliability
DOI:
https://doi.org/10.24235/4vv5bx14Keywords:
Instrument Development, Social Anxiety Disorder, Content Validity, Item Distinction Power, Cronbach's Alpha, School CounselingAbstract
This study aims to develop and examine the psychometric quality of an instrument for measuring social anxiety disorder among high school students. The instrument was developed using the Research–Development–Research (RDR) model, which consists of three main stages. The construct was developed based on the diagnostic criteria outlined in the DSM-5-TR, encompassing three dimensions: Fear of Negative Evaluation, New Situation Anxiety, and Peer Anxiety. The initial instrument consisted of 31 items, comprising favorable and unfavorable items. Content validity was assessed by two psychometric experts and a Guidance and Counseling Center using Aiken’s V coefficient. The results showed coefficients ranging from 0.833 to 1.000, indicating that all items demonstrated adequate theoretical relevance. Empirical item validity was subsequently tested involving 145 high school students. The results indicated that 28 items met the validity criteria, with Corrected Item-Total Correlation coefficients ranging from 0.181 to 0.733, exceeding the critical value of 0.163. Three items (P5, P9, and P11) were excluded due to ambiguous behavioral descriptions and potential cognitive bias associated with negation in reverse-worded items. Reliability analysis of the 28 valid items yielded a Cronbach’s Alpha coefficient of 0.904, indicating very high internal consistency. However, the high reliability coefficient may also suggest potential item redundancy, providing an opportunity for further item refinement. Overall, the 28-item instrument demonstrates satisfactory validity and reliability and can be utilized by school counselors as an early detection instrument for identifying social anxiety among high school students.
Downloads
References
Ahmed, S., & Ishtiaq, S. (2021). Reliability and validity analysis of psychometric scales: Addressing misinterpretations of Cronbach's alpha. Journal of Applied Measurement, 22(3), 245–259.
Allen, M.S., Iliescu, D., & Greiff, S. (2022). Single-item and short-form measurement in psychological assessment: A position statement. European Journal of Psychological Assessment, 38(1), 1–5. https://doi.org/10.1027/1015-5759/a000694
Antony, M.M., Coons, M.J., McCabe, R.E., & Ashbaugh, A.R. (2018). Psychometric properties of the Social Phobia Inventory (SPIN) in a clinical sample. Journal of Anxiety Disorders, 20(8), 1042–1056. https://doi.org/10.1016/j.janxdis.2006.01.002
Carleton, R.N., McCreary, D.R., Norton, P.J., & Asmundson, G.J. (2006). Brief Negative Evaluation Fear Scale-revised. Depression and Anxiety , 23(4), 217–228. https://doi.org/10.1002/da.20142
Connor, K.M., Davidson, J.R., Churchill, L.E., Sherwood, R.H., Foa, E.B., & Weisler, R.H. (2000). Psychometric properties of the Social Phobia Inventory (SPIN): A new self-rating scale. The British Journal of Psychiatry , 176(4), 379–386. https://doi.org/10.1192/bjp.176.4.379
Cordier, R., Speyer, R., Chen, Y.W., & Wilkes-Gillan, S. (2021). Evaluating the psychometric properties of social anxiety assessments in children and adolescents: A systematic review. Assessment , 28(2), 512–530. https://doi.org/10.1177/1073191119858380
DeVellis, RF (2017). Scale development: Theory and applications (4th ed.). SAGE Publications.
Heimberg, R.G., Mueller, G.P., Holt, C.S., Hope, D.A., & Liebowitz, M.R. (1992). Assessment of anxiety in social situations and its relationship to the Liebowitz Social Anxiety Scale. Journal of Psychopathology and Behavioral Assessment, 14(1), 53–69. https://doi.org/10.1007/BF00960444
Hussey, I., Hughes, S., & Arslan, R.C. (2025). Rethinking Cronbach's alpha in psychological measurement: Limitations and modern alternatives. Psychological Methods, 30(1), 88–104. https://doi.org/10.1037/met0000512
Kamila, AN, & Widodo, PB (2025). The effect of negative words on the reliability and factor structure of psychological instruments in adolescents. Indonesian Journal of Psychometrics and Assessment, 9(1), 15–28.
Kim, S.Y., & Lecavalier, L. (2025). High Cronbach's alpha and item redundancy in pediatric clinical scales: A psychometric evaluation. Journal of Clinical Child and Adolescent Psychology, 54(2), 178–191. https://doi.org/10.1080/15374416.2024.2312001
La Greca, A.M., & Stone, W.L. (1993). Social Anxiety Scale for Children-Revised (SASC-R): Development and initial psychometric properties. Journal of Clinical Child Psychology, 22(1), 17–27. https://doi.org/10.1207/s15374424jccp2201_2
Liebowitz, MR (1987). Social phobia. Modern Problems of Pharmacopsychiatry, 22, 141–173. https://doi.org/10.1159/000414022
Majeed, S., Munawar, A., & Gillani, M. (2022). Validation of the Social Interaction Anxiety Scale (SIAS) for adolescents in educational settings. International Journal of School and Educational Psychology , 10(3), 321–333. https://doi.org/10.1080/21683603.2021.1901112
Mattick, R.P., & Clarke, J.C. (1998). Development and validation of a measure of social phobia surveillance fear and social interaction anxiety. Psychotherapy and Psychosomatics, 67(1), 28–35. https://doi.org/10.1159/000012258
Ranta, K., Junttila, N., Laakkonen, E., Uhmavaara, A., La Greca, A.M., & Niemi, P.M. (2012). Social Anxiety Scale for Adolescents (SAS-A): Measuring social anxiety in Finnish adolescents. Child Psychiatry & Human Development, 43(4), 574–591. https://doi.org/10.1007/s10578-012-0285-3
Saifuddin, A. (2024). Psychological scale development: Principles, theories, and comprehensive applications (Revised Edition). Kencana.
Stefana, A., Young, S.N., & Gabbard, G.O. (2025). The impact of reversed items on survey response fidelity in adolescent populations. Assessment & Evaluation in Higher Education, 50(2), 201–215. https://doi.org/10.1080/02602938.2024.2301122
Tse, WS, Fong, TC, & Chan, WC (2023). Early screening for social anxiety disorder in school counseling: A psychometric comparison of brief scales. British Journal of Guidance & Counseling, 51(4), 589–602. https://doi.org/10.1080/03069885.2022.2091234
Weeks, JW, Heimberg, RG, Fresco, DM, Hart, TA, Turk, CL, Schneier, FR, & Liebowitz, MR (2005). Empirical validation and psychometric evaluation of the Brief Negative Evaluation Fear Scale in patients with social anxiety disorder. Psychological Assessment, 17(2), 179–190. https://doi.org/10.1037/1040-3590.17.2.179
Whiteside, S.P., Gryczkowski, M.R., Biggs, B.K., Farris, J.R., & Nielsen, B.A. (2012). Validation of the Social Phobia Inventory (SPIN) in a clinical sample of adolescents. Journal of Anxiety Disorders, 26(1), 111–116. https://doi.org/10.1016/j.janxdis.2011.09.006
Zubeidat, I., Salinas, J. M., & Sierra, J. C. (2008). Exploration of the psychometric properties of the Social Anxiety Scale for Adolescents in a Spanish sample. Journal of Adolescence, 31(6), 861–876. https://doi.org/10.1016/j.adolescence.2007.11.004
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Proposed Policy for Journals That Offer Open Access
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
Proposed Policy for Journals That Offer Delayed Open Access
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication, with the work [SPECIFY PERIOD OF TIME] after publication simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
