A Multi-Dimensional Comparison of CSAT English Reading Passages before and after the Use of Criterion-Referenced Assessment

Chang, Ji-Yeon

doi:10.30961/lr.2022.58.1.61

Lang. Res. 58(1):61-89

pISSN: 0254-4474, eISSN: 2586-7113

DOI: https://doi.org/10.30961/lr.2022.58.1.61

Article

절대평가 도입 전후 수능 영어 읽기 지문의 다차원 비교

장지연 ¹ ^, ^†

A Multi-Dimensional Comparison of CSAT English Reading Passages before and after the Use of Criterion-Referenced Assessment

Ji-Yeon Chang ¹ ^, ^†

Author Information & Copyright ▼

¹명지대학교

¹Myongji University

^†Corresponding Author: 부교수(전문), 미래융합대학 창의융합인재학부, 명지대학교, 03674 서울특별시 서대문구 거북골로 34, E-mail: jchang@mju.ac.kr

ⓒ Copyright 2022 Language Education Institute, Seoul National University. This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Received: Feb 17, 2022 ; Revised: Apr 11, 2022 ; Accepted: Apr 18, 2022

Published Online: Apr 30, 2022

ABSTRACT

This study aims to compare register variation in the reading passages of the CSAT (College Scholastic Ability Test) English section before and after the use of criterion-referenced assessment by applying an additive multi-dimensional analysis based on Biber (1988). To this end, it compiled a corpus of the CSAT English reading passages of the 2013-2022 academic years, using the computer software MAT (Multidimensional Analysis Tagger, Version 1.3.2) (Nini, 2019) for analysis. The median dimension scores indicate that the passages were informational, non-narrative, and abstract. The passages did not tend to persuade their readers, and the referents were specified in an explicit manner. However, the wide range and distribution of the dimension scores suggest that various registers were used in the reading passages. Finally, there was no difference between the two assessment groups in all dimensions except for Dimension 3, implying that the consistency of the CSAT English reading passages has been relatively well maintained regarding register variation.

Keywords: additive multi-dimensional analysis; MAT; register variation; dimensions; CSAT English reading passages

References

Al-Surmi, M. (2012). Authenticity and TV shows: A multidimensional analysis perspective. TESOL Quarterly, 46(4), 671-694.

Biber, D. (1988). Variation across speech and writing. Cambridge: Cambridge University Press.

Biber, D., & Finegan, E. (2001). Intra-textual variation within medical research articles. In S. Conrad & D. Biber (Eds.), Variation in English: Multi-dimensional studies (pp. 108-123). New York: Routledge.

Chang, J.-Y. (2018). A comparison of 2017-2018 CSAT reading passages via Coh-Metrix: Focusing on descriptive, readability, and easibility measures. Foreign Languages Education, 25(4), 81-106.

Chang, J.-Y. (2019a). A comparison of lexical complexity in 2016-2019 CSAT English reading passages by the types of assessment (norm-referenced assessment vs. criterion-referenced assessment). The SNU Journal of Education Research, 28(3), 85-110.

Chang, J.-Y. (2019b). A comparison of syntactic complexity in CSAT reading passages before and after the introduction of criterion-referenced evaluation. Journal of the Korea English Education Society, 18(2), 161-188.

Conrad, S. (2001). Variation among disciplinary texts: A comparison of textbooks and journal articles in biology and history. In S. Conrad & D. Biber (Eds.), Variation in English: Multi-dimensional studies (pp. 94-107). New York: Routledge.

de Souza, R. C. (2014). Dimensions of variation in TIME magazine. In T. B. Sardinha & M. V. Pinto (Eds.), Multi-dimensional analysis, 25 years on: A tribute to Douglas Biber (pp. 177-193). Philadelphia: John Benjamins.

Helt, M. E. (2001). A multi-dimensional comparison of British and American spoken English. In S. Conrad & D. Biber (Eds.), Variation in English: Multi- dimensional studies (pp. 171-183). New York: Routledge.

10.

Kim, G.-H., & Cha, K.-W. (2021). An analysis on the vocabulary of recent CSAT's English listening assessment: Lexical scales, word difficulty, and high frequency content words. English Language Teaching, 33(2), 23-36.

11.

Kim, J.-r. (2017). A diachronic analysis of English KSAT reading passages. Journal of the Elementary Education Society, 27, 63-78.

12.

Koh, N., & Shin, J.-A. (2017). A comparison of the level of difficulty in the English reading part of the CSAT: Before and after the EBS-CSAT linkage policy. Secondary English Education, 10(4), 3-24.

13.

Korea Institute for Curriculum and Evaluation. (2021, March 16). Announcement of the basic plan for the implementation of the 2022 College Scholastic Ability Test [Press release].

14.

Lee, B. C. (2013). A study on the optimization of the CSAT categorization framework with a focus on item analysis. Modern English Education, 14(1), 267-296.

15.

Lu, X. (2014). Computational methods for corpus annotation and analysis. New York: Springer.

16.

Ministry of Education, Science and Technology. (2012, June 15). Announcement of the basic plan for the 2014 College Scholastic Ability Test [Press release].

17.

Moon, J.-H., & Kim, H.-D. (2017). An analysis of the linguistic elements of the text in the English reading section of the College Scholastic Ability Test. Modern English Education, 18(1), 193-211.

18.

Nini, A. (2019). The multi-dimensional analysis tagger. In T. B. Sardinha & M. V. Pinto (Eds.), Multi-dimensional analysis: Research methods and current issues (pp. 67-94). London: Bloomsbury Academic.

19.

Park, K., & Hwang, E. (2019). A multi-dimensional factor analysis of pre-university students' argumentative essays. Modern English Education, 20(3), 129-138.

20.

Park, K., & Kim, E.-J. (2016). A multi-dimensional analysis of Korean college English learners' speech transcript. Studies in Foreign Language Education, 30(3), 25-52.

21.

Park, K., & Kim, E.-J. (2019). A study of university English learners' oral presentation with multi-dimensional analysis: Focusing on informative and persuasive presentation. Studies in Foreign Language Education, 33(3), 1-26.

22.

R Core Team (2021). R: A language and environment for statistical computing (Version 4.1.2) [Computer software]. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from https://www.R-project.org/.

23.

Reppen, R. (2001). Register variation in student and adult speech and writing. In S. Conrad & D. Biber (Eds.), Variation in English: Multi-dimensional studies (pp. 187-199). New York: Routledge.

24.

Rey, J. M. (2001). Changing gender roles in popular culture: Dialogue in Star Trek episodes from 1966 to 1993. In S. Conrad & D. Biber (Eds.), Variation in English: Multi-dimensional studies (pp. 138-156). New York: Routledge.

25.

Sardinha, T. B., Pinto, M. V., Mayer, C., Zuppardi, M. C., & Kauffmann, C. H. (2019). Adding registers to a previous multi-dimensional analysis. In T. B. Sardinha & M. V. Pinto (Eds.), Multi-dimensional analysis: Research methods and current issues (pp. 165-186). London: Bloomsbury Academic.

26.

Shin, Y. (2019). Analyzing CSAT reading passages by using Coh-Metrix and VocaProfile: Focusing on four years from 2016 to 2019. Journal of Language Sciences, 26(4), 109-127.

27.

Toutanova, K., Klein, D., Manning, C. D., & Singer, Y. (2003). Feature-rich part-of-speech tagging with a cyclic dependency network. In M. Hearst & M. Ostendorf (Eds.), Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology (Vol. 1, pp. 173-180). Edmonton, Canada: Association for Computational Linguistics.

28.

Yoon, H. (2012). A multi-dimensional analysis of oral registers in revised middle school English textbooks. Korean Journal of English Language and Linguistics, 12(4), 777-799.