A Validity Argument for the Interpretation and Use of Revised TEPS Scores

Jun, Heesung

doi:10.30961/lr.2019.55.S.65

Lang. Res. 55(S):65-74

pISSN: 0254-4474, eISSN: 2586-7113

DOI: https://doi.org/10.30961/lr.2019.55.S.65

Linguistics

개정 TEPS 점수해석 및 사용에 대한 타당도 논증

전희성 ¹ ^, ^†

A Validity Argument for the Interpretation and Use of Revised TEPS Scores

Heesung Jun ¹ ^, ^†

Author Information & Copyright ▼

¹서울대학교

¹Seoul National University

^†Corresponding Author : hsjun@snu.ac.kr

ⓒ Copyright 2018 Language Education Institute, Seoul National University. This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Published Online: Nov 01, 2019

ABSTRACT

Validity is an evaluative judgment of the appropriateness of evidence supporting test score interpretation and use (Messick, 1989). As a first step toward arguing for the validity of TEPS score interpretation and use, this paper presents an interpretation/use argument (IUA) for the revised TEPS, following Kane’s argument-based approach to validation (Kane, 2013). The IUA for the revised TEPS outlines seven inferences—domain definition, evaluation, generalization, explanation, extrapolation, utilization, and consequence implication—with a warrant and one or more assumptions underlying each inference. Backing or evidence that is needed to support each assumption is also stated in the IUA. As a second step, a validity argument for the revised TEPS is made by describing the backing that has already been collected while pointing out the backing that is still needed. The paper concludes with further research that needs to be conducted to support the validity argument more fully.

Keywords: interpretation/use argument; validity argument; argument-based approach to validation; revised TEPS

References

AERA, APA, & NCME. (1966). Standard for educational and psychological testing. Washington DC: American Psychological Association .

AERA, APA, & NCME. (1999). Standard for educational and psychological testing. Washington DC: American Psychological Association .

Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice. Oxford: Oxford University Press .

Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice. Oxford: Oxford University Press .

Chapelle, C. A. (2008). The TOEFL validity argument. In C. A. Chapelle, M. K. Enright, & J. M. Jamieson. (Eds), Building a validity argument for the Test of English as a Foreign LanguageTM (pp. 319-352). New York: Routledge .

Chapelle, C. A. (2012). Conceptions of validity. In G. Fulcher & F. Davidson (Eds.), The Routledge handbook of language testing (pp. 21-33). New York: Routledge .

Chapelle, C. A., Enright, M. K., & Jamieson, J. M. (Eds.). (2008). Building a validity argument for the Test of English as a Foreign LanguageTM. New York: Routledge .

Choi, I.-C. (2009). Comparability study on TEPS & IELTS (Research Report No. 50). Seoul: SNU Language Education Institute .

Jun, H., Lim, E., & Lee, Y.-W. (2019, March). Enhancing the interpretability and usefulness of TEPS section scores through alignment with the CEFR. Paper presented at the 41st Language Testing Research Colloquium. Atlanta, GA .

10.

Jun, Y. C., Ryu, D.-S., Park, Y. J., Lee, Y., Shin, S.-H., Jun, H. . . . Byun, J. (2014). Research for the development of TEPS 2.0 (Research Report No. 77). Seoul: SNU Language Education Institute .

11.

Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527-535 .

12.

Kane, M. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed.) (pp.17-64). Westport, CT: Greenwood Publishing .

13.

Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1-73 .

14.

KELTA. (2018). A study on the compatibility and score conversion between the revised TEPS and other certified language proficiency tests. Seoul: KELTA .

15.

Kim, J. Y. (2016). Reliability and test length (internal document). Seoul: TEPS Center .

16.

Kim, J. Y., & Jun, H. (2017). An investigation of local item dependence in testlets and its causes in a large-scale English proficiency test. Journal of Educational Evaluation, 30(4), 837-858 .

17.

Kim, M.-W., Jeong, S., Kim, J.-W., Lee, Y.-W., Lee, Y., Shin, S.-H. . . . Lee, G. (2012). Basic research for a complete revision of the regular TEPS (Research Report No. 73). Seoul: SNU Language Education Institute .

18.

Kwon, H., Lee, Y.-W., Lee, Y., Park, Y.-J., Kim, J., Jun, H., . . . Park, H. (2018). Development and validation of a pilot test form for the revised TEPS (Research Report No. 80). Seoul: SNU Language Education Institute .

19.

Lee, B., Kim, C., Park, Y. J., So, Y.-S., Lee, Y., & Jun, H. (2016). Verification of preparation for New TEPS (Research Report No. 79). Seoul: SNU Language Education Institute .

20.

Lee, B., Park, Y. J., So, Y.-S., Lee, Y.-J., Kim, C., Jun, H., . . . Yeom, S. (2015). The development of New TEPS (Research Report No. 78). Seoul: SNU Language Education Institute .

21.

Lee, Y.-W., Kwon, H., Lee, Y., Lim, E., Jun, H., Song, M., . . . Min, S. (2019). Research for the stabilization and validation of the New TEPS (Research Report No. 81). Seoul: SNU Language Education Institute .

22.

Lim, E., Jun, H., & Lee, Y.-W. (2019). Standard setting to relate an English reading comprehension test to the CEFR: A comparison of modified Angoff and Bookmark methods. Journal of Educational Evaluation, 32(3), 523-548 .

23.

Lim, E., Jun, H., Kim, J., Kwon, H., Wu, R. Y. F., Liao, C., . . . Pan, L. (2018, October). Alignment of the standards-based tests in Asia: TEPS and GEPT comparison. Paper presented at the 5th Asian Association of Language Assessment Conference. Shanghai, China .

24.

Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed.) (pp.13-103). Washington, DC: American Council on Education/Macmillan .

25.

Seong, T. (2002). Modern educational evaluation. Seoul: Hakjisa .

26.

TEPS Center. (2019a). TEPS item writer's guide (internal document). Seoul: TEPS Center .

27.

TEPS Center. (2019b). TEPS reviewer's guide (internal document). Seoul: TEPS Center .

28.

TEPS Center. (2019c). TEPS technical report: 2018 administration (internal document). Seoul: TEPS Center .

29.

Toulmin, S. E. (2003). The uses of argument. Cambridge: Cambridge University Press .