Correcting for Scale Usage Differences among Latin American Countries, Portugal, and Spain in PISA
##plugins.pubIds.doi.readerDisplayName##:
https://doi.org/10.7203/relieve.22.1.8282关键词:
Extreme response style, overclaiming, anchoring vignettes, comparability, validity.摘要
This paper investigated the effects of corrections for scale usage preference in seven Latin American countries, Portugal and Spain in student self-reports in the 2012 Programme for International Student Assessment (PISA). These targeted countries tend to show a strong tendency of expressing strong opinions and self-enhancement, which can pose serious validity threats in cross-cultural comparisons of self-reports. We examined to what extent score corrections, that have been proposed, would change the patterning of the cross-cultural differences. We corrected for the scale usage preferences in a measure of teacher support among 39,045 students in nine countries, based on extreme response style, overclaiming, and anchoring vignettes, respectively. These measures showed different effects: (1) All correction methods helped to improve measurement invariance, although the correction based on anchoring was less effective in reaching scalar invariance compared with the correction of extreme response style and overclaiming; (2) controlling for extreme response style and overclaiming changed the mean score of Spain to a greater extent than other countries, which seems to present a more realistic patterning, whereas the changes on correlations with other measures were rather limited. The use of anchored scores led to drastic changes both in means and correlations. A firm conclusion about which method is to be preferred cannot be given as there is no evidence which method enhances the validity of scores in these countries more. We discuss the necessity and practicability of correction methods.
参考
Arbuckle, J. L. (2006). AMOS user's guide. Chicago, IL: SPSS.
Brown, A., & Maydeu-Olivares, A. (2011). Item response modeling of forced-choice questionnaires. Educational and Psychological Measurement, 71, 460-502. doi: http://dx.doi.org/ http://dx.doi.org/10.1177/0013164410375112
Chen, C., Lee, S.-y., & Stevenson, H. W. (1995). Response style and cross-cultural comparisons of rating scales among East Asian and North American students. Psychological Science, 6, 170-175. doi: http://dx.doi.org/10.1111/j.1467-9280.1995.tb00327.x
Cheung, G. W., & Rensvold, R. B. (2000). Assessing extreme and acquiescence response sets in cross-cultural research using structural equations modeling. Journal of Cross-Cultural Psychology, 31, 187-212. doi: http://dx.doi.org/10.1177/0022022100031002003
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255. doi: http://dx.doi.org/10.1207/s15328007sem0902_5
Diamantopoulos, A., Raeynolds, N. L., & Simintiras, A. C. (2006). The impact of response styles on the stability of cross-national comparisons. Journal of Business Research, 59, 925-935. doi: http://dx.doi.org/10.1016/j.jbusres.2006.03.001
Goldstein, H. (2004). International comparisons of student attainment: Some issues arising from the PISA study. Assessment in Education Principles Policy and Practice, 11, 319-330. doi: http://dx.doi.org/10.1080/0969594042000304618
Gorur, R. (2014). Towards a sociology of measurement in education policy. European Educational Research Journal, 13, 58-72. doi: http://dx.doi.org/10.2304/eerj.2014.13.1.58
Harzing, A.-W. (2006). Response styles in cross-national survey research: A 26-country study. International Journal of Cross Cultural Management, 6, 243-266. doi: http://dx.doi.org/10.1177/1470595806066332
He, J., van de Vijver, F., J. R., Domínguez, A. d. C., & Mui, P. H. C. (2014). Toward a unification of acquiescent, extreme, and midpoint response styles: A multilevel study. International Journal of Cross-Cultural Management, 14, 306-322. doi: http://dx.doi.org/10.1177/1470595814541424
He, J., & van de Vijver, F. J. R. (2015a). Effects of a general response style on cross-cultural comparisons: Evidence from the Teaching and Learning International Survey. Public Opinion Quarterly, 79, 267-290. doi: http://dx.doi.org/10.1093/poq/nfv006
He, J., & Van de Vijver, F. J. R. (2015b). The motivation-achievement paradox in international educational achievement tests: Toward a better understanding. In R. B. King & A. B. I. Bernardo (Eds.), The psychology of Asian learners: A festschrift in honor of David Watkins (pp. 253-268). Singapore: Springer.
He, J., & van de Vijver, F. J. R. (2015c). Self-presentation styles in self-reports: Linking the general factors of response styles, personality traits, and values in a longitudinal study. Personality and Individual Differences, 31, 129-134. doi: http://dx.doi.org/10.1016/j.paid.2014.09.009
Hofstede, G. (1980). Culture's consequences: Comparing values, behaviors, institutions and organizations across nations. Beverly Hills, CA: Sage.
Hofstede, G. (2009). Dimension data matrix. Dimension Data Matrix Retrieved 03/02/2001 http://www.geerthofstede.eu/dimension-data-matrix
King, G., Murray, C. J. L., Salomon, J. A., & Tandon, A. (2004). Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review, 98, 191-207. doi: http://dx.doi.org/ doi: http://dx.doi.org/10.1017/S000305540400108X
King, G., & Wand, J. (2007). Comparing incomparable survey responses: Evaluating and selecting anchoring vignettes. Political Analysis, 15, 46-66. doi: http://dx.doi.org/10.1093/pan/mpl011
Kyllonen, P. C., & Bertling, J. P. (2014). Innovative questionnaire assessment methods to increase cross-country comparability. In L. Rutkowski, M. v. Davier & D. Rutkowski (Eds.), Handbook of international large-scale assessment: Background, technical issues, and methods of data analysis (pp. 277-286). Boca Raton, FL: CRC Press.
OECD. (2013a). PISA 2012 Assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy. Paris, France: OECD Publishing.
OECD. (2013b). PISA 2012 technical report. Paris, France: OECD Publishing.
Paulhus, D. L. (1991). Measurement and control of response biases. In J. Robinson, P. Shaver & L. Wrightsman (Eds.), Measures of personality and social psychological attitudes (Vol. 1, pp. 17-59). San Diego, CA: Academic Press.
Paulhus, D. L., Harms, P. D., Bruce, M. N., & Lysy, D. C. (2003). The over-claiming technique: Measuring self-enhancement independent of ability. Journal of Personality and Social Psychology, 84, 890-904. doi: http://dx.doi.org/10.1037/0022-3514.84.4.890
Rutkowski, L., von Davier, M., & Rutkowski, D. (Eds.). (2014). Handbook of international large-scale assessment: Background, technical issues, and methods of data analysis. Boca Raton, FL: CRC Press.
Smith, P. B. (2004). Acquiescent response bias as an aspect of cultural communication style. Journal of Cross-Cultural Psychology, 35, 50-61. doi: http://dx.doi.org/10.1177/0022022103260380
Smith, P. B. (2011). Communication styles as dimensions of national culture. Journal of Cross-Cultural Psychology, 42, 216-233. doi: http://dx.doi.org/10.1177/0022022110396866
Uskul, A. K., Oyserman, D., & Schwarz, N. (2010). Cultural emphasis on honor, modesty or self-enhancement: Implications for the survey response process. In J. A. Harkness, M. Broun, B. Edwards, T. P. Johnson, L. Lyberg, P. P. Mohler, B.-E. Pennell & T. W. Smith (Eds.), Survey methods in multinational, multiregional and multicultural contexts (pp. 191-201). New York, NY: Wiley.
van de Vijver, F. J. R., & Leung, K. (1997). Methods and data analysis of comparative research. Thousand Oaks, CA: Sage.
Wand, J., & King, G. (2007). Anchoring vignetttes in R: A (different kind of) vignette. Retrieved from http://wand.stanford.edu/anchors/doc/anchors.pdf
##submission.downloads##
已出版
期
栏目
##submission.license##
Los autores ceden de forma no exclusiva los derechos de explotación de los trabajos publicados a RELIEVE (a los solos efectos de favorecer la difusión de los artículos publicados:firmar contratos de difusión, de integración en bases de datos, etc.) y consienten que se distribuyan bajo la licencia de Creative Commons Reconocimiento-Uso No Comercial 4.0 International (CC-BY-NC 4.0), que permite a terceros el uso de lo publicado siempre que se mencione la autoría de la obra y la fuente de publicación, y se haga uso sin fines comerciales.
Los autores pueden llegar a otros acuerdos contractuales adicionales e independientes, para la distribución no exclusiva de la versión del trabajo publicado en esta revista (por ejemplo, incluyéndolo en un repositorio institucional o publicándolo en un libro), siempre y cuando se cite claramente que la fuente original de publicación es esta revista.
A los autores se les anima a difundir su trabajo después de publicado, a través de internet (por ejemplo, en archivos institucionales en línea o en su página web) lo que puede generar intercambios interesantes y aumentar las citas del trabajo.
La mera remisión del artículo a RELIEVE supone la aceptación de estas condiciones.