Propuesta de un nuevo cuestionario de evaluación de los profesores de la Universidad del País Vasco. Estudio psicométrico, dimensional y diferencial

Autores/as

DOI:

https://doi.org/10.7203/relieve.23.2.10436

Palabras clave:

Evaluación de la actividad docente por estudiantes, Dimensionalidad, Cuestionario, Evaluación del desempeño docente, Competencias docentes

Resumen

El objetivo de este artículo es analizar el proyecto de nuevo cuestionario diseñado por la Universidad del País Vasco (UPV/EHU) para realizar la evaluación de sus docentes (SET). Se analizan las respuestas de una muestra de 941 estudiantes y se estudia la fiabilidad del cuestionario, la dimensionalidad, la validez de constructo y criterial, finalizando con un estudio diferencial tomando en cuenta variables como el género, el campo disciplinar, el nivel percibido de dificultad o el interés de las materias. Los resultados permiten afirmar que se trata de un instrumento de alta consistencia interna y esencialmente unidimensional pero que, a la vez, se ajusta a las dimensiones teóricas usadas para su diseño y construcción lo que posibilita un uso formativo de la información.

Biografía del autor/a

Luis Lizasoain-Hernández, Universidad del País Vasco

PhD in Philosophy and Educational Sciences. Tenured lecturer in Statistics at the Department of Research and Diagnostic Methods in Education of the UPV-EHU at its Donostia-San Sebastián campus, he teaches graduate and masters’ degree course in Human and Social Sciences, both at the UPV-EHU and in various Latin American universities. He has lectured at the Complutense University in Madrid and worked as a visiting lecturer at the Universidad Autónoma in Baja California (Mexico)

Juan Etxeberria-Murgiondo, Universidad del País Vasco

PhD in Educational Sciences and a graduate in Exact Sciences (specialising in statistics). Chair in Statistics at the Department of Research and Diagnostic Methods in Education of the UPV-EHU at its Donostia-San Sebastián campus. Since 1997 he has been the Deputy Director of the University of the Basque Country Summer Courses, organised in the city of Donostia-San Sebastián.

José Francisco Lukas-Mujika, Universidad del País Vasco

PhD in Pedagogy, a graduate in Philosophy and Educational Sciences (Pedagogy section) and a qualified teacher. Tenured lecturer in Educational Evaluation at the Department of Educational Research and Diagnostic Methods at the University of the Basque Country (UPV-EHU), he graduate and masters’ degree courses in Human and Social Sciences and is a specialist in Evaluation in Education with special dedication given over to topics regarding the evaluation of educational programmes and centres and the construction of measuring instruments. His postal address is: Department of Research and Diagnostic Methods in Education, Faculty of  Philosophy and Educational Sciences, University of the Basque Country, Avenida de Tolosa, 70. San Sebastián–20018.  He is Section Editor at RELIEVE

Citas

Adams, Meredith J.D., & Umbach, Paul D. (2012). Non response and online student evaluations of teaching: Understanding the influence of salience, fatigue, and academic environments. Research in Higher Education, 53(5), 576-91. doi: https://doi.org/10.1007/s11162-011-9240-5

Addison, William E., Best, John & Warrington, John D. (2006). Students’ Perceptions of Course Difficulty and Their Ratings of the Instructor. College student journal, 40(2), 409-16.

Alvarado, Elías, Morales Dionicio & Aguayo, Ernesto (2016). Percepción de la calidad educativa: caso aplicado a estudiantes de la Universidad Autónoma de Nuevo León y del Instituto Tecnológico de Estudios Superiores de Monterrey. Revista de la Educación Superior, 45(180), 55-74. doi: https://doi.org/10.1016/j.resu.2016.06.006

Apodaca, Pedro & Grad, Héctor (2002). Análisis dimensional de las opiniones de los alumnos universitarios sobre sus profesores: comparación entre técnicas paramétricas y no-paramétricas. Revista de Investigación Educativa, 20(2), 385-409.

Apodaca, Pedro & Grad, Héctor (2005). The dimensionality of student ratings of teaching: integration of uni-and multidimensional models. Studies in Higher Education, 30(6), 723-48. doi: https://doi.org/10.1080/03075070500340101

Basto, Mário & Pereira, José Manuel. (2012). An SPSS R-Menu for ordinal factor analysis. Journal of Statistical Software, 46(4), 1-29. doi: https://doi.org/10.18637/jss.v046.i04

Berk, Ronald A. (2013). Should Global Ítems on Student Rating Scales Be Used for Summative Decisions?. The Journal of Faculty Development, 27(1), 63-68.

Burdsal, Charles A. & Harrison, Paul D. (2008). Further evidence supporting the validity of both a multidimensional profile and an overall evaluation of teaching effectiveness. Assessment & Evaluation in Higher Education, 33(5), 567-76. doi: https://doi.org/10.1080/02602930701699049

Caldera, Juan F., Carranza, María del R., Jiménez, Alma A. & Pérez, Ignacio (2015). Actitudes de los estudiantes universitarios ante la tutoría. Diseño de una escala de medición. Revista de la Educación Superior, 1(173), 103-124. doi: https://doi.org/10.1016/j.resu.2015.04.004

Casero, Antonio (2010). Factores moduladores de la percepción de la calidad docente. RELIEVE, 16(2). doi: http://doi.org/10.7203/relieve.16.2.4135

Cattell, Raymond B. (1966). The Scree Test For The Number Of Factors. Multivariate Behavioral Research, 1, 245-276. doi: https://doi.org/10.1207/s15327906mbr0102_10

Chen, Guo-Hai & Watkins, David. (2010). Stability and correlates of student evaluations of teaching at a Chinese university. Assessment & Evaluation in Higher Education, 35(6), 675-85. doi: https://doi.org/10.1080/02602930902977715

Choi, Bo-Keum & Kim, Jae-Woong. (2014). The Influence of Student and Course Characteristics on Monotonic Response Patterns in Student Evaluation of Teaching in South Korea. Asia Pacific Education Review, may, 1-10. doi: https://doi.org/10.1007/s12564-014-9332-y

Darby, Jenny A. (2007). Are course evaluations subject to a halo effect? Research in Education, 77(1), 46-55. doi: https://doi.org/10.7227/RIE.77.4

De Juanas Oliva, Angel & Beltrán Llera, Jesús A. (2014). Valoraciones de los estudiantes de ciencias de la educación sobre la calidad de la docencia universitaria. Educación XX1, 17(1), 59-82.

Fernández Rico, J. Esteban, Fernández Fernández, Samuel, Álvarez Suárez, Alberto & Martínez Camblor, Pablo (2007). Éxito académico y satisfacción de estudiantes con la enseñanza universitaria. RELIEVE, 13(2). doi: http://doi.org/10.7203/relieve.13.2.4207

Ginns, Paul, Prosser, Michael & Barrie, Simon. (2007). Students’ perceptions of teaching quality in higher education: The perspective of currently enrolled students. Studies in Higher Education, 32(5), 603-615. doi: https://doi.org/10.1080/03075070701573773

Glorfeld, Louis W. (1995). An Improvement on Horn's Parallel Analysis Methodology for Selecting the Correct Number of Factors to Retain. Educational & Psychological Measurement, 55, 377-393. doi: https://doi.org/10.1177/0013164495055003002

González López, Ignacio (2003). Determinación de los elementos que condicionan la calidad de la universidad: aplicación práctica de un análisis factorial. RELIEVE, 9(1). doi: http://doi.org/10.7203/relieve.9.1.4351

Haarala-Muhonen, Anne, Ruohoniemi, Mirja, Katajavuori, Nina & Lindblom-Ylänne, Sari. (2011). Comparison of students’ perceptions of their teaching–learning environments in three professional academic disciplines: A valuable tool for quality enhancement. Learning Environments Research, 14(2), 155-69. doi: https://doi.org/10.1007/s10984-011-9087-x

Hooper, Daire, Coughlan, Joseph & Mullen, Michael (2008). Structural Equation Modelling: Guidelines for Determining Model Fit. The Electronic Journal of Business Research Methods, 6(1), 53-60.

Hoyuelos, F. J.; Ibáñez, J; Jerónimo, E.; San Martín, S. & Santamaría, M. (2014). Variables definitorias del perfil del profesor/a universitario/a ideal desde la perspectiva de los estudiantes pre-universitarios/as. Educación XX1, 17(2), 193-215.

Kaplan, David (2009). Structural Equation Modeling (2nd ed.): Foundations and Extensions. Thousand Oaks, USA: Sage.

Kember, David & Leung, Doris YP. (2011). Disciplinary differences in student ratings of teaching quality. Research in Higher Educatio, 52(3), 278-99. doi: https://doi.org/10.1007/s11162-010-9194-z

Kline, Rex (2011). Principles and practice of structural equation modeling (3rd ed.). New York, London: The Guilford Press.

Lance, C., Butts, M. & Michels, L.. (2006). The Sources of Four Commonly Reported Cutoff Criteria What Did They Really Say? Organizational Research Methods, 9(2), 202-20. doi: https://doi.org/10.1177/1094428105284919

Ledesma, Rubén Daniel & Valero-Mora, Pedro (2007). Determining the Number of Factors to Retain in EFA: An Easy-to-Use Computer Program for Carrying Out Parallel Analysis. Practical Assessment, Research & Evaluation, 12(2), 1-11.

Lemos, M. S., Queirós, C., Teixeira, P.M. & Menezes, I. (2011). Development and validation of a theoretically based, multidimensional questionnaire of student evaluation of university teaching. Assessment & evaluation in higher education, 36(7), 843-64. doi: https://doi.org/10.1080/02602938.2010.493969

(Referencia eliminada para evaluación por pares)

Molero López Barajas, David (2007). Rendimiento académico y opinión sobre la docencia del alumnado participante en experiencias piloto de implantación del Espacio Europeo de Educación Superior. RELIEVE, 13(2), art. 2. doi: http://doi.org/10.7203/relieve.13.2.4205

Mortelmans, D. & Spooren, P. (2009). A revalidation of the SET37 questionnaire for student evaluations of teaching. Educational Studies, 35(5), 547-552. doi: https://doi.org/10.1080/03055690902880299

Muñoz Cantero, J.M., Ríos de Deus, M.P & Abalde, E. (2002). Evaluación Docente vs. Evaluación de la Calidad. RELIEVE, 8(2), art. 4. doi: http://doi.org/10.7203/relieve.8.2.4362

Otani, Koichiro, B., Joon Kim & Jeong-IL Cho (2012). Student evaluation of teaching (SET) in higher education: How to use SET more effectively and efficiently in public affairs education. Journal of Public Affairs Education, 18(3), 531-544.

Palmer, Stuart (2012). The performance of a student evaluation of teaching system. Assessment & Evaluation in Higher Education, 37(8), 975-985. doi: https://doi.org/10.1080/02602938.2011.592935

Pascual Gómez, Isabel (2007). Análisis de la Satisfacción del Alumno con la Docencia Recibida: Un Estudio con Modelos Jerárquicos Lineales. RELIEVE, 13(1), art. 6. doi: http://doi.org/10.7203/relieve.13.1.4216

Pepe, Julie W., & Wang, Morgan C. (2012). What Instructor Qualities Do Students Reward. College Student Journal, 46(3), 603-14.

Peres-Neto, Pedro R., Jackson, Donald A. & Somers, Keith M. (2005). How Many Principal Components? Stopping Rules for Determining the Number of Non-Trivial Axes Revisited. Computational Statistics & Data Analysis, 49, 974-997. doi: https://doi.org/10.1016/j.csda.2004.06.015

Rantanen, Pekka (2013). The number of feedbacks needed for reliable evaluation. A multilevel analysis of the reliability, stability and generalisability of students' evaluation of teaching. Assessment & Evaluation in Higher Education, 38(2), 224-239. doi: https://doi.org/10.1080/02602938.2011.625471

Revelle, William & Rocklin, Thomas (1979). Very Simple Structure. Alternative Procedure for Estimating the Optimal Number of Interpretable Factors. Multivariate Behavioral Research, 14(4), 403-414. doi: https://doi.org/10.1207/s15327906mbr1404_2

Schreider, J., Stage, F., King, J., Nora, A. & Barlow, E. (2006). Reporting structural equation modeling and confirmatory factor analysis results: a review. The Journal of Education Research, 99(6), 323-337. doi: https://doi.org/10.3200/JOER.99.6.323-338

Spooren, Pieter, Brockx, Bert & Mortelmans, Dimitri. (2013). On the Validity of Student Evaluation of Teaching The State of the Art. Review of Educational Research, 83(4), 598-642. doi: https://doi.org/10.3102/0034654313496870

Stark-Wroblewski, Kimberly, Ahlering, Robert F. & Brill, Flannery M. (2007). Toward a more comprehensive approach to evaluating teaching effectiveness: Supplementing student evaluations of teaching with pre–post learning measures. Assessment & Evaluation in Higher Education, 32(4), 403-15. doi: https://doi.org/10.1080/02602930600898536

Stout, William F. (1990). A new ítem response theory modeling approach with applications to unidimensionality assessment and ability estimation. Psychometrika, 55(2), 293-325. doi: https://doi.org/10.1007/BF02295289

Stout, William F. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52(4), 589-617. doi: https://doi.org/10.1007/BF02294821

Tomkiewicz, Joseph & Bass, Kenneth. (2008). Differences between Male Students’ and Female Students’ Perception of Professors. College Student Journal, 42(2), 422-430.

Uttl, B., White, C. A. & Gonzalez, D. W. (2016). Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22-42. doi: http://dx.doi.org/10.1016/j.stueduc.2016.08.007

Velicer, Wayne F., Eaton, Cheryl A. & Fava, Joseph L. (2000). Construct explication through factor or component analysis: A review and evaluation of alternative procedures for determining the number of factors or components. En Problems and solutions in human assessment, 41-71. Springer. doi: https://doi.org/10.1007/978-1-4615-4397-8_3

Wood, James M., Tataryn, Douglas J. & Gorsuch, Richard L. (1996). Effects of under-and overextraction on principal axis factor analysis with varimax rotation. Psychological methods, 1(4), 354. doi: https://doi.org/10.1037/1082-989X.1.4.354

Zerihun, Zenawi, Beishuizen, Jos & Van Os, Willem. (2012). Student learning experience as indicator of teaching quality. Educational Assessment, Evaluation and Accountability, 24(2), 99-111. doi: https://doi.org/10.1007/s11092-011-9140-4

Zhao, Jing & Gallant, Dorinda J. (2012). Student evaluation of instruction in higher education: Exploring issues of validity and reliability. Assessment & Evaluation in Higher Education, 37(2), 227-35. doi: https://doi.org/10.1080/02602938.2010.523819

Zwick, William R. & Velicer, Wayne F. (1986). Comparison of five rules for determining the number of components to retain. Psychological bulletin, 99(3), 432-442. doi: https://doi.org/10.1037/0033-2909.99.3.432

Descargas

Publicado

2017-12-15

Número

Vol. 23 Núm. 2 (2017)

Sección

Artículos

Licencia

Los autores ceden de forma no exclusiva los derechos de explotación de los trabajos publicados a RELIEVE (a los solos efectos de favorecer la difusión de los artículos publicados:firmar contratos de difusión, de integración en bases de datos, etc.) y consienten que se distribuyan bajo la licencia de Creative Commons Reconocimiento-Uso No Comercial 4.0 International (CC-BY-NC 4.0), que permite a terceros el uso de lo publicado siempre que se mencione la autoría de la obra y la fuente de publicación, y se haga uso sin fines comerciales.

Los autores pueden llegar a otros acuerdos contractuales adicionales e independientes, para la distribución no exclusiva de la versión del trabajo publicado en esta revista (por ejemplo, incluyéndolo en un repositorio institucional o publicándolo en un libro), siempre y cuando se cite claramente que la fuente original de publicación es esta revista.

A los autores se les anima a difundir su trabajo después de publicado, a través de internet (por ejemplo, en archivos institucionales en línea o en su página web) lo que puede generar intercambios interesantes y aumentar las citas del trabajo.

La mera remisión del artículo a RELIEVE supone la aceptación de estas condiciones.