Psychometric analysis of a test to assess the digital competence of compulsory education students

Authors

DOI:

https://doi.org/10.7203/relieve.26.2.17611

Keywords:

assessment, educational technology, basic education, psychometry

Abstract

Valid and reliable information-gathering tools are necessary if research on digital competence is to provide valuable information that is able to guide education policies on the development of digital competence and the integration of information and communication technologies (ICT) into the educational systems of different countries.

The aim of the present work is to validate a test that assesses the digital competence of students in compulsory education (ECODIES) in the areas of general knowledge, ability and attitude, using the Development and Understanding of Digital Competence in Europe Framework (DigComp) as the foundation for this research. This tool, designed by a university research team, was used on a sample of 771 students. The easiness/difficulty index and dimensions of the test were analyzed, as well as its validity and reliability. It was concluded that ECODIES is an original and novel test that features good psychometric properties that qualify it as a reliable and valid instrument for directly measuring digital competence by reflecting on real situations and problem solving. 

Author Biographies

Sonia Casillas-Martín, University of Salamanca

Phd in Pedagogy from the Pontifical University of Salamanca. Teacher in the Department of Didactics, Organization and Research Methods in the Faculty of Education of the University of Salamanca. Member of the Research Group in Educational Technology of the University of Salamanca. Main research lines: "digital competence", "educational innovation", "collaborative learning and ICT". University of Salamanca. Faculty of Education. Paseo de Canalejas, 169, 37008, Salamanca (Spain).

Marcos Cabezas-González, University of Salamanca

Phd in Pedagogy from the Pontifical University of Salamanca. Teacher in the Department of Didactics, Organization and Research Methods in the Faculty of Education of the University of Salamanca. Member of the Research Group in Educational Technology of the University of Salamanca. Main research lines: “ICT in Education”, “collaborative learning and ICT”, “digital competence”. University of Salamanca. Faculty of Education. Paseo de Canalejas, 169, 37008, Salamanca (Spain).

Ana García-Valcárcel Muñoz-Repiso, University of Salamanca

Phd in Pedagogy from Comillas Pontifical University (Madrid). Professor of Educational Technology in the Department of Didactics, Organization and Research Methods in the Faculty of Education of the University of Salamanca. Director of the research group GITE-USAL and Coordinator of the master’s degree in Information and Communication Technologies in Education at the Faculty of Education of the University of Salamanca. Main research lines: “ICT in Education”, “collaborative learning and ICT”, “digital competence”. University of Salamanca. Faculty of Education. Paseo de Canalejas, 169, 37008, Salamanca (Spain).

References

Agudo, A.A., García, E.G., & Martínez Heredia, N. (2020). Desafíos para una ciudadanía inclusiva: Competencia digital entre adultos mayores y jóvenes. Comunicacao Midia e Consumo, 17(48), 11-33. https://doi.org/10.18568/cmc.v17i48.2247
Alvarado Martínez, E. (2020). La formación tecno-pedagógica de los profesores universitarios de lengua extranjera. Revista Lengua y Cultura, 1(2), 58-63. https://doi.org/10.29057/lc.v1i2.5423
Armengol, C., Castro, D., Jariot, M., Massot, M., & Sala, J. (2011). El prácticum en el Espacio Europeo de Educación Superior (EEES): mapa de competencias del profesional de la educación. Revista de Educación, 354, 71-98. http://www.univnova.org/documentos/492.pdf
Basantes-Andrade, A.V., Cabezas-González, M., & Casillas-Martín, S. (2020). Digital Competences Relationship with Gender and Generation of University Professors. IJASEIT, International Journal on Advanced Science, Engineering and Information Technology, 10(1), 205-211. https://doi.org/10.1857/ijaseit.10.1.10806
Bonnes, C., Leiser, C., Schmidt‐Hertha, B., Rott, K.J., & Hochholdinger, S. (2020). The relationship between trainers’ media‐didactical competence and media‐didactical self‐efficacy, attitudes and use of digital media in training. International Journal of Training and Development, 24(1), 74-88. https://doi.org/10.1111/ijtd.12171
Cabezas, M., & Casillas, S. (2018). Social Educators: A Study of Digital Competence from a Gender Differences Perspective. Croatian Journal of Education-Hrvatski Casopis za Odgoj i obrazovanje, 20(1), 11-42. https://doi.org/10.15516/cje.v20i1.2632
Carretero, S., Vuorikari, R., & Punie, Y. (2017). DigComp 2.1. The digital competence framework for citizens. Publications Office of the European Union. https://doi.org/10.2760/38842
Casillas-Martín, S., Cabezas-González, M., & García-Peñalvo, F.J. (2020). Digital competence of early childhood education teachers: attitude, knowledge and use of ICT. European Journal of Teacher Education, 43(2), 210-223. https://doi.org/10.1080/02619768.2019.1681393
Casillas-Martín, S., Cabezas-González, M., Sanches-Ferreira, M., & Teixeira Diogo, F. L. (2018). Psychometric Study of a Questionnaire to Measure the Digital Competence of University Students (CODIEU). Education in the Knowledge Society, 19(3), 69-81. https://doi.org/10.14201/eks20181936981
Domínguez-Lara, S. (2018). Fiabilidad y alfa ordinal. Actas Urológicas Españolas, 42(2), 140-141. https://doi.org/10.1016/j.acuro.2017.07.002
Ferrari, A. (2013). DigComp: A framework for developing and understanding digital competence in Europe. Publications Office of the European Union. https://doi.org/10.2788/52966
Frailon, J., Schulz, W., Friedman, T., Ainley, J., & Gebhardt, E. (2013). International Computer and Information Literacy Study. Amsterdam: IEA Secretariat. https://doi.org/10.15478/uuid:b9cdd888-6665-4e9f-a21e-61569845ed5b
García-Valcárcel, A., Hernández, A., Basilotta, V., Cabezas-González, M., Casillas-Martín, S., González Rodero, L., Iglesias, A., Martín del Pozo, M., Mena Marcos, J.J., & Salvador Blanco, L. (2019). Modelo de indicadores para evaluar la competencia digital de los estudiantes tomando como referencia el modelo DigComp (INCODIES) [Universidad de Salamanca].  Repositorio Institucional GREDOS.  https://gredos.usal.es/handle/10366/139409
García-Valcárcel, A., Casillas-Martín, S., & Basilotta, V. (2020). Validation of an Indicator Model (INCODIES) for Assessing Student Digital Competence in Basic Education. Journal of New Approaches in Educational Research, 9(1), 110-125. https://doi.org/10.7821/naer.2020.1.459
García-Valcárcel, A., Salvador Blanco, L., Casillas-Martín, S., & Basilotta, V. (2019). Evaluación de las competencias digitales sobre seguridad de los estudiantes de Educación Básica. Revista de Educación a Distancia, 19(61), 1-34. https://doi.org/10.6018/red/61/05
González-Segura, C.M., García-García, M., Menéndez-Domínguez, V.H. (2018). Análisis de la evaluación de competencias y su aplicación en un Sistema de Gestión del Aprendizaje. Un caso de estudio. RED. Revista de Educación a Distancia, 58(3), 1-20. http://dx.doi.org/10.6018/red/58/3
He, T., Huang, Q., Yu, X., & Li, S. (2020). Exploring students’ digital informal learning: the roles of digital competence and DTPB factors. Behaviour & Information Technology, published online, 1-11. https://doi.org/10.1080/0144929X.2020.1752800
Hea, T., Huanga, Q., Yub, X., & Li, S. (2020): Exploring students’ digital informal learning: the roles of digital competence and DTPB factors. Behaviour & Information Technology, published online. https://doi.org/10.1080/0144929X.2020.1752800
ISTE (2016). ISTE Standards for Students. A Practical Guide for Learning with Technology. Stanstebook.
Kaiser, H. F. (1974). An index of factorial simplicity. Psychometrika, 39, 32-36. https://doi.org/10.1007/BF02291575
Krumsvik, R. J. (2011). Digital competence in Norwegian teacher education and schools. Högre Utbildning, 1(1), 39-51. https://www.researchgate.net/publication/305360830_Digital_competence_in_the_Norwegian_teacher_education_and_school
Larionova, V., Brown, K., Bystrova, T., & Sinitsyn, E. (2018). Russian perspectives of online learning technologies in higher education: An empirical study of a MOOC. Research in Comparative and International Education, 13(1), 70-91. https://doi.org/10.1177/1745499918763420
Lores Gómez, B., Sánchez Thevenet, P., & García Bellido, M.R. (2019). La formación de la competencia digital en los docentes. Profesorado, Revista de Currículum y formación del profesorado, 24(4), 234-260. https://doi.org/10.30827/profesorado.v23i4.11720
Martínez, A., Cegarra, J.G., & Rubio, J.A. (2012). Aprendizaje basado en competencias: Una propuesta para la autoevaluación del docente. Profesorado, Revista de currículum y formación del profesorado, 16(2), 373-386. http://www.ugr.es/~recfpro/rev162COL5.pdf
McMillan, J.H., & Schumacher, S. (2010). Investigación educativa. Pearson Addison Wesley.
Mishra, P., & Koehler, M. (2006). Technological Pedagogical Content Knowledge: A Framework for Teacher Knowledge. State University. https://doi.org/10.1111/j.1467-9620.2006.00684.x
Moreno Rodríguez, M.D., Gabarda Méndez, V., & Rodríguez Martín, A.M. (2018). Alfabetización informacional y competencia digital en estudiantes de magisterio. Profesorado, Revista de currículum y formación del profesorado, 22(3), 253-270. http://dx.doi.org/10.30827/profesorado.v22i3.8001
Morales, P.; Urosa, B., & Blanco, A. (2003). Construcción de escalas de actitud tipo Likert: una guía práctica. Cuadernos de Estadística.
Nowak, B.M. (2019). The development of digital competence of students of teacher training studies-Polish cases. International Journal of Higher Education, 8(6), 262-266. https://doi.org/10.5430/ijhe.v8n6p262
Oliden, P. E., & Zumbo, B. D. (2008). Coeficientes de fiabilidad para escalas de respuesta categórica ordenada. Psicothema, 20(4), 896‐901. http://www.psicothema.com/psicothema.asp?id=3572
Otzen, T., & Manterola, C. (2017). Técnicas de Muestreo sobre una Población a Estudio. International Journal of Morphology, 35(1), 227-232. http://dx.doi.org/10.4067/S0717-95022017000100037
Pérez-Rodríguez, A., Delgado-Ponce, A., Marín-Mateos, P., & Romero-Rodríguez, L.M. (2019). Media Competence in Spanish Secondary School Students. Assessing Instrumental and Critical Thinking Skills in Digital Contexts. Educational Sciences: Theory and Practice, 19(3), 33-48. https://doi.org/10.12738/estp.2019.3.003
Punie, Y. (2017). European Framework for the Digital Competence of Educators (DigCompEdu). Publications Office of the European Union. https://doi.org/10.2760/159770
Terry, J., Davies, A., Williams, C., Tait, S., Condon, L.  (2019). Improving the digital literacy competence of nursing and midwifery students: A qualitative study of the experiences of NICE student champions. Nurse Education in Practice, 34, 192-198. https://doi.org/10.1016/j.nepr.2018.11.016
Tobón, S., Pimienta, J.H., & García Fraile, J.A. (2010). Secuencias didácticas: Aprendizaje y evaluación de competencias. Pearson Educación.
Torres-Hernández, M., Pessoa, T., & Gallego-Arrufat, M.J. (2019). Intervención y evaluación con tecnologías de la competencia en seguridad digital. Digital Education Review, 35, 111-129. https://revistes.ub.edu/index.php/der/article/view/27399/pdf
Unión Europea (2018, 22 de mayo). Recomendación C 189 Relativa a las Competencias Clave Para el Aprendizaje Permanente. Diario Oficial de la Unión Europea, 4 de junio de 2018.
Valverde Berrocoso, J., Revuelta Domínguez, F.I., & Fernández Sánchez, M.R. (2012). Modelos de evaluación por competencias a través de un sistema de gestión de aprendizaje. Experiencias en la formación inicial del profesorado. Revista Iberoamericana de Educación, 60, 51-62. https://doi.org/10.35362/rie600443
Valverde Crespo, D., De Pro Bueno, A., & González Sánchez, J. (2018). La competencia informacional-digital en la enseñanza y aprendizaje de las ciencias en la educación secundaria obligatoria actual: una revisión teórica. Revista Eureka sobre Enseñanza y Divulgación de las Ciencias, 15(2). 1-15. https://doi.org/10.25267/Rev_Eureka_ensen_divulg_cienc.2018.v15.i2.2105
Vuorikari, R., Punie, Y., Carretero, S, & Van den Brande, L. (2016). DigComp 2.0: The Digital Competence Framework for Citizens. Update Phase 1: The Conceptual Reference Model. Publication Office of the European Union. https://doi.org/10.2791/11517
Welch, S., & Comer, J. (1998). Quantitative methods for public administration: Techniques and applications. Brooks/Cole.
Xu, S., Yang, H. H., MacLeod, J., & Zhu, S. (2019). Social media competence and digital citizenship among college students. Convergence, 25(4), 735-752. https://doi.org/10.1177/1354856517751390
Zumbo, B. D., Gadermann, A. M., & Zeisser, C. (2007). Ordinal versions of coefficients alpha and theta for Likert rating scales. Journal of modern applied statistical methods, 6(1), 20-29. https://doi.org/10.22237/jmasm/1177992180

Downloads

Additional Files

Published

2020-12-08

Issue

Vol. 26 No. 2 (2020): MONOGRÁFICO. Edición científica: 25 años de Relieve

Section

Research Articles

License

The authors grant non-exclusive rights of exploitation of works published to RELIEVE and consent to be distributed under the Creative Commons Attribution-Noncommercial Use 4.0 International License (CC-BY-NC 4.0), which allows third parties to use the published material whenever the authorship of the work and the source of publication is mentioned, and it is used for non-commercial purposes. 

    The authors can reach other additional and independent contractual agreements, for the non-exclusive distribution of the version of the work published in this journal (for example, by including it in an institutional repository or publishing it in a book), as long as it is clearly stated that the Original source of publication is this magazine.  

    Authors are encouraged to disseminate their work after it has been published, through the internet (for example, in institutional archives online or on its website) which can generate interesting exchanges and increase work appointments. 

   The fact of sending your paper to RELIEVE implies that you accept these conditions.