Psychometric properties and standard-setting study of the Piano Performance Test for prospective teachers

Authors

DOI:

https://doi.org/10.7203/LEEME.0.27217

Keywords:

Music-education, piano performance, pre-service teachers, inter-rater reliability, rubric, standard-setting.

Abstract

The purpose of this study was to investigate the psychometric properties of the Piano Performance test for Elementary School Teachers and to undertake a standard-setting study for this scale. This study included three groups of participants: students (n=100), raters (n=2) used to test the psychometric features of the musical instrument performance test and experts (n=6) for the standard-setting study of the test. In this study, the researchers developed a music performance test and analytical rubric. The results showed that the one-factor structure was appropriate for the musical instrument performance test, which explained 66% of the total variance. The Cronbach Alpha coefficient showed that the internal consistency of the scale was acceptable (.83). Moreover, generalizability studies and the intra-class correlation coefficient indicated excellent rater reliability for the scale. The results of the item discrimination analysis show that the musical instrument performance test is capable of discriminating participants who had high and low levels of ability to play the piano.

References

Abeles, H.F. (1973). Development and validation of a clarinet performance adjudication rating scale. Journal of Research in Music Education21, 246-255. https://doi.org/10.2307/3345094

Alessandri, E., Williamson, V.J., Eiholzer, H. y Williamon, A. (2016). A critical ear: analysis of value judgments in reviews of Beethoven’s piano sonata recordings. Frontiers in Psychology, 7, 391. https://doi.org/10.3389/fpsyg.2016.00391

Andrade, H.G. (2000). Using rubrics to promote thinking and learning. Educational Leadership, 57(5),13-18. https://www.researchgate.net/publication/285750862_Using_rubrics_to_promote_thinking_and_learning

Barret, P. y Kline, P. (1981). The observation to variable ratio in factor analyses. Journal of Personality and Group Behavior, 2, 94-98.  https://www.researchgate.net/publication/232561774_The_Observation_to_Variable_Ratio_in_Factor_Analysis

Bejar, I.I. (2008). Standard setting: What is it? Why is it important? (R&D Connections No. 7.). https://www.ets.org/Media/Research/pdf/RD_Connections7.pdf

Ben-David, M.F. (2000). AMEE guide no.18: Standard setting in student assessment. Medical Teacher, 22(2), 120-130. https://doi.org/10.1080/01421590078526

Bergee, M.J. (2003). Faculty interjudge reliability of music performance evaluation. Journal of Research in Music Education, 51(2), 137-150. https://doi.org/10.2307/3345847

Bergee, M.J. (2007). Performer, rater, occasion, and sequence as sources of variability in music performance assessment. Journal of Research in Music Education, 55(4), 344-358. https://doi.org/10.1177/0022429408317515

Boud, D. (1995). Enhancing Learning Through Self-Assessment. Routledge.

Boyle D.J. y Radocy, R.E. (1987). Measurement: Evaluation of musical experiences. Schirmer Books.

Brennan, R.L. (2001). Statistics for social science and public policy generalizability theory. Springer.

Brown, T.A. (2006). Confirmatory factor analysis for applied research. Guilford Publications, Inc.

Cizek, G.J. y Bunch, M.B. (2007). Standard setting: A guide to establishing and evaluating performance standards on tests. Sage Publications.

Crocker, L. y Algina, J. (1986). Introduction to classical and modern test theory. Cengage Learning.

Diederich, P.B. (1973). Short-cut statistics for teacher-made tests. Educational Testing Service.

Duerksen, G.L. (1972). Some effects of expectation on evaluation of recorded musical performance. Journal of Research in Music Education., 20, 268-272. https://doi.org/10.2307/3344093

Elliott, D. y Silverman, M. (2015). Music matters: A Philosophy of music education. Oxford University Press.

Fraenkel, J.R., Wallen, N.E. y Hyun, H.H. (2015). How to design and evaluate research in education (9th ed.). McGraw Hill Education.

Guven, E.D. (2020). Piano-accompanied solfège reading experiences of preservice music teachers. Research Studies in Music Education, 43(3), 417-433. https://doi.org/10.1177/1321103x19871078

Gliner, J.A., Morgan, G.A. y Leech, N.L. (2009). Research methods in applied settings: An integrated approach to design and analysis. Routledge.

Haladyna, T.M. (1997). Writing test items to evaluate higher order thinking. Allyn and Bacon.

Hooper, D., Coughan, J. y Mullen, M. (2008). Structural equation modeling: Guidelines for determining model fit. The Electronic Journal of Business Research Methods, 6(1), 53-60. http://mural.maynoothuniversity.ie/6596/

Howell, D.C. (2013). Statistical methods for psychology. Cengage Learning.

Juslin, P.N. (2003). Five facets of musical expression: a psychologist’s perspective on music performance. Psychology of Music, 31, 273-302. https://doi.org/10.1177/03057356030313003

Kamış, Ö. y Doğan, C.D. (2018). An investigation of reliability coefficients estimated for decision studies in generalizability theory. Journal of Education and Learning, 7(4), 103-113. https://doi.org/10.5539/jel.v7n4p103

Kim, S., Park, J.M., Rhyu, S., Nam, J. y Lee, K. (2021). Quantitative analysis of piano performance proficiency focusing on difference between hands. PLoS ONE, 16(5), 1-28. https://doi.org/10.1371/journal.pone.0250299

Kutlu, Ö., Doğan, C.D. y Karakaya (2014). Ölçme ve değerlendirme: Performansa ve portfolyoya dayalı durum belirleme [Measurement and evaluation: An assessment based on performance and portfolio]. Pegem Academy.

Lee, Y. (2009). Music practices and teachers’ needs for teaching music in public preschools of south korea. International Journal of Music Education, 27(4), 356-371. https://doi.org/10.1177/0255761409344663

Leech, N.L., Barrett, K.C. y George, A.M. (2005). SPSS for intermediate statistics: Use and interpretation (2nd ed.). Lawrence Erlbaum Associates, Publishers.

Mak, H.W. y Fancourt, D. (2019). Arts engagement and self‐esteem in children: Results from a propensity score matching analysis. Annals of the New York Academy of Sciences1449(1), 36-45. https://doi.org/10.1111/nyas.14056

McMillan, J.H. y Hearn, J. (2008). Student self-assessment: The key to stronger student motivation and higher achievement. Educational Horizons, 87(1), 40-49. https://www.jstor.org/stable/42923742

Moskal, M.B. (2000). Scoring rubric: what, when and how? Practical Assessment, Research & Evaluation, 7, 1-5. https://doi.org/10.7275/a5vq-7q66

Nichols, J.P. (1991). A factor analysis approach to the development of a rating scale for snare drum performance. Dialogue Instrumental Music Education, 15, 11-31. Michigan University Press.

Palmer, C. (1996). On the assignment of structure in music performance. Music Perception, 14, 23-56. https://doi.org/10.2307/40285708

Price, H.E. y Burnsed, V. (1989). Classroom teachers’ assessments of elementary education music methods. Update: Applications of Research in Music-education, 8(1), 28-32. https://doi.org/10.1177/875512338900800107

Robinson, N.R. (2016). Developing a critical consciousness for diversity and equity among preservice music teachers. Journal of Music Teacher Education, 26(3), 11-26. https://doi.org/10.1177/1057083716643349

Russell, B.E. (2010). The Empirical Testing of a Musical Performance Assessment Paradigm. PhD. University of Miami.

Schellenberg, E.G. (2018). Music and cognitive abilities. En S. Hallam, I. Cross y M. Thaut (Eds.), The Oxford handbook of music psychology (2nd ed.) (pp.261-275). Oxford University Press.

Scripp, L. y Kaufman, B. (2019). Music Learning as Youth Development. https://doi.org/10.4324/9780429436956-11

Shavelson, R.J. y Webb, N.M. (1991). Generalizability theory a primer. Sage Publications.

Southgate, L., Hays, R.B., Norcini, J., Mulholland, H., Ayers, B., Woolliscroft, H., Cusimano, M., McAvoy, P., Ainsworth, M., Haist, S. y Campbell, M. (2001). Setting performance standards for medical practice: A theoretical framework. Medical Education, 35(5), 474-481.  https://doi.org/10.1046/j.1365-2923.2001.00897.x

Standley, J.M. (2016). Music and early childhood development. En A.C. Lehmann, J.A. Sloboda y R. Woody (Eds.), Psychology for musicians: Understanding and acquiring the skills (pp.103-118). Oxford University Press.

Stanley, M., Brooker, R. y Gilbert, R. (2002). Examiner perceptions of using criteria in music performance assessment. Research Studies in Music Education, 18(1), 46-56. https://doi.org/10.1177/1321103x020180010601

Stevens, D.D. y Levi, A. (2005). Introduction to rubrics: An assessment tool to save time, convey effective feedback, and promote student learning. Styus.

Sümer, N. (2000). Yapısal eşitlik modelleri: Temel kavramlar ve örnek uygulamalar [Structural equation modeling: Basic concepts and applications]. Türk Psikoloji Yazıları, 3(6), 49-74.  https://www.researchgate.net/publication/281981476_Yapidotlesssal_esitlik_modelleri_Temel_kavramlar_ve_ornek_uygulamalar

Swanwick, K. (2016). Music education and human development. En R. Colwell y C. Richardson (Eds.), The new handbook of research on music teaching and learning (pp.3-23). Oxford University Press.

Tabachnick, B.G. y Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Pearson Education.

Thompson, S. y Williamon, A. (2003). Evaluating evaluation: Musical performance assessment as a research tool. Music Perception, 21(1), 21-41. https://doi.org/10.1525/mp.2003.21.1.21

Thompson, W.F., Diamond, C.T.P. y Balkwill, L.L. (1998). The adjudication of six performances of a Chopin etude: A study of expert knowledge. Psychology of Music, 26(2), 154-174. https://doi.org/10.1177/0305735698262004

Urbina, S. (2004). Essentials of psychological testing. John Wiley & Sons.

Wapnick, J., Ryan, C., Lacaille, N. y Darrow, A.-A. (2004). Effects of selected variables on musicians’ ratings of high-level piano performances. International Journal of Music Education, 22(1), 7-20. https://doi.org/10.1177/0255761404042371

Wapnick, J., Ryan, C., Campbell, L., Deek, R., Lemire, R. y Darrow, A.A. (2005). Effects of excerpt tempo and duration on musicians' ratings of high-level piano performance. Journal of Research in Music Education, 53, 162-176. https://doi.org/10.1177/002242940505300206

Watkins, J.G. y Farnum, S.E. (1954). The Watkins-Farnum performance scale, form A; A standardized achievement test for all band instruments. Hall Leonard Music Press.

Wesolowski, B.C. (2012). Understanding and developing rubrics for music performance assessment. Music Educators Journal, 98(3), 36-42. https://doi.org/10.1177/0027432111432524

Wesolowski, B.C., Wind, S.A. y Engelhard, G. (2016). Examining rater precision in music performance assessment: An analysis of rating scale structure using the Multifaceted Rasch Partial Credit Model. Music Perception: An Interdisciplinary Journal, 33(5), 662-678. https://doi.org/10.1525/mp.2016.33.5.662

Wesolowski, B.C., Wind, S.A. y Engelhard, G. (2015). Rater fairness in music performance assessment: Evaluating model-data fit and differential rater functioning. Musicae Scientiae, 19(2), 147-170. https://doi.org/10.1177/1029864915589014

Wrigley, W.J. y Emmerson, S.B. (2011). Ecological development and validation of a music performance rating scale for five instrument families. Psychology of Music, 41(1), 97-118. https://doi.org/10.1177/0305735611418552

Zdzinski, S.F. y Barnes, G.V. (2002). Development and validation of a string performance rating scale. Journal of Research in Music Education, 50, 245-255. https://doi.org/10.2307/3345801

Additional Files

Published

2023-12-04