Psychometric properties and standard-setting study of the Piano Performance Test for prospective teachers
DOI:
https://doi.org/10.7203/LEEME.0.27217Keywords:
Music-education, piano performance, pre-service teachers, inter-rater reliability, rubric, standard-setting.
Abstract
The purpose of this study was to investigate the psychometric properties of the Piano Performance test for Elementary School Teachers and to undertake a standard-setting study for this scale. This study included three groups of participants: students (n=100), raters (n=2) used to test the psychometric features of the musical instrument performance test and experts (n=6) for the standard-setting study of the test. In this study, the researchers developed a music performance test and analytical rubric. The results showed that the one-factor structure was appropriate for the musical instrument performance test, which explained 66% of the total variance. The Cronbach Alpha coefficient showed that the internal consistency of the scale was acceptable (.83). Moreover, generalizability studies and the intra-class correlation coefficient indicated excellent rater reliability for the scale. The results of the item discrimination analysis show that the musical instrument performance test is capable of discriminating participants who had high and low levels of ability to play the piano.
References
Abeles, H.F. (1973). Development and validation of a clarinet performance adjudication rating scale. Journal of Research in Music Education, 21, 246-255. https://doi.org/10.2307/3345094
Alessandri, E., Williamson, V.J., Eiholzer, H. y Williamon, A. (2016). A critical ear: analysis of value judgments in reviews of Beethoven’s piano sonata recordings. Frontiers in Psychology, 7, 391. https://doi.org/10.3389/fpsyg.2016.00391
Andrade, H.G. (2000). Using rubrics to promote thinking and learning. Educational Leadership, 57(5),13-18. https://www.researchgate.net/publication/285750862_Using_rubrics_to_promote_thinking_and_learning
Barret, P. y Kline, P. (1981). The observation to variable ratio in factor analyses. Journal of Personality and Group Behavior, 2, 94-98. https://www.researchgate.net/publication/232561774_The_Observation_to_Variable_Ratio_in_Factor_Analysis
Bejar, I.I. (2008). Standard setting: What is it? Why is it important? (R&D Connections No. 7.). https://www.ets.org/Media/Research/pdf/RD_Connections7.pdf
Ben-David, M.F. (2000). AMEE guide no.18: Standard setting in student assessment. Medical Teacher, 22(2), 120-130. https://doi.org/10.1080/01421590078526
Bergee, M.J. (2003). Faculty interjudge reliability of music performance evaluation. Journal of Research in Music Education, 51(2), 137-150. https://doi.org/10.2307/3345847
Bergee, M.J. (2007). Performer, rater, occasion, and sequence as sources of variability in music performance assessment. Journal of Research in Music Education, 55(4), 344-358. https://doi.org/10.1177/0022429408317515
Boud, D. (1995). Enhancing Learning Through Self-Assessment. Routledge.
Boyle D.J. y Radocy, R.E. (1987). Measurement: Evaluation of musical experiences. Schirmer Books.
Brennan, R.L. (2001). Statistics for social science and public policy generalizability theory. Springer.
Brown, T.A. (2006). Confirmatory factor analysis for applied research. Guilford Publications, Inc.
Cizek, G.J. y Bunch, M.B. (2007). Standard setting: A guide to establishing and evaluating performance standards on tests. Sage Publications.
Crocker, L. y Algina, J. (1986). Introduction to classical and modern test theory. Cengage Learning.
Diederich, P.B. (1973). Short-cut statistics for teacher-made tests. Educational Testing Service.
Duerksen, G.L. (1972). Some effects of expectation on evaluation of recorded musical performance. Journal of Research in Music Education., 20, 268-272. https://doi.org/10.2307/3344093
Elliott, D. y Silverman, M. (2015). Music matters: A Philosophy of music education. Oxford University Press.
Fraenkel, J.R., Wallen, N.E. y Hyun, H.H. (2015). How to design and evaluate research in education (9th ed.). McGraw Hill Education.
Guven, E.D. (2020). Piano-accompanied solfège reading experiences of preservice music teachers. Research Studies in Music Education, 43(3), 417-433. https://doi.org/10.1177/1321103x19871078
Gliner, J.A., Morgan, G.A. y Leech, N.L. (2009). Research methods in applied settings: An integrated approach to design and analysis. Routledge.
Haladyna, T.M. (1997). Writing test items to evaluate higher order thinking. Allyn and Bacon.
Hooper, D., Coughan, J. y Mullen, M. (2008). Structural equation modeling: Guidelines for determining model fit. The Electronic Journal of Business Research Methods, 6(1), 53-60. http://mural.maynoothuniversity.ie/6596/
Howell, D.C. (2013). Statistical methods for psychology. Cengage Learning.
Juslin, P.N. (2003). Five facets of musical expression: a psychologist’s perspective on music performance. Psychology of Music, 31, 273-302. https://doi.org/10.1177/03057356030313003
Kamış, Ö. y Doğan, C.D. (2018). An investigation of reliability coefficients estimated for decision studies in generalizability theory. Journal of Education and Learning, 7(4), 103-113. https://doi.org/10.5539/jel.v7n4p103
Kim, S., Park, J.M., Rhyu, S., Nam, J. y Lee, K. (2021). Quantitative analysis of piano performance proficiency focusing on difference between hands. PLoS ONE, 16(5), 1-28. https://doi.org/10.1371/journal.pone.0250299
Kutlu, Ö., Doğan, C.D. y Karakaya (2014). Ölçme ve değerlendirme: Performansa ve portfolyoya dayalı durum belirleme [Measurement and evaluation: An assessment based on performance and portfolio]. Pegem Academy.
Lee, Y. (2009). Music practices and teachers’ needs for teaching music in public preschools of south korea. International Journal of Music Education, 27(4), 356-371. https://doi.org/10.1177/0255761409344663
Leech, N.L., Barrett, K.C. y George, A.M. (2005). SPSS for intermediate statistics: Use and interpretation (2nd ed.). Lawrence Erlbaum Associates, Publishers.
Mak, H.W. y Fancourt, D. (2019). Arts engagement and self‐esteem in children: Results from a propensity score matching analysis. Annals of the New York Academy of Sciences, 1449(1), 36-45. https://doi.org/10.1111/nyas.14056
McMillan, J.H. y Hearn, J. (2008). Student self-assessment: The key to stronger student motivation and higher achievement. Educational Horizons, 87(1), 40-49. https://www.jstor.org/stable/42923742
Moskal, M.B. (2000). Scoring rubric: what, when and how? Practical Assessment, Research & Evaluation, 7, 1-5. https://doi.org/10.7275/a5vq-7q66
Nichols, J.P. (1991). A factor analysis approach to the development of a rating scale for snare drum performance. Dialogue Instrumental Music Education, 15, 11-31. Michigan University Press.
Palmer, C. (1996). On the assignment of structure in music performance. Music Perception, 14, 23-56. https://doi.org/10.2307/40285708
Price, H.E. y Burnsed, V. (1989). Classroom teachers’ assessments of elementary education music methods. Update: Applications of Research in Music-education, 8(1), 28-32. https://doi.org/10.1177/875512338900800107
Robinson, N.R. (2016). Developing a critical consciousness for diversity and equity among preservice music teachers. Journal of Music Teacher Education, 26(3), 11-26. https://doi.org/10.1177/1057083716643349
Russell, B.E. (2010). The Empirical Testing of a Musical Performance Assessment Paradigm. PhD. University of Miami.
Schellenberg, E.G. (2018). Music and cognitive abilities. En S. Hallam, I. Cross y M. Thaut (Eds.), The Oxford handbook of music psychology (2nd ed.) (pp.261-275). Oxford University Press.
Scripp, L. y Kaufman, B. (2019). Music Learning as Youth Development. https://doi.org/10.4324/9780429436956-11
Shavelson, R.J. y Webb, N.M. (1991). Generalizability theory a primer. Sage Publications.
Southgate, L., Hays, R.B., Norcini, J., Mulholland, H., Ayers, B., Woolliscroft, H., Cusimano, M., McAvoy, P., Ainsworth, M., Haist, S. y Campbell, M. (2001). Setting performance standards for medical practice: A theoretical framework. Medical Education, 35(5), 474-481. https://doi.org/10.1046/j.1365-2923.2001.00897.x
Standley, J.M. (2016). Music and early childhood development. En A.C. Lehmann, J.A. Sloboda y R. Woody (Eds.), Psychology for musicians: Understanding and acquiring the skills (pp.103-118). Oxford University Press.
Stanley, M., Brooker, R. y Gilbert, R. (2002). Examiner perceptions of using criteria in music performance assessment. Research Studies in Music Education, 18(1), 46-56. https://doi.org/10.1177/1321103x020180010601
Stevens, D.D. y Levi, A. (2005). Introduction to rubrics: An assessment tool to save time, convey effective feedback, and promote student learning. Styus.
Sümer, N. (2000). Yapısal eşitlik modelleri: Temel kavramlar ve örnek uygulamalar [Structural equation modeling: Basic concepts and applications]. Türk Psikoloji Yazıları, 3(6), 49-74. https://www.researchgate.net/publication/281981476_Yapidotlesssal_esitlik_modelleri_Temel_kavramlar_ve_ornek_uygulamalar
Swanwick, K. (2016). Music education and human development. En R. Colwell y C. Richardson (Eds.), The new handbook of research on music teaching and learning (pp.3-23). Oxford University Press.
Tabachnick, B.G. y Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Pearson Education.
Thompson, S. y Williamon, A. (2003). Evaluating evaluation: Musical performance assessment as a research tool. Music Perception, 21(1), 21-41. https://doi.org/10.1525/mp.2003.21.1.21
Thompson, W.F., Diamond, C.T.P. y Balkwill, L.L. (1998). The adjudication of six performances of a Chopin etude: A study of expert knowledge. Psychology of Music, 26(2), 154-174. https://doi.org/10.1177/0305735698262004
Urbina, S. (2004). Essentials of psychological testing. John Wiley & Sons.
Wapnick, J., Ryan, C., Lacaille, N. y Darrow, A.-A. (2004). Effects of selected variables on musicians’ ratings of high-level piano performances. International Journal of Music Education, 22(1), 7-20. https://doi.org/10.1177/0255761404042371
Wapnick, J., Ryan, C., Campbell, L., Deek, R., Lemire, R. y Darrow, A.A. (2005). Effects of excerpt tempo and duration on musicians' ratings of high-level piano performance. Journal of Research in Music Education, 53, 162-176. https://doi.org/10.1177/002242940505300206
Watkins, J.G. y Farnum, S.E. (1954). The Watkins-Farnum performance scale, form A; A standardized achievement test for all band instruments. Hall Leonard Music Press.
Wesolowski, B.C. (2012). Understanding and developing rubrics for music performance assessment. Music Educators Journal, 98(3), 36-42. https://doi.org/10.1177/0027432111432524
Wesolowski, B.C., Wind, S.A. y Engelhard, G. (2016). Examining rater precision in music performance assessment: An analysis of rating scale structure using the Multifaceted Rasch Partial Credit Model. Music Perception: An Interdisciplinary Journal, 33(5), 662-678. https://doi.org/10.1525/mp.2016.33.5.662
Wesolowski, B.C., Wind, S.A. y Engelhard, G. (2015). Rater fairness in music performance assessment: Evaluating model-data fit and differential rater functioning. Musicae Scientiae, 19(2), 147-170. https://doi.org/10.1177/1029864915589014
Wrigley, W.J. y Emmerson, S.B. (2011). Ecological development and validation of a music performance rating scale for five instrument families. Psychology of Music, 41(1), 97-118. https://doi.org/10.1177/0305735611418552
Zdzinski, S.F. y Barnes, G.V. (2002). Development and validation of a string performance rating scale. Journal of Research in Music Education, 50, 245-255. https://doi.org/10.2307/3345801
Downloads
Additional Files
Published
Issue
Section
License
Authors who have publications with this journal agree to the following terms:
a) Authors will retain their copyright and will grant the journal the right of first publication of their work, which will simultaneously be subject to the Creative Commons Attribution License that allows third parties to share the work as long as its author and first publication in this journal are indicated.
b) Authors may enter into other non-exclusive licensing arrangements for the distribution of the published version of the paper (e.g. depositing it in an institutional telematic archive or publishing it in a monographic volume) provided that initial publication in this journal is indicated.
c) Authors are allowed and encouraged to disseminate their work via the Internet (e.g. in institutional telematic archives or on their website) before and during the submission process, which may lead to interesting exchanges and increase citations of the published work.
d) They agree to act as reviewers, if requested by the journal's editorial team.
Language
Information