NSF Awards: 1029756
2020 (see original presentation & discussion)
Grades K-6, Grades 6-8, Grades 9-12, Undergraduate, Graduate
This video describes a survey instrument that can be used to assess and evaluate students’ motivation-related perceptions of STEM activities and classes. The instrument is available for free at www.theMUSICmodel.com. This project provided validation evidence for the scores produced by the middle/high school version of the instrument. In three studies (Chittum & Jones, 2017; Chittum, Jones, Akalin, & Schram, 2017; Jones, Sahbaz, Schram, & Chittum, 2017), we documented that the instrument provided valid scores for use with middle school students in science class. We also demonstrated how the instrument could be used to assess students’ motivation-related perceptions of STEM activities (e.g., Chittum et al., 2017; Jones et al., 2015). In this video, we describe the instrument and provide examples of how it can be used by researchers and teachers to assess students’ motivation in STEM activities and classes.
Chittum, J. R., & Jones, B. D. (2017). Identifying pre-high school students’ science class motivation profiles to increase their science identification and persistence. Journal of Educational Psychology, 109(8), 1163-1187. doi:10.1037/edu0000176
Chittum, J. R., Jones, B. D., Akalin, S., & Schram, A. B. (2017). The effects of an afterschool STEM program on students’ motivation and engagement. International Journal of STEM Education, 4(11), 1-16. doi:10.1186/s40594-017-0065-4
Jones, B. D., Chittum, J. R., Akalin, S., Schram, A. B., Fink, J., Schnittka, C.,…Brandt, C. (2015). Elements of design-based science activities that affect students’ motivation. School Science and Mathematics, 115(8), 404-415. doi:10.1111/ssm.12143
Jones, B. D., Sahbaz, S., Schram, A. B., & Chittum, J. R. (2017). Using psychological constructs from the MUSIC Model of Motivation to predict students’ science identification and career goals: Results from the U.S. and Iceland. International Journal of Science Education, 39(8), 1089-1108. doi:10.1080/09500693.2017.1319093
Alan Peterfreund
Executive Director
Brett: What has been the experience of using this as a pre- and post survey? Has it been use enough in this mode to gain an appreciation for what size variations one might find as significant?
Brett Jones
Professor
Thanks for the great question! When comparing students over time or comparing groups (or classes) in quasi-experimental studies, we've been able to detect differences. I'm going to make a big generalization here (because it depends on sample size and variance, obviously), but differences of about 0.3 or 0.4 on the 6-point scale have often been statistically significant (p < .05).
Kristin Flaming
Brett Jones
Professor
The MUSIC Inventory measures students' perceptions of a class or activity (perceptions of eMpowerment, Usefulness, Success, Interest, and Caring). These five perceptions provide a multidimensional approach to motivation and are consistent with current motivation theories. Higher perceptions of these MUSIC components are indicators that students were, or are, motivated to engage in the class or activity. Here are some questions we've been considering, what do you think?
Jeffrey Ram
Thanks for doing the necessary formal validation work on your MUSIC inventory. Some of my colleagues complain that the pre- v post comparison survey that we do has "too many questions" for students. My response is the necessity for internal validation, such as you've analysed for MUSIC. I intend to look at your instrument to compare to one that we've been using derived from another source. BTW, we administer our survey tool to students electronically on tablets using a Qualtrics based system. Instead of seeing a long sheet of questions for which there might be a temptation to "just mark them all the same," students see only one question at a time. We think it keeps them from being intimidated by a large number of questions, and, really, it doesn't take them that long to go through the whole battery of questions. The Qualtrics tablet survey can run independently of the internet (sometimes necessary when the school system has certain sites blocked). When we get the tablets back "out" to where we have internet, the results are uploaded to Qualtrics and we also save the labor (and prevent errors) involved in transcribing paper surveys. Our project uses this tool to evaluate impact of a field trip visit to a public aquarium. Our showcase video focuses on the teacher PD and field trip and doesn't mention the research tools we use, but I think it is very similar to MUSIC.
Brett Jones
Professor
Thanks for the message, I like the way you collect data on the tablets, we haven't tried that, but we should. For university students we use the MUSIC Inventory online through Qualtrics. We've been using paper surveys with the middle school kids though. I'll check out your video...
Henry Yang
Thank you Dr. Jones for the succinct introduction to the MUSIC model. I'm eager to apply the inventory to pinpoint the problems in my students' motivation and adopt relevant strategies to help them stay motivated.
Patrick Honner
Teacher
Thanks for the presentation, and for making your materials freely available. As a K-12 classroom teacher, I'm wondering what this would look like for me: Say I give the survey, compile the results, and then perhaps notice that some scores are lower than others. After that, are there set next steps I take? Or is this more of a tool to create dialogue in schools and departments?
Brett Jones
Professor
Thanks for asking Patrick. The MUSIC Inventory can be used for any of those things you've noted. As a teacher, you would see which MUSIC components were lower and think about strategies you could use to improve them. I've provided a few strategies in this article and this book, but if you email me (brettdjones@gmail.com), I can send you some other things. We've found that it's also really beneficial to ask at least one open-ended question for each MUSIC component, such as "What could be changed to make this class more interesting." (Or, make you feel more successful, or make it more useful, etc.) In combination, the inventory and open-ended items will give you good feedback. We've found the MUSIC model helps teachers think more comprehensively about motivation. Instead of just thinking about what "motivates" students generically, it gets them to realize that empowerment/autonomy, usefulness, success perceptions, interest, and caring are all important (of course most teachers know these things, but the model reminds them and sort of forces them to think intentionally about these research-based components).
patrick honner
Bradley Allf
I had not heard of the MUSIC construct before but I found this very interesting. One factor I don't see in the construct is something related to norms. I wonder whether students' feeling about how their peers experience a classroom positively or negatively might impact how they themselves think about their experience. But I would love to hear your team's thoughts on the use of norms in this context!
Brett Jones
Professor
Good point, that's not something we've studied much explicitly, but in the MUSIC model, students are affected by internal factors (cognition, affect, needs/desires, identity beliefs, personality characteristics, etc.) and external factors (teachers, students, family, peers, culture, society, etc.). These factors somehow intermingle and affect students' perceptions at any particular moment in time. We conducted one quasi-experimental study (McGinley & Jones, 2014) in which students in the experimental group were asked to work in teams of 3 or 4 students and discuss their perceptions and feelings about the class (including why they thought the course was useful and interesting). In our post-survey, compared to students in the control classes, students in the experimental classes found the class more interesting (p = .02) and rated usefulness higher (although not statistically, p = .09). This study would lend support to your idea, at least when students are specifically asked to share their perceptions and feelings about a course with one another.
David Bergin
Brett: you need to go to Iceland to do further validity work! Nice video and research. I like your comment about including open ended questions to seek further information from respondents.
Brett Jones
Professor
Haha, thanks David, you always have good ideas! I missed seeing you at AERA this year.
Elizabeth Edmondson, Ph.D.
Thank you. This is fascinating. I believe it has many possibilities.
Nathan Auck
STEM Coordinator
Thanks for sharing your work!
In looking at the measures on the scale, I wonder whether there are opportunities for an equity lens to ensure that ideas such as cultural relevancy are captured and whether there rubric indicators on would capture cultural differences in demonstrating engagement. Do you have any insight in this regard?
Brett Jones
Professor
We have validated the inventory in many different countries and languages, but we have not attempted to modify it to capture cultural differences. I'm not quite sure how to do that, but it would definitely be interesting to try. We have thought about these ideas a little bit, but have been too busy with current projects to think about it in earnest. Maybe someone else out there can try it!
Kristin Flaming
Thank you for sharing. I can see us adapting this to use as a post-survey for our faculty workshops on https://passiondrivenstatistics.com and with our students after boot camp or full semester courses.
Brett Jones
Professor
Yes, that's a good idea, we've used it for things like that as well. I've worked with many faculty at different universities over the past few years as they've interpreted the results from their courses. I hope it will be helpful!
Parvaneh Mohammadian
Thank you for sharing your project! Is it possible to get a copy of the survey?
Jessica Chittum
Senior Scholar
Thank you for your interest in the MUSIC Inventory! You can find a user guide including the full survey and instructions at https://www.themusicmodel.com/. Navigate to the Questionnaires tab to download the PDF.
Parvaneh Mohammadian
Excellent! As you know, many studies have reported lack of student participation in many project activities. Are you familiar with any questionnaires to assess the lack of student participation? The MUSIC inventory assesses the student motivation while in the class. Thank you!
Jessica Chittum
Senior Scholar
We've done some research investigating students' self-reported engagement (here's one example including a measure of engagement along with the MUSIC model: Chittum, J. R., & Jones, B. D. (2017). Identifying pre-high school students’ science class motivation profiles to increase their science identification and persistence. Journal of Educational Psychology, 109(8), 1163-1187. doi:10.1037/edu0000176). Feel free to reach out to brettdjones@gmail.com and he can send you some engagement scales.
Parvaneh Mohammadian
Excellent! Will do.
Thanks,
Zoe Higheagle Strong
This was a helpful video, and I appreciate the relational aspect of the motivation measure. I would also be interested in the engagement scales, so thanks for providing the email address. Has any of these measures been used with Indigenous or Native American students? Thank you, Zoe
Brett Jones
Professor
Thanks for your comment. To my knowledge, the MUSIC Inventory has not been used specifically with Indigenous or Native American students. Let me know if you want to try it and have access to these students! :) brettdjones@gmail.com
Hiller Spires
Great video! I actually use the MUSIC Inventory in my current project to assess students' motivation before and after engaging in project-based inquiry. I have found the survey to be particularly useful in having students consider specific aspects of their motivation to learn, and I recommend its usage to anyone looking to more deeply understand motivation. It is very straightforward to use, and the user guide makes it very simple to interpret the results. Thank you for sharing the video; I hope others find this tool as helpful as I have!
Brett Jones
Professor
Thanks for the message and I'm glad that you've found the MUSIC Inventory useful!
Suzanne Ruder
Thanks for the video and information. I like Hiller's suggestion about assessing motivation before and after active learning projects. I had not heard of MUSIC before and will definitely check out the survey and other information.
Further posting is closed as the event has ended.