Hi Gerard, We are currently building a dashboard to help teachers view information about their students' progress, and finalizing what features/student characteristics to display in real time versus which to provide after a game play episode. We used a principled process using the Evidence Centered Design framework to come up with specific assessment targets for Data and Analysis - what we call FKSAs or Focal Knowledge, Skills and Abilities we would like to elicit evidence of through the game. We are attempting to capture information about as many of those FKSAs as we can from the log data and display them in a teacher-friendly manner using our dashboard.

Jill, 

Thanks for that comment. One place we have found the Standards challenging to work with is "Store data" -- in our music game context, we were able to give students to show their thinking about "collect" "visualize" and "model/make inferences" -- but what is appropriate for Store for middle school kids in the context. Would love to know what you think!

jeremy

I'm very interested in learning more about your project - musician who worked in the music biz for a while before a career change into cybersecurity...And as a practitioner turned researcher, I'm not sure what Standard you are referring to ---and this may be offroading a bit, but what occured me was approaching the "Store data" concept --particularly since this game models the entertainment business-- as a way for the students to think about data lifecycle (creation > use > transmission > storage) issues, and the requirements of storage: do you have sufficient file server space for data storage? Can copies be made and stored elsewhere, e.g. USB drive? how long should stored data be retained? And thinking about data protection requirements as well: who has access - is access recorded/monitored? Are changes to stored data allowed, and is stored data monitored for changes? (data integrity)? Is the stored data of high value (e.g., is it Big Artist's unreleased single you don't want released early / for free!) and should it be encrypted, monitored for access? Is it financial investment data about your record company that not everyone should have access to? These are the types of questions we will be addressing with cybersecurity curricula for middle school students in our project.

Hi Laurin,

Thanks for spending so much time thinking about this -- your ideas are generative and helpful to us. The standards are these CSK12 framework statements about data. In particular for grades 6-8

pplications store data as a representation. Representations occur at multiple levels, from the arrangement of information into organized formats (such as tables in software) to the physical storage of bits. The software tools used to access information translate the low-level representation of bits into a form understandable by people.  Its true that lots of this pertains to music file formats, but it doesn't seem players of our game need to get into that level of thinking to play the game. We didn't find a way to make these sorts of things germane and interesting. 

Computers can represent a variety of data using discrete values at many different levels, such as characters, numbers, and bits. Text is represented using character encoding standards like UNICODE, which represent text as numbers. All numbers and other types of data are encoded and stored as bits on a physical medium. Lossy and lossless data formats are used to store different levels of detail, but whenever digital data is used to represent analog measurements, such as temperature or sound, information is lost. Representations, or file formats, can contain metadata that is not always visible to the average user. There are privacy implications when files contain metadata, such as the location where a photograph was taken

I second this interest in finding high quality interest/attitude metrics. 

One interesting data point from our pilot studies that I think *does* speak to "interest/attitude towards..." (though maybe not CS/data science) is that students spend a lot of time tinkering with the combination of mood, topic, and song name as they record. In other words, though they may choose the "angry" mood for data-specific reasons, they often attempt to find a song topic and a song title that matches this angry mood or the specific artist that's recording. We see this as evidence that players are bringing their real world interest in music into the game and engaging with mechanics in a personal way.

This doesn't point to an interest in CS or data science, but I think it does suggest deep engagement and interest in the game (perhaps more so than having students indicating on a likert scale whether or not they thought the game was fun).

Thanks Rachel! Assessing students ability to make decisions about how to collect data, visualize data, and draw inferences from the data is the primary goal of the game. How students interact with data representations throughout the game, and what kind of predictions they make from these representations are logged. These logs are analyzed live and key information is presented to the teacher on a dashboard. While this dashboard system is still in development, our goal is to provide teachers with actionable and formative information about how players are encountering different kinds of data representations, how they are making decisions about those representations, where they are getting stuck etc.

Thanks for the kind words Rebecca! The game is currently freely available at play.beatsempire.org. While the game is available for you and your students to play, it's worth mentioning that we are currently conducting small scale studies throughout NYC schools to evaluate and refine the game. The teacher dashboard is in development but not yet useable.

If you do decide you'd like to try the game out with some students, please let us know so we can be in contact with you to get feedback!

Thank you Feng Liu! This is super helpful, as I am not expert in measuring attitudes.

jeremy

Hi Joi,

Thank-you for your kind words and your interest in our project!

Yes, we have been thinking of students with limited funds of knowledge and interest in traditional CS. While Beats Empire has been designed as a formative assessment game and not as a learning game per say, we have developed a video where we have interviewed real life professionals and experts working in the music industry who have described similarities of the game with what they do in their day-to-day professional lives. Our hope is that the video will help students recognize various non-traditional career opportunities involving the application of computer science and data analytics.

Students do not get to run analytics on data in the current version of the game. They can choose to view data visualizations of most popular types of songs in a  particular borough, or the trend for a particular genre or mood of music, and then make inferences from these visualizations to decide what song to record, what artist to hire, where to release a song, etc.

As for the question about where we're piloting in NYC, I think our NY-based collaborators can answer that better than I can. I'll pass on your question to the rest of the team.

Thanks!

Joi,

One of our goals during the design phase was absolutely to create a game that would tap into the interests and experiences of NYC Middle School students. We conducted multiple focus groups with students throughout NYC to talk to them about their interest in music, how they listen to it, where they find it, how they talk about it with their friends, etc. These conversations were incredibly insightful and directly informed the design of game mechanics and aesthetic.

One way we try to tap into players' funds of knowledge is by using artist names that sound a lot like the names of real artists (for example, "Beyonde" and "Half Dollar") and music genres with which students will likely be familiar. We've found that some students spend a lot of time tinkering with the song topic, mood, and title to record songs that fit their experiences with music outside of the game. While this activity isn't always about optimizing decisions based on data, it is a meaningful choice for players and we wanted to be sure Beats Empire supported this play!

Hi Nancy,

While we are still developing our teacher dashboard and finalizing what information we will provide to teachers via the dashboard, here are a few things that are likely candidates:

- Whether a student attempts to access/view data before making a decision about what kind of song to record or which artist to hire

- Whether a student makes correct inferences from provided data visualizations

- Which kind or kinds of data visualizations is a student relying on in order to make decisions? (bar graph, line graph, a combination)

- Is a student balancing the frequency of data collection with the storage requirements?

We have also been developing some follow-up activities for the game, and teachers will get additional formative information through those activities. For example, can students identify a trend? Can students identify what data needs to be collected and in what representation to answer a question or generate a given visualization? Do students realize when it is better to process data manually versus program the computer to process data? How much data do students think is sufficient to collect for making a prediction about a broad population?

Thank-you!

Thanks, James.

It is meant to be general -- in our pilot context in NYC, lots of different materials are in use.

I don't think we have a game ending, but others on the team may know more.

Our overall plan for addressing assessment subtleties (like the one you mention in your #3) is to give teachers guidance on how they should follow up to learn more about how their students' think with data. The game can serve as a context to elicit student thinking both in game play and in follow-up, unplugged discussions.

best, jeremy

Thanks Dennis! Not exactly, but one of the interesting observations of students playing the game has been how interested many of them are in aligning their game play with their real world experience with music. We've found that some students spend a lot of time tinkering with the song topic, mood, and title to record songs that fit their expectations about what kind of music their artists *should* record--such as having the artist "Beyonde" record some awesome angry love songs. It's been very important for us that the game support this kind of play! The trick now is how to capture/account for this meaningful, though maybe not data-based, play on the assessment side.