Hi Andee,

What fun to see this great application of CODAP. Who suggests the investigations for Data Clubs? How much preparation do leaders need to succeed?

Take care,

Marcia

Hi Marcia,

We did some focus groups with kids to get the themes for Data Clubs, but of course we also have to make sure there are good public data sets available!  For example, we thought we could get data on what happens to animals in shelters, but there's not a good national data set on that. So we have to find the sweet spot where kids are interested and data sets are available!

In terms of preparing leaders, we are doing a DBR process where we teach the modules ourselves.  But we're also writing up  guides for leaders. As we move forward with this work, we want it to work with a variety of leaders, both in school and out of school.

So needed!  And really helpful in understanding and explaining to others -- their parents, for example -- how the COVID-19 data are real (even with the many errors in testing and reporting) and the models are meaningful.  "Data club":  Does the name of the club put any of your students off?  Sounds so nerdy!

Hi Jeffrey. Thanks for your kind comments. We so agree that data science is super important in today's world!  I laughed at your comment about "Data Clubs" as as a name for an out-of-school program for middle schoolers. In a summer camp context where kids are already signing up for classes like robotics we have an easy time recruiting. However, we did have to change our name to "Data Divas" to compete with "Beach Buddies...." offered at the same time in the context of another after-school program. We also recognize that it is important to lead with the fun topics they will be exploring, not the data science piece itself, when describing what the "club" is all about.  However, after doing Data Clubs, we follow up with kids and ask what activities they enjoyed the most. The majority of kids talk about being able to freely explore the data in CODAP and they are delighted to be able to create multiple graphs as they explore their own questions. In fact, they often start with one question, but then as they visualize the data, related question emerge or they refine their questions further, which leads them to different data moves to create new graphs, which leads to additional questions...Some have referred to how different this feels from school where the goal of a task would typically be to create a specific graph, it was a whole new discovery to realize that graphs are tools for making sense of data and that sometimes you may make multiple graphs to explore a question, or that as you explore, your questions can shift, and this can lead to ideas for different ways to slice, organize, and visualize the data, which give you different insights that you wouldn't have if you stopped at the first graph.

This is very cool work.  I think a lot of undergrads would benefit from being in the club!  Was it hard for the kids to learn to use CODAP?

Thanks for the comment Barbara. Great question. I think CODAP is a very accessible and intuitive tool for students. We also structured the CODAP activities to help them learn how to use it effectively. Before teaching, we uploaded data into tables in CODAP, so students could visit a link and have all the data they needed there, ready for them to explore (for an example of what this looked like in the Ticks and Lyme module, see http://bit.ly/lyme17). We then had students pair up and work through "CODAP challenge cards" we created to help them learn the basic features of CODAP. Then we moved on to having them explore the data in more depth and make graphs to answer questions they had about the data. I think having them work in pairs was really helpful because if they get stuck with something they can more easily work it out together. Facilitators can of course also helped point students in the right direction with CODAP with suggestions and hints about how to do what they wanted to do. Another thing that was really useful was having all the students and facilitators go around the room at the end of each activity in CODAP, look at the work others had done, and ask questions about it. This gave us all new ideas about the types of things you could do to explore the data in CODAP.

Just to add to Jacob's response - we would love to figure out how to influence the undergraduate curriculum, as our sense is that statistics is often taught as a series of recipes, rather than with the goal of making meaning of and with data.

This sounds wonderful!  Thanks for the explanation!

You could study their process....

Yes, many possible research projects to follow this up!  Thanks for your input, Barbara.

I was also wanted to add that we keep the data contextualized.  Students spend time engaging with the topic, looking at how attributes were measured, and generating their own questions about the data. They also do a related mini project that engages them in generating attributes to measure, gathering data, creatively representing the data, and examining the patterns they see with different representations. They create visualizations both in CODAP with given datasets and using paper and pencil with their own small dataset. The visualizations become a tool for making sense of the data, rather than a final output. We try to demystify the whole process and put sense making and reasoning front and center. 

Thanks for your comment Dave. Jill Denner, below, also asked about how we see our work connecting with the K-12 curriculum. I encourage you to take a look at Andee's response. Our work has focused on exploring what is possible in out-of-school environments. How do middle schoolers interact with authentic somewhat "messy" datasets?  How do they use CODAP tools to explore the data?  What sorts of questions do they ask of the data and what do they notice when they explore data using visualization? Most importantly, what gets kids of this age excited about data science and wanting to dig deeper?  We have thought about lots of next steps--from scaling the modules, to creating PD materials for teachers, to developing units for in-school use.  What forms of scaling are you thinking about and what are some directions you'd like to see our work take?

Thanks for the comment Ed. We have held several Data Clubs in the Boston area, including in Lynn and Malden. Your work at Dearborn STEM Academy sounds great. In Data Clubs we are also working on getting underrepresented youth into STEM, including girls, urban youth, and youth in rural settings in Maine.

To follow up on Jacob's response, Ed -- we should talk more, as TERC is not far from you.  I recently had a conversation with one of your co-principals, Dana Brown, about Dearborn.  Our conversation focused on high school, but it sounds like talking with your faculty about middle school might be really generative.  Let's keep in touch!

Thanks. Yes, we should definitely keep in touch. Pre-pandemic, I passed by the TERC office all the time since my best friends and godson live in North Cambridge off of Rindge Ave. Eli Tucker-Raymond was also helpful is giving us some very helpful feedback on our ITEST proposal a couple years back. I believe he's now at BU. And glad you connected with Dana. 

Great question Elizabeth.  Early on we ran some focus groups with students to learn more about their interests. In some of these sessions we gave them choices between topics and had them select the one that sounded more interesting. Not surprisingly, they were interested in topics that connected with their lives--like the use of technology and social media.  We also learned that as long as students are given opportunities to personally connect and engage with a topic, they are open and curious about a lot of things. What then becomes important is finding authentic but accessible datasets with interesting attributes that give students choice--so that different students can look at the same dataset but decide to explore different relationships. They feel like data detectives and love sharing their discoveries.

I love this question! I think it is important to help the students connect with the data personally. They need opportunities to interact with the attributes so that the data they see in the table aren't just a bunch of numbers but have meaning for them. I also think it is very easy to start them in on questions of their own. We may seed a question by asking, "I wonder if there are any differences between boys and girls on any of these attributes."  Students may start with one question, but then as they explore, new questions emerge. They get excited when they realize they can ask more than one question and that they can choose different graphs to look at the data in different ways. Having multiple attributes allows them to move beyond the end point of simply creating a graph, now that graph becomes part of their reasoning, and spurs them to create another graph, and that opens up some more wonderings, and so they try slicing the data a different way, and so on. They get to experience the excitement of teasing out what the data can tell them. When given a very small number of attributes to analyze, students miss out on the exploration that is what makes data science exciting.  

Let me just add to what Traci said:  In interviews, kids who are part of Data Clubs talk about the power they have to examine patterns and look at relationships that involve many attributes. They like being able to make choices, then ask the next question based on what they find. We generally have 5-7 attributes that they're looking at, and this seems to be in a ballpark that's developmentally on target.

Traci and Jan,

Thank you both for these insightful responses. I think I'll print them out and post them where I can easily reread them from time to time to reaffirm my belief that kids this age are interested in more things about data than the mean and median! (Actually, they're probably rarely interested in the mean and median.)

Have you yet written anything about this? If so, can you point me to it? If not, I hope you will.

I concur. I particularly like Traci's discussion of questioning.  Asking interesting and useful questions about the data is such an important skill. Or is it a habit and disposition? Maybe both. I agree that it's essential to engage student on issues they connect with, whether its social media use or the spread of the coronavirus. And there are potential cross disciplinary opportunities spanning math, IT, social studies, and civics. 

Here is a link to an article we wrote about our first implementation of the Teens and Technology module:

Hands On article on Data Clubs

The question of integrating into schools is an interesting and complicated one.  We decided to start our work outside of the school context so that we could explore topics, datasets, tools and approaches without the constraints of subject matter boundaries.  Our first module (teens and time) would fit best into social studies, while the second one (Lyme and other tick-borne diseases) would fit best in science or even health.  The current curriculum is crowded and one of the challenges is finding a place for working with data.  Is it a separate course? Is it integrated into subject areas? Is it part of mathematics?  Or computer science (which is itself trying to fit into a crowded curriculum)?  Many of these issues are out of our control, so for the immediate future, I don't think there's a one-size-fits-all solution.

The PD we're planning for this summer will mostly be for science teachers, especially those whose students might want to do data-based projects for science fairs.  I think uptake of data science techniques is likely to be easiest for science teachers, who are used to fostering quantitative inquiry.  But we are still exploring just where the kind of work we're doing would be most easily integrated into school structures.

Thanks for the comment. I see the benefits of beginning in an after-school contexts then moving into integrating this to the curriculum. Like many other statistics and data science educators, I've also grappled with the question of "where does data science belong?" As an interdisciplinary field, I could see an argument for presenting it as its own course or incorporating it into other classes at the K12 level (e.g., statistics, computer science, math, science, etc.).  

I am curious to know more about the different problem solving and investigatory practices that emerged in the Data Clubs. For example, you noted that students developed a curiosity for exploring the data before, tapping into exploratory data analysis and connecting. Students also played around with visualizations to help communicate ideas and connected the data and inferences back to the real-life application. Were there other practices that were developed in the Data Clubs? How do you think that this would compare to a traditional or AP introductory statistics course? 

Thanks for your comment, Kevin - I'm glad there are a lot of people grappling with the issue of situating data science in the curriculum.  Maybe with some concerted and collaborative effort, we will be able to make some progress!

One of the practices that we work on in Data Clubs is question posing and, in particular, knowing what questions can be investigated with a particular data set.  We (and others) have found that generating questions about patterns in data (as opposed to questions like: what is the maximum value for a particular variable?) is not a skill that students or teachers have had much practice with.  So we have to model asking questions about the shapes of distributions (where are most of the values? where are there gaps in the distribution?),  group comparisons (how do boys and girls compare on this variable?), and trends over time (is this quantity generally increasing, decreasing or neither?).  We believe this is a core skill in data science - and inquiry in general.  More attention to question posing would likely improve most traditional statistics classes too!

Thank you for the question Susan. Before diving into tables of data, the students get to know the attributes by engaging with them personally.  In a module focused on data on how teens spend their time, they complete a Time Use Diary of their own, getting a taste of what it would have been like to be part of the study.  In another module students survey each other about their experiences with ticks and play a tick life cycle to learn about the conditions that ticks like best. It is important that the data is not just numbers, but that those numbers have context and meaning. They spend time on and off the computer and work with traditional as well as non traditional graphs.  They develop informal language to talk about the features that they are noticing. Our datasets are rich with many interesting attributes. They provide fertile ground for many wonderings. So I think much of our strategy is to focus on the meaning making and keep the data contextualized. We want students to be interacting with the data and in some way seeing themselves in it. We also want them asking questions of the data. And we ask them to articulate what they are noticing. I think when they are engaged in that way, they are able to help each other make sense of the data. 

You also ask about supporting teachers.  Ours is a design project focused on exploring what is possible in various out of school settings. We work with facilitators that are specifically interested in the materials. However, we think this work is of interest to a lot of teachers and are in the process of planning a professional development workshop based on our work with students. Jan Mokros talks about some of this in more detail above in a reply to an earlier question. I hope you have a chance to check out her reply above:) 

Thanks for the comment and question, DeLene. From my experience helping implement Data Clubs modules on two different topics and going through recordings of students' post-module interviews, it seems many middle school youths' previous experience learning about data has been fairly prescriptive. They have learned to make graphs, but they are likely told exactly what to put on the x and y axes, etc.--so they are following simple instructions rather than exploring the data and trying to answer their own questions about it. And students rarely are looking at meaningful real-world data. In Data Clubs, we use carefully curated real world data sets on topics that interest students, like how teens use technology. We support students in learning how to explore the data, how to ask good questions about it, and how to try to answer those questions, both on paper and using an accessible digital tool (CODAP). So what I think is most needed in data education, and what we try to do in Data Clubs, is to give students time to explore meaningful real-world data, talk about it, ask questions of the data, and try out different types of data representations using tools that are intuitive and easy to learn (such as CODAP).

This is such a great question and response. Completely agree that much of the data experiences to date have been not only prescriptive, but the data itself has been scrubbed and cleaned and buffed to be easy to interpret. As part of the behind the scenes team in this work I would only offer that maybe because the tools are so intuitive and easy to learn we spend time making sure kids understand what a particular action means once it's applied and they see the result. As Jacob says, this is really important with these authentic, messy data sets.

I agree... great response.  The last sentence especially "says it all"  Thanks Jacob.  As a teacher, I will strive to give students time to explore meaningful real-world data, talk about it, ask questions of the data, and try out different types of data representations using tools that are intuitive and easy to learn (such as CODAP).  

Thanks for the comment Kathryn. Getting a diverse range of kids in with data science at an earlier age is definitely something we think is important and is a reason why we work with middle school youth and aim to get girls, rural youth, and urban youth participating in Data Clubs.

I just watched your video on StatPREP. What a great initiative to foster PD for educators in data science! We will have to look more at your resources as we are planning future Data Clubs modules. One thing we are planning now is a virtual PD workshop series for high school science teachers to get them comfortable doing the types of things we have done in our Data Clubs modules, helping their students do projects with real-world data sets. We are also thinking about COVID-19 data (who isn't right now?) and are planning to have one session of our PD workshop focus on this topic. 

Fantastic!  I love the idea of a high school teachers online training.  The COVID data class would be particularly exciting to be part of... I would think teachers would want to connect to this for students.  They need support in processing all this and with careful instruction, this personal topic could really create an understanding through data. 

Oh, I think maybe we have different takes on this!  My sense is that while they quickly learn to ask good questions, make graphs, and interpret findings, students may not take enough time to examine their findings before they ask the next question. Another thing that's hard is doing their own representations (on paper) of multi-attribute data in the way that Lupi and Posavec do in "Dear Data."  

Jeffrey, you're right that kids can be misled by looking at a small amount of data and trying to predict a trend.  So can statisticians, of course!  We don't do anything with statistical analysis of significance, though we allude to it so that students know it would be the next step.  Partly, this is because data clubs are short (8-10 hours, typically) and they always take place in out-of-school environments.  There's a lot to do in this short time, and we've prioritized looking at the data distributions, asking questions, and making representations to compare groups or determine patterns.

Thanks for your comment, Susan.  Data Clubs is definitely for ALL students - we are particularly interested in students who are underrepresented in STEM fields - in our case, that means urban students (mostly students of color), girls and rural students.  Our project is partly in Massachusetts and partly in Maine, so that has given us the opportunity to work with a wide range of students and to find topics that are interesting to students in a variety of settings.  One of our field test sites was an after-school program that serves girls in a low-income community; that experience taught us a lot about how to design materials for participants who have not had much experience with data before.  I like the old adage: low floor, high ceiling - accessible to all with lots of room for students to explore.

Excellent! Thank you for the explanation.

Thanks for your comment Chris. They do indeed get caught up exploring their questions with visualizations and making sense of the patterns they find!  We typically have 8-10 hours with the kids in an out-of-school environment. We stay focused on interacting with and making sense of data in a way that is meaningful and engaging to middle schoolers. We want kids to develop their sense making, reasoning, and questioning. Much of this is done by keeping the data contextualized, using visualization as a tool of analysis, and having students describe in their own words what they are noticing as they look at the data. The end goal is for students to be comfortable interacting with data, sorting and slicing it in different ways, being able to look at distributions and notice meaningful features such as differences in spread, clumping, where there are holes, whether it skews to one side, and then wonder why, and then be motivated to create more visual representations to further investigate "what is going on". Or they may look at a scattergram with a lot of variance in the middle but with interesting patterns as you move toward the edges--students might wonder about this and examine those more extreme cases in more detail--in the case of state level rates of Lyme disease student found that at a certain average temperatures, the rate of disease dropped off dramatically. So we are not trying to teach formal mathematical techniques, but rather introduce them only when the interactions with the data motivate them (in our teens and time module, students are drawn to questions of what is "typical" when comparing gender differences in time spent on certain activities and in this context they explore "median" as one way of characterizing a group--but this is seen as one of many ways to describe what they are noticing). So the reasoning behind meaningful mathematics is most definitely being developed, and I think they will draw on these experiences when they are introduced to more formal mathematical techniques and terminology in school, but you are right, students have told us that Data Clubs feels different than any work they have done with data or graphs in school. More over, they love working with CODAP and one of the things they report learning is that you can look at the same data in different ways with different graphs. They felt that being able to ask their own questions and create their own graphs helped them understand the data in a way they wouldn't have been able to had they just all done the same graph.

Thanks for your comment, Leanne.  On the topic of data representations, we love using a tool like CODAP that makes it easy for students to create data representations that help them answer their own questions.  But even a great tool like CODAP can hide some of the process of mapping attribute values to representational forms -- and limit the creative palette that students might want to explore.  The Dear Data book has been inspiring to all of us on the team and we find that it's inspiring to Data Clubs participants as well.  If you haven't seen it, we highly recommend taking a look!

Thanks for the positive comment Leanne. I do think that giving students the opportunity to create their own data visualizations, both in CODAP and by hand, is really important. That experience engages them in thinking about the case attribute structure of the data and in making sense of the representational elements in data visualizations. When they work on creating visualizations by hand they are given a topic and as a group generate and define their own attributes, gather their own data, and individually create their own data visualizations with the goal of exploring the relationship between attributes in their data.  When creating data visualizations in CODAP they are given a dataset that they have gotten to know well and first create graphs to answer our questions but are then given lots of time to explore their own questions. We find that they tend to start with one question, create a graph, but then what they see in that graph may spur more questions, which may lead them to a different way to slice the data, and a new set of graphs to explore "what is going on". Being able to flexibility interact with the data, perform different data moves, create their own graphs, and then ask more questions and create even more graphs, gives them that opportunity to think like a data scientist and begin to see data visualizations as sense making tools and not just end products of an assignment. The feedback we have gotten from the kids is that they love being able to pose their own questions and explore them with graphs in CODAP.  In fact, I think having the CODAP platform makes a big difference because it gives kids a chance to interact with authentic real life data and quickly and easily create graphs that are really fun and interesting to make sense of. 

Thanks for your comment, Frank.  I know you're thinking of the Statistics for Action project, which worked with community environmental action groups to help them use data to further their arguments for environment justice.  I don't think we've gotten quite that far with our Data Clubs participants yet!  However, the Tick and Lyme module certainly helped participants focus on the impacts of geography and local conditions on the incidence of Lyme disease.  Similarly, participants in the teens and time module were quick to note that the data about social media use that we shared with them in the original implementation (collected in 2014) didn't reflect their experience and immediately tried to figure out why that might be the case.  So, I see Data Clubs participants developing a sensitivity to the context of the data and how that might be an important determinant of the patterns they are seeing.  I don't know that they yet have a deep understanding of the importance of social and cultural experiences on data - but we've taken first steps in that direction.

Thanks for your vote of confidence, Mitchell.  We're hoping to integrate some of our techniques and what we've learned into disciplinary courses such as science and health with other curriculum developers who take kids' interests seriously.

Hi Bill - great to hear from you!  The analog (i.e hand-drawn) representations were quite a different process, as we wanted students to create non-traditional representations (i.e. not just two-axis graphs or pie charts).  So they first had to wrap their heads around what was possible other than the representations they were used to - that's where the examples in Dear Data came in, as models.  Getting to the idea that a single "symbol" could have multiple characteristics that reflect multiple attributes was a big leap and we found students puzzling over that process.  Combined with that, students enjoyed using different materials - nice pens and markers - that convey different esthetic values than the digitally produced graphs.  I'm sure there are things we haven't thought about that a conversation with you would help us focus on!

Hi Chris - No surprise, since Jan and I worked on that unit!  I'm often sad that those early Investigations units are no longer in use...

Thanks, Raphael!  We certainly hope the skills participants gain in Data Clubs will prepare them for more encounters with data.

Hi Sue - thanks for your enthusiasm!  Yes, I know the Data Jam people - and have been working with them to beef up the data part of their work, as many of the participating students (and teachers) don't have much experience with data before this experience.  In both that project and Data Clubs, we have found that the process of generating interesting questions is a challenge for both students and teachers and that understanding which questions a dataset can help you answer and which it can't is a non-trivial pedagogical goal.