Hi Barbara, we do have a few middle school teachers that started the boot camp model in STEM/STEAM and other similar elective courses. I worked with them to tie in the the Arts & Humanities through the visual displays of their posters. We also have a new partner in NY at a technology school that has a program for high school students working toward a nursing degree that take dual enrollment courses through the technology school. When the students are done at the technology school they will complete their CNA work and move on to a nursing program.

Thank you Rebecca! Feel free to email me, kristin.flaming@gmail.com, to gain access to our free faculty resources that are password protected. Some of our early publications examine learning experiences and outcomes in URM vs Non-URM. While URM students considered the material more difficult than non-URM students, URM students demonstrated similar levels of increased confidence in applied skills and interest in follow up courses as non-URM students. URM students were found to be twice as likely as nonURM students to report that their interest in conducting research increased. 

Dierker, L., Alexander, J., Cooper, J., Selya, A., Rose, J. Dasgupta, N. (2016) Engaging diverse students in statistical inquiry: A comparison of learning experiences and outcomes of under-represented and non-underrepresented students enrolled in a multidisciplinary project-based statistics course, International Journal for the Scholarship of Teaching and Learning, 10(1), 1-9.

Hi Lelli. Liberal arts colleges, large state universities, regional colleges/universities, medical schools, community colleges, and high schools have all successfully implemented the model. We have also published work showing that the project-based model attracts more underrepresented students than a traditional math statistics course...

Dierker, L. ., Cooper, J. ., Selya, A. ., Alexander, J. ., & Rose, J. . (2015). Evaluating access: Comparing enrollment patterns in traditional versus multidisciplinary, project-based introductory statistics courses. Journal of Interdisciplinary Studies in Education, 4(1), 22–37. Retrieved from https://www.ojed.org/index.php/jise/article/view/1599

and that the project-based curriculum attracts higher rates of women and underrepresented students compared to an introductory programming course. 

Cooper, J., & Dierker, L. (2017). Increasing exposure to programming: A comparison of demographic characteristics of students enrolled in introductory computer science programming courses vs. a multidisciplinary data analysis course. International Research in Higher Education, 2(1), 92-100. DOI: 10.5430/irhe.v2n1p92

Great question about the barriers. The single greatest barrier is that instructors who teach statistics and research methods courses often do not have the skills needed to support students in doing authentic research. This is particularly true in departments of mathematics where large numbers of students encounter statistics. Though we support many different statistical software platforms, it is important that students learn a platform that helps them answer their research question and apply their new knowledge. We run faculty workshops helping to support this effort. 

Hi there Lisa. Your comment here really stood out to me: "The single greatest barrier is that instructors who teach statistics and research methods courses often do not have the skills needed to support students in doing authentic research". That has been my experience as well, and it's why I decided to drop the traditional stats courses in my graduate program (Environmental Studies Masters program at Virginia Commonwealth University) and replace them with in-house versions. Our students tend to be overwhelmed and disappointed by the mechanical presentation they typically get from a Statistics professor, where the class may include students from 20 different departments. Now, all of our grad students take an initial "Environmental Data Literacy" class that does dual duty as introductory stats and R programming. We make things very conceptual and do our best to make the R content feel more like a puzzle than programming per se. I'm quite pleased at the initial success with this approach. But there remains at least one fundament component that seems to be a stumbling block for many of our students: stats will never make sense, no matter how well its taught, if a student does not have a somewhat intuitive grasp of probability. For practical purposes, just about any statistical procedure can be explained with simple drawings of probability distributions. But if those distributions themselves seem scary and students tune out, things become much more difficult. I wonder if you've struggled with this (granted, your project is dealing with much younger students than my grad students) and if so, what sorts of solutions you've come up with?

Daniel, Thanks so much for your post. I love what you describe. Without question statistics needs to be integrated with the statistical platform. We need to get real tools like R into students hands much earlier than we are. Focusing on the conceptual and on the questions students care about within their discipline or their lives is also critical to creating empowerment. Great that you are embedding it within the discipline. As for statistics making sense, that is such a great topic of discussion. From my own experience of taking many statistics classes as a grad student and post doc, it is the general explaining alone that was the barrier to my understanding. When I look at what I know and ask myself how I got here, the answer is not that instructor X explained it to me. All of the instructors drown me in details that helped me see the trees and miss the forest. Understanding came from watching data behave over and over and over again. I knew the definition of multicollinearity, but I did not understand it until my multivariate models were not converging and I had to start descriptively exploring what was going on with the variables that were providing strange estimates. Then I could see in the data that I had two variables practically measuring the same thing. Statistical power is another good example. I could define it for you, but until I started getting small p values within large data sets and then seeing that the differences between groups were not all that interesting, the definition did not really make sense to me. It is in those moments that I remind students about probability and how large/small numbers affect how large the differences need to be for something to be considered statistically significant. I did not even get the relevance of the coin flip concept in the work I was doing until I was a post doc! Crazy I know but I think it demonstrates that understanding comes in application of the tools and explanations that are given just in time and in the context of real data. We need instructors to guide students and react to what the data is doing and what it is showing. It isn't magic, but I find it gets students there much more quickly. I am also going to drop a comment on your video because I think marrying what your group is doing (getting students out in the world doing science and collecting data) and what we are doing (teaching data analysis and applied statistics in the context of a question) would be a very powerful combination.

Great project. I appreciate the project's focus to target underrepresented communities. You mentioned that “the single greatest barrier is that instructors who teach statistics and research methods courses often do not have the skills needed to support students in doing authentic research.”

How might you all tackle this barrier while also utilizing the assets of the school, communities and students that you serve?

Your suggestions on this front are warmly welcomed! Resources in the schools are limited and not surprisingly instructors have many competing demands on their time. We use NSF support to run both online and in person workshops that take everyone exactly where they are at and provide the kind of individualized support that allows instructors to begin to fill in gaps they may have in their skill set. Still, learning takes time and whenever possible we try to implement a apprenticeship model where instructors can learn along with students in the classroom setting. 

Hi Joan, 

As Lisa mentioned the eBook is the best way to see the course topics. Since each instructor takes their own approach with the model they have the flexibility to remove content as well as add content that they believe is important for their students. For example, some remove the confounding chapter and add in additional types of factorial ANOVA's.

The Guidelines for Assessment and Instruction in Statistics Education (GAISE) college report was used when this model was conceptualized https://www.amstat.org/education/guidelines-for-assessment-and-instruction-in-statistics-education-(gaise)-reports.

In terms of the data we collect, our focus is on the student attitudes and less on testing their knowledge of the content. In the class many of our partners use M.C. quizzes to assess their learning of the lecture video, then applied M.C. questions at the end of a chapter and end of the unit. The final exam used by many requires the students to analyze a dataset new to them and answer applied M.C. questions.

When I compare my students abilities at the end of a course to my students that I taught the more traditional calculation and canned exercises approach I find the project-based students can see the use and interpret statistics and research outside of the classroom setting. It has created a pipeline of students entering into faculty labs as sophomore's instead of junior's and senior's. I have many students that will send me an email or stop by my office to tell me how their experience in a Passion-Driven Statistics classroom developed many transferable skills for future classes and in their workplace. 

A very popular data set has been AddHealth. With this data students have gravitated toward questions about religion, depression, school connectedness, etc. We find that students pick something very personal. One student grew up without a father and wanted to learn about mental health outcomes based on the presence of a father in ones life. Another student's mom told her that  religion protects us from depression and so that student explored the relationship between religion and depression in her project to see if the hypothesis could be supported. This model is incredibly useful for getting students thinking about socially important topics and framing the conversation with them around empirical data!

Hi Ambika,

Yes we essentially are doing that cycle without gathering data instead we use large, real, archival data sets. We find that having this context of their project to work through each statistical concept and analysis allows the student a frame of reference throughout the semester. Then after they leave the classroom the students report that they use their project to refer back to when they are reading research articles or working on a research project. 

We all have had the experience of learning more when we are using the skills rather than canned projects whether it is in a statistics class or some other content course. This course allows that immersion into the research project to gain those real world skills.

Hi Joi. To date, the Gear Up teams have handled recruitment for the summer programs and everyone who was interested was able to participate. No academic requirements and no prior experience needed! We even had some students who were newly learning English. Since we have a low student-teacher ratio, we are able to provide the kind of one-on-one support that is needed. 

What the kids bring with them that is so valuable is curiosity. Our curriculum capitalizes on this and students begin to drive their own work with our role being simply as a mentor and guide. Students also have an intuitive sense of how to describe data. The ideas behind choosing and using inferential statistics are more challenging. 

The use of R and other statistical platforms is actually a window into statistical thinking. With the right kind of support and a very applied approach students learn this very quickly. The vast majority of students have never done any programming in the past even at the college level. This is one of the most valuable aspects of the curriculum, that is, engaging students in applied programming experiences. This is so important early and often in ones educational journey. 

The data sets we use are large and rich (e.g. the National Longitudinal Study of Adolescent Health) and students have no trouble developing questions that they are interested in pursuing. Given that students are new to data analyses, we do not have them locate their own data sets, but instead provide them with options. 

Thanks so much for all of these great questions. We are excited that this curriculum has been used across so many different educational contexts. If you are starting students off with real world data, having them choose the research questions to pursue, and everything you teach them is in support of helping them to answer those questions, you are doing Passion-Driven Statistics!

Joi to add on to what Lisa said...

You mentioned the students that are not performing need the extra-curricular experiences. I am not sure how familiar you are with Gear-Up. That program is implemented when a school has a minimum of 50% of a school's student body must be enrolled in the federal free and reduced-price lunch (FRL) program. A person working for Gear-Up is placed in the school to work with all the students not just those on the FRL. When the Gear-Up personnel recruits for the summer programs it is offered to all students not just those on the FRL. For the summer program we had many students that were high performers without financial struggles based on what they reported their parents did for work. However, the other half when describing their research topics or they would bring up in some way throughout the week I would guess did not get access to many extra-curricular without the Gear-Up program. 

Most of our instructors offer either one or just a few data sets for students to choose from and when we do the summer boot camp we only offer one data set with younger students. We do look for new data sets to try out with students but we are not finding specific data sets for each students because we use such rich data sets that have a diverse number of variables. For example, AddHealth that Lisa mentioned has over 2300 variables that are from over 20 major topics. 

In terms of software, how we lessen the learning curve is by have engaging well developed lecture videos students watch prior to class that show step by step the software with an example. Then in class we provide generic code that they then edit just the necessary parts. As an instructor, each day I work through a project of my own showing a step then having the students do the same step. This back and forth of me demonstrating, discussion and the students following along with their project reduces the anxiety of learning a statistical software program. I use SAS on demand, free cloud based, because I find it easier for students to learn the R, but even with R we reduce the learning curve. 

Thank you for your support of our project. 

Do your bioinformatics students use archival data or gather data?

I am sure you experience challenging conversations in the classroom with students choosing diagnosis of someone they know. What successful strategies have you found to turn the conversation to positive or productive?

That is great you have found success with the GAISE recommendations. We have found these work even more so with the non-statistics major. I really enjoy when these types of students have a positive attitude change toward statistics and research rather than just getting through the course.

We have a condensed version of our model we call the boot camp model. It is essentially 15 hours of contact with the project, that can be all in class or some in class and then homework.

They choose two variables they are interested in learning if there is a relationship from a large, real world, archival dataset. For example, like AddHealth https://addhealth.cpc.unc.edu/ We have conversations about the research process, their data, software, etc. We take a just in time teaching of only what you need to know in a flipped classroom.

Next we run frequency tables, sort each variable by the other in frequency tables, univariate and bivariate graphs to allow the students to visualize and understand in percentages the relationship. We typically use SAS on demand code based, free, cloud based program but have also use R and other software. 

We then add a third variable to understand the relationship in the subsamples of the third variable. 

Gradually while exploring the data we take breaks to work on developing a research poster. The background is just why they chose the two variables, then method we give them some generic that they edit for their variables. Next they start doing the results and uses graphics that help tell their story. 

Throughout the process we support them one-on-one and have group conversations to have the students tell the story their data is telling. 

This culminates into a research poster presentation session much like you would see at a research conference. We invite in people from the university and community to explore the projects so the students have a new audience. 

We have done this boot camp as a week intensive course or at the end of a course to wrap it up. If we have more time then we add in inferential statistics, doing a review of the literature, etc. depending on how much time we have with the students.

Thank you for this detailed response-- it is very helpful to get this view into your boot camp model!