Thank you for your comments and support, Sondra.

Hi Dr. Demetrikopoulos, 

Thank you for the feedback! We have a control and an experimental class for each subject. The study skills videos and related activities/assignments are embedded into the curriculum of the experimental class. The control class is taught without the study skills materials. Regarding student performance in the modified courses, we just finished our first semester of implementation so our sample size isn't large enough yet to determine results. We should have this information in the next year. 

Hi Nancy~  Thanks for your comments.  

To answer your first question, yes we are using the NSF grant, in part, to buy release time from teaching, so we could develop the modules.  We are a 100% teaching institution, much like a community college, and have heavy teaching loads contrasted with university professors.  The grant will continue to support faculty as we collect and analyze data, among other things. 

We did not model our work on existing "programs", but before we built the models, we embarked on professional development for those of us on the team.  We adapted a lot of work from Dr. Barbara Oakley who wrote A Mind for Numbers, How to Excel in Math and Science (Even if you Flunked Algebra).  Additionally, all team members took her MOOC.  Additional ideas came from the literature.

As to the faculty buy-in...  So far, we only have 7 faculty at our school using the material formally in the study.  Of course, the modules are open to all (link in Welcome Message) and we know some faculty are also using them but aren't collecting data.  If the data shows the intervention to be effective, we will roll this out to as broad of an audience as possible.  Even though we focused on study strategies for STEM, the modules contain strategies that will cross over to other disciplines as well.

Thanks for viewing and your comments.  Holly

Hello Phillip,

That's a good question.  Since we are a inter-disciplinary group, we all approach our andra/pedagogy uniquely.  So, I can't say if or how many formative assessments my colleagues have/give.  They can chime in here to provide some more clarity for you.

I teach students chemistry and have a lot of formative assessments.  Do I individually coach students after formative assessments and suggest study strategies?  Yes.  Our class sizes are small (24) so I can do that.

We also give surveys at the beginning of the term and at the end to measure student attitudes/efficacy.

We only have preliminary data, but it looks like our students are surprisingly confident that they already have effective study skills when they walk into the course. This was a surprise for us.  That's why your question ties in nicely with our experience.  

We think discussing study strategies before an assessment is probably not smart from a timing standpoint.  Here's why:  If students are confident in their abilities, they may not think the study strategy modules have anything to offer them.  However, if they have empirical evidence that there's some inefficiency there, they may be more open to engaging with the material.

To be honest, poor performance on formative assessments is sometimes "discounted" by students.  In fact, I have students who will leave a short quiz blank except for writing "I don't know how to do this but I'll figure it out by test day."  

Sometimes, it's a disappointing performance on a summative assessment that gets the attention and while that is better than nothing, it's disappointing as a teacher because STEM subjects build upon themselves and now that student has to dig out of the hole while simultaneously moving forward.  

In closing, your question raises a point we hadn't thought about prior to the study.  Namely, is the effectiveness of the intervention dependent on when the subject is addressed/assigned?  We don't have an answer, but our hypothesis is yes.  Maybe that's something to study in round 2.  

Thanks for your viewing and comment.  Holly

Thank you for the feedback, Dr. Eaglin! Our initial survey data shows that students are overconfident when entering our courses -- they've expressed that they didn't have to study or were not taught study skills in high school. This issue becomes magnified with rigorous STEM courses and students don't feel like they have what it takes to succeed. 

I think the individual coaching aspect is a great idea and would likely be very effective -- just like one-on-one tutoring, but with study skills. Unfortunately, we currently don't have the resources to develop and scale something like that. We are incorporating the study skills into our curricula without forgoing content so we are already using every minute of instructional time. This is probably something we could tackle with a larger grant and additional personnel. If you have any specific ideas, we'd love to hear them. 

Ivetta:

You mention over-confidence in their own abilities. We see it too with students in a more-selective enrollment population, and we've had to intervene one on one with a lot students to teach them study skills.

Holly also mentioned "poor performance on formative assessments is sometimes "discounted" by students.  In fact, I have students who will leave a short quiz blank except for writing "I don't know how to do this but I'll figure it out by test day."  We found one way to help improve this is to reframe the formative assessment. We tell students that it is a diagnostic tool, and that we are trying to discover the gaps they have (kind of like the calibration gap in https://stemforall2019.videohall.com/presentations/1431). Attacking how they think about the formative assessment process helped us improve effort. This only worked though when we shared what we learned and followed through with the students.

Definitely want to look deeper into your strategy.

Hello Holly and Ivetta,

Thank you for sharing your project! Are you considering having students utilize the study skills modules in groups and compare to individual use? just wonder if this would impact student reflection on using the modules and modifying study habits when working in groups vs. alone. 

Hi, Danielle.

That's a great question! We have not considered that. This event has generated a lot of good things to think about when we plan an extension or second phase of the study. Students, and people in general, certainly do better in groups.

Even though I have enough equipment for students to do their IT labs individually, I always organize the labs in groups so they can learn from each other. The interaction is always enlightening. As a community college, our students often come to class and leave directly afterwards, so they don't usually socialize with each other. But I actually see bonds forming during labs!

My guess is that students would form a support/accountability system if they were studying together, which would help them learn the material. This would also emphasize the soft skills that are integrated into our IT courses.

The focus groups sounds like the way to go.

Thank you for your feedback, Marcelo. We are going to hold student focus groups in the fall. 

Dear Melissa,

I see the benefit of the retrospective questions - thanks for that tip. 

It would be a big help if you could point me to some literature (yours or others) that show the U-shaped curve for self efficacy.  Where should I start looking for that?

Holly

Melissa brings up a great point. We do not have published data, but plenty of course evaluation data showing similar U-shaped confidence curves.

Have you considered using something other than Likert scales? Maybe ask students to complete a 3-4 sentence reflection, then pull out themes to see if the students are saying WHY they feel less confident?

Hi, Daniel.

It appears that we did not answer you question about using something other than Likert scales to determining why students may feel less confident. That is certainly something we can consider if we do a second phase of the project. Also, after we finish this round of data collection. our external evaluator will conduct focus groups to draw out some qualitative data. This could be one of the things on which the evaluator focuses.

Hi Sandy,

About the creation of the modules:  We are a team that represents many disciplines: math, biology, chemistry, IT, engineering, earth and space science and the videos were both a team and individual effort.  After we selected the topics, individuals volunteered to make videos for topics that they felt strongly about.  The individual put together the graphics and scripts.  Then, both were shared with the greater group to get feedback and after incorporating the feedback, the videos were recorded.

Some team members made the visuals and scripts but passed on the recording part, so someone else did that part.  

We also had support from our marketing team as they did the logos which appeared on the PowerPoints and the library staff helped by creating the library guide and getting the videos uploaded there.  In case you didn't see it, the link to the lib guide is in the Welcome Message and you can watch and use the videos, if you think they might be helpful for your students.

We talk a lot about team work and interdisciplinary efforts, but I must say this is the first time I've been involved with a project that requires such a huge amount of team work with my colleagues.  Additionally, our school is a 100% teaching institution, so to execute a federally funded research grant, we also have to work closely with the grants office, finance, administration, etc. 

It hasn't been all rainbows and unicorns...  but we all put the project first.  We make compromises.  It's a privilege to serve with this team.

Thanks for viewing and your comments.  Holly

Sandy~  

After watching your video (Great idea, by the way.) I'm thinking maybe you are more interested in the technical aspect of making the video...

Our videos were made by recording a screen cast (using Quicktime) of a PowerPoint presentation we played on our computers.  

Holly

Hi, Sandy.

Some of us also used MediaSpace by Kaltura to capture the PowerPoint presentations.

Some members of the team spoke directly into their computer microphones to capture the audio, while others recorded their audio at our campus Digital Media Studios. In the case of the latter, the PowerPoint videos were exported as JPEG files. The audio and the JPEG files were then compiled as a video via Adobe Elements.

I hope we answered your question!

Chris

Dear Ellis,

The data!  Yes, we are also very interested to see how it turns out.  Of course, we'd like this to be the "secret sauce" that moves the needle on student performance, but we are also scientists and are trying to remain neutral about the outcome.  Like all evidence gathering endeavors we must be open to the possibility that the groups will not be statistically different.  We must go where the data leads.  We will learn a lot about our students, ourselves and our institution, no matter how it turns out.

There is a single set of modules (see link in Welcome Message) and our interdisciplinary team tried to incorporate the best topics for the widest STEM audience.

I'm not sure I understand what you mean by quantitative skills.  My best guess is you mean math remediation.  If not, please help me out.

Our work is modeled primarily on the work of Dr. Barbara Oakley with a few additional ideas from the literature. We did not select quantitative skills as a module.  Now, I know for chemistry, I must do math remediation for my students, but our idea was to focus more broadly on study strategies, so that the modules would transfer better across disciplines.  Even though we focused on strategies that are well suited for STEM fields, the modules would apply in non-STEM disciplines as well. 

Hope that answers your question.  Thanks for viewing and for your comment.  Holly

Thank you, Michelle. We really appreciate your feedback. 

Hi Joanne~  The link to our modules are in the Welcome message.  If you think they would benefit your students, you are welcome to use them.  Holly

Thanks for suggesting that I go to your modules Holly. They are fantastic! and I will definitely use them with my students next year. This is such important information and it is so well presented! Thank you so much for sharing! I hope everyone will check out your links!

Thanks for the comments Jose.  Holly

Thank you, Bruce! Please feel free to do so. Here is the link to our website, which houses our study skills video and other supplemental information: 

https://guides.fscj.edu/BOSS

We'd also love to hear feedback from any of your partners that visit our site and/or implement our techniques.

Thanks, Sharon!

Thanks for the feedback, Jonathan! Will do! 

Thank you for your comments, Tameiko.