The current proposal will deliver three cost-effective courses developed through the collaboration of Embry-Riddle Aeronautical University (ERAU); Adams State University (ASU), an Hispanic-Serving Institution (HSI); Hampden-Sydney College (HSC), a small liberal-arts college; and Bethune-Cookman University, an Historically Black University (HBU).
A team of representatives from each institution will develop and promote interdisciplinary Computational and Data-Enabled Science and Engineering (CDSE) education in a sustainable manner via cyber-learning technologies. A broadened focus will demonstrate how pioneering methods in cyber-learning can be used in multiple STEM disciplines. The common goal is to produce students who have the scientific and critical thinking skills necessary to excel in a scientific environment increasingly dominated by big data and interdisciplinary problems.
The key goals and objectives of this project are to: 1) Course Design and Development: Develop and implement high quality and relevant CDSE courses using active learning and research-based methodologies that promote inter-institutional and interdisciplinary collaboration; 2) Student Learning Assessment: Develop and implement innovative learning assessment tools to gauge student achievement, including a web-based learning analytics system, formative learning assessment through blog assignments, and a virtual educational observatory to help track student demographics, and student learning strategies. 3) Expansion and Sustainability of CDSE Pedagogical Network: Develop, implement, and test an expanded CDSE pedagogical network in which resource sharing allows institutions of various sizes and types to sustainably offer CDSE courses.
Emerging Technologies, Research / Evaluation, Broadening Participation
Embry-Riddle, Hampden-Sydney Coll, Adams State Univ
Improving Undergraduate STEM Education (IUSE)
Related Content for Coalition for Computational Science & Engineering Education
Discussion from the 2019 STEM for All Video Showcase (19 posts)
Matthew Iklé
Lead Presenter Professor
May 12, 2019 | 10:07 p.m.
Welcome to the Coalition for Computation & Engineering (CDSE) Education!
Our goal was to solve common STEM educational barriers at small institutions by creating ofvirtual departments consisting of coalitions of institutions. The coalition built the teaching efficiency, research opportunities, and capacity to offer CDSE degree programs and student enrollments to sustain them. Our project provides insights into effective cyber-learning and teaching strategies that promote diversity within the STEM workforce. Students benefited from the new courses and learning assessment strategies as performance data and student and faculty input drove course improvement.
Hope you enjoy seeing and hearing some of our successes!
Matthew Iklé, PI, Adams State University
Hong Liu, co-PI, Embry-Riddle Aeronautical University
Raphael Isokpehi, co-PI, Bethune-Cookman University
Michael Wolyniak, co-PI, Hampden-Sydney College
Michael Spector, External Evaluator, University of North Texas
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Hong Liu
Jonathan Spector
Co-Presenter Professor
May 13, 2019 | 10:09 a.m.
As the evaluator on this effort, I noticed the significance of design on the courses - specifically, designing activities for students to help them master complex concepts ... and what really made the effort successful was scaffolding for individual students - giving struggling students the support they needed to achieve some level of success that then led to continued engagement.
Joni Falk
Co-Director of CSR at TERC
May 13, 2019 | 04:17 p.m.
I like the fundamental ideas driving this project - to expand opportunities to those who do not have access by connecting institutions and universities. I think I heard that those who take the course virtually have a mentor available to them... is that right? Wondering if these courses are equally well rated by those who are physically in attendance and those present through the screen? Is the drop-out rate similar in each group? Do the students (in class and virtually) interact with each other through some online portal? Jonathan you mention offering struggling students support. What does this look like?
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Danielle Watt
Matthew Iklé
Lead Presenter Professor
May 13, 2019 | 04:31 p.m.
Hi Joni,
Thanks for your comments and questions!
Yes, professors at both the broadcasting and receiving institutions help mentor students. The success rates tend to be similar at both the broadcasting and receiving institutions.
We are constantly striving to improve interaction among all the constituents, professors and students. The difficult part is, of course, breaking down the barriers between the institutions. While we have made progress in this direction, there is still a lot of work to do. Much ultimately depends upon the instructors' abilities to make the classrooms at both ends feel like a single larger classroom despite the distance: Inviting the (reluctant) students at a distance into the broadcasting classroom. Having projects composed of teams from both institutions also helps somewhat but is also not a panacea. Hong, Raphael, Michael, and Michael can provide additional insights.
2
Discussion is closed. Upvoting is no longer available
Found helpful:
Danielle Watt
Joni Falk
Dr. Hong Liu
Co-Presenter Professor
May 13, 2019 | 04:37 p.m.
Dr. Falk, thank you for your comments.
This is Hong, a Co-PI of the project. To answer your first question, Yes, the students who are learning from the remote classroom have either a Co-Teacher sitting with them when the number of students is large or a TA with them when there is only a couple of students. The Co-Teacher not only helps to address the immediate/emergent need of the students, but also gains an opportunity to serve as a peer review of the teaching, and learn/improve teaching for the next round when the co-teacher takes turns to teach the same course. The student evaluation data (drop rate) shows that the local students and remote students of the primary teacher have no noticeable difference. We have a course website and encourage students to collaborate online with the teammate in remote campuses. But the students mostly prefer to team-up with the local students if they have the choices. To address this issue, we are building a Computer-Supported Collaborative Learning platform based on AI-technology (webbot) currently.
I hope that my answers make sense to you. As an evaluator, Mike (Jonathan), probably can answer your second question better.
2
Discussion is closed. Upvoting is no longer available
Found helpful:
Danielle Watt
Joni Falk
Dr. Hong Liu
Co-Presenter Professor
May 13, 2019 | 04:36 p.m.
Dr. Falk, thank you for your comments.
This is Hong, a Co-PI of the project. To answer your first question, Yes, the students who are learning from the remote classroom have either a Co-Teacher sitting with them when the number of students is large or a TA with them when there is only a couple of students. The Co-Teacher not only helps to address the immediate/emergent need of the students, but also gains an opportunity to serve as a peer review of the teaching, and learn/improve teaching for the next round when the co-teacher takes turns to teach the same course. The student evaluation data (drop rate) shows that the local students and remote students of the primary teacher have no noticeable difference. We have a course website and encourage students to collaborate online with the teammate in remote campuses. But the students mostly prefer to team-up with the local students if they have the choices. To address this issue, we are building a Computer-Supported Collaborative Learning platform based on AI-technology (webbot) currently.
I hope that my answers make sense to you. As an evaluator, Mike (Jonathan), probably can answer your second question better.
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Joni Falk
Phillip Eaglin, PhD
Facilitator Founder and CEO
May 13, 2019 | 05:53 p.m.
What a great way to provide STEM learning opportunities to universty students who would not otherwise have them! And the BOT captains idea sounds like an exciting way to get the students to communicate about their studies! Question: Can you share more about what the BOT captains concept is (the tools used for collaboration) and how it works (what types of projects do the students collaborate on and if/how that collaboration is part of their course outcomes)?
Dr. Hong Liu
Co-Presenter Professor
May 13, 2019 | 06:38 p.m.
Thanks for your interest to BOT Dr. Philip,
The web bot is a chatbot built on the Microsoft Bot Framework. It serves as a virtual TA who monitor the student teamwork under teamwork social media platform such as GroupMe, and Slack, etc. It collects data from the teamwork environment. Therefore, the instructors can evaluate the teamwork process and individual performances besides the team project outcomes.
The details can be found on the linked article under review.
Hong
2
Discussion is closed. Upvoting is no longer available
Found helpful:
Phillip Eaglin, PhD
Danielle Watt
Marcelo Worsley
Facilitator Assistant Professor
May 13, 2019 | 11:36 p.m.
Thank you for sharing this work. The description mentioned a web-based learning analytics system. Can you say more about what the system is able to do. For example, what the types of data is being collected and how are instructors and students using that data.
Dr. Hong Liu
Co-Presenter Professor
May 14, 2019 | 08:53 a.m.
Dr. Worsley, Thanks for your question and interest.
The purpose of the Bot Captain is to make the performance and contribution of the teamwork transparent to the instructors as well as the students in the same way as those of athletes are transparent to the coaches and spectators. The system can help to mitigate the two extremes, hitchhikers and heroism. It aims to promote peer collaborative learning and make each teammate more accountable. The Bot Captain logs three types of formative learning assessment data, 1) the student efforts & motivation based on presences and absences of meetings, the roles and tasks, and progress report on each weekly online meeting records, 2) social relational data based on the positive/negative words in communications, 3) cognitive assessment data based on the frequencies of conceptual keywords, and stages of the projects (e.g. initiating a topic thread, topic diverging, topic converging, and resolution). The system sends emails to remind students of the deadlines of the tasks, meetings, etc. and collects data such as task decomposition, the leading role and supporting role for each task. Since the team project counts as 40% of the course grades (homework, quizzes and test count 60%), the scores of the team project consist of two parts: 60% for the summative assessment of the project report, presentation, and codes that all teammates shared the same. The other 40%, however, is evaluated differently for each individual based on the logged data from Bot Captain, and peer evaluations. The data analytics components have not integrated with the Bot Captain at this stage. The team of students will integrate the two components with another third-party database called Learning Record Store built by the ADL (Advanced Distributed Learning ) group of ARL. The data interoperability is based on xAPI (eXperience API), a JSON (JavaScript Object) format. I am not sure if I have answered your question. More details can be found on the linked article and future publication. As a product, it will have a debut in fall 2019 working more like an Amazon Mechanical Turk, instead of full automation.
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Danielle Watt
Marcelo Worsley
Facilitator Assistant Professor
May 14, 2019 | 09:57 p.m.
Thank you. This is what I was wondering.
Danielle Watt
Facilitator Director of Education, Outreach, Diversity
May 14, 2019 | 01:52 p.m.
Thank you for sharing your project, it's a great model. Can you comment more on the outcomes from the student assessment? Is there an increase in content mastery and performance in students who take this course verse students who do not?
Jonathan Spector
Researcher
May 16, 2019 | 12:33 p.m.
As Dr. Hong has said, we are still pursuing how best to assessment student outcomes. One methodology involves an approach that involves capturing how a student thinks about a complex problem in the form of an annotated concept map or causal influence diagram and then analyzing that representation in comparison with a reference or expert model (see Spector & Koszalka, 2003; see https://www.nsf.gov/awardsearch/showAward?AWD_ID=0335644). Unfortunately the DEEP software is not longer available. Another method is to examine longer-term impact such as how students progress in their studies and what courses and degrees they pursue. The DEEP software is being revised and we also support the longitudinal approach, although as yet we do not have strong indicatioins other than student and faculty reports on surveys which are a weaker meythod.
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Danielle Watt
Dr. Hong Liu
Co-Presenter Professor
May 15, 2019 | 08:48 a.m.
Dear Dr. Watt
Thanks for the comments and we appreciate your question. I will try to answer it from a teacher's point of view.
First of all, we compared the Computational & Data Science courses that we taught, with other courses such as Calculus & Differential Equation, Computer science courses, the DFW rates of CDS courses are much lower. For example, the Data Mining Courses, we had zero DFW rate for three semesters. But, the most important difference is the depth of learning, especially to solve complex problems. Our students coauthored five publications and gave dozen of presentations at the professional conference since we start to offer the course from 2014-2019. We also tracked the success of many graduates from the program. They have 100 retentions in the STEM field or work for a technical career. For example. We tracked a new graduate from ERAU Mr. Alex Knoyha. He is working as a Space Systems Engineer at Lockheed Martin. He said at his Linked-in "I incorporated skills learned in the Data Mining & Visualization class to perform statistical analysis and create predictive models of student retention rates."
Great project! I’m interested to learn more about the business case that will support cross-university registrations and what your thinking is around that piece. Collaborative, cross-university projects can be so valuable and informative, and you’ve obviously done a great job, with amazing outcomes. Where are your next steps with this initiative and what are your insights about the potential to scale it beyond the current parameters? Thanks so much! - Dav
Dr. Hong Liu
Co-Presenter Professor
May 15, 2019 | 09:58 a.m.
Dear Dr. Miller,
It is a very nice question. We really walked around all business troubles in a way similar to that our kids share a ride to school by carpool. We just take turns to teach (drive) and co-teach (free ride).
More specifically, the students in remote campus (e.g. at Adams State University at Colorado, ASU) always register a course from their own school - ASU (e.g. a special topic MA499/CS499 if the ASU does not have the same title). But their class is delivered from another university (Embry-Riddle at Daytona, ERAU) through teleconference tools when the teacher teaches his own students face-to-face at ERAU. Typically, a co-teacher is sitting at the remote classroom (at ASU Colorado). The role of the co-teacher is to serve as a peer reviewer of the course, and also learn how to teach a new subject or teach better by learning each other. When the semester ends, the teacher of the face-to-face class at ERAU submit the grades of the remote students to the co-teacher at ASU. Then, the co-teacher will assign (adjust based on the school expectation) the grades of ASU for the remote students at ASU. Next term, the teach and co-teach exchange role for another class, and their students also exchange from face-to-face students to online students.
Not sure, if this is clear. If you have more questions, please let me know.
Thanks for the reply. Interesting model. What sorts of challenges did you encounter with recruiting faculty at other institutions to embrace this model?
Matthew Iklé
Lead Presenter Professor
May 15, 2019 | 10:11 a.m.
Another excellent question. In fact, we are currently setting up a workshop from 4-6 June 2019 for the purposes of disseminating our results, as well as for recruiting new faculty and scaling up our efforts for the next stages of collaborations. Although we have a limited number of spots, we encourage interested faculty to contact us for additional information.
Matthew Iklé
Professor
Welcome to the Coalition for Computation & Engineering (CDSE) Education!
Our goal was to solve common STEM educational barriers at small institutions by creating ofvirtual departments consisting of coalitions of institutions. The coalition built the teaching efficiency, research opportunities, and capacity to offer CDSE degree programs and student enrollments to sustain them. Our project provides insights into effective cyber-learning and teaching strategies that promote diversity within the STEM workforce. Students benefited from the new courses and learning assessment strategies as performance data and student and faculty input drove course improvement.
Hope you enjoy seeing and hearing some of our successes!
Matthew Iklé, PI, Adams State University
Hong Liu, co-PI, Embry-Riddle Aeronautical University
Raphael Isokpehi, co-PI, Bethune-Cookman University
Michael Wolyniak, co-PI, Hampden-Sydney College
Michael Spector, External Evaluator, University of North Texas
Hong Liu
Jonathan Spector
Professor
As the evaluator on this effort, I noticed the significance of design on the courses - specifically, designing activities for students to help them master complex concepts ... and what really made the effort successful was scaffolding for individual students - giving struggling students the support they needed to achieve some level of success that then led to continued engagement.
Joni Falk
I like the fundamental ideas driving this project - to expand opportunities to those who do not have access by connecting institutions and universities. I think I heard that those who take the course virtually have a mentor available to them... is that right? Wondering if these courses are equally well rated by those who are physically in attendance and those present through the screen? Is the drop-out rate similar in each group? Do the students (in class and virtually) interact with each other through some online portal? Jonathan you mention offering struggling students support. What does this look like?
Danielle Watt
Matthew Iklé
Professor
Hi Joni,
Thanks for your comments and questions!
Yes, professors at both the broadcasting and receiving institutions help mentor students. The success rates tend to be similar at both the broadcasting and receiving institutions.
We are constantly striving to improve interaction among all the constituents, professors and students. The difficult part is, of course, breaking down the barriers between the institutions. While we have made progress in this direction, there is still a lot of work to do. Much ultimately depends upon the instructors' abilities to make the classrooms at both ends feel like a single larger classroom despite the distance: Inviting the (reluctant) students at a distance into the broadcasting classroom. Having projects composed of teams from both institutions also helps somewhat but is also not a panacea. Hong, Raphael, Michael, and Michael can provide additional insights.
Danielle Watt
Joni Falk
Dr. Hong Liu
Professor
Dr. Falk, thank you for your comments.
This is Hong, a Co-PI of the project. To answer your first question, Yes, the students who are learning from the remote classroom have either a Co-Teacher sitting with them when the number of students is large or a TA with them when there is only a couple of students. The Co-Teacher not only helps to address the immediate/emergent need of the students, but also gains an opportunity to serve as a peer review of the teaching, and learn/improve teaching for the next round when the co-teacher takes turns to teach the same course. The student evaluation data (drop rate) shows that the local students and remote students of the primary teacher have no noticeable difference.
We have a course website and encourage students to collaborate online with the teammate in remote campuses. But the students mostly prefer to team-up with the local students if they have the choices. To address this issue, we are building a Computer-Supported Collaborative Learning platform based on AI-technology (webbot) currently.
I hope that my answers make sense to you. As an evaluator, Mike (Jonathan), probably can answer your second question better.
Danielle Watt
Joni Falk
Dr. Hong Liu
Professor
Dr. Falk, thank you for your comments.
This is Hong, a Co-PI of the project. To answer your first question, Yes, the students who are learning from the remote classroom have either a Co-Teacher sitting with them when the number of students is large or a TA with them when there is only a couple of students. The Co-Teacher not only helps to address the immediate/emergent need of the students, but also gains an opportunity to serve as a peer review of the teaching, and learn/improve teaching for the next round when the co-teacher takes turns to teach the same course. The student evaluation data (drop rate) shows that the local students and remote students of the primary teacher have no noticeable difference.
We have a course website and encourage students to collaborate online with the teammate in remote campuses. But the students mostly prefer to team-up with the local students if they have the choices. To address this issue, we are building a Computer-Supported Collaborative Learning platform based on AI-technology (webbot) currently.
I hope that my answers make sense to you. As an evaluator, Mike (Jonathan), probably can answer your second question better.
Joni Falk
Phillip Eaglin, PhD
Founder and CEO
What a great way to provide STEM learning opportunities to universty students who would not otherwise have them! And the BOT captains idea sounds like an exciting way to get the students to communicate about their studies! Question: Can you share more about what the BOT captains concept is (the tools used for collaboration) and how it works (what types of projects do the students collaborate on and if/how that collaboration is part of their course outcomes)?
Dr. Hong Liu
Professor
Thanks for your interest to BOT Dr. Philip,
The web bot is a chatbot built on the Microsoft Bot Framework. It serves as a virtual TA who monitor the student teamwork under teamwork social media platform such as GroupMe, and Slack, etc. It collects data from the teamwork environment. Therefore, the instructors can evaluate the teamwork process and individual performances besides the team project outcomes.
The details can be found on the linked article under review.
Hong
Phillip Eaglin, PhD
Danielle Watt
Marcelo Worsley
Assistant Professor
Thank you for sharing this work. The description mentioned a web-based learning analytics system. Can you say more about what the system is able to do. For example, what the types of data is being collected and how are instructors and students using that data.
Dr. Hong Liu
Professor
Dr. Worsley, Thanks for your question and interest.
The purpose of the Bot Captain is to make the performance and contribution of the teamwork transparent to the instructors as well as the students in the same way as those of athletes are transparent to the coaches and spectators. The system can help to mitigate the two extremes, hitchhikers and heroism. It aims to promote peer collaborative learning and make each teammate more accountable. The Bot Captain logs three types of formative learning assessment data, 1) the student efforts & motivation based on presences and absences of meetings, the roles and tasks, and progress report on each weekly online meeting records, 2) social relational data based on the positive/negative words in communications, 3) cognitive assessment data based on the frequencies of conceptual keywords, and stages of the projects (e.g. initiating a topic thread, topic diverging, topic converging, and resolution). The system sends emails to remind students of the deadlines of the tasks, meetings, etc. and collects data such as task decomposition, the leading role and supporting role for each task. Since the team project counts as 40% of the course grades (homework, quizzes and test count 60%), the scores of the team project consist of two parts: 60% for the summative assessment of the project report, presentation, and codes that all teammates shared the same. The other 40%, however, is evaluated differently for each individual based on the logged data from Bot Captain, and peer evaluations. The data analytics components have not integrated with the Bot Captain at this stage. The team of students will integrate the two components with another third-party database called Learning Record Store built by the ADL (Advanced Distributed Learning ) group of ARL. The data interoperability is based on xAPI (eXperience API), a JSON (JavaScript Object) format. I am not sure if I have answered your question. More details can be found on the linked article and future publication. As a product, it will have a debut in fall 2019 working more like an Amazon Mechanical Turk, instead of full automation.
Danielle Watt
Marcelo Worsley
Assistant Professor
Thank you. This is what I was wondering.
Danielle Watt
Director of Education, Outreach, Diversity
Thank you for sharing your project, it's a great model. Can you comment more on the outcomes from the student assessment? Is there an increase in content mastery and performance in students who take this course verse students who do not?
Jonathan Spector
As Dr. Hong has said, we are still pursuing how best to assessment student outcomes. One methodology involves an approach that involves capturing how a student thinks about a complex problem in the form of an annotated concept map or causal influence diagram and then analyzing that representation in comparison with a reference or expert model (see Spector & Koszalka, 2003; see https://www.nsf.gov/awardsearch/showAward?AWD_ID=0335644). Unfortunately the DEEP software is not longer available. Another method is to examine longer-term impact such as how students progress in their studies and what courses and degrees they pursue. The DEEP software is being revised and we also support the longitudinal approach, although as yet we do not have strong indicatioins other than student and faculty reports on surveys which are a weaker meythod.
Danielle Watt
Dr. Hong Liu
Professor
Dear Dr. Watt
Thanks for the comments and we appreciate your question. I will try to answer it from a teacher's point of view.
First of all, we compared the Computational & Data Science courses that we taught, with other courses such as Calculus & Differential Equation, Computer science courses, the DFW rates of CDS courses are much lower. For example, the Data Mining Courses, we had zero DFW rate for three semesters. But, the most important difference is the depth of learning, especially to solve complex problems. Our students coauthored five publications and gave dozen of presentations at the professional conference since we start to offer the course from 2014-2019. We also tracked the success of many graduates from the program. They have 100 retentions in the STEM field or work for a technical career. For example. We tracked a new graduate from ERAU Mr. Alex Knoyha. He is working as a Space Systems Engineer at Lockheed Martin. He said at his Linked-in "I incorporated skills learned in the Data Mining & Visualization class to perform statistical analysis and create predictive models of student retention rates."
Of course, this empirical data is not a formal statistics study. We had two papers, one is on the Journal of Computational Science Education, the other in Journal of Computer Application for Engineering Education, that summarize the student accomplishment so far.
Sorry that my answer might be too long. Our external evaluator Dr. Mike Spector can answer the question better.
Hong
Danielle Watt
Danielle Watt
Director of Education, Outreach, Diversity
Thank you Dr. Liu for the info. Dr. Spector and you answered my question.
Dave Miller
Great project! I’m interested to learn more about the business case that will support cross-university registrations and what your thinking is around that piece. Collaborative, cross-university projects can be so valuable and informative, and you’ve obviously done a great job, with amazing outcomes. Where are your next steps with this initiative and what are your insights about the potential to scale it beyond the current parameters? Thanks so much! - Dav
Dr. Hong Liu
Professor
Dear Dr. Miller,
It is a very nice question. We really walked around all business troubles in a way similar to that our kids share a ride to school by carpool. We just take turns to teach (drive) and co-teach (free ride).
More specifically, the students in remote campus (e.g. at Adams State University at Colorado, ASU) always register a course from their own school - ASU (e.g. a special topic MA499/CS499 if the ASU does not have the same title). But their class is delivered from another university (Embry-Riddle at Daytona, ERAU) through teleconference tools when the teacher teaches his own students face-to-face at ERAU. Typically, a co-teacher is sitting at the remote classroom (at ASU Colorado). The role of the co-teacher is to serve as a peer reviewer of the course, and also learn how to teach a new subject or teach better by learning each other. When the semester ends, the teacher of the face-to-face class at ERAU submit the grades of the remote students to the co-teacher at ASU. Then, the co-teacher will assign (adjust based on the school expectation) the grades of ASU for the remote students at ASU. Next term, the teach and co-teach exchange role for another class, and their students also exchange from face-to-face students to online students.
Not sure, if this is clear. If you have more questions, please let me know.
Dave Miller
Thanks for the reply. Interesting model. What sorts of challenges did you encounter with recruiting faculty at other institutions to embrace this model?
Matthew Iklé
Professor
Another excellent question. In fact, we are currently setting up a workshop from 4-6 June 2019 for the purposes of disseminating our results, as well as for recruiting new faculty and scaling up our efforts for the next stages of collaborations. Although we have a limited number of spots, we encourage interested faculty to contact us for additional information.
Further posting is closed as the event has ended.