In this video, we will provide an overview of csedresearch.org, a site designed to provide resources to those in engaged in primary and secondary computer science (CS) education research. This site features a repository of manually-curated data on over 500 articles focusing on primary and secondary CS education research across ten targeted venues, 2012-2018, and over 150 evaluation instruments, many of which are focused on computing. Users can search for articles by filtering various curriculum and activity components, including student demographics, instructor demographics, and program components. Users can also search for evaluation instruments on criteria such as type, year published, constructs measured in the instrument, and whether the instrument has been validated or tested for reliability. We will provide a brief summary of the rationale for the site, the current state of the site, and how educators and researchers are using the site to date. We will also discuss future plans for the site.
Computer Science, Research / Evaluation, Broadening Participation
SUNY Buffalo, Knox College
Innovative Technology Experiences for Students and Teachers (ITEST), Improving Undergraduate STEM Education (IUSE), STEM + Computing Partnerships (STEM+C)
Related Content for csedresearch.org: Resources for K-12 Computing Education
Discussion from the 2019 STEM for All Video Showcase (19 posts)
Monica McGill
Co-Presenter Associate Professor
May 12, 2019 | 05:09 p.m.
Thank you for taking the time to watch our video! csedresearch.org is in its formative phase. Our research team would be interested in your feedback, especially in regard to the following:
Have you used csedresearch.org? If so, how?
As csedresearch.org evolves, what types of additional resources would be most useful for you?
Thanks for stopping by!
1
Discussion is closed. Upvoting is no longer available
Thank you for building a great resource for the CS education community. ECEP Alliance state leaders and NSF PIs in our network are often looking for research that will help them make the case for CS education. CSedresearch.org is one of the first places that I point them to. We've posted your repository on the ECEP website so more people can find it and use it.
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Monica McGill
Mac Cannady
Facilitator Research Group Director
May 13, 2019 | 09:13 p.m.
Hi, thanks for the video! I was unfamiliar with your website and it looks like a great resource!
Just wondering, do you actively search for tools or articles to include in your system, or should users nominate/submit their own work to be included?
Thanks!
Monica McGill
Co-Presenter Associate Professor
May 13, 2019 | 10:06 p.m.
Hi Mac! We curate as much as we can as we are able. But we welcome submissions as well. You can submit directly through the site or you can email us.
We are particularly actively seeking evaluation instruments--especially those that have been shown to be reliable and/or validated by others.
As far as sustainability, we currently have over two more years left on our grant. There are a couple of models we are looking at for continuing the work after this grant ends (additional funding, community editors, etc.). We haven't formulated a formal plan yet, but expect to over the next year.
Monica
Susan McKenney
Facilitator Professor
May 14, 2019 | 08:09 a.m.
This is wonderful - thank you for sharing! I have taken the liberty of sharing this with some of my colleagues here in the Netherlands. I am also wondering, how relevant would you say this resources is for educators outside of the US?
Kind regards, Susan
Monica McGill
Co-Presenter Associate Professor
May 14, 2019 | 09:08 a.m.
Hi Susan, thanks for stopping by. The article summaries are from ten venues, all international. Many of the instruments and articles right now are US-centered, but I still believe many of the items within the constructs measured could be easily used outside of the U.S.
As we grow the collection of data, we would LOVE to have more instruments and articles from outside of the U.S. If you know of any you can either email me directly (mmmcgill@knox.edu) or submit using the form on the site.
Monica
1
Discussion is closed. Upvoting is no longer available
This is a great video! I've used the evaluation tools (A lot) and the research articles. When might the research guides be ready? And who would the target audience for these be?
Monica McGill
Co-Presenter Associate Professor
May 14, 2019 | 12:09 p.m.
Hi Rebecca! Thanks for stopping by. The guides will be a work in progress, so the guides will be developed over the next 1-2 years. However, the next set will be on data collection methods and I'm targeting summer for those.
The intended target audience is anyone new to computing education research or evaluation.
Monica
K. Renae Pullen
Facilitator Elementary Science Specialist
May 14, 2019 | 10:38 p.m.
Yes! I love the idea of having access to CS research and tools all in one place. What metrics will you use to measure the impact of csedresearch.org?
Adrienne Decker
Lead Presenter
May 15, 2019 | 09:53 a.m.
Right now, we are focused on use of site in terms of visitors and time spent using the resources. As we move into the next phase of this work, we are hoping to expand the functionality and add other ways for users to interact with the materials. We are looking towards hosting evaluation instruments and possibly data. Implementing those features will also help in measuring impact.
Long term, if we go back to the literature and start to see improvements in areas of reporting, use of validated instruments, and better research methodology in this area, there is impact.
1
Discussion is closed. Upvoting is no longer available
This looks like a great resource. Do you also have research on effective teaching of CS and/or the preparation of elementary vs secondary science and mathematics teachers to incorporate CS into their classrooms and courses?
Adrienne Decker
Lead Presenter
May 15, 2019 | 04:20 p.m.
Currently, the article summaries are from 10 selected venues, primarily focused on computing education. If there are articles in those venues that deal with the issues you mention, they would have been included if focused in the pre-college domain. We are now providing ways for additional articles to be recommended for inclusion in the site by the community and we would be interested in articles of this sort for consideration.
Adrienne and Monica, great video and fantastic resource. Watching the video pointed me to a computational thinking resource that I am about to email to an ECEP Alliance state leader! Thanks.
Monica McGill
Co-Presenter Associate Professor
May 19, 2019 | 12:53 p.m.
Thanks, Sarah! Glad you find it useful. Look for more eval instruments to be posted over the next few weeks.
Dear Adrienne and Monica, Thank you so much for putting together this fantastic resource. It's super useful for both education and research. Since learning about your project from SIGCSE, I've been pointing students to your site to look for assessment tools. I especially appreciate the filtering ability. One side benefit I notice is it highlights the value of age-specific tools and validation, because of how these attributes are made available as filtering options. This saves me a lot of time to explain to students the importance of choosing age appropriate assessment tools, rather than simply adopting a popular and one-size-fit-all tool.
Perhaps one feature I'd like to see would be a way for the community to self-report how they may have used a particular assessment tool and to share experiences. Also, a way to connect to the authors or people with experiences of a particular tool would be nice, so that users can receive advice or offer feedback.
Adrienne Decker
Lead Presenter
May 20, 2019 | 03:35 p.m.
Thanks for visiting our video and for the feedback! So glad you have found the site useful.
I think the suggestion you give here is an important one for us to consider going forward. It is not in our immediate plans to provide this type of functionality to the site, but since we have started this project, there has been a lot of discussion of giving the community a voice in more of the content (particularly as you mention around ratings/reviews or comments).
I think the challenge to that is to find a way to make the feedback and reports meaningful in the same way that the curated content is. We have taken a lot of time to make sure the content being presented is of high quality and would like addition items (like reports or feedback) to be of the same useful quality
Monica McGill
Associate Professor
Thank you for taking the time to watch our video! csedresearch.org is in its formative phase. Our research team would be interested in your feedback, especially in regard to the following:
Thanks for stopping by!
Sarah Dunton
Sarah Dunton
Thank you for building a great resource for the CS education community. ECEP Alliance state leaders and NSF PIs in our network are often looking for research that will help them make the case for CS education. CSedresearch.org is one of the first places that I point them to. We've posted your repository on the ECEP website so more people can find it and use it.
Monica McGill
Mac Cannady
Research Group Director
Hi, thanks for the video! I was unfamiliar with your website and it looks like a great resource!
Just wondering, do you actively search for tools or articles to include in your system, or should users nominate/submit their own work to be included?
Thanks!
Monica McGill
Associate Professor
Hi Mac! We curate as much as we can as we are able. But we welcome submissions as well. You can submit directly through the site or you can email us.
We are particularly actively seeking evaluation instruments--especially those that have been shown to be reliable and/or validated by others.
Monica
Adrienne Decker
I'll add in the direct link to submit to this thread: https://csedresearch.org/submit-to-repository/, but email does work as well.
Adrienne
Laura Bofferding
Wow! Sounds like a great resource. I can't wait to explore it more. How do you choose the articles and how do you plan to sustain the site over time?
Monica McGill
Associate Professor
Our about page (https://csedresearch.org/about/) explains how we curate the data.
As far as sustainability, we currently have over two more years left on our grant. There are a couple of models we are looking at for continuing the work after this grant ends (additional funding, community editors, etc.). We haven't formulated a formal plan yet, but expect to over the next year.
Monica
Susan McKenney
Professor
This is wonderful - thank you for sharing! I have taken the liberty of sharing this with some of my colleagues here in the Netherlands. I am also wondering, how relevant would you say this resources is for educators outside of the US?
Kind regards,
Susan
Monica McGill
Associate Professor
Hi Susan, thanks for stopping by. The article summaries are from ten venues, all international. Many of the instruments and articles right now are US-centered, but I still believe many of the items within the constructs measured could be easily used outside of the U.S.
As we grow the collection of data, we would LOVE to have more instruments and articles from outside of the U.S. If you know of any you can either email me directly (mmmcgill@knox.edu) or submit using the form on the site.
Monica
Susan McKenney
Rebecca Zarch
This is a great video! I've used the evaluation tools (A lot) and the research articles. When might the research guides be ready? And who would the target audience for these be?
Monica McGill
Associate Professor
Hi Rebecca! Thanks for stopping by. The guides will be a work in progress, so the guides will be developed over the next 1-2 years. However, the next set will be on data collection methods and I'm targeting summer for those.
The intended target audience is anyone new to computing education research or evaluation.
Monica
K. Renae Pullen
Elementary Science Specialist
Yes! I love the idea of having access to CS research and tools all in one place. What metrics will you use to measure the impact of csedresearch.org?
Adrienne Decker
Right now, we are focused on use of site in terms of visitors and time spent using the resources. As we move into the next phase of this work, we are hoping to expand the functionality and add other ways for users to interact with the materials. We are looking towards hosting evaluation instruments and possibly data. Implementing those features will also help in measuring impact.
Long term, if we go back to the literature and start to see improvements in areas of reporting, use of validated instruments, and better research methodology in this area, there is impact.
K. Renae Pullen
Ed Geary
This looks like a great resource. Do you also have research on effective teaching of CS and/or the preparation of elementary vs secondary science and mathematics teachers to incorporate CS into their classrooms and courses?
Adrienne Decker
Currently, the article summaries are from 10 selected venues, primarily focused on computing education. If there are articles in those venues that deal with the issues you mention, they would have been included if focused in the pre-college domain. We are now providing ways for additional articles to be recommended for inclusion in the site by the community and we would be interested in articles of this sort for consideration.
Sarah Dunton
Adrienne and Monica, great video and fantastic resource. Watching the video pointed me to a computational thinking resource that I am about to email to an ECEP Alliance state leader! Thanks.
Monica McGill
Associate Professor
Thanks, Sarah! Glad you find it useful. Look for more eval instruments to be posted over the next few weeks.
Tom Yeh
Dear Adrienne and Monica, Thank you so much for putting together this fantastic resource. It's super useful for both education and research. Since learning about your project from SIGCSE, I've been pointing students to your site to look for assessment tools. I especially appreciate the filtering ability. One side benefit I notice is it highlights the value of age-specific tools and validation, because of how these attributes are made available as filtering options. This saves me a lot of time to explain to students the importance of choosing age appropriate assessment tools, rather than simply adopting a popular and one-size-fit-all tool.
Perhaps one feature I'd like to see would be a way for the community to self-report how they may have used a particular assessment tool and to share experiences. Also, a way to connect to the authors or people with experiences of a particular tool would be nice, so that users can receive advice or offer feedback.
Adrienne Decker
Thanks for visiting our video and for the feedback! So glad you have found the site useful.
I think the suggestion you give here is an important one for us to consider going forward. It is not in our immediate plans to provide this type of functionality to the site, but since we have started this project, there has been a lot of discussion of giving the community a voice in more of the content (particularly as you mention around ratings/reviews or comments).
I think the challenge to that is to find a way to make the feedback and reports meaningful in the same way that the curated content is. We have taken a lot of time to make sure the content being presented is of high quality and would like addition items (like reports or feedback) to be of the same useful quality
Further posting is closed as the event has ended.