There were statistically significant differences in the groups and the results are being published in an ITiCSE 2019 paper in July (not available yet). In this stage of the project, we are not measuring other impacts other than academic knowledge. This work is being done with undergraduate students and were designed for that audience, but could likely be used as-is for upper level high school students in an introductory programming course. 

The materials are being designed for an introductory programming course and tested with courses for intended majors.  However, the audience is not necessarily strictly that population (majors). The technique could be moved into another course for non-majors, provided that the concepts being taught were the same. While we are testing with undergraduate students at this time, we believe that the material could be used for upper-level high school students in introductory programming courses (like AP CSA).

Our work does not directly address this issue, but others have looked into this problem. It is a complicated problem and no one has yet found the ultimate cause. As a field, we have some ideas, which include varying levels of preparation for the course caused by many factors prior to college, general college factors, and potentially things about the course or material itself. 

We have not yet heard pushback. In fact, many who see the materials remark that they use similar language and technique when explaining the concept, even if they do not use this exact methodology. We have teachers who are willing to pilot and interest in getting more subgoals for different languages and more topics, so I would say general support.

Not exactly - we lose 1/3 from the course. This may lead to them dropping out of the major, but it is simply pointing that 1/3 of the students are not successfully completing the course.

At this point, we don't have plans to expand into high school yet, but we will keep you in mind as our conversations progress.

We have done some work prior to this particular project comparing telling students the subgoals to asking students to identify subgoals for themselves. In general, the more novice students do better if you tell them the subgoals and the more experienced students do better if you let them apply their knowledge to identifying the subgoals for themselves. For this project, we decided to design for the more novice students and tell them, but the subgoals that we created could also be used as feedback for students who are identifying the subgoals for themselves.

Thank you for the response! I like the idea of students generating their own subgoals as it would encourage them to think through the processes for decomposing a problem and solution; however, it makes sense to me that your prior work found novice students might struggle with such an approach. Thanks again!