NSF Awards: 0962863
2015 (see original presentation & discussion)
Grades K-6, Grades 6-8, Grades 9-12
This video gives a short overview of the goals, activities and results of the Poincare Institute. We built a program for middle school in service Mathematics teachers that develops mathematical thinking and pedagogical skills for teachers in grades 5-9. The common thread of our three on-line courses is the relationship of the topics in the middle school curriculum with the use of functions and their representation. We also present the results of a few studies that show the progress of participating teachers and their students in standarised tests.
Brian Drayton
Very cool. I am interested in two possible knock-on effects: Are some of your teachers, stimulated by this work, reconceptualizing other math areas that they’re teaching? And are there any effects on teachers in grade levels above or even below your participants’? It seems to me that this change in habits of mind might have systemic impacts at least within a school.
David Carraher
Hi Brian,
Thanks for your comments.
We know that the impact of the Poincaré program on students (via their teachers) is rather large and statistically significant but we don’t yet know which sub-areas of mathematics have benefited most.
The three graduate courses for teachers offered by the Poincaré Institute focus heavily on functions and on the algebraic underpinnings of middle school math. These themes are introduced as a means of uniting a wide range of standard topics and exploring them more deeply. In this broad sense we aim to improve teaching and learning across the middle school mathematics curriculum.
A close examination of our course notes, videos, software and activities for teachers will reveal concerns and aims of a more specific nature. For example, we spend much time considering the transition from counting number to measuring number and to real numbers. As you might imagine, this entails helping students gradually move “onto the real line” and “into the Cartesian plane”. Throughout this evolution, students will need to expand their sense of number from ordinal and cardinal number, to points, and later, transformations that entail processes and such as displacement, dilation, reflection, and shears. They will also be focusing on variables and relations, such as covariation, among variables. All the while, we need to constantly shift back and forth between operations on pure numbers and those involving physical quantities and magnitudes. Graphs become the representatons of functions. Slope comes to be reconceptualized as the rate of change rather than the inclination of a line.
Given that these aims touch on so many topics of middle school mathematics, I’m not sure precisely where to look for differential impact. In any case, where we are aiming is one thing, where we are effectively making a difference can only be determined with the help of additional data.
David (co-PI)
montserrat Teixidor-i-Bigas
Professor of Mathematics
Brian,
Our program takes an algebraic approach to Mathematics. There is not much algebra in the MCAS (MA testing system) for the lower grades, so the program may be having an impact on how the teachers teach a number of topics.
We have only been analyzing test results for the grades that we cover. We hope this will have an effect on the higher grades, as the students move up. I would not expect much if any change in lower grades, as these are taught by different teachers, often in different schools
montserrat
Carolina Milesi
I wonder if your design has changed over time. Will you be able to apply insights from your work with the first three cohorts to the upcoming work with the fourth cohort of teachers?
Analucia Schliemann
Hi Carolina,
Some of the changes we made, based on teachers’ feedback and our evaluation of course materials offered to the first cohort of teachers are described in the following paper: Teixidor-i-Bigas, M., Carraher, D. W. & Schliemann, A. D. (2013). Integrating Disciplinary Perspectives: The Poincaré Institute for Mathematics Education. The Mathematics Enthusiast, 10(3).
Changes were quite substantial from cohort 1 to cohort 2, in terms of mathematical content, presentation of materials, and the pedagogical activities teachers participate in.
Further improvements were implemented in the materials for cohort 3. These will be offered to cohort 4 teachers in grades 6 or 7 to 9, with very minor changes. We will also design a set of adapted materials for teachers in grades 3 to 5 or 6, following a district’s overwhelming interest in expanding the program to elementary school.
Ana
Tamara Moore
I love seeing multiple representations highlighted! How are you researching that aspect of your work?
Furthermore, you described that this is targeted to a few Poincare school districts. How will you continue this work? Are teachers free to choose to participate?
montserrat Teixidor-i-Bigas
Professor of Mathematics
Hi Tamara,
Our external evaluator looked at multiple representations. In a separate study, members of our team looked at the teaching projects designed by the teachers that were part of the coursework and there was a noticeable increase in the number of representations used from the beginning to the end of a semester.
Teacher enrolled through their districts. In some districts there was some administrative pressure to participate but in general it was up to the teachers themselves to volunteer. Because the districts that were pressing participation were also providing a lot of support to the participants, this group was slightly more successful than the un-pressured group.
We are looking on how to continue the work. Funding is of course a big issue once the MSP ends. The course materials are ready to use but the faculty support is essential to the success of the program
montserrat
Dacid Lustick
I really like the ‘foreshaddowing’ aspect of this curriculum. It seems like a sensible approach. While the correlationis modest, how many teachers make up the data set? What were some of the rival hypotheses that might explain the observed gains? Was there any relationship between how well a teacher performed in the online course and the observed impact on student learning?
David Carraher
Hi David L,
Thanks for your questions. Montserrat addressed the first and third ones.
I’ll respond here to your second question.
\
Let’s consider some rival hypotheses for explaining the results, that is, for explaining why the students in the Poincaré districts significantly outperformed students in the comparison districts.
\
Rival hypothesis #1. Perhaps, from the start, the students in the Poincaré districts were stronger in mathematics than students from the matching districts.
Assessment: There is no support for this. Past MCAS performance was one of the criteria for matching districts to the Poincaré districts. Before teachers took part in Poincaré there were no differences in students’ performance levels.
\
Rival hypothesis #2. The students in the Poincaré districts may have been better poised to improve their mathematics performance. For example, the matching districts may have had different demographics, income levels, district size, or % of students on free lunches as compared to the Poincaré districts.
Assessment: Again, this seems not to be the case, since these very criteria was used (in addition to past MCAS performance) for deciding whether a district constituted a good match for a given Poincaré district.
\
Rival hypothesis #3. Perhaps the sample sizes were too small and the observed differences were due to chance factors.
Assessment: This hypothesis can never be definitively ruled out, regardless of the significance levels obtained. However, it is worth mentioning that for each testing year there were roughly 5000 students in the Poincaré districts and 25000 students in the matching districts. (There were 5 matching districts for each Poincaré district.) A Wilcoxin test on the group differences yielded W=144, z=2.68, and two-tailed p = 0.0074. The Spearman Rank correlation between % of Poincaré teachers in a district and performance advantage (over matched districts) was 0.54, with p =0.003. (The corresponding Pearson correlation was 0.577 but we reported the smaller of the two values. Also, the Spearman test seemed more appropriate given the small number of district-grade ordered pairs.)
\
If you can think of any additional rival explanations, please share them with us.
\
At the moment our attention is focused on the challenging issue of determining what features of the Poincaré program may be responsible for the observed differences.
\
Best,
\
David C
Dacid Lustick
How about overall teacher quality or parental support were better for those students in the Poincare program? I am wondering how you can issolate the program as the most likely variable responsbilble for the observed differences. We know how complex the learning environment of a classroom can be. So, variables such as teacher quality (years experience, education, preparation, resources), admin support, community involvement, or parent involvement—-any of which might be influential. I am sure you examined these possibilites.
Your analysis and communication of your analysis is quite wonderful.
Congratulations on such great work.
davod
David Carraher
At first blush it seems reasonable to raise the issue that the overall teacher quality and perhaps parental support were, from the start, better for the students of teachers in the Poincaré program. (We only interacted with the teachers, not their students. It gets confusing because the Poincaré in-service teachers are also students, i.e. grad students.)
\
But then one has to wonder, How is it that each of the 5 MA Poincaré districts moved ahead of the matched districts? Was this by chance? If the students in the Poincaré districts had more support, why were the districts not performing better than peer districts in the past?
\
There is evidence that Poincaré teacher quality improved during and after enrollment in the Institute. We have RTOP observations from our independent evaluator showing very promising (and significant) gains during and 6 mod and 1 year after. (According to RTOP observers, they were more responsive to student reasoning, used more representations for a given problem, for example.) The mean gain for Poincaré teachers was approximately eight/tenths of a standard deviation upward.
\
We’re trying to improve our measures of teachers’ understandings of mathematics. It has taken us several years to be able to clearly identify the topics and issues we have, from the start, implicitly expected teachers to make progress in.
\
In the past, we employed items for which there was, unfortunately, a ceiling effect; too many teachers scored at the highest levels on items. This was not necessarily due to having had extraordinarily gifted teachers. Rather, the items were too easy (quite a few of the items came from our earlier work in Early Algebra much of which covered grades 3-5). We’ll be applying the revised assessment in Fall, 2015. 18 months later we should have a better read of the evolution of Poincaré Teachers’ mathematical and pedagogical expertise.
David C
montserrat Teixidor-i-Bigas
Professor of Mathematics
Hi David,
There were about 60 teachers per cohort and the third third is about to complete the program. Each of our districts was matched to 5 districts similar in terms of size and socioeconomic background. Because we only have aggregate data of MCAS for each grade and district, we are not able to distinguish among the students that were with a particular teacher or even among the students that were with teachers in the program and those that were not.
In our internal evaluations, we do look at the correlation between teacher and student. All of the teachers and students in the district completed pre-assessment and all of the students completed assessments at the end of the academic year. In a study of seventh grade students, we could check that the students of teachers in the program gained a lot more than those that were not. When sorted by teacher ability in the pre-assessment, those students who were with teachers with the lowest ability at the start and in the program made the highest gains as compared with students who were with teachers with the lowest ability at the start and the teachers were not in the program. Students with teachers in the program outperformed those not in the program regardless of the teacher’s initial level
montserrat
David Carraher
Hi again, David L,
\
Below are some additional thoughts on the size of the correlation between the percentage of teachers in a Poincaré district and the size of the performance edge enjoyed by students…
\
For there to be a substantially greater correlation (than 0.54) between the percentage of teachers in a Poincaré district and the size of the performance edge enjoyed by students, there would have to be a uniform Poincaré benefit for students. There is good reason to suppose that this will not be the case. Instead, one expects some degree of individual variation among Poincaré teachers and schools in their impact on their students’ MCAS scores.
\
There is another way to assess the impact of the Poincaré Institute on students’ performance on the MCAS.
When we perform a linear regression on the added value of Poincaré as a function of the percentage of Poincaré teachers in a district, we obtain a regression line that allows us to make the following prediction: if 100% of the math teachers in a district were Poincaré graduates, the added value to the district’s MCAS scores would be just over 18%.
\
Let’s examine what his means.
Suppose a 100% Poincaré district were compared to matched districts before and after its teachers enrolled in Poincaré. Let’s suppose further that 50% of the students in all of these districts were considered to be either Proficient or above in mathematics, according to the MCAS’s official classification. Then, if the matched districts’ performance percent did not change, that is, if it remained at 50%, then the most likely outcome for the Poincaré district would be to have 68% (50% + 18%) of their students classified as proficient or above in the post-intervention assessment.
\
David C
Dacid Lustick
David,
thank you for the additional analysis. I think your project is quite promising and relevant to today’s STEM teachers—especially in MA. The correlation is meanigful, but not necessarily indicative of causality. Do you think that a randomized assignment might be possible some day? Your suggestions of using matched pairs might help to further clarify the effect size of the intervention.
Best of luck,
david
David Carraher
Thanks for your kind words.
\
You are correct about the perils of inferring causality from correlation. I would like to think that our choice of initial districts was not systematically biased, other than the fact that we favored needy districts. But random assignment is a tall order. It assumes that we can determine which districts receive treatment and which do not. (I suppose that is not entirely impossible; we could in principle find 10 willing districts and then randomly select 5 of them and follow up on all 10.)
\
But a special opportunity recently presented itself. We came across a district for which well over half (69) of its math teachers enrolled, from grades 3 to 9. We accepted them all (as we have done in the past). They can serve as their own controls. If we find that the teachers/their students make substantial progress over the 18 months, that will be compelling evidence, I believe, that it is related to the Poincaré program. Now, there will be additional factors at play. For example, we expect the culture of the school to undergo some changes.
\
In Educational research, even in a carefully controlled study, one always needs to go beyond the data to construct a model of causality. It’s easier to construct such a model if we have systematic and trustworthy data. But any model that aspires to generalization also entails theoretical considerations.
Analúcia Schliemann
David L. in the thread of postings above asks:
“How about overall teacher quality or parental support were better for those students in the Poincare program? I am wondering how you can isolate the program as the most likely variable responsible for the observed differences. We know how complex the learning environment of a classroom can be. So, variables such as teacher quality (years experience, education, preparation, resources), admin support, community involvement, or parent involvement—-any of which might be influential. I am sure you examined these possibilities.”
We’ve been discussing the possible role of variables that might be termed methodological contaminants. I’d like to turn the discussion to some of the variables we think are more likely to be responsible for improvements in student performance. We would collectively refer to these as teachers’ deep understanding of mathematical content and of the ways students reason about specific concepts and representations.
Our teachers take three graduate level courses, taught across three semesters. The courses are the result of an intense collaboration between mathematicians, mathematics educators, and physicists, and aim at integrating mathematical and pedagogical content. Special attention is given to multiple representations and to how mathematical reasoning and concepts can emerge from actions and reasoning about worldly phenomena. Most importantly, it adopts a functions approach to mathematics and to algebra that enriches and integrates the existing mathematics content across the grades.
(There are several things that distinguish our approach to functions from other, unsuccessful programs from the past such as the New Math. For the moment, I would highlight the idea of introducing formal representations, e.g. algebraic notation, after students have made some progress in making sense of problem situations based on their existing knowledge, representations, and intuitions.)
Teachers dedicate around 10 hour per week to course activities, reading notes, solving problems, watching videos, working with software apps, interviewing students about their ways of solving problems, designing, implementing, and analyzing videotaped classroom activities, interacting online and engaging in weekly meetings with other teachers and instructors.
Our extensive matching criteria, especially those based on mathematical proficiency, income, and ethnicity, make it reasonable to assume that teachers’ years of experience, education and resources, administrative support, community involvement, and parent involvement would not differ much between the two groups. What did differ were the nature and degree of the teachers’ preparation: Poincaré district teachers had 18 months of extra preparation, of a very unique sort, that was not offered to teachers in the other districts. I am inclined to think that the Poincaré program is the most likely variable (or set of variables) responsible for the differences we found in the MCAS results.
Elizabeth VanderPutten
Nice. thanks for posting
Analúcia Schliemann
Hi Elizabeth,
Thanks for visiting. This was a nice opportunity to describe our work in 3 minutes. And then, in the discussion, we can add more information.
Ana
Elizabeth Barrett
I am a participant in the next cohort that will begin in August. This is a wonderful overview of the program. I am excited to get started and am looking forward to seeing the data added at the end of our cohort to the results.
Analúcia Schliemann
Welcome, Elizabeth, to the fourth cohort of Poincaré teachers. I’m glad you liked the video. We are looking forward to working with nearly all teachers in your district.
Laurie Brennan
This is a very interesting model and design. If the change in teacher learning and student performance continue to be positive, are there plans for further scaling up?
David Carraher
Hi Laurie,
We’d like to scale up, and have been giving it thought for some time, but there are still several pieces of the puzzle to find and put together.
David
Myriam Steinback
Your results are impressive. What a great way to truly give access to all – starting with the teachers. Congratulations. It’ll be interesting to follow those students through their other math courses – I imagine their understanding and knowledge of algebra will help them.
Analúcia Schliemann
Hi Myriam,
We’re also pleased with our results. We won’t be able to follow individual students beyond middle school, as we did for our Early Algebra, Early Mathematics Project, as participants progressed from elementary to middle school. But we’’ll continue collecting district data to find out about possible long term effects of teachers’ participation in Poincare courses.
Further posting is closed as the event has ended.