13180 Views (as of 05/2023)
  1. Amy Shelton
  2. Professor, Associate Dean for Research and Doctoral Programs
  3. Presenter’s NSFRESOURCECENTERS
  4. Johns Hopkins University
  1. Cathryn Cortesa
  2. Postdoctoral Fellow
  3. Presenter’s NSFRESOURCECENTERS
  4. Johns Hopkins University
  1. Gregory Hager
  2. http://www.cs.jhu.edu/~hager
  3. Professor
  4. Presenter’s NSFRESOURCECENTERS
  5. Johns Hopkins University
  1. Jonathan Jones
  2. Presenter’s NSFRESOURCECENTERS
  3. Johns Hopkins University
  1. Sanjeev Khudanpur
  2. https://www.clsp.jhu.edu/~sanjeev
  3. Associate Professor
  4. Presenter’s NSFRESOURCECENTERS
  5. Johns Hopkins University
  1. Barbara Landau
  2. Dick and Lydia Todd Professor
  3. Presenter’s NSFRESOURCECENTERS
  4. Johns Hopkins University
Facilitators’
Choice
Public
Choice

Characterizing and improving children's block-building skills: Interdisciplin...

NSF Awards: 1561278

2017 (see original presentation & discussion)

Informal / multi-age

Spatial construction—creating novel spatial arrangements or copying existing ones—is a hallmark of human spatial cognition. Spatial construction appears early in development, predicts later spatial and mathematical skills, and is used throughout life. This project focuses on block building, an accessible and adaptable skill for young children in both formal and informal learning contexts. We use the block-building process as a window to understand how this complex spatial skill develops, how it is linked to academic learning more generally, and how it can be nurtured, moving children from "novice" to "expert" builders. More specifically, we use traditional cognitive methods along with machine learning techniques such as computer vision and inertial measurement units embedded in the blocks to characterize in detail the process of children’s block building and its development. This innovative approach not only provides a better understanding of early spatial skills but also has the potential to change the conceptual framework through
which we understand block building by establishing novel metrics for performance. We aim to characterize the detailed processes of block
building among children of different ages and levels of experience, ranging from gifted/talented children to children with spatial impairments, broadening the potential implications for education. This project will also work toward generating scalable tools that can be used by scientists and educators to characterize, analyze, and promote development of block-building skills.

This video has had approximately 1,167 visits by 819 visitors from 222 unique locations. It has been played 771 times as of 05/2023.
Click to See Activity Worldwide
Map reflects activity with this presentation from the 2017 STEM for All Video Showcase: Research and Design for Impact website, as well as the STEM For All Multiplex website.
Based on periodically updated Google Analytics data. This is intended to show usage trends but may not capture all activity from every visitor.
show more
Discussion from the 2017 STEM for All Video Showcase (23 posts)
  • Icon for: Cathryn Cortesa

    Cathryn Cortesa

    Co-Presenter
    Postdoctoral Fellow
    May 15, 2017 | 08:40 a.m.

    Welcome! We are excited to share this first stage of our research about how children building with blocks provides rich insight into their early spatial skills. We hope to have a rich discussion with comments and questions from parents, teachers, and fellow researchers. We think that the skills we are studying are widely applicable to other tasks (chess, for example) and development in STEM. We want to hear from you: what other domains might our task relate to? What elements or skills should we be considering when measuring how novices build compared to experts?

    Ask our team how we measure the ways children build with blocks, the different skills we think our task is measuring, or how this technology might be useful for classrooms and parents in the future!

  • Icon for: Gladys Krause

    Gladys Krause

    Facilitator
    Research Associate
    May 15, 2017 | 09:52 a.m.

    I would like to hear more about the skills you are measuring, how you measure them, and the age group of the children in the study. -- Thank you!!

  • Icon for: Cathryn Cortesa

    Cathryn Cortesa

    Co-Presenter
    Postdoctoral Fellow
    May 15, 2017 | 11:35 a.m.

    Great question!

    Currently, we are studying children aged 4-8 years old. Our preliminary results show that there seems to be significant differences in the ways in which younger children construct their block copies compared to older children. At the same time, there are some young 4-year-olds in our study whose construction patterns look strikingly adult-like, and some older children who struggle. We think that there are very interesting questions we can address regarding how individual differences in this spatial construction task relate to their abilities on other types of tasks. In particular, we hypothesized that spatial skills such as mental rotation or pattern matching may be particularly relevant to the block copying task.

    Our study includes several additional measures to assess children’s math abilities, pattern matching abilities, approximate number system, mental rotation abilities, and verbal and non-verbal reasoning and knowledge. Some of the standardized measures we use include the pattern construction subtest of the Differential Ability Scales (DAS), the Kaufman Brief Intelligence Test (KBIT), and the Wide Ranging Achievement Test (WRAT).

  • Icon for: Wendy Smith

    Wendy Smith

    Facilitator
    Associate Director
    May 15, 2017 | 07:23 p.m.

    Very interesting project! How close are you to having computers that can "watch" and assess children building blocks? What kinds of strategies have you developed that seem to be effective with students who struggle more with spatial awareness? Is there a learning progression or trajectory for spatial awareness that teachers can use to help accelerate students' skills?

     
    1
    Discussion is closed. Upvoting is no longer available

    Amy Shelton
  • Icon for: Amy Shelton

    Amy Shelton

    Lead Presenter
    Professor, Associate Dean for Research and Doctoral Programs
    May 16, 2017 | 12:21 p.m.

    Great questions!  Our current set up is not yet automated for coding the data that is coming from the video and accelerometers, but we are collecting the necessary data to get us there.  Stay tuned!  One of my engineering colleagues may wish to jump in and elaborate.

    With respect to what to do with children who struggle, this is a major part of our ongoing research question.  We know that experience is critical, so the key may be to offer experiences that stem from the child's interests.  In one of our other ongoing studies, we are trying to essentially sneak spatial skills into an activity that does not necessarily feel spatial on the surface to see if it will engage students who might otherwise be intimidated by what they perceive as a task that will be "too hard for me".

    In another line of work under the JHU Science of Learning Institute, my colleagues and I are actively exploring ways to bring spatial skills into the elementary curriculum.  That work is aimed at both taking advantage of student skills and embedding activities in curriculum that will help to build those skills.

  • Icon for: Gregory Hager

    Gregory Hager

    Co-Presenter
    Professor
    May 16, 2017 | 09:19 p.m.

    Just to add a bit more on the ability of computers to  "watch and assess."  Our blocks have accelerometers built in, and they are viewed by a special camera that returns both video and range data. With these three modalities, and based on our past experience with video tracking of objects, we're confident we'll be able to track the blocks and determine their placement automatically. The challenge is whether the system can be made reliable enough to be fully automatic, or if it will still need a human coder to check its work.  As Amy says, stay tuned; we hope to have an answer within the next few months.

     
    1
    Discussion is closed. Upvoting is no longer available

    Amy Shelton
  • Icon for: Christopher Whitmer

    Christopher Whitmer

    Researcher
    May 16, 2017 | 05:38 p.m.

    Interesting project, are you only looking at physical block building or have you looked at virtual environments or games where the same skills are used?

    Best,

    Chris Whitmer

     
    1
    Discussion is closed. Upvoting is no longer available

    Amy Shelton
  • Icon for: Amy Shelton

    Amy Shelton

    Lead Presenter
    Professor, Associate Dean for Research and Doctoral Programs
    May 16, 2017 | 06:32 p.m.

    The current project is physical block building, but we have plans to test out the relationship between real and virtual block building using a gaming format.  Some of the challenges are exciting to consider--type of interface, how much should we incorporate true physical constraints, are there order effects, etc.  We do collect information about the children's experiences with games and other types of play.  It will be interesting to see similarities and differences with real and virtual activities, especially as a function of age and digital experience.  Do you have thoughts or insights about what we might expect to see?

  • Ben Reynolds

    May 18, 2017 | 10:54 a.m.

    Virtual building sounds like Minecraft to me.

  • Icon for: Amy Shelton

    Amy Shelton

    Lead Presenter
    Professor, Associate Dean for Research and Doctoral Programs
    May 18, 2017 | 11:29 a.m.

    Yes.  The interesting thing about standard Minecraft is that it partially preserves real physics in some ways but not others. For example, you have to build things on top of other things, but you can remove blocks to leave items hanging in the air.  It's also quirky in that you can't move things around; you can only add or remove (usually by smashing!).  This opens the door for asking a lot of questions about these kinds of features and how they affect and/or interact with a child's (or adult's) cognitive ability.  We see a lot to be learned from the virtual space.

  • Icon for: Miriam Sherin

    Miriam Sherin

    Facilitator
    Professor, Associate Dean of Teacher Education
    May 16, 2017 | 10:51 p.m.

    What an interesting project! Really exciting to hear about the development of automated techniques for examining students' approaches to the blocks task. I will definitely be interested to see how the project develops. I'm wondering about the blocks task - I understand the connection to spatial reasoning but am wondering about the process of copying an existing collection of blocks and what aspects of spatial reasoning it does and does not tap into. Are there any task modifications that you are considering exploring in the future?

     
    1
    Discussion is closed. Upvoting is no longer available

    Amy Shelton
  • Icon for: Amy Shelton

    Amy Shelton

    Lead Presenter
    Professor, Associate Dean for Research and Doctoral Programs
    May 16, 2017 | 11:10 p.m.

    Your question is definitely something with which we have wrestled.  Even in a simple copying task, there is a lot happening.  Children need to be able to parse the model, match parts, consider the orientations of the blocks and the model itself.  We started with the simplest version because we recognized just how complex it would be.  Despite this apparent simplicity, the degree of individual differences we observe is quite remarkable.  There are children for whom this seemingly simple task is quite hard.  Part of our project is to try to understand which spatial skills, as well as other cognitive skills, have the greatest influence on performing this task.

    That said, there are manipulations that would allow us to target more specific spatial skills. For example, what would happen if we gave them more blocks than they needed? Determining which blocks are necessary is part of the parsing process that we have constrained. In the current version, they might still have to choose when to use a green square versus a green rectangle, but they currently know that all blocks will be used. Similarly, how much would the individual differences spread out if we did not allow the children (or adults) to rotate the model during the task?  We already have some evidence for a relationship between mental rotation ability and these kinds of building tasks, so one could imagine those relationships being even more dramatic.

    We have a whole host of specific manipulations underway or that we hope to try as we keep this project going, but I would also love to hear other people's thoughts on what could be learned through variations on our tasks.

  • Icon for: Miriam Sherin

    Miriam Sherin

    Facilitator
    Professor, Associate Dean of Teacher Education
    May 16, 2017 | 11:19 p.m.

    Super interesting! So much to think about! Thanks for sharing these different ideas that you have been considering

  • Icon for: Barbara Landau

    Barbara Landau

    Co-Presenter
    Dick and Lydia Todd Professor
    May 17, 2017 | 09:03 a.m.

    Just a few more thoughts on what the block task requires:  We tend to think of it as a spatial task-- requiring the child to understand the spatial relationships among the blocks, the global structure of the entire construction, etc..  But it is actually much more.  For example, the task also requires use of memory-- the child must look at the model and then move her attention to the copy space and replicate part of the model, and then do this repeatedly, each time, taking in a part of the model and replicating it.  The fact that children (and adults) tend to carry out copy tasks like this by duplicating small parts and then returning to the model to get more information also tells us that so-called executive functions are involved-- the child must know when to look back at the model to get more information, and must also understand that he/she should check back to the model periodically, to make sure that the on-going copy is still correct.  So the task is both highly complex, and also taps into many cognitive processes.  It seems likely that this is part of the reason that block building (especially this copy task) is a good predictor of later academic achievement.

     
    1
    Discussion is closed. Upvoting is no longer available

    Amy Shelton
  • Icon for: Miriam Sherin

    Miriam Sherin

    Facilitator
    Professor, Associate Dean of Teacher Education
    May 17, 2017 | 12:26 p.m.

    Yes! this makes sense. Perhaps your analysis can tap into some of these different aspects of what it means to work on the task by looking at the range of processes participants engage in.

     
    1
    Discussion is closed. Upvoting is no longer available

    Amy Shelton
  • Icon for: Cathryn Cortesa

    Cathryn Cortesa

    Co-Presenter
    Postdoctoral Fellow
    May 17, 2017 | 05:54 p.m.

    Miriam, yes, we think that by describing in great detail the processes the children engage while building, that we will better be able to understand the underlying cognitive processes that are engaged during this task. In fact, we are conceptualizing the building process as a "path" that the builder traverses as they add blocks, and perhaps backtracks when they remove blocks or fix a mistake. By drawing a path of each builder's trajectory, we can observe places where most builders behave similarly (for example, we have observed that many participants start by building the base of the structure). In addition, we can observe places where certain builders diverge from the most common path, perhaps because they place a block incorrectly, or perhaps they use an unconventional method of putting the blocks together, like building from the top down.  Do you think there are other ways we could conceptualize the processes participants engage in while they build?

     
    1
    Discussion is closed. Upvoting is no longer available

    Amy Shelton
  • Icon for: Miriam Sherin

    Miriam Sherin

    Facilitator
    Professor, Associate Dean of Teacher Education
    May 17, 2017 | 05:59 p.m.

    great, it will be very interesting to see what kinds of paths emerge. I wonder if the paths will suggest different conceptualizations of the task - as a set of smaller tasks vs. one large task, for example. I'm excited to see what you learn!

     
    1
    Discussion is closed. Upvoting is no longer available

    Amy Shelton
  • Icon for: Selma Sabanovic

    Selma Sabanovic

    Researcher
    May 18, 2017 | 12:16 a.m.

    You have a very informative, well done and clear video, and an exciting project! This is a question particularly for Gregory Hager, perhaps. Are you considering using the information gleaned from observing, coding and analyzing these block tasks done by children to provide machines with the capability of performing similar cognitive tasks like parsing, copying, etc. It seems like this might be a good way to give robots the ability to do similar actions. Thank you for your video and good luck with your future work! 

  • Icon for: Barbara Landau

    Barbara Landau

    Co-Presenter
    Dick and Lydia Todd Professor
    May 18, 2017 | 09:47 a.m.

    Very interesting question.  This project actually has its roots in a project that Greg Hager and colleagues have been working on to characterize surgeons' ability to use a robotic surgery device.  This is still one step away from having a robot do the surgery, but it asks questions about how a skilled surgeon learns to carry out the actions that control the robotic device.  The idea of using what we learn to inform the development of robotic "skills" themselves is something we have definitely discussed.  As one of the cognitive scientists on the team, I think we are most interested in how to characterize what human cognitive abilities are required to carry out complex tasks that involve using actions in the service of a specific goal (like block construction, among many other tasks).  But that understanding could certainly inform our ability to create robots who carry out complex action tasks "like humans".  Of course, we may not actually want to create robots who embody all of the cognitive skills -- and limitations-- that humans have; we may want robots that do the task in a more efficient way, etc.  But here again, insights from human cognition can help us define what the limitations are in these tasks. As one example, humans have to deal with limited attention and memory, so the most efficient way to build a block model might be to separate it into conceptual parts, and assemble each sub-part one at a time.  With no memory limitations (as might be true of a robot), the most efficient path of construction could be quite different.  The bottom line here is that insights from human cognition can inform robotics, and robotics can help us understand human cognition.  That's why we need to collaborate across disciplines!

     
    1
    Discussion is closed. Upvoting is no longer available

    Amy Shelton
  • Icon for: Gregory Hager

    Gregory Hager

    Co-Presenter
    Professor
    May 18, 2017 | 12:59 p.m.

    Just as an addendum to Barbara's response, from a computational point of view, as Barbara says, we see the opportunity to learn "skills" from people as a path toward developing robots that are capable of functioning as teachable assistants. The blocks project is a beautiful sandbox to play with this problem, marrying both the science of understanding human cognition, and the analytics needed to transfer human capability to a robot. In fact, we have "played" with these blocks using a robot in the lab; but its very hard to program a robot to actually do the physical assembly -- that type of fine dexterity and force-based manipulation is very difficult. It would be a fun exercise to see if the robot could learn how to attach blocks to each other from the data we have. Going from basic assembly to parsing and copy is actually the simpler step, believe it or not.  Thanks for your question!

     
    1
    Discussion is closed. Upvoting is no longer available

    Amy Shelton
  • Icon for: Selma Sabanovic

    Selma Sabanovic

    Researcher
    May 19, 2017 | 03:45 p.m.

    Thank you Gregory and Barbara for your detailed answers. You had a wonderful project, and I'm sure it will lead to many exciting extensions.

  • Icon for: Kathy Kennedy

    Kathy Kennedy

    PISA2 Program Manager
    May 22, 2017 | 08:35 a.m.

    Block building seems to be a task that most children have an interest in when they are younger but which wanes as time goes on. Children who retain an interest in clock building are often thought of as having an inclination for STEM fields. Will there be any examination of how to maintain interest in tasks such as those which you are working with?

     
    1
    Discussion is closed. Upvoting is no longer available

    Amy Shelton
  • Icon for: Amy Shelton

    Amy Shelton

    Lead Presenter
    Professor, Associate Dean for Research and Doctoral Programs
    May 22, 2017 | 09:04 a.m.

    Very interesting insight.  In our experience talking with parents and pre-k teachers, there are a lot of children who show very little interest in blocks, but I can definitely imagine that among those who do show an interest, not all continue to engage in these kinds of activities.  Finding ways to foster and maintain an interest in skills that benefit academic performance has been part of our motivation, but I like your suggestion that we also need to be thinking about when it is also important to try to maintain that interest.  I have not mulled over this yet, but I think the key is keeping the interest long enough for learners to achieve 2 goals: (1) To develop the minimum level of competency in some areas that benefit from these skills--I think of this is the need for all learners to be well-rounded and informed consumers of information; and (2) To discover the areas that they do want to pursue more deeply.

    One might counter the idea of maintaining interest by saying that not all kids should maintain an interest in STEM, and I would agree with that at the career level.  However, I do think that many young learners lose interest in some areas too early and for the wrong reasons (the way we teach it, lack of relevance to their world, social factors/identity issues) limiting their chance to discover an interest at a higher level.  I have definitely thought about these issues in some of my work, but we could definitely dig deeper and by more thoughtful about it.

     I would love to hear any ideas you have about maintaining interest and what data we might need (from our work but also more broadly) that would help us get there.   Thanks for the provocative question!

  • Further posting is closed as the event has ended.