In Games to Schools project we are studying ways to support learning of mathematics with embedded embodied game interactions. Yesterday I had an opportunity to follow first graders’ music lesson in Las Lomitas Elementary School in Atherton, California. It was a nice surprise that most of the singing and rhythm exercises of the lesson relied so much on embodied cognition approach. The children used different kind of gestures to concretize the pitch, the tempo and the loudness of the music. It seemed that children really liked to combine movements to the music and the movements also helped children to perform better on musical tasks. I hope that we can design as natural and intuitive practices also for mathematics games.

There is a crowing body of research about embodied cognition in mathematics. One of the most studied and debated aspects of embodied cognition is finger-counting practice. Fingers provide an external representation that help children to relieve working memory when counting. It has been argued that finger counting provides ‘the missing tool’ between sensorimotor experience and abstract mathematical concepts. In general, the research has shown that embodied representations of number (magnitude) exist, are not limited to finger-based representations, and influence number processing in such a systematic and functional way that it can be used to foster the efficiency of numerical trainings

Our project is not about finger counting, but focuses on gestures that help children to concretize mental number line. One recent study showed that embodied number line training with whole-body movements (Kinect based UI) resulted in better learning outcomes than control training (mouse based UI). The mental number line is assumed to be spatially oriented from left-to-right, which might account for the findings about a relationship between numbers and spatial information. The results have also revealed that embodied numerical training seems to be beneficial in particular for children with lower general cognitive ability and lower working memory capacity. To conclude embedded embodied cognition approach assumes that seemingly abstract representations may be based on bodily experiences and movement may help children to understand abstract concepts better and the perceptual and interactive richness of games provides opportunities for alleviating cognitive load.

Tilting the tablet controls the character on the number line.

We are developing a Semideus game in which we utilize the ideology of embedded embodied cognition. The gameplay of Semideus is founded on tasks that require working with number lines implemented as walkable platforms of the mountain. We see number line as a powerful user interface to represent and solve numerical problems. In our game the player can control the Semideus character by tilting the tablet device. When tilting the tablet right Semideus walks right on the platform (number line), which in western countries refers to increment of the number magnitude. Whereas when tilting the tablet left Semideus walks left, which refers to decrement of the number magnitude. The approach is quite similar than the one that I saw in the music lesson: the students used vertical movements to concretize the pitch of a new melody. Even the bare gestures included in Semideus’ user interface (tilting right and left) could facilitate the numerical processing, but the rich visualization of the game complements the gestures and helps the player to really concretize the mental number line and alleviate cognitive load of processing (embedded cognition). In other words, the game helps player to imagine how he or she walks on the number line and that way the player can experience the distances on the number line in a concrete way.

Walking on the number line

According to embodied cognition approach concepts are grounded in the re-activation of specific neural patterns in multiple modalities (motor system, visual system etc.). Thus, simply seeing a character walking on the number line can activate motor areas of brains associated with moving (walking). Thus, the processing of numerical concepts with virtual manipulatives like Semideus is more comprehensive than working on static number lines. Of course, the real walking would make the experience even more realistic and activate the motoric processes more deeply. Therefore, we have developed also a user interface in which the player has to really walk with the tablet to control the Semideus character. In this UI version tilting is still required to determine the direction of walking. We are eager to study how the real walking changes the user experience and how it influences on learning outcomes.

To conclude, with the uptake of touch interfaces, accelerometer based systems, and whole body systems the understanding of implications of embodied cognition for interaction design will become increasingly relevant for designers. However, human-computer interaction research has not been adequately informed by understanding of children’s motor, cognitive, perceptual, and social development and thus more research on embodied interaction is needed. And of course we need also good products that lead the way to more effective and meaningful learning. Hopefully, Semideus game will be one of those.