Evaluating the effectiveness of a game-based rational number training

Study 1: Recently, we published a new open access article in Computers & Education journal in which we studied the effectiveness of the beta version of the Semideus School rational number game. In the study, Ninety-five fourth graders were assigned to either a game-based training group (n = 54) who played a digital Semideus School game for five 30-minutes sessions or a control group (n = 41) who attended regular math curriculum. Conceptual rational number knowledge was assessed in a pre- and posttest session. Additionally, the game groups’ playing behavior was evaluated. Results indicated that the game-based training group improved their conceptual rational number knowledge significantly more strongly than the control group. In particular, improvement of the game-based training group was driven by significant performance increases in number magnitude estimation and ordering tasks. Moreover, results revealed that in-game metrics, such as overall game performance and maximum level achieved provided valid information about students’ conceptual rational number knowledge at posttest. Therefore, results of the current study not only suggest that aspects of conceptual rational number knowledge can be improved by a game-based training but also that in-game metrics provide crucial indicators for learning.

Kiili, K., Moeller, K., & Ninaus, M. (2018). Evaluating the effectiveness of a game-based rational number training – In-game metrics as learning indicators, Computers & Education, Volume 120, 2018, 13-28. https://doi.org/10.1016/j.compedu.2018.01.012

Study 2: Based on the study 1 we made some changes in the game and conducted a new study. We were particularly interested in evaluating the effectiveness of a new feedback mechanism implemented into comparison tasks. The new version, the official Semideus School  game, provided feedback of compared magnitudes also as bar charts or number line visualizations to trigger reflective processes on compared magnitudes and to foster the development of rational number understanding. We expected that this new visual feedback channel would facilitate learning gains in comparison tasks. 70 Finnish 5th graders played the game approximately 2.5 hours. As we expected, a paired t-test showed that students’ comparison performance was significantly higher in posttest (M=92.60%, SE=1.20) than in the pretest (M=76.97, SE=2.54), t(69)=6.855, p<.001, Cohen’s d=.94, indicating that the new implementation of our comparison task worked well.

Posted in Game-based learning, Math games, Publications

Games to Schools: Project Postmortem

As the year changed to 2018, TUT Game Lab’s Games to Schools -project funded by Academy of Finland was finished. The primary purpose of that project was to study the learning effectiveness of digital math games, identify design principles for game-based learning solutions as well as to explore ways to implement learning analytics and physical activities to math games. Next I will shortly highlight some outcomes of the project.


Can we use games for assessment purposes?

One of the main outcomes of the project is the Semideus School math game that we have used in several research contexts – the game played a significant role in 13 scientific publications and opened up fruitful collaboration with international partners. During the project we studied the usefulness of several different user interface solutions and ways to map educational theories to the game mechanics. The most challenging aspect of our project was to validate the Semideus School game with respect to learning assessment. After several iterations we managed to validate the game as an assessment tool. Our results indicated that hallmark effects of rational number magnitude processing typically observed in basic numerical cognition research, such as the numerical distance effect and whole number bias, were successfully replicated in game-based assessment context. More importantly, the results revealed that game-based assessment lowered test anxiety and increased engagement which is likely to decrease assessment bias caused by test anxiety. In addition, the results showed that earlier playing experience and gender did not influence the game-based test score suggesting fairness of the game-based assessment approach. In line with this, over 80% of the students reported that they would like to complete a game-based math test instead of a traditional paper-based math test. To conclude, game-based assessment seems very promising educational approach that can be used in both formative and summative assessment contexts.

Are game-based learning solutions useful?

After the validation phase we conducted several studies to investigate the learning effectiveness of the Semideus game. As a summary, the studies showed that players’ rational number understanding increased significantly even in informal competitive contexts. In addition to significant learning gains we managed to collect great impact stories. For example, one student told us that after the game competition he got his very first 9 from a math test. In fact, the two large scale math game competitions were success stories. It is very promising that intrinsically motivating factors such as enjoyment and perceived learning gains rather than success in the competition predicted students’ willingness to participate in math game competitions again. Most of the teachers also appreciated the math games and the game competitions. The interviews revealed that especially the learning analytics dashboard increased the usefulness of the games a lot.

Can we utilize flow theory in game-based learning?

As a part of the project we conducted also a systematic literature review about flow experience in game-based learning. The review revealed that the prior literature has focused mainly on conceptual considerations about flow in game-based learning, but no robust empirical evidence about the meaning of flow in game-based learning exists. Our project contributed to this research gap and flow research in several ways. First, the results indicated that flow had a significant positive relation with the game performance. This is to say, that it is possible to increase learning outcomes by designing games that facilitate flow experience. Second, our results indicated that technology acceptance measures such as perceived usefulness and perceived ease of use predicted both flow experience (quality of the playing experience) and learning success. Third, we created a flow framework for analyzing the quality of educational games and demonstrated how flow theory can be used as a design framework.

What next?

Although the official funding period of the Games to Schools –project ended we still continue to report results of the project. For, example we have just collected large data set for evaluating the usefulness of three different user interface solutions of the Semideus game from which one is physically activating. Moreover, we still have two ongoing projects that will utilize math games that are based on the Semideus School game. Our overall aim of the year 2018 is to distribute our math games to daily school practices at large scale and collect huge data sets at the same time. The math game competitions that we organized in 2017 facilitated the distribution of the games, but the challenge is to find even better distribution methods. Maybe, community based challenges might work for us. We will see.



Tagged with:
Posted in Game-based learning, Math games

Assessment and Training with a Game Competition

We conducted a study to evaluate the usefulness of a math game competition for engaging, assessing, and training rational numbers in schools. In particular, we investigated whether playing a digital number line based Semideus game would improve students’ magnitude understanding of rational numbers and whether playing behavior of a competition can be used for assessment purposes. Finnish fourth (n = 59) and sixth-graders (n = 105) participated in a math game competition relying on intra-classroom cooperation and inter-classroom competition. Students were allowed to play the game as much as they wanted during a three-week period in order to improve their teams’ (classes) rank in the competition and improve their rational number knowledge at the same time. Participants were able to check the status of the competition online. Results indicated that students benefited significantly from participating in the math game competition with respect to rational number knowledge. In particular, the Semideus game was effective with students who started with lower rational number knowledge. Overall, the study demonstrated that math game competitions seem to be a useful approach to assess and support the development of students’ math knowledge.

Screen Shot 2017-04-03 at 15.25.51

Furthermore, one aim of the study was to test several new mechanics – skills that can be activated with in-game currency. Some of the skills made the task easier, for example by removing enemies or time pressure while others provided direct assistance to the magnitude estimation itself. The skills seemed to work well and engaged the students.

More detailed results of the study will be published in scientific publications during 2017.

Posted in Uncategorized

The Semideus School is now available

The Semideus School is now available on the App Store for iPad! We are looking forward to make research collaboration with schools.


The Semideus School game is a really fun way to learn fractions, decimals, and percentages. By playing Semideus, learners improve their ability to perceive and estimate rational numbers in multiple forms. The adventure happens in Mount Olympos. The task of the player, Semideus, is to find the gold coins that Gobolas the Goblin has stolen from Zeus. On the way to Mount Olympos, Semideus has to chase the goblin, seek the coins, avoid traps, and crush math based enemies.

– Rational number magnitude estimation, comparing, and ordering
– Numerator / Denominator notation
– Proper fractions, improper fractions, mixed numbers, decimal numbers, and percentages
– Equivalence
– Density of rational numbers
– Adding, subtracting, expanding, and reducing fractions

Attention! Semideus School is targeted for our research collaborators (a limited number of teachers and researchers) that conduct research in schools. In order for a student to play the Semideus School game, a teacher/researcher must register to the Semideus School service and obtain game access codes for students.

Visit support site to get more information about obtaining game access codes to students:

Posted in App Store, Math games

Embedded embodied cognition in math games

In Games to Schools project we are studying ways to support learning of mathematics with embedded embodied game interactions. Yesterday I had an opportunity to follow first graders’ music lesson in Las Lomitas Elementary School in Atherton, California. It was a nice surprise that most of the singing and rhythm exercises of the lesson relied so much on embodied cognition approach. The children used different kind of gestures to concretize the pitch, the tempo and the loudness of the music. It seemed that children really liked to combine movements to the music and the movements also helped children to perform better on musical tasks. I hope that we can design as natural and intuitive practices also for mathematics games.

There is a crowing body of research about embodied cognition in mathematics. One of the most studied and debated aspects of embodied cognition is finger-counting practice. Fingers provide an external representation that help children to relieve working memory when counting. It has been argued that finger counting provides ‘the missing tool’ between sensorimotor experience and abstract mathematical concepts. In general, the research has shown that embodied representations of number (magnitude) exist, are not limited to finger-based representations, and influence number processing in such a systematic and functional way that it can be used to foster the efficiency of numerical trainings

Our project is not about finger counting, but focuses on gestures that help children to concretize mental number line. One recent study showed that embodied number line training with whole-body movements (Kinect based UI) resulted in better learning outcomes than control training (mouse based UI). The mental number line is assumed to be spatially oriented from left-to-right, which might account for the findings about a relationship between numbers and spatial information. The results have also revealed that embodied numerical training seems to be beneficial in particular for children with lower general cognitive ability and lower working memory capacity. To conclude embedded embodied cognition approach assumes that seemingly abstract representations may be based on bodily experiences and movement may help children to understand abstract concepts better and the perceptual and interactive richness of games provides opportunities for alleviating cognitive load.

Semideus tilting UI

Tilting the tablet controls the character on the number line.

We are developing a Semideus game in which we utilize the ideology of embedded embodied cognition. The gameplay of Semideus is founded on tasks that require working with number lines implemented as walkable platforms of the mountain. We see number line as a powerful user interface to represent and solve numerical problems. In our game the player can control the Semideus character by tilting the tablet device. When tilting the tablet right Semideus walks right on the platform (number line), which in western countries refers to increment of the number magnitude. Whereas when tilting the tablet left Semideus walks left, which refers to decrement of the number magnitude. The approach is quite similar than the one that I saw in the music lesson: the students used vertical movements to concretize the pitch of a new melody. Even the bare gestures included in Semideus’ user interface (tilting right and left) could facilitate the numerical processing, but the rich visualization of the game complements the gestures and helps the player to really concretize the mental number line and alleviate cognitive load of processing (embedded cognition). In other words, the game helps player to imagine how he or she walks on the number line and that way the player can experience the distances on the number line in a concrete way.

Walking on number line

Walking on the number line

According to embodied cognition approach concepts are grounded in the re-activation of specific neural patterns in multiple modalities (motor system, visual system etc.). Thus, simply seeing a character walking on the number line can activate motor areas of brains associated with moving (walking). Thus, the processing of numerical concepts with virtual manipulatives like Semideus is more comprehensive than working on static number lines. Of course, the real walking would make the experience even more realistic and activate the motoric processes more deeply. Therefore, we have developed also a user interface in which the player has to really walk with the tablet to control the Semideus character. In this UI version tilting is still required to determine the direction of walking. We are eager to study how the real walking changes the user experience and how it influences on learning outcomes.

To conclude, with the uptake of touch interfaces, accelerometer based systems, and whole body systems the understanding of implications of embodied cognition for interaction design will become increasingly relevant for designers. However, human-computer interaction research has not been adequately informed by understanding of children’s motor, cognitive, perceptual, and social development and thus more research on embodied interaction is needed. And of course we need also good products that lead the way to more effective and meaningful learning. Hopefully, Semideus game will be one of those.

Tagged with: , , , ,
Posted in Math games

Published book: Finnish Innovations and Technologies in Schools: A Guide towards New Ecosystems of Learning

Our article “Learning by Creating Educational Exergames: Creative Pedagogy: That Gets Students Moving” is included in the Finnish Innovations and Technologies in Schools: A Guide towards New Ecosystems of Learning book.

The article proposes a new pedagogical approach, learning by creating educational exergames, which entails user-generated content and gets children moving during school hours. In the article we report results of a pilot study in which primary school students created educational exergames for their peers. Students created games with Magos Lite game creation platform that we have developed in MAGICAL project (Making Games in Collaboration for Learning, http://www.magical-project.net/). Check out the book for more information about the approach and the pilot.

Kiili, K. Tuomi, P., Koskela, M. & Earp, J. (2014). Learning by Creating Educational Exergames: Creative Pedagogy That Gets Students Moving. In Niemi, H., Multisilta, J., Lipponen, L. & Vivitsou, M. (Eds.). Finnish Innovations and Technologies in Schools: A Guide Towards New Ecosystems of Learning. (pp. 87-96) Rotterdam: Sense Publishers. www.sensepublishers.com/catalogs/bookseries/other-books/finnish-innovations-and-technologies-in-schools/

Finnish Innovations and Technologies in Schools

Tagged with: , , , , ,
Posted in Publications

Rational Number Knowledge: The gatekeeper of higher mathematics

The acquisition of mathematical skills is crucial for today’s society. It has been argued that in individual level, insufficient mathematical competencies may be even more harmful to career prospects than reading or spelling deficiencies. This is to say, that in society level mathematical deficiencies can lead to immense costs. In fact, OECD has argued that improvements in arithmetic skills influence positively on economic growth.

Research has shown that the extent to which elementary school students master rational numbers is a strong predictor of future success in mathematics. However, there is a great deal of evidence that understanding of rational numbers is very difficult for children and that even after considerable mathematics instruction many children fail to perform adequately even in simple rational number tasks.

Mathematics education researchers have admitted that most of the students’ difficulties with rational numbers can be attributed to inadequate instruction. The problem is that the recent advances in modeling numerical development have not been exploited to practices of teachers and the instruction tends to emphasize procedural instead of conceptual knowledge.

Thus, one of my current interests is to design games for supporting the development of rational number conceptual knowledge. In the near future I will devote much time to design a Semideus rational number game that is based on magnitude estimation approach on number lines.  In forthcoming series of posts I will consider the theoretical foundation of the game and report also results about the effectiveness of the game. In meanwhile see Semideus tab for more information.

Tagged with: , , , ,
Posted in Math games