We held our first playtest On October 10 at Mountainview Elementary School with around 65 students from grades 3 – 5.
The main questions we wanted to address were:
- Did players take up the objectives we intended?
- Did players understand our mechanics?
- Did players retain information from the game?
To examine this, we collected data through observation, a post-test survey, and an API we developed to track certain events in-game.
We asked players before and after playing to write down animals they thought would live in the arctic tundra to gauge if players remembered animals they encountered in the game
Out of 57 responses only 8 students mentioned wolves and 2 students mentioned caribou before playing. However, after playing 46 students mentioned wolves and 25 students mentioned caribou, indicating that our format had some success for content retention immediately after playing. In future playtests when we have more animals, we plan to evaluate this again, assess if this retention lasts over time, and explore if other content in the game sticks with players.
When asked about what they felt they were supposed to do, 55% of responses included explicit goals we tried to directly communicate to the player in-game, such as finding animals, and 48% included implicit goals we hoped the player would take up, such as collecting data and investigating.
When we asked, “Did you feel you understood how to play?” 78% of players said responded positively, but this data did not reflect what we observed. While many players grasped the basics, there were a significant number that seemed unclear on some interactions, especially when beginning play. Additionally, we tracked in-game the time it took players to complete the first mission we provided, and while many of the players completed it near our projected time (5 minutes) there were quite a few players that took a while longer or who didn’t complete it at all (although these numbers were affected by teachers playing and some other factors so they may not be reliable). Regardless, observation and interviews following the playtest indicated that confusion over the interface or mechanics was one of the main sources of frustration, along with the perceived movement speed, and feeling lost.
To summarize our takeaways, we feel we are going in the right direction for communicating our desired content, intonating objectives to players, and giving them exciting moments. However, our results indicated we need to iterate on our usability, and refine our mechanics to give players more fun and less frustration