So What Do Virtual Reality Games Have to Offer Computer Science Education?

By Miriam Harries, Producer and Designer for Team Wonderland

We’ve spent this semester attempting to answer this question. My team and I have researched, rapid-prototyped, and playtested, and we’ve noticed some interesting things. I wouldn’t say that we found THE answer, but we did find some promising potential, as well as some surprises.

I’ll be honest here, at the beginning of the semester I didn’t actually think virtual reality had much to offer computer science education. In my research I was seeing how VR was already being used in other areas of education, and it was always focused on the real, the tangible. Virtual field trips, lecture halls, and simulations. I couldn’t find anything that dealt with the abstract; all the examples I found were trying to make the inaccessible, accessible rather than trying to make the intangible, tangible. This wasn’t terribly encouraging, since we were trying to take abstract concepts that don’t necessarily correlate to anything in the real world and help beginners understand them.

As a side note, we did find some examples of coding in virtual reality. There have been a few attempts to put traditional development environments into VR. Unfortunately they are, well, pretty awful. Almost every single one is a text editor on a plane in a pretty VR environment, and they all rely either on touch-typing or on having a camera on the headset and rendering that image in the virtual environment, at which point the whole thing starts to feel pretty pointless. Ultimately, they were largely useless to us except as confirmation that that was not the problem we wanted to solve this semester and that no one had yet figured out how coding and VR go together on the screen instead of behind it.

Since other examples of educational VR and coding in VR weren’t really helping us, we started taking inspiration from existing computer science education games. Our client is all about “head fake” learning, where students deal with and come to understand major concepts without necessarily being aware that they’re learning something. In computer science, this typically means hiding the syntax behind a fun activity. In the Alice software (which our client developed), it takes the form of having kids create animations with blocks that correspond to code but don’t really look like code.  There are plenty of other games out there where you have to write or select particular pieces of code in order to progress. One game that is particularly popular with my team is Manufactoria, in which the player learns basic binary logic by correctly processing input strings (this is a terrible description but the game is really cool).

Somewhat to our surprise, the designs that we based on those existing games were kind of disasters. We didn’t develop a single one. I think this was for three major reasons:

1. We were putting a fancy VR covering on techniques that were well established, and our client wanted something new.
2. These games were built for, and work well in, 2D. We had assumed they would somewhat naturally transition to 3D without our doing a lot of extra design work, which is just not true.
3. Most of those games didn’t really help people understand anything beyond the very basic levels of computer science. Programming a little character to move in a certain direction or fight something is great for sequence, conditionals, loops, and not much else. Maybe functions, in the higher levels.

After struggling to find a virtual reality experience or game that would teach something about computer science, we made what I think was the most crucial decision of the entire semester – and the one that finally put us on the right track. We decided as a team that the question we were trying to answer was not, in fact, “how can we teach computer science in virtual reality?” as we had been assuming. Rather, it was “how can we illustrate abstract computer science concepts in virtual reality?”

This seems like a subtle, largely semantic distinction, but it turned out to be crucial. When we were trying to answer the first question, we were starting with the method of getting information across or the framing device. So we were coming up with things like “following a recipe to make a dish correctly” and “create and put together components of a robot that will go fight mutant plants” and “let’s literally just put this existing computer science game in VR.”

Whereas when we started trying to answer the second question, we started thinking about the essential qualities of the concept we wanted to convey and how best to show them in an easily understandable way. We started coming up with things like “let’s have the player act as a pointer and physically interact with this data structure” and “what if the player literally acts out this algorithm” and “let’s make something with distinct elements a teacher could refer back to during a lesson about this.” See the difference?

By the end of the semester, we had six prototypes. The concepts we chose to illustrate were Classes and Objects, Functions, Modular Programming, Linked Lists, Binary Search, and Parameter Passing. The first two topics were chosen partly at random and partly because we thought they would be well suited to being visually demonstrated in virtual reality. The other four were chosen based on feedback from teachers who answered a survey about what they found difficult to teach in computer science.

As we finished the prototypes, we started showing them to local computer science and technology teachers to get feedback on their potential educational value. Their responses were encouraging, enlightening, and surprising. Many teachers, especially younger ones, were quite excited about our work and the potential of experiences like these. Most teachers said they would like to use these in their classes and all of them thought their students would love to try them out. Some teachers could picture exactly how these experiences would fit into their curriculum, while others thought they would be best suited to a maker space environment where students could use them on their own time. Several teachers were really only interested in these experiences if they either encapsulated a lesson or came with one already designed. Overall, however, we got the impression that even though not all teachers would want these, there are plenty out there who would.

Later in the semester we also showed these prototypes to grade school students. Due to the circumstances of our playtests these ended up being mostly middle school students with a few high school students mixed in. Their reactions varied, but nearly all of them were at least excited to try out virtual reality and told us that they would be interested in taking classes that included experiences like these. We found that older students tended to be more aware of the patterns in what they were doing, regardless of whether or not they had any computer science experience. Younger students could generally start to understand the experience itself, but only rarely started to notice the patterns and systems behind it. However, according to the teachers we talked to these experiences could still be a way to introduce more advanced concepts earlier.

A quick summary of things we’ve discovered:

1. Don’t start with teaching, start with illustrating the concept in an interactive way. The teaching will come later, largely on its own.
2. The more fundamentally an experience is connected to its concept, the more effective it will be and the more potential it will have to be extended and include additional detail.
3. People get way more out of exploratory experiences than procedural ones. Having room to explore and experiment with an idea can be quite powerful.
4. These are best suited to high school and up as far as pure teaching tools go, but middle school students like them to and they could function as early exposure to some concepts.
5. These sorts of experiences could ABSOLUTELY be useful in higher level programming courses where key concepts become more abstract and harder to visualize in 2D.
6. Lots of computer science teachers lack formal computer science training, and for many of those teachers these sorts of tools are only interesting if they come with or are pre-made lessons.
7. At the same time, there are also lots of teachers who would love incorporate these sorts of experiences into their curriculum.

Some of the most interesting feedback that we’ve gotten is that the teaching methods that these experiences open up are ones teachers have already been trying to use. A few of our prototypes involve players enacting an algorithm themselves, which is already a reasonably common practice but is often constrained by physical logistics of materials or number of students. Generally, one student gets to work through the algorithm while the others watch. Our prototypes may be the first version of something that each student could do for themselves without needing other students or materials to act out the rest of the data. One teacher even told us “I didn’t know it before, but this is what I was looking for.”

So what can virtual reality games offer computer science education? I think they primarily offer two connected things:

1. A way to visualize abstract concepts that don’t make much sense in two dimensions.
2. A way for students to explore, experiment with, and discover these concepts for themselves at the fundamental conceptual level, completely separately from the syntax involved (to be reconnected later).

Lots of teachers are already finding ways to give their students access to these two opportunities, but virtual reality experiences are not subject to traditional physical constraints like gravity and materials (aside from the VR equipment, of course, which will probably remain a significant hurdle for a while), and could therefore be a powerful tool for teachers to add to their educational tool kits.