DTOX Semester Post-Mortem
I. Overview
DTOX is a semester-long project at Carnegie Mellon’s Entertainment Technology Center in collaboration with Games for Change (G4C hereafter). The team includes:
- Jamie Cui – Programmer / Artist
- Derrick Pemberton Jr. – Designer / Co-Producer / Writer
- Parker Ramsey – Co-Producer / Programmer
- Stefani Taskas – Programmer
- Sherry Zhang – Designer / Programmer
- Brenda Harger – Faculty Advisor
- Dave Culyba – Faculty Advisor
- Raul Carvajal – Client / G4C Festival Producer
The team was tasked to create a live interactive experience to be showcased at the Games for Change festival in NYC summer 2020. The transformational experience we’ve created explores online toxicity along with potential solutions to foster positive online communities. The experience was initially supposed to be presented to an in-person audience of 40-80 people at the festival but due to social distancing the festival, and this project, shifted to a remote experience.
Online toxicity is a growing concern for our ever more “online” society. This includes toxicity prevalent in social media and games. The DTOX team chose to focus on just toxicity in gaming communities. There are a couple reasons for this decision. The first is that most of the cutting edge research on online toxicity is happening at the world’s largest game devs like Riot Games and Blizzard Entertainment. Social media companies don’t publish much research on the subject so the team decided to focus on what is available. The second reason is gaming toxicity was more personally interesting to the team. The team spent the first month or so conducting research and developing the following mission statement:
Encourage a greater understanding of online toxicity and healthier community moderation practices.
With this mission statement guiding us we designed what ultimately became an interactive lecture where players took on the role of online community moderators. The experience uses a simulation of online gaming communities that the players moderate. Throughout the experience, players will review reports and decide appropriate punishments, as well as unlock some of the latest methods or tools used in the industry today. By the end of the experience, the audience should have a better understanding of the struggles that online moderators face, the belief that healthy online communities are possible, and the understanding that robust moderation systems are worth the investment. Put a different way, the transformations we wished to see in the audience are shown in Fig A.
II. What Went Well
Scope
As mentioned in the overview, the team decided to only focus on online gaming toxicity because of the greater amount of research on that topic as compared to social media toxicity. The team spent a considerable amount of time developing a base of research. This solid foundation, also including the mission statement, guided the team through the semester. By halves, the team had solidified the structure of the experience as an alternating cycle of gameplay and reflection. The team also started developing the simulation that players interact with in the gameplay segments allowing for even more iterations on that aspect of the experience even before the COVID hit.
Iteration
After the team made the initial design changes, we pushed to playtest the experience as soon as we could. The team eventually held six playtests where we ran the experience live for around 20 concurrent viewers on average. While most of these playtests were focused on general usability, game flow, and tech. We also conducted a stress test specifically aiming to check our networking capabilities as that was one of the hurdles we most often encountered.
Client Relationship
We worked closely with our client, Raul throughout the project. Raul has remained supportive of our decisions before and after the pivot. We developed a delightful working relationship through our weekly meetings. Raul has expressed his pleasure with the outcome of the project and how excited he is to present the experience this summer.
III. What Went Wrong (aside from *gestures broadly*)
Networking
We discovered many networking problems through our playtests. Though we were able to fix most of them, there is still inherent spookiness with the systems we implemented. In retrospect, it would’ve been prudent for us to research alternative systems a bit more.
Playtesting Scale
We were never able to playtest with as many players as we expect on the day of the festival. We saw no errors on our last stress test but that was only an attendance of 37 players, about half of our anticipated audience. To prepare for this we implemented a couple elements like tools that help us control the outcomes of the simulation if things start to go awry. We have also improved our networking capabilities by subscribing to a premium service for our NGROK hosting service.
IV. Lessons Learned
Lay a Solid Foundation
The solid foundation we laid with our research, mission statement, transformations, and concept held strong through a huge pivot. We were more easily able to replace features made obsolete by this pivot like the physical room ecosystem and replace it with our twitch integration. This experience absolutely taught us the importance of thorough pre-production.
Embrace Agile
When the world changed and the team had to pivot, this solid structure held strong as we changed out elements that became obsolete in a remote presentation of the experience. For example, in our early live tests we encountered errors when only 20 players attempted to connect to our game at the same time, shown here at the bottom right. This problem was only discovered through live testing, and, because we were regularly running tests, we were able to improve our solutions over time. It goes to show you that agile is especially useful in developmental cycles like ours–cycles filled with many unknown problems that can only be discovered by frequent playtesting and iteration.
Remote Connection
After the world changed, we found ourselves feeling a little bit like this picture.
We were still closely working with our teammates, but the physical presence is missing. Losing focus, motivation, or team cohesion is much easier during remote work, so we had to find ways to keep the team connected and communicating effectively. We started using a task journal channel in our team slack. In this thread we posted status updates on our tasks. Oftentimes these would spin off into juicy and helpful discussions. We also met on video calls daily for scrum meetings. Seeing each other alive and moving helped us feel less isolated. We often started leaving these calls running as we were working with no pressure to have cameras on. This effectively simulated the feeling of being in the same room together where we could quickly call out for a question or overhear relevant conversations.
V. Conclusion
The team will test the experience on G4C’s chosen platform, hopin.to, in the weeks following finals. This gives us two months until the virtual Games for Change Festival in July where we will run the experience.
Recent Comments