Coming to 1/4 of our semester, we had our quarters walkaround this Wednesday. It is our first presence to the guests and therefore we had prepared 4 different stations for them to get to know our project better.
We separated our space into 4 different parts. Introduction to reinforcement learning/ Unity ML-Agent tech demo/ Design process/ Our 6 ideas.
Introduction to reinforcement learning (RL)
Our guests may have different ideas of what is reinforcement learning or what it is different from machine learning, we want our guests to be on the same page as us. Therefore, we show how reinforcement work by taking suitable action to maximize reward in the situation. We also mentioned some of the recent game example of using reinforcement learning and since most of them are competition game, our goal of the project is to think outside of that area.
Unity ML-Agent tech demo
After introducing the concept of RL, we would like to show a small demo of utilizing the Unity ML-Agent plugin to show that we have the knowledge of setting up the training model and the pipeline.
We trained the board to balance the ball, first we could see the amount of time we spent to train the model will really make a difference of the performance. Also, If we trained at the smaller board and put it at the bigger board environment, the ball tempts to not fall off since it is restricted when it trains. From this, we learned that environment is also an important factor in RL.
Our client after looking at the demo brought up an interesting idea of changing the ball into player and the board is the environment, since the board is trained, the player will have to try to balance onto the board. It helps us to think reversely and to view agent not just as a tangible character but also other abstract form.
Design process
Before going into details of our 6 ideas, we would like to explain a bit of our process and how we get to our 6 ideas.
The board shows the three stage we have been through from discover to defince and to design which is the stage we are at right now. At discovery stage, we tried to figure out the technology, the goal and looking at different genres of games. At define stage, we summarized our multiple game research into 6 kinds of ideas by comparing the feasibility of RL and consulting from the people at EA in their machine learning innovation fair. At design stage, we dig deeper to what specific part of RL is used in the 6 ideas that we had.
Our 6 ideas
We then explanined and showed our 6 ideas with what is the problem we want to solve by using RL and what RL would apply on to them. More specific game play, we have mentioned in week 3 blog!
Some quarters setup:
We had very much enjoyed the Quarters, and it is nice talking to some alumni, EA employee and our clients! We also got valuable feedback from them.
Some feedback we have got:
- Party games seem like an relatively easy direction to start for prototyping, howerver if the agent is not satisfying, the player might blame the game.
- Raising up your kid simulation seems to be a good idea to get player attached to their agent emotionally and the guests mentioned they would love to see them grow up. However, this requires training in real time, which for our case is not an optimal choice.
- Responsive canvas is unique from other ideas, however, it is hard to define the reward and it seems more like an edge detection problem.
- Escaping from the fire scene has an interesting take-turn relationship for the agent, and it would be better to have more scripting to know the story and how RL would apply. Also, Real-life situation is not a good theme – if the agent behaves wired, it’s neither funny nor delightful.
- For Coop puzzle game, most of the guest does not know how RL could give hints for the player, and how to define the reward, however, they are interested in using stickers or visual images to communicate with agents. And building relationship with agent is vague but unique.
We have asked our guests to pick 2 to 3 ideas that they thought is more interesting and most of them have picked the party game and the coop puzzle game. And therefore after quarters, we will look into these two directions and try to design based on these two directions. Since our client has an interesting idea after seeing our demo, we also looked at agent as environment.
