We created a physical set-up for our LEGO interactive experience using a miniature projector and a Kinect 2 camera sensor hanging from a makeshift stand. We are looking into options for creating a final installation setup, focusing on local fabrication specialists that can help create our custom table and mount structure.
For the narrative component, one notable piece of feedback centered on the conga-line of dogs, with preferences for a variety of CMU-related characters and references instead of just a single one. We are revisiting the story of our experience as from playtesting we realized that the context helps players to understand what they should build, alleviating that initial ‘blank canvas’ hesitation.
Art Progress
Since we had to expand out from just Scotty as a CMU-specific feature, our artists started looking into more CMU icons we could use, such as the famous buggy race that CMU holds! Here’s some of the assets created for that!
Through testing using our quick structure we realized that we would need a much brighter projector in order for our content to stay visible in ambient room lighting conditions. We noticed that while the UI elements showing the detected LEGO blocks were easy to read on the LEGO base board, digital assets with finer details like the dog character would not display easily.
Additionally, a lot of faculty commented that if the Scotty dog is white, it isn’t Scotty. So we have gone back to the black design and thinking more about how to make this version of the dog readable and clear in the top-down, projected display. Our 3D artist went back and re-textured the model to make it clearer and more visible, despite the dark fur.
Tech Progress
On the tech side, as we merged the color detection and depth detection to work together, we realised we had a very serious memory leak in our program. This was largely due to us using a C++ plugin (OpenCV) in our C# environment. C# has automatic garbage collection, so we didn’t pay attention to releasing unused memory. However, it turns out we had to still mind the C# wrapper for the plugin. We’re working on fixing this serious issue (our program runs for barely 5 minutes before crashing the computer because of using too much RAM). Additionally, our frame rate is incredibly slow right now due to the depth correction being a per-pixel bilinear interpolation operation being calculated on the CPU. We are thinking of moving the depth correction to the GPU as this will also help with our memory leak.
