Tangible Programming

Digital Dream Lab is using a system termed tangible programming. A set of physical puzzle blocks is used with an interactive table to create objects in the virtual space, and alter their attributes within our virtual world!

The Table

The table on the right is the interactive surface that allows the blocks to connect to the virtual world. It houses all of the necessary equipment for the exhibit, except for the projector. We have cut windows in the front wall to help demystify the magic behind the experience. The windows let the children (and parents) explore the equipment inside, and see what the web camera sees! We have also set up a monitor to show the ReacTIVision engine.

The Blocks

The picture to the left shows all of the basic blocks that accompany our interactive table. The object block is the key, and allows children to place a grey scale object anywhere in the virtual world. Where they place the block on the table corresponds with where the character shows up in the virtual world.

The other basic blocks serve as modifiers to the character in the virtual world. The color block adds a color texture to the character, and the colors in the physical and virtual worlds match. Changing the color simply requires rotating the block 90 degrees. The size block scales the character up from its original size, and functions as a dial. Instead of working in 90 degree increments, the size block scales fluidly as it is turned. The last basic modifier is the action block. This block only requires simple placement in its correct position, and there are three to choose from.

 

 

 

The variable tray is the next step in layering programming concepts into the exhibit. It functions as a template that surrounds the construction created from the four basic blocks. Characters placed within the variable tray may be copied through the use of matching tokens. Changes within the tray will also cause changes in all of the copies. Each token can also be moved and rotated in the same way that the basic object block can.

 

 

 

 

 

How did we do that?

The link between the physical world and the virtual world uses a standard web camera(eyes). The camera is looking at the surface of the table and watching for the fiducial markers on our blocks. The fiducial markers are images that the computer understands, and is part of a computer vision system. We are using ReacTIVision, which is a vision system that takes the images from the camera and translates them to a set of numbers. ReacTIVision has a library of images that correspond to a number, and each of our blocks has a different image on it.

So now we have a system where each block generates a number for the computer to use! What’s next?

The computer needs to use this information in a useful way. We have decided to use these numbers to help create and alter a virtual world. A virtual world is a place where anything can happen, and it utilizes a game engine to make it run. We are running a game engine called Unity 3D, which allows us to bind actions in the virtual world to numbers collected from the physical world.

There is one other thing that we need to make the experience work, and that is a link between ReacTIVision and Unity 3D. The link is called a library, which is a set of compiled functions that translates information from one system into information that can be used by the next. We are using the Uniducial library written at MIT to accomplish this task.

How to learn more?

What’s the best way to extend the learning you started at the museum? How about trying one of these platforms with your child:

• Scratch: Scratch is a free visual programming environment that was a huge inspiration in our project, and functions in much the same way! The environment allows you to create, share, and remix interactive stories, art, or even games!
• Lego MINDSTORMS: Lego MINDSTORMS is a great extension of the familiar brick building foundation. It gives a brain to the building blocks, and lets kids start to learn logic and programming concepts through a physical interface. They also provide a visual programming interface to learn and prototype with.