Heavy Metal

Our client Caterpillar Inc, is developing remote and autonomous vehicle control systems for their heavy equipment.  All of that means that they are developing remote control bulldozers.  They have already developed a system where an operator can control a vehicle that is in line of sight of the operator.  However their eventual goal is to allow an operator to control a vehicle that is potentially hundreds of miles away from their location.

There one specific issue that arises with remote control bulldozers. Once you take the operator of a bulldozer from the cabin of the bulldozer itself, how do you provide that operator the necessary situational awareness of what his bulldozer, the terrain, and what he is doing to that terrain.

This is where we come in.  The obvious solution to this problem is to provide a real-time visualization of what the bulldozer is doing to the terrain.  Caterpillar has come to the ETC to see what techniques and technologies used by the video game industry might help them enhance and augment the system they already have.

To that end we are providing 3 deliverables to Caterpillar:

• A Real-time Terrain Deformation Prototype
• A redesign of their User Interface
• A cost benefit analysis of various game engines

So why is Caterpillar developing remote control bulldozers? There are actually 3 reasons why they are doing this:

First and foremost is safety.

As you can see, these vehicles sometimes operate in hazardous and dangerous environments.  Removing the operator from the vehicle means they don’t have to be in these dangerous environments to do their work.

Second, Caterpillar hopes to increase the productivity of the operators.  There are many advantages to having an operator control a vehicle from a remote location.  They don’t have to drive nearly as far to get to a far-flung worksite, they don’t have to take as many mandatory breaks and the working conditions are much more comfortable.

* Potential Setup for a central office control terminal as seen at Bauma 2010.

Lastly, as a more long term-goal, Caterpillar wants to allow operators to run more than one vehicle at time.  This means that the vehicles themselves would be semi-autonomous and the operator would intervene when there are more complex or difficult tasks to perform.  This would make the operator more efficient and allow more work to be completed overall.

* These vehicles are all being controlled by individual operators

So one of our main goals this semester is to do real-time terrain deformation, but what is terrain deformation?

A simple explanation of terrain deformation is moving the terrain up and down within our virtual environment.  It is however, more complex than that.  In most virtual environments and games, terrain is rendered using a technique called height maps.

A height map is a grayscale image where the white values of the image would be higher elevations of the terrain and the black values would be the lower elevations of the terrain.  The computer would take that image and apply it to a flat plane.  Then it would extrude or stretch the terrain from that flat plane based on the information in the height map image.  Below is an image of a 2D height map image and the terrain that is generated from that height map.

* the height map is on the left the resulting terrain on the right.  The bright white area on the height map is the plateau you see in the middle of the terrain.

As I said before deformation is moving terrain up and down within a virtual environment.  The way we are doing this is changing the height map to reflect what the new terrain should look like.  Essentially we are ‘painting’ the new terrain onto the height map and then updating the 3D model based on the changed height map.

How does a bulldozer deform terrain?

Terrain deformation for this project is not enough.  We have to deform it the way a bulldozer would change terrain in the real world. A bulldozer actually does this in very specific ways.  There are two major categories of terrain deformation that we will be dealing with.

First is positional deformation.  This means that we will deform the terrain based on the position of the bulldozer and its blade.  This may seem obvious, but we need to take into account a number of factors.

If we know that the bulldozer is below the level of our virtual terrain then we need to deform the terrain to match that level.

* terrain lowered to meet bulldozer, shown in blue

If we know that the bulldozer is above the level of our terrain then we need to raise the level of the terrain to bring it to the bottom of the bulldozer.

* terrain moved up to meet bulldozer, shown in blue

If the bulldozer moves along and encounters some dirt, we need to push that dirt in front of the bulldozer.  If we are pushing dirt and go over a portion of the terrain that is lower than our bulldozer, then we need to fill that portion with the dirt.

* blade moves across the terrain and levels it as it goes along, removed material shown in red, deposited material shown in blue

The second type of deformation is what we call approximate or predictive deformation.  Because the bulldozer can’t give us exact information on how much dirt it is pushing or carrying, we are approximating the amount of material that is being moved based on information like the volume the blade can hold.

This particular bulldozer can lift and carry material so we need to approximate how much it can carry and then raise the terrain when it drops the material from the bucket.

*dropped material shown in blue

As the bulldozer moves along the terrain and has the maximum capacity of material in the blade, it will leave a falloff or wake of dirt to sides of the bulldozer.  We will approximate that information as well.

* top down view of bulldozer, dirt being displaced to the side shown in blue

All illustrations drawn by Ashley Johnson