How do you push the boundaries of simulation in terms of what is possible visually?
We learn a lot from the game industry. There are entire classes of problems — like optimizations in modeling complex things — that are big problems in robotics, but which are solved problems in high-end video game development.
The game industry has done a remarkable job in getting a lot of mileage out of a computer in terms of simulating complex systems. A great example is how video games model the physics of this world on the graphics card. They create sophisticated visualization of complex virtual environments that are extremely engineering-heavy tasks to implement, and they do this all on the graphics card — specifically on the GPUs. The game industry lives on the graphics card and the graphics card industry lives on the game business. That connection has been so strong for so many years. In the last few years, graphics cards — or GPUs — have become more relevant in autonomy and robotics because the hardware is becoming smaller and more affordable. This has enabled the robotics industry to take advantage of the parallel computing capabilities of GPUs which are more accessible from a software development perspective.
Why haven’t the game and robotics industries partnered on simulation development in the past?
There is an enormous culture gap that emerges in really obvious ways. For instance, in the game industry there are people called technical artists. Technical artists are a key element of game design because they are the bridge between the engineers and the artists. In video games there are the visual assets which artists make, and then there are the engineers who put the game mechanics together and add physics, such as gravity. To create the game, the static asset that the artist made has to come to life in the environment which the engineer has created, and that’s the job of the technical artist. It’s a big job. None of these games would be what they are if there weren’t these technically savvy, creative bridge-makers called technical artists. For their part, however, many roboticists are unfamiliar with the role of a technical artist — the role hasn’t previously existed in their realm. This absence is what leads to robotics simulation engines looking as they have for the past five years. If you have a technical artist, on the other hand, these environments start to look more photo-realistic, more dynamic, full of detail and nuance. Technical artists and their contribution is a good example of this culture gap.
Another example is that in the game industry speed is everything. If a player is constantly waiting for the game to load, then the game is not going to sell. So the game industry uses many sophisticated mechanisms and skill sets around squeezing one more frame out of a computer. Few people in academia have mastered those skills in the way that the folks in the game industry have. And it’s a shame because those mechanisms can be quite clever, mathematically intense tricks. In academia, the engineering solution to the problem would be to simply buy a bigger computer. But in the game industry that’s not possible because whoever bought the game already has a computer and that’s where the game has to run. So that world of optimization is huge and robotics is going to benefit a lot from it.
