MIT creates video you can prod - it even lets Pokemon realistically bounce off bushes
An imaging technique created by researchers at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) could find applications in business and even pave the way for a more realistic Pokemon Go experience.
The imaging technique allows viewers to push and prod objects in a video and cause them to react as they would in a real-world interaction.
MIT researchers believe the technique will have useful applications in major engineering projects, such as bridge construction, but could also offer a boost to augmented reality games and special effects in movies.
The appeal of the technique partly lies in potentially offering a cheaper way to simulate objects than by building 3D models. The other advancement on existing imaging techniques is that it can be used to create realistic simulations with just five seconds of video.
The technique requires video footage captured with a standard camera and an algorithm that analyzes 'vibration modes' -- the ways an object responds to vibrations at different frequencies. Using this data, the researchers can predict how objects will move in new situations.
CSAIL PhD student Abe Davis, who is publishing the work this month, demonstrated the modeling method with a bush moving under a breeze, and playground equipment that was kicked. Using data from these samples, he simulated the effect of Pokemon Go characters bouncing off the objects.
"If you want to model how an object behaves and responds to different forces, we show that you can observe the object respond to existing forces and assume that it will respond in a consistent way to new ones," Davis said.
The technique could also encourage augmented- and virtual-reality developers to think of the simulation in a different way.
"When you look at VR companies like Oculus, they are often simulating virtual objects in real spaces," Davis said. "This sort of work turns that on its head, allowing us to see how far we can go in terms of capturing and manipulating real objects in virtual space."