MIT researchers develop new technique that allows you to reach in and touch objects in videos

MIT researchers develop new technique that allows you to reach in and touch objects in videos

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed an imaging technology called Interactive Dynamic Video (IDV) that allows you to reach in, push, pull, and poke and prod objects in videos.

The technology works with the use of traditional cameras and algorithms. Using these mediums, IDV looks at the almost invisible vibrations of an object to come up with video simulations that can be used by users to virtually interact with.

Researcher Abe Davis said that by making videos interactive, they can predict how objects will respond to unknown forces and come up with new ways to be hooked with videos. According to Davis, IDV has many uses, including filmmakers can use it to produce new types of visual effects and architects can use the technology to find out if buildings are structurally solid.

IDV has the potential to make virtual objects like Pokemon to interact with their environments in realistic ways. In order to simulate the objects, the research team has assessed video clips to look for vibration modes at different level of frequencies.

Each frequency tracks ways that an object can move. Researchers need to identify these modes’ shapes and then they can predict how these objects will move in new situations. In the study, Davis has used IDV on videos of different objects, including a bridge and jungle gym.

He was able to push and pull the image, bend and move it in different directions. Davis said, “If you want to model how an object behaves and responds to different forces, we show that you can observe the object respond to existing forces and assume that it will respond in a consistent way to new ones”.

Category: