Microsoft Research is all about machine intelligence. There’s Cortana, of course, which is powered by
Bing. There’s
Project Oxford that delves into all sorts of image and sound identification possibilities. And now we have Microsoft Research working with Carnegie Mellon University to
teach machines to more deeply analyze images and act much more human.
The current work builds on previous efforts to automatically caption images, which involved a system that can recognize the elements in a scene and provide meaning by captioning an image in the same way as a human might do. That’s only a first step, however, because a caption does not provide context for how to act within the context of what a scene involves.
The new system goes further, combining computer vision, deep learning, and language understanding to identify the elements of a scene and understand relationships between them. One example given is a system mounted on a bicycle that continuously evaluates the surroundings and asks questions that could be pertinent to a human rider.
