Microsoft turns photo albums into navigable 3D worlds
Representatives from Microsoft Live Labs and a University of Washington work group on Wednesday discussed the Photosynth browser during a speech at the Siggraph show about the university's Photo Tourism system. Live Labs is developing Photosynth from the Photo Tourism technology as part of Microsoft's push into Internet-based personal services.
"Photosynth will be released as a Web client that people can use to explore large collections of photos," said Richard Szeliski, who leads the Interactive Visual Media Group at Microsoft. Photosynth has the potential to create a complete virtual world made up of the world's photos, Szeliski said.
Photosynth lets people import photos, either from their own collection or via a search of a photo-sharing site like Yahoo's Flickr. Drawing from the imported shots, Photosynth organizes the photos into a virtual 3D collage, using an algorithm that measures location information, identifiable features and subsequent 3D points for similarities, differences and viewpoints.
"Unlike (other groups) who have worked on this before, Photosynth combines image-based rendering with search controls. Our scene reconstruction automatically estimates the position, orientation and focal length of the cameras that took the photo. Once the scene is reconstructed, the user can explore using the photo browser features," said Noah Snavely, a member of University of Washington's Charles Simonyi Graphics and Imaging Laboratory, which developed the Photo Tourism technology.
Viewers can choose to see a detail of the constructed photo scene by clicking on an object or dragging a box around a group of objects. The highest-scoring photo for that particular object or view is blown up on the screen. Photos are scored based on the best resolution, lighting and view. Head-on views score higher, as do daytime shots.
Once in detail, viewers can move around and "step back," and they have the option of seeing other images of the same view within the collection. Thumbnails of these alternates run along the bottom of the screen.
The program also renders transitional views from one photo to the next within the 3D space, giving the illusion that viewers are panning the scene with a video camera. Users can also annotate a particular photo, or portion of the scene. Annotations are shared among images based on the level of detail the image covers. For example, a tag describing the Rome's statue of Neptune at the Trevi Fountain would carry over to other views of Neptune but not necessarily panoramic views of the entire Trevi Fountain.
Eventually, the Photosynth team plans to include a time dimension that would let viewers specify the time of day, season or even year they'd like to see with a specific view. The sample shown at Siggraph was a contemporary photo of Yosemite National Park, with an Ansel Adams alternative available for the same vantage point.
Because the tool measures the focal length of each image, Photosynth can determine the exact location the photographer was standing in when she took the photo. 3D overview mode offers a 3D photo scene with these vantage points plotted throughout. Clicking on a vantage point on the rotunda of St. Peter's Basilica brings the viewer to a panoramic view of St. Peter's Square looking out from the rotunda.
According to
For one, the program cannot reliably match photos if they are too dark or of too low a resolution. The program currently lacks the power to create extremely large-scale renderings of the sort the team would like to achieve. For example, a 3D rendering of New York's Times Square is possible, but not one for all of Manhattan.