Microsoft HoloLens: How mixed reality is finding fans in the hard hat gang

In business, mixed and augmented reality makes a lot more sense than virtual reality.
Written by Mary Branscombe, Contributor

So far, the people who are really using mixed reality headsets like Microsoft's Hololens are in industry, which is why the integrated hard hat was such a keenly-awaited accessory.

Aviad Almagor is director of the mixed-reality program at land survey and construction company Trimble, which has been working with Microsoft on Hololens.

Almagor worked with an architect in California who was presenting the redesign of the abandoned Packard plant in Detroit at the Venice Biennale design festival in Italy.

"It's a project that's half a mile long, it's got pretty complex geography; it's in several segments, all of which are complex spaces," said Almagor. "With mixed reality, you can move from the tabletop scale view, which helps you understand the relationships between the different segments in the project, to full immersive view at a one to one scale, so you can physically walk through the design and see how a visitor will experience the design. Moving between the two gave them a great perspective of how the design was progressing."

Using HoloLens let architect Greg Lynn try out ideas and immediately see what impact they would have throughout the giant structure, rather than having to wait months to make those decisions.

Being able to experience a space remotely is also useful for working in harsh environments where you don't want to spend a lot of time on site, like open pit mines.

See also: Executive's guide to the business value of VR and AR (free ebook)

Trimble is working with partners like Caterpillar to put sensors and scanners onto mining equipment to get a detailed 3D view of the mine structure, and the locations of the machinery for monitoring and planning.

"No-one wants to be in the mine. It's far better if you can monitor it from a remote location, that improves safety," said Almagor.

"If you take the 3D geometry and the sensor data and add mixed reality, you get a solution that allows the mining engineer to sit in their office, which might be thousands of miles away from the actual mine and visualise the actual mine almost in real time.

"You can visualise the geometry of the mine at tabletop scale or you can jump to the immersive view and actually be inside the mine. You can see changes in the slopes of the walls as well as the progress of mining activity. You can visualise the machinery almost in real time. You can visualise the blast holes, or see where materials are taken from and where they're stored. The engineering data you require to effectively manage the mine is there in mixed reality."

What Trimble calls a 'materials rainbow' arches from where the material is mined to where it's stored. The mining engineer can also replay the movements of mining equipment to check if operators are following procedures properly, or if there's a more efficient way to do things.

The mining system has similar benefits to those architects are seeing from HoloLens, he says: "It's about visualisation, communications and collaboration -- remotely and co-located."

Almagor also pointed out that building those kind of experiences is getting easier. When Trimble started bringing HoloLens and Sketchup to design and engineering firms, they needed to run multiple pilot programs and offer support services to help them fit it into their workflow.

They will still offer those if required, "but now we have customers purchasing them as a commercial solution," says Almagor. "They buy the software and the HoloLens and they don't need us."

Authoring is getting easier thanks to tools like PTC's ThingWorx Studio MR and the new Simplygon rendering service on Azure. ThingWorx Studio is a web app that lets you build 3D scenes (for HoloLens or mobile AR) by dragging and dropping existing CAD models, connecting them to sensor data and adding gesture and voice controls for interacting with the models. Simplygon can significantly reduce the file size of 3D assets and still keep the fidelity for HoloLens.

Personal and collaborative

KPMG's data and analytics team is using a HoloLens app for data visualisation in the Insights Center in Frankfurt where it presents to groups of corporate executives from its Fortune 500 clients.

The 3D visualisations can show more dimensions of data and make it easier to understand, and the whole group can see them at once. The HoloLens system will come to KPMG Insight Centers in Hong Kong, London, Paris, and New York later this year.

Being able to collaborate is important in consumer scenarios as well, and it's something HoloLens enables rather better than a VR headset, which is more isolating.

"Yes, I can design my kitchen in VR, but I can't interact with the sales associate while I'm doing it. They need the customer to be actively working with them to make design decisions," says Forrester analyst J P Gownder.

That's easier to do when you can see who you're talking to, and not just in a store.

"Anywhere where you have interactivity between multiple workers who are together in a real space, doing 3D visualisation is possible in VR but it's more desirable in mixed reality. There is still something about the social aspect of working together that mixed reality is going to be better because you want to have that social interaction."

Download now: New equipment budget policy

The issue with that kind of 3D design system is getting the 3D assets for all the parts and components you want to show; Gownder mentions an online furniture retailer that carries many thousands of different products but only has high quality 3D designs for some 15,000 of them.

Architects and engineers who are already designing projects in 3D in their CAD systems have an advantage here because they can reuse those assets for mixed and virtual reality advertising. Some high-end condo developers selling units off plan are already using 3D walkthroughs.

Other interesting HoloLens apps are coming. CLAIRITY is testing its augmented reality SmartBinocular air traffic control system at Budapest International Airport, and a HoloLens version called HoloTower is in development.

UK police forces are currently looking at Black Marble's TuServ command and control centre, which lets the officers co-ordinating a major incident view information on multiple, virtual screens, according to the company. Officers in these forces already use the mobile TuServ app to capture incident reports and evidence on phones and tablets and that feeds directly into the command and control interface, alongside maps and other relevant information. "HoloLens is expensive, but a command and control room that you only use a few times a year is more experience," points out Rob Hancock, Black Marble's development director.

Where HoloLens truly shines is in situations where you have no other way of seeing something 3D. Professor Mark Griswold of Case Western Reserve University, which is teaching medical students anatomy with HoloLens, says that despite having spent decades studying MRIs of the human brain "I never truly understood their 3D structure until I saw them in HoloLens."

And one geologist at NASA told Gownder that "he learned more in a week with HoloLens than he had in six months with 2D imagery".

Recent and related coverage

Toshiba's AR glasses skip HoloLens charm in putting Windows to work

The Glass-like eyewear from the venerable laptop pioneer relies on a wearable PC for a range of field tasks, from warehousing to maintenance. It can even start up a video chat for remote assistance. But overlaying fancy graphics requires looking into the future.

Microsoft makes its HoloLens headset available in 29 more countries

HoloLens will now be available in 39 countries worldwide, including Belgium, Denmark, Italy and Spain.

Microsoft's next-generation HoloLens processor will feature an AI coprocessor

Microsoft's next-generation Holographic Processing Unit, currently in development, will add a coprocessor for implementing deep-neural networks.

How Honeywell hopes to train tomorrow's factory workers with an AR/VR simulator based on Microsoft HoloLens (TechRepublic)

A new AR/VR simulator from Honeywell uses Microsoft HoloLens to help train industrial employees and could even aid companies in closing the skills gap.

Ford adds Microsoft HoloLens to development toolbox (CNET)

Mix reality tech will improve global collaboration and cut development times.

Editorial standards