Augmented reality in the operating theatre: How surgeons are using Microsoft's HoloLens to make operations better
While the fact that each person is different from the next may be testament to the infinite variety of humankind, that variety can be a real challenge for surgeons.
AR + VR
While anyone who's thumbed through a biology textbook may be under the impression that all human bodies are wired up the same, the reality is very different: the plumbing of our blood vessels can vary significantly from person to person.
Take reconstructive surgery using what's known as a perforator flap, for example, where surgeons take a chunk of leg with perforating vessels attached to fill in an area of tissue missing elsewhere.
When reconstructive plastic surgeons want to fix up a nasty injury from a road traffic accident, they'll often use tissue from one part of the body to repair the damage in another. The tissue they choose needs to have a good blood supply to ensure it flourishes when it's grafted on to its new location. And that's where those unique variations in layout of our bodies can cause all sorts of problems.
"It's not like the anatomy books," says Matt Ives, consultant plastic reconstructive and plastic surgeon at St Mary's Hospital in the Imperial College Healthcare Trust, tells ZDNet.
Ives and his team, however, now have a new tool to help them find the vessels they're looking for, and trace them exactly as they travel through a patient's leg: Microsoft's HoloLens.
SEE: Executive's guide to the business value of VR and AR (free ebook)
A research project at Imperial is using the mixed reality headset to help plan plastic surgery in patients' lower legs, identifying which tissue and veins can be used in reconstructive operations. Effectively, the system allows surgeons to see inside the leg before they've even picked up their scalpel.
The mixed reality system has been used in a handful of cases at the hospital, picking out the posterior tibial or medial seural vessels to be used in perforating flap reconstruction surgery.
Patients are given the given the standard contrast-enhanced CT scans -- contrast is a material that helps light up blood vessels on imaging to make them easier to spot.
3D reconstruction software is then used to model the leg and the vessels inside, and the data is then fed through to the HoloLens.
Surgeons wearing the headset can see a projection of the blood vessels, bones, and muscles overlaid on the patient's leg, plus the radiologists' markings of which vessels the surgeons need to take. With all that information available in their field of vision, the surgeons know exactly where to make an incision and how deep it should be.
"The good thing about this project is we have surgery, radiology, and technology all in equal measure," says Philip Pratt, research fellow in the department of Surgery & Cancer at Imperial College, who is leading the research project.
SEE MORE ON OUR NHS REPORT: VR, AR and the NHS: How virtual and augmented reality will change healthcare
Traditionally, doppler ultrasound is used to confirm location of blood vessels in a patient's leg before their operation. However, the technique isn't 100 percent reliable for picking out perforating vessels. In some operations, surgeons found the results from the doppler too inaccurate, and would prefer to skip it in favour of investigating its location once an incision has been made in the leg.
Due to their tricky nature, perforator flap surgeries "are sometimes much maligned in other units", said Ives, but they "work very well for us, because our information is way better than what we were dealing with two years ago or five years ago. Now people are getting much more bespoke, much more elegant solutions because we have much more information," says Ives.
"We're saving the patient a lot of anaesthesia time, and in a busy unit like this where we are seeing five or six open tibias in a week sometimes, we can get two patients done on a list [the surgeries scheduled for a day] where before we'd only get one done," he adds.
That time saving is not only useful for the hospital, it's good news for the patient too. "Pre-HoloLens, it could be a time-consuming process to find these vessels, and a lot of these procedures could be very long anyway -- 10 or 12 hours typically -- and all that time the patient's under anaesthesia. Anything we can do to reduce that time is of benefit to the patient," Pratt says.
SEE: Special report: How to automate the enterprise (free ebook)
Having proved that the HoloLens system could work in a handful of cases, other surgical departments are also looking into how it could be deployed in their specialties: it's already been used in an operation to remove colorectal cancer, and research is ongoing into using it for plotting the location of burr holes through which bleeds on the brain could be removed.
"There's a wealth of things we can apply the initial prototype to," Pratt says.
There are a number of issues that the researchers are hoping to address before the system can be deployed more widely, like shortening the time between the scans being taken and the images appearing on surgeons' HoloLens -- currently around an hour.
"When the patient has a scan, you want to put your HoloLens on and you want to start planning or operating immediately -- you want to reduce the amount of time it takes someone in between to prepare the data," said Pratt.
Similarly, in order to achieve the sub-millimetre accuracy that certain operations need, the kit may require an additional tracking system to the Kinect-based one found in the HoloLens.
Interoperability between the HoloLens and the hospital's own software will also need to be improved. "It's definitely the ambition for it to become a standard tool but there are barriers to making that happen. They're quite simple to solve, but pretty tedious to do. The minute you want to connect a device to a hospital network, you have to jump through a lot of hoops. The ambition is for it to be a standard tool and talk to the hospital directly," Pratt said.
And, as you'd expect from any new technology that could potentially require NHS funding, Imperial's HoloLens system will need a bit more bedding in before it can be used more broadly.
"I think we need to demonstrate the benefit," said Ives. Used in everyday practice, the system should ultimately cut the time it takes to mark up patients and perform the surgery. However, HoloLens' novelty means it's taking currently a little more time. "Now, it takes longer because when you use it everyone goes 'wow, can I have a go?' and it goes around the whole [operating] theatre."
Surgeons are using Microsoft's mixed-reality headset to "see inside" patients before they operate.
RECENT AND RELATED COVERAGE
Smart watches, fitness trackers and the NHS: Are wearables just what the doctor ordered?
While more and more of us are wearing fitness trackers, the real benefits of wearables may come from another quarter.
Why the NHS is killing paper records to save lives
The NHS still relies on paper for many patient records. Getting rid of it could free up time, and money.
Robots and the NHS: How automation will change surgery and patient care
The rise of robots is inevitable in healthcare, but for now, keeping it simple is just what the doctor ordered.
VR, AR and the NHS: How virtual and augmented reality will change healthcare
Against a background of growing enterprise adoption of virtual reality, mixed reality and similar technologies are beginning to gain a foothold in the NHS.
AI and the NHS: How artificial intelligence will change everything for patients and doctors
The rise of artificial intelligence is set to reshape the health sector as we know it, from back office to doctor's office.
IoT and the NHS: Why the Internet of Things will create a healthcare revolution
The Internet of Things must be one of the most high-profile technology trends of the last five years. Could IoT be the backbone of the NHS of the future?
NHS and technology: Making the case for innovation
Physician, reboot thyself! The health service is caught between its creaky past and a shiny future. But change is needed, and fast.
How human microchip implants could lead to the 'democratization of healthcare' via IoT (TechRepublic)
A Swedish biohacker has created RFID microchips that can be implanted in someone's hand. He believes the innovation could lead to a revolution in the IoT health industry.
CES 2018 gets serious about health, wellness and medical tech (CNET)
With the slew of self-care, fitness and sleep devices on show at CES, the health and medical industries are making big rumbles in consumer tech.