Virtual Surgery, 3-D Security Cameras, and Other Glimpses of Our Augmented Future
(Photo: University of Maryland)
When virtual-reality headsets aren’t being stigmatized as playthings for game addicts and porn hounds, they’re being made to look weird and impractical.
(Remember that notorious Time cover featuring Oculus founder Palmer Luckey jumping for virtual joy? It did nothing to help VR’s image problem. And the PR pitch I just got about innovations in adult video virtual-reality — I swear I’m not making that up — won’t either.)
But — as I learned at a recent open house at the University of Maryland’s “Augmentarium” — the ability to immerse yourself in a realistic recreation of a real or imagined scene by strapping on a headset like an Oculus Rift could, in fact, have some serious and practical applications
UMD set up this facility on its College Park campus (with funding from the National Science Foundation and graphics-chipmaker Nvidia) to experiment with productive uses for virtual reality and its cousin augmented reality (in which a projection of data is overlaid on your current view of the real world).
Amitabh Varshney, director of the University’s Institute for Advanced Computer Studies, led off by noting how increasingly cheap sensors, cameras, headsets and processing power could make VR and AR everyday experiences.
“We can now, at a very low cost, deliver these virtual experiences to everyone,” he said.
And with the right software, it can also bring insight and maybe order to some complicated situations.
Augmented security
One example began with the security cameras right outside the facility. They provide video feeds to a room on UMD’s campus, where security officers must monitor dozens of screens for signs of suspicious behavior.
Varshey explained that you could instead fuse these video feeds and then overlay them on 3-D models of structures. The result: “We can make buildings transparent,” allowing officers to track a suspicious person continuously instead of waiting for him to disappear from one screen and then appear on another.
However, I will admit that Varshey lost me with his next statement: “Imagine a near future where instead of these cameras [being] fixed to buildings, they are attached to moving drones.”
(Of course, we had no idea at the time that, not long after the open house ended, there’d be yet another horrible incident in which suspicious people with guns were not caught in time.)
Hands-free health care
We also got some hints about how VR could improve health care.
Part of that, Baltimore-based surgeon Sarah Murthi explained, means improving conventional health care by freeing doctors’ hands and getting them away from tablet or laptop computers: “Most of our time is interacting with the electronic medical record, not actually looking at the patient.”
Yes, this was a pitch for Google Glass-style interactivity, providing a heads-up display of patient information during office visits and medical procedures. But I get the appeal of having hands-free access to charts and other patient data; studies have already found that surgeons can do better work this way.
(Photo: Philips.com)
The other part way VR could help medicine is by improving doctor training by letting them experience surgery as vividly as possible without actually crowding into an operating room. The shaky 3D video we watched — shot at Murthi’s hospital with GoPro cameras — was a decent proof of concept. But the ultimate goal would be to fuse video from hundreds of cameras into an interactive environment trainees could explore via VR. (Presumably, no drones would be required.)
3-D audio
Three-dimensional audio could be also a part of those immersive environments, to judge from a couple of demos I had of technology developed by a UMD spinoff named VisiSonics.
One part of their system is a round ball stuffed with five video cameras and 64 microphones — sort of like an Imperial interrogation droid from Star Wars, except mounted on a post and absent the scary needles — that creates a 360-degree model of a setting’s sights and sounds. You can then browse that model on a computer. One early customer: auto manufacturers looking to find sound leaks in a car’s cabin.
Visionics’ virtual-reality recorder. (Photo: Rob Pegoraro)
The other half of the system combines software and sensors to create audio that, even when heard through a standard pair of headphones, appears to move around you. The effect is uncannily realistic; I was convinced the sound was tracing orbits around my head, even as the headphones stayed clamped on my ears and I stayed put in a chair.
The last demo I saw seemed the most conventional: a polished take on the old Asteroids game I played through an Oculus Rift. The graphics looked great — it was impressive how these shiny rocks caught the reflection of ambient light as they tumbled towards my spaceship — but it wasn’t clear how this demo connected to what I’d seen so far.
Then I caught up on the explanation that I’d missed earlier: This game was created by middle-school girls in a computer camp. They’d taken some basic game code and other assets and crafted this simulation.
I’m still not quite sure where this technology will take us in 10 or 15 years, but at least I now have a better idea of who I’m going to be working for then: those kids.
Email Rob at rob@robpegoraro.com; follow him on Twitter at @robpegoraro.
More stories from the future:
