Augmented reality: Not just a game, but a game changer
Connecting state and local government leaders
The announcement of the OnSight application jointly developed by NASA and Microsoft is just the latest experiment by government to harness augmented reality to advance agency missions.
The announcement of the OnSight application jointly developed by NASA and Microsoft is just the latest experiment by government to harness augmented reality to advance agency missions.
The augmented reality headset HoloLens is the brainchild of Alex Kipman, the creator of the Kinect. Like the game controller, HoloLens weaves computer generated images into the real world where the user can interact with them.
But the government has been using Kinect technology with some success for several years.
Recently, researchers at the Naval Research Laboratory used Kinect to reinforce training that IED-sniffing dog handlers receive. They built a game interface based on the Virtual Battlespace platform, in which the player (or dog handler) communicates with the virtual dog using a Kinect controller.
And the Army Research Laboratory developed the Augmented Reality Sand Table, which combines a small sand box, Microsoft Kinect’s video game motion sensor and an off-the-shelf projector to create realistic topographical maps.
Combining 3D mapping technology with the sandbox allows users to create a 3D representation of the space. The map instantly changes as sand is moved around the box, and is faster than traditional sand tables, Charles Amburn, senior instructional systems specialist for the lab’s Simulation and Training Technology Center, told Army Times.
“With a traditional sand table, you’ve got to create the grid, and then somebody’s got to go take that map and say, ‘in this grid, there’s a hole here,’” Amburn said. “By the time it’s done, you’ve spent an hour setting up for an exercise or a scenario.”
The system can project real maps from Google Earth and similar technologies. The lab is now working on developing visual cues, such as arrows, to help troops shape the sandbox to the map’s topography. The maps can be easily reset and scenarios “rewound,” which isn’t possible on traditional static sand tables, Amburn said.
The 3D projections are already helping with language and communications barriers, said Amburn. When it comes to training third-party nationals, “if you project the plan with the real map, everyone gets it,” Amburn said. “You look at a map of their village and you show it to them and they get it.”
Potential plans for the 3D mapping technology includes examining the feasibility of remotely conducting joint training and operations; creating large scale models that could cover the space of a gymnasium floor for battalion briefings; and a smartphone version for pocket-sized projections on the go. The military is also studying how it can manipulate maps using hand gestures alone, said Amburn.
Combining Kinect with sandboxes is only one way governments are using Kinect to address problems.
Northrop Grumman’s Virtual Immersive Portable Environment (VIPE) Holodeck uses Microsoft’s Kinect motion sensor for simulation and training, mission rehearsal and data visualization, according to a report in Wired.
And Business Insider reported in February hat South Korea is using Kinect in select areas along the demilitarized zone (DMZ), which separates the country from North Korea, to help spot threats.
Self-taught South Korean programmer Jae Kwan Ko developed the Kinect-based software, which identifies objects crossing the DMZ and can tell the difference between human and animal movement. After identifying a possible threat, the system sends a warning to the closest manned outpost. Officials began using the technology at the border in 2013, but it wasn’t publicized until recently.
Healthcare is another sector taking advantage of the 3D technology. Avanande is using Kinect to create virtual healthcare services. Using Kinect, doctors are able to remotely meet with a patient, “just as if they were visiting their office,” the company said. “Doctors can view and analyze patient symptoms in an interactive, three-dimensional view. The patient data is then stored securely in the cloud for future reference.”
More uses for Kinect are to come. In October, Microsoft announced the full release of its Kinect for Windows software development kit 2.0, the ability for developers to create Kinect apps for the Windows Store and a Kinect Adapter for Windows.
Additional information on Kinect for Windows is available at http://www.microsoft.com/en-us/kinectforwindows/.