ASU graduate student Dylan Kerr partnered with Luke Air Force Base airmen to develop a 3D augmented reality system that could revolutionize the U.S Air Force pilot training process.
Kerr, along with Maj. Kevin Hawkins, 56th Operations Support Squadron wing intelligence chief and 1st Lt. Adam Treece, 56th OSS intelligence readiness chief is now a top-five finalist among over 300 entries for the annual Spark Tank Air Force innovation challenge.
At Spark Tank, the team pitched its plan to use data captured from flights and turn it into 3D interactive animations using Microsoft HoloLens 2 smart glasses.
For the final round of the competition, the team will present an updated three-minute pitch of their idea to a panel of U.S. Air Force leadership and industry experts at the virtual Air Force Association Aerospace Warfare Symposium on Feb. 26.
The prototype, called "Next-Gen Debrief,'' will display an instant replay of missions on the smart glasses so the airmen can feel as though they’re in the space they’re viewing.
"You are still looking at the real world as opposed to virtual reality where you are completely removed from the real world," Kerr said.
The airmen said the system could transform post-flight debriefs at Luke Air Force Base by allowing them to come in after their flight missions and easily put on the headsets rather than having to use a variety of different software to talk about what they just did.
Following each Luke Air Force flight, in a meeting called a "debrief," pilots evaluate mission details that could be further developed. During current post-flight debrief sessions, they collect data from sensors on the F-16 Fighting Falcon and F-35A Lightning II aircraft and put that data into a software system called Air Combat Maneuver Instrumentation (ACMI) data.
“The ACMI data comes in as a running data set using projectors and different screens hosted by a computer to preview a top-down view of where the planes were and at what point in time,” Kerr said.
They conduct additional debrief material using whiteboards, maps, and model planes glued together with sticks.
The Air Force continues to use new technology in the field, but it has not yet improved its debriefing process which is "arguably just as important, if not more, to understanding and becoming better at what they do," according to Kerr.
Hawkins said when they fly, everything is happening all around, so it would be much more natural and efficient to review their post-flight details in 3D.
The Luke airmen wanted to expand their idea, so Treece contacted the National Security Innovation Network (NSIN) in October and an ASU representative put them in contact with Kerr, a graduate research assistant for the Luminosity Lab who specializes in design.
"We are doing a lot of heaving lifting on the informative piece of the design, you know, bringing our expertise and sharing what would be useful in building the software," Treece said. "But Dylan is actually going in there and programming the device."
In order to continue with ASU’s innovation, NSIN provided the project with direct funding to further develop the software. Hawkins said they plan on this device being successful and have a goal to make the process mobile, in order to communicate across Air Force bases in different locations.
“Because of the way the Air Force wants to fight in the future, we need to be able to achieve that mobility and distributed capability,” Hawkins said.
Reach the reporter at jqmoya@asu.edu and follow @JudahMoya on Twitter.
Like The State Press on Facebook and follow @statepress on Twitter.
Continue supporting student journalism and donate to The State Press today.
