Holograms are awesome. Their power lies in allowing humans to share imaginations, which is why they work great in movies. But what are holograms well suited for, other than for entertainment purposes like Star Wars special effects or resurrecting Tupac at Coachella in 2012? Well, while what we are calling ‘Holograms’ are technically just an optical illusion called Pepper’s Ghost, they are very useful for other applications. From here on out we will refer to ‘pseudo’ holograms as Augmented Reality.
As Tupac rapped, “Reality is wrong. Dreams are for real.” I think he meant that dreams allow us to simulate all our hopes and fears and consider the best outcomes. Augmented Reality allows us to ‘dream’ into the future and make really smart decisions affecting our present.
Can AR be used for business application? Sure! AR is powerful for visually exploring predictive models that are registered to our physical surroundings.
For example, take an air traffic controller who just started a position in the all new, $126 million control tower that goes online July 28 at San Francisco International (SFO).
Did you know that SFO as well as many other air traffic control units have been operating by paper strips until last year?
The FAA is currently implementing an upgraded air traffic control system called NexGen in ‘pilot’ airports (pun intended). But this is still limited to a screen. Air traffic controllers will still need switch between looking at information regarding the plane and then the plane itself. They also have to look into the future.
As part of NexGen, why not simplify the air traffic control job by melding information about the plane with a hologram of the plane itself and allow for direct manipulation of the scenario through modern UI? Enter Holograms.
The Use Case:
SFO air traffic controllers are directing planes during the holidays. They need the ability to look into the future and see where issues may arise. For instance a storm might be affecting arrival times for planes coming in from the east. Our controller needs to see where margins are thin for collisions and potentially reroute.
What Is On The Horizon?
Planes that are far off in the distance can be represented by white dots of varying size. Size will represent spatial distance. When planes come within a predetermined distance they can be rendered as low-fidelity holograms. This plane is orange since it is triggering a warning. Other planes are white since they are all speculative.
With an augment reality interface, a controller can reach out to a series of radars showing simulated or real-time inbound and outbound planes by time of day. There is no clicking to do, just grabbing and moving hologram-like objects.
For example, here is a radar timeline that projects all the events and planes happening in real time. They are arranged in a circular dial that spins and expands to show you the radar at the specific time you are interested. The white box corresponds to the largest circle with two orange dots. If you dialed the radar back to the blue circle (current-real time) then the box would move to the ‘Now’ indicator. Obviously the present is always moving so the word ‘Now’ would move with time. Anything projected or in the past takes on a white color. Once controllers select a radar matching the busiest time of day, they can verbally tell their computer to, “Show projected warnings” or grab the two orange dots that would slightly expand as their arm approaches.
Once they interactively activate the warning dots, controllers would see low fidelity planes landing, departing and taxing along the runway. Gazing or moving an arm towards a plane would populate an information window displaying information and a warning for that asset, where relevant.
Controllers can grab the virtual plane by reaching out, at which point its display quality would increase to a CAD model with more detail. The philosophy behind this design is that you start at a high level and then are offered more detail as you interact with items. This alleviates the need to have an explicit file structure. You can spin the plane, open it to see section, or even view pilot and system information.
Augmented reality software engineer Tyler Lindell says, “Using augmented reality and interacting with real-world objects will take us beyond the barrier we have known for the last 40 years, the personal computer.”
Computers should feel natural. You should be able to use any interface and feel like you have always used it. No more opening applications. No more going through file systems. No more explicit metaphors trapped in 2D ‘planes’ like icons and spreadsheets trapped in a PC.
As imaginative as this prototype AR interface is, it is likely to have myriad shortcomings and flaws. That is why I’m asking you, the reader, to complete this survey regarding this UI. Please be brutally honest.