Augmented reality has come a long way from a science-fiction concept to a science-based reality. Until recently the costs of augmented reality were so substantial that designers could only dream of working on design projects that involved it — today things have changed and augmented reality is even available on the mobile handset. That means design for augmented reality is now an option for all shapes and sizes of UX designers.
The Current State of Play in Augmented Reality (The Present)
Augmented reality is achieved through a variety of technological innovations; these can be implemented on their own or in conjunction with each other to create augmented reality. They include:
General hardware components — the processor, the display, the sensors and input devices. Typically a smartphone contains a processor, a display, accelerometers, GPS, camera, microphone etc. and contains all the hardware required to be a an AR device.
Displays — while a monitor is perfectly capable of displaying AR data there are other systems such as optical projection systems, head-mounted displays, eyeglasses, contact lenses, the HUD (heads up display), virtual retinal displays, EyeTap (a device which changes the rays of light captured from the environment and substitutes them with computer generated ones),Spatial Augmented Reality (SAR — which uses ordinary projection techniques as a substitute for a display of any kind) and handheld displays.
Sensors and input devices include — GPS, gyroscopes, accelerometers, compasses, RFID, wireless sensors, touch recognition, speech recognition, eye tracking and peripherals.
Software — the majority of development for AR will be in developing further software to take advantage of the hardware capabilities. There is already an Augmented Reality Markup Language (ARML) which is being used to standardize XML grammar for virtual reality. There are several software development kits (SDK) which also offer simple environments for AR development.
There are apps available for or being researched for AR in nearly every industrial sector including:
Archaeology, Art, Architecture
Construction, Industrial Design
Emergency Management, Disaster Recovery, Medical and Search and Rescue
Games, Sports, Entertainment, Tourism
The Future of Augmented Reality
Jessica Lowry, a UX Designer, writing for the Next Web says that AR is the future of design and we tend to agree. Already mobile phones are such an integral part of our lives that they might as well be extensions of our bodies; as technology can be further integrated into our lives without being intrusive (a la Google Glass) — it is a certainty that augmented reality provides opportunities to enhance user experiences beyond measure
This will almost certainly see major advances in the much-hyped but still little seen; Internet of Things. UX designers in the AR field will need to seriously consider the questions of how traditional experiences can be improved through AR — just making your cooker capable of using computer enhancements is not enough; it needs to healthier eating or better cooked food for users to care.
The future will belong to AR when it improves task efficiency or the quality of the output of an experience for the user. This is the key challenge of the 21st century UX profession.
How AR Creates Value
AR creates business value in two broad ways: first, by becoming part of products themselves, and second, by improving performance across the value chain — in product development, manufacturing, marketing, service, and numerous other areas.
Though engineers have been using computer-aided design (CAD) capabilities to create 3-D models for 30 years, they have been limited to interacting with those models through 2-D windows on their computer screens, which makes it harder for them to fully conceptualize designs. AR allows 3-D models to be superimposed on the physical world as holograms, enhancing engineers’ ability to evaluate and improve designs. For example, a life-size 3-D hologram of a construction machine can be positioned on the ground, and engineers can walk around it, peer under and over it, and even go inside it to fully appreciate the sight lines and ergonomics of its design at full scale in its intended setting.
AR also lets engineers superimpose CAD models on physical prototypes to compare how well they match. Volkswagen is using this technique — which makes any difference between the latest design and the prototype visually obvious — to check alignment in digital design reviews. This improves the accuracy of the quality assurance process, in which engineers previously had to painstakingly compare 2-D drawings with prototypes, and makes it five to 10 times faster.
We expect that in the near future AR-enabled devices such as phones and smart glasses, with their embedded cameras, accelerometers, GPS, and other sensors, will increasingly inform product design by exposing when, where, and how users actually interact with the product — how often a certain repair sequence is initiated, for example. In this way the AR interface will become an important source of data.
In manufacturing, processes are often complex, requiring hundreds or even thousands of steps, and mistakes are costly. As we’ve learned, AR can deliver just the right information the moment it’s needed to factory workers on assembly lines, reducing errors, enhancing efficiency, and improving productivity.
In factories, AR can also capture information from automation and control systems, secondary sensors, and asset management systems and make visible important monitoring and diagnostic data about each machine or process. Seeing information such as efficiency and defect rates in context helps maintenance technicians understand problems and prompts factory workers to do proactive maintenance that may prevent costly downtime.
Iconics, which specializes in automation software for factories and buildings, has begun to integrate AR into its products’ user interfaces. By attaching relevant information to the physical location where it will be best observed and understood, the AR interfaces enable more-efficient monitoring of machines and processes.
Warehouse operations are estimated to account for about 20% of all logistics costs, while picking items from shelves represents up to 65% of warehouse costs. In most warehouses, workers still perform this task by consulting a paper list of things to collect and then searching for them. This method is slow and error-prone.
The logistics giant DHL and a growing number of other companies are using AR to enhance the efficiency and accuracy of the picking process. AR instructions direct workers to the location of each product to be pulled and then suggest the best route to the next product. At DHL this approach has led to fewer errors, more-engaged workers, and productivity gains of 25%. The company is now rolling out AR-guided picking globally and testing how AR can enhance other types of warehouse operations, such as optimizing the position of goods and machines in layouts. Intel is also using AR in warehouses and has achieved a 29% reduction in picking time, with error rates falling to near zero. And the AR application is allowing new Intel workers to immediately achieve picking speeds 15% faster than those of workers who’ve had only traditional training
Marketing and sales
AR is redefining the concept of showrooms and product demonstrations and transforming the customer experience. When customers can see virtually how products will look or function in a real setting before buying them, they have more-accurate expectations, more confidence about their purchase decisions, and greater product satisfaction. Down the road, AR may even reduce the need for brick-and-mortar stores and showrooms altogether.
When products can be configured with different features and options — which can make them difficult and costly to stock — AR is a particularly valuable marketing tool. The construction products company AZEK, for instance, uses AR to show contractors and consumers how its decking and paver products look in various colors and arrangements. Customers can also see the simulations in context: If you look at a house through a phone or a tablet, the AR app can add a deck onto it. The experience reduces any uncertainty customers might feel about their choices and shortens the sales cycle.
In e-commerce, AR applications are allowing online shoppers to download holograms of products. Wayfair and IKEA both offer libraries with thousands of 3-D product images and apps that integrate them into a view of an actual room, enabling customers to see how furniture and decor will look in their homes. IKEA also uses its app to collect important data about product preferences in different regions.
This is a function where AR shows huge potential to unlock the value-creating capabilities of SCPs. AR assists technicians serving customers in the field in much the same way it helps workers in factories: by showing predictive analytics data generated by the product, visually guiding them through repairs in real time, and connecting them with remote experts who can help optimize procedures. For example, an AR dashboard might reveal to a field technician that a specific machine part will most likely fail within a month, allowing the tech to preempt a problem for the customer by replacing it now.
At KPN, a European telecommunications service provider, field engineers conducting remote or on-site repairs use AR smart glasses to see a product’s service-history data, diagnostics, and location-based information dashboards. These AR displays help them make better decisions about how to resolve issues, producing an 11% reduction in overall costs for service teams, a 17% decrease in work-error rates, and higher repair quality.
AR dramatically reduces errors and increases productivity in factories.
Xerox used AR to connect field engineers with experts instead of providing service manuals and telephone support. First-time fix rates increased by 67%, and the engineers’ efficiency jumped by 20%. Meanwhile, the average time it took to resolve problems dropped by two hours, so staffing needs fell. Now Xerox is using AR to connect remote technical experts directly with customers. This has increased by 76% the rate at which technical problems are resolved by customers without any on-site help, cutting travel costs for Xerox and minimizing downtime for customers. Perhaps not surprisingly, Xerox has seen its customer satisfaction rates rise to 95%.
Early AR adopters like DHL, the U.S. Navy, and Boeing have already discovered the power of delivering step-by-step visual worker training on demand through AR. AR allows instruction to be tailored to a particular worker’s experience or to reflect the prevalence of particular errors. For example, if someone repeatedly makes the same kind of mistake, he can be required to use AR support until his work quality improves. At some companies, AR has reduced the training time for new employees in certain kinds of work to nearly zero and lowered the skill requirements for new hires.
This is especially advantageous for the package delivery company DHL, which faces surges in demand during peak seasons and is heavily dependent on the effective hiring and training of temporary workers. By providing real-time training and hands-on guidance on navigating warehouses and properly packing and sorting materials, AR has reduced DHL’s need for traditional instructors and increased the onboarding speed for new employees.
AR and Strategy
AR will have a widespread impact on how companies compete. As we’ve explained in our previous HBR articles, SCPs are changing the structure of almost all industries as well as the nature of competition within them — often expanding industry boundaries in the process. SCPs give rise to new strategic choices for manufacturers, ranging from what functionality to pursue and how to manage data rights and security, to whether to expand a company’s scope of products and compete in smart systems.
The increasing penetration of AR, along with its power as the human interface with SCP technologies, raises some new strategic questions. While the answers will reflect each company’s business and unique circumstances, AR will become more and more integral to every firm’s strategy.