Augmented reality starts with a camera-equipped device—such as a smartphone, a tablet, or smart glasses—loaded with AR software. When a user points the device and looks at an object, the software recognizes it through computer vision technology, which analyzes the video stream.

The device then downloads information about the object from the cloud, in much the same way that a web browser loads a page via a URL. A fundamental difference is that the AR information is presented in a 3-D “experience” superimposed on the object rather than in a 2-D page on a screen. What the user sees, then, is part real and part digital.

AR can provide a view of the real-time data flowing from products and allow users to control them by touchscreen, voice, or gesture. For example, a user might touch a stop button on the digital graphic overlay within an AR experience—or simply say the word “stop”—to send a command via the cloud to a product. An operator using an AR headset to interact with an industrial robot might see superimposed data about the robot’s performance and gain access to its controls.

As the user moves, the size and orientation of the AR display automatically adjust to the shifting context. New graphical or text information comes into view while other information passes out of view. In industrial settings, users in different roles, such as a machine operator and a maintenance technician, can look at the same object but be presented with different AR experiences that are tailored to their needs.

A 3-D digital model that resides in the cloud—the object’s “digital twin”—serves as the bridge between the smart object and the AR. This model is created either by using computer-aided design, usually during product development, or by using technology that digitizes physical objects. The twin then collects information from the product, business systems, and external sources to reflect the product’s current reality. It is the vehicle through which the AR software accurately places and scales up-to-date information on the object.