Using Mixed Reality to Unlock the Potential of Industrial IoT

Mixed Reality is a boon to data visualization, but making sure it’s integrated into existing workflows is key.

At T4G, we love to help companies derive value from their data. And in order to help companies realize the incredible value their data holds, we often look to various methods of data visualization to illustrate and display the information and situation at hand.

When we talk about data visualization, it’s usually in the context of building out reports or dashboards or leveraging data science. But it’s often also about incorporating new insights within existing and new business processes. Our clients are busy and the speed at which they do business is always increasing. They don’t need another tool or report that lives in a silo. From experience, we know that the most useful data visualization is one which has been integrated, with context, into an existing workflow.

In the discovery phase of a project, we like to about what is possible, both now and in the future, and focus on the importance of visualizing data in context and discovering how we collectively think the data visualization space is developing.

Telling Stories with Data

Whether we’re talking about data visualization on a broad scale or actually leveraging it in a work process, a form of storytelling takes place. To tell this kind of story requires integrated data points rather than plot points. And, as with any good story, the key insights and actions should be revealed at the moments when their impact is greatest.

Data scientists and developers at T4G have been experimenting with mixed reality as a way to tell engaging data stories and to demonstrate the importance of integrated systems in the application of data visualization.

We believe mixed reality can, in the right situation, add immense value to the presentation of the data. With mixed reality, the user explores the data by interacting with it in engaging ways. They are able to receive and act on the data in real time as they are immersed in the experience. The ability of users to handle and to be able to immediately act on data in a tangible way is one of the most exciting potentials of mixed reality.

How Far Can Mixed Reality Take Us?

Will we get to a place where we can use mixed reality to visualize jet engines and other complex machinery to help drive insights, predict problems, and suggest solutions? As form factors are developing, computing is pushing towards the edge, and complex analytics can be driven to the point of decision in a manageable form, we’re getting closer to the answer being “Yes”.

But mixed reality still must provide an improvement on the current experience, which is a computer screen. What is produced in this next step must be consumable. It must be relevant. And, perhaps most importantly, it must be integrated with existing tools and systems.

Scorecards with a multitude of information points aren’t necessarily the most effective or understandable tools for users in the field. Process improvements that require multiple systems to activate aren’t necessarily easy to operate.

This is where data visualization becomes key. Visualization aims to improve the clarity and appeal of the displayed information. It should allow a person to understand large amounts of data and interact with it. And dependent on the use case and stage in the insight process, the visualization must be distilled to the most pertinent information available

How Data Visualization Has Changed

Visualization methods have evolved from simple graphs on spreadsheets to near works of art.

We’ve moved past static images into animations, TED talks on population growth, and highly animated cluster graphs depicting relationships across millions of people. The only limit to what can be visualized and how it is presented seems to be human imagination.

Now, mixed reality is a next logical step. There have already been attempts to make AR/VR a workable solution to some of the constraints of data visualization. But will an immersive experience ever be as intuitive as a mouse and monitor, or even a tablet?

There are still a number of challenges with AR/VR technology. These include limited fields of vision, or in the case of massive multi dimension graphs, the ability of the user to take it all in.

We think analysis and discovery is still best served near the compute and rapid form factor of a laptop or tablet. However, these systems often operate independently or remotely of the systems they’re meant to analyze. More importantly, we think mixed reality will drive some of the benefits from data visualization into the field or remote workplaces and allow contextual data discovery more easily among teams and remote workers.

Unlocking the Potential of Mixed Reality Industrial IoT

At T4G, we’ve been working with Microsoft to prototype some examples of how automation can present a vision of the possible. It requires the right data, presented in the right context, driven from existing technology capabilities (like predictive maintenance driven through AI, IoT enabled systems and sensor connectivity) and work process automation. We also believe mixed reality will be a fundamental component of the digital twin process.

The use case we present here demonstrates the effectiveness of information presented in context and integrated with multiple systems. Although use cases will vary widely, we think some of the best current use cases are in design, remote discovery by multiple teams, and education and training.

Using Microsoft’s Hololens, Azure Dynamics, and Power BI, T4G prototyped an industrial use case for mixed reality. In the prototype, a technician is dispatched to investigate a water issue at a commercial building. Through mixed reality, the technician is able to identify and correct a faulty water valve hidden behind a wall without actually seeing or touching it. Attendees at the Big Data Congress 2017 were invited to become the technician and ‘solve’ the problem.

In the demo, we’ve attempted to demonstrate some of the capabilities of mixed reality, from information overlay on new and existing models to the integration of traditional dashboards in a contextually aware manner and integration of mixed reality into a pre-existing workflow.

The Power of Mixed Reality for Data Visualization

Mixed reality shows the near future of data visualization: a real benefit can be driven through increased productivity, access to data, the availability of computer power, the required integration of systems, and ease of use of the form factor.

It’s important to recognize is that even in a rapidly constructed prototype like our maintenance use case, the concepts of data integration, system integration, device integration, and user design are crucial. Form factors for mixed reality will change and evolve, but the key will be to ensure the information is presented in a manner that improves the experience, provides contextual and relevant information, and is well suited to the environment. As Nelson Kunkel and Steve Soechtig state,

“The goal should be evolving engagement—building more intuitive, immersive, and empowering experiences that augment and amplify individual users, leading to new levels of customer intimacy, and creating new solutions to reshape how employees think and feel about work.”

Gary Filan

Gary leads the strategy and growth of T4G’s Analytics practice through continuous development, deployment, and commercialization of leading edge data driven solutions and applications supported by an amazing team of multi-faceted practitioners. His current focus on Applied AI and associated platforms has been developed over 25 years by working with companies around the world, helping them drive value from data by leveraging both leading technologies and business methodologies. His professional background crosses many industry verticals, solving a variety of data-driven challenges through multiple lenses of solution development, software, hardware, and start-up companies.