Avoiding digital transformation icebergs: Can industrial DataOps steer projects away from disaster?

Nov 21, 2024 by admin

In short

Industrial digital transformation projects, though as grand and promising as they are, face many obstacles along their journeys, such as data management and integration complexity.
Among the industrial connectivity technologies covered in IoT Analytics’ latest market report on the topic, 3 can help steer projects away from project icebergs and reach their final destinations: 1. Industrial DataOps, 2. Protocol converters, 3. Message brokers.

Why it matters

Understanding the challenges of digital transformation projects and solutions to help overcome them can help organizations improve their chances of success.

Industrial digital transformation: An adventurous voyage with unforeseen challenges

Digital transformation is like embarking on the Titanic for a daring voyage across the North Atlantic—full of potential but also accompanied by significant risks. While many companies set sail on this journey with high expectations—believing their digital initiatives to be robust and well-prepared—unforeseen connectivity challenges can emerge like icebergs, threatening to turn even the most promising projects into a Titanic-like disaster.

Much like the iceberg that the Titanic failed to navigate around in 1912, these challenges—ranging from data silos and incompatible systems to protocol complexity—can cause even the most well-planned digital transformation strategies to fall short or even falter. In 2020, research from the Boston Consulting Group found that 70% of digital transformation projects fail to reach their objectives. IoT Analytics research continues to show that major obstacles remain.

Recognizing and addressing these hidden obstacles is crucial to ensuring that digital transformation initiatives stay on course and achieve their intended outcomes. Fortunately, lessons learned from industrial digital transformation case studies have helped others navigate the journey, just as lessons learned from the Titanic have led to modern safety measures to prevent such tragedies.

Note: Insights in this article are from the 290-page Industrial Connectivity Market Report 2024–2028 (published July 2024), which shares industrial connectivity market data and projections, trends, challenges, and how companies are addressing their connectivity needs.

Key industrial digital transformation challenges in 2024

Nearly two-thirds (62%) of organizations face severe, unforeseen challenges—metaphorical icebergs—during their industrial digital transformation journeys, according to Microsoft’s June 2024 report, Accelerate industrial transformation: How manufacturers prepare shop floors for a future with AI, which was supported by an IoT Analytics survey conducted for ongoing research into smart factory adoption (N=499).

The following is the share of survey respondents who marked each challenge as either severe or major:

Cybersecurity at 58%*
Data management at 49%*
Integration complexity at 48%*
Change management or cultural resistance at 45%
Skill gaps at 44%
Regulatory compliance at 44%
Legacy system modernization at 43%*
Reliability and uptime at 40%*
Scaling solutions or devices at 39%*
Updating devices, assets, or systems at 38%*
Energy management and sustainability at 37%

Note: IoT Analytics plans to publish the Smart Factory Adoption Report 2024 in Q4 2024 with further related data points. Those interested in accessing this report when it is released can sign up for IoT Analytics’ IoT Research Newsletter at the top right of this article to receive updates on the release of these and other reports.

The role of industrial connectivity

Connectivity at the core of industrial digital transformation. Connectivity is one of the foundational elements of industrial digital transformation, be it connecting devices and equipment in factories or in the field. Of the 11 challenges listed above, 7 relate to industrial connectivity (as denoted by asterisks), indicating that connectivity significantly contributes to industrial digital transformation challenges. Connectivity enables companies to collect operational data for automation and decision-making. The overall market size for industrial connectivity hardware and software reached $89 billion in 2023, according to the Industrial Connectivity Market Report 2024–2028 (the largest portion of the market being industrial automation hardware like PLCs or I/O modules).

Navigating hazardous seas: 3 technologies that could help avoid industrial digital transformation project icebergs

3 technologies address digital transformation challenges. The report dives into 3 industrial connectivity software technologies that help manage the issues related to connected OT and IT equipment outlined above:

Industrial DataOps
Protocol converters
Message brokers

The following is a deeper look into these 3 technologies and how they address digital transformation challenges.

1. Industrial DataOps

“The importance of industrial connectivity for making the vision of our smart factories a reality is underestimated. A unified shopfloor connectivity architecture is the key to realizing various use cases that we want to implement.”

Director of Digitalization Production at Porsche

Industrial DataOps definition (according to IoT Analytics)

The process of enhancing data quality by providing structure and context for accurate, logical data representation, ensuring usability by downstream applications

Industrial DataOps on the rise. According to the industrial connectivity report, the growing importance of industrial DataOps is the #1 general industrial connectivity trend. Manufacturing landscapes are complex, with numerous sensors, machines, and systems interacting. Industrial DataOps helps capture these intricate relationships and derive meaningful insights by processing, cleaning, and transforming data into usable formats, making it easier for downstream applications to leverage the data for analytics, machine learning, and other purposes. By standardizing how industrial assets are modeled, industrial DataOps brings uniformity across the industrial data landscape. This consistency simplifies the integration and consumption of data by various applications, reducing errors.

From a simplified, high-level perspective, industrial DataOps consists of two core elements: data quality management and data modeling. The former ensures reliable OT data for downstream applications through profiling, verifying, and cleaning, while the latter is the process of creating a logical representation of assets, processes, and systems.

As the established core element, data modeling provides a framework that allows different systems to communicate and integrate seamlessly, facilitating data flow and analytics. Several data modeling standards exist in industrial settings, such as OPC-UA, Sparkplug B, Asset Administration Shell, and Web of Things.

These standards help with two core subelements of data modeling:

Structuring – This involves organizing entities—i.e., distinct components representing real-world concepts that have data associated with them—into hierarchies—i.e., structures that organize entities into various levels, establishing relationships and dependencies.
Contextualizing – This adds attributes, standardizes values via specific transformation (and normalization), and details relationships both within the model and across interconnected models. This process converts raw data into valuable information by clarifying what entities represent, how they relate to one another, and their roles in a broader network of models. Contextualization is achieved via attributes, transformation, and relationships.

Ultimately, all of this leads to the creation of a Unified Namespace (UNS). This centralized, real-time framework allows data from all systems, machines, and sensors to be organized and accessed seamlessly. A UNS serves as a single source of truth across the organization, making data from various sources instantly available, structured, and contextualized for use in analytics, automation, and decision-making. By integrating the principles of industrial DataOps and leveraging UNS, organizations can achieve greater interoperability, reduce data silos, and ensure that data flows freely and efficiently throughout the entire digital ecosystem.

Selection of challenges that industrial DataOps can help address

Data management – Industrial DataOps ensures that data is cleaned, validated, and standardized as it moves through different systems, reducing errors and inconsistencies. This makes data reliable for analytics, decision-making, and operational processes.
Integration complexity – With differing data structures from different connected devices, normalizing incoming data into a consistent, usable format helps make data easier for downstream applications to consume and analyze.

Selected industrial DataOps trend from the Industrial Connectivity Market Report 2024–2028

Vendors are creating products that combine IT, ET, and OT data. Operational (OT) data is a hygiene requirement in today’s transformed industrial space. It is needed for all industrial use cases. In the vendor community, there is a growing realization of the need for conventional (IT), engineering (ET), and even location and social data to implement the use cases better to achieve the desired results. Industrial software vendors and OEMs are focusing on providing/integrating connectors to access these varied data sources.

Example: Norway-based data modeling software company Cognite offers Cognite Data Fusion, a data operations platform for production, asset maintenance, and sustainability use cases in asset-intensive industries. The platform aggregates, cleans, and contextualizes real-time and historical data from OT, IT, and ET sources from a suite of pre-built ‘extractors’.

DOWNLOAD REPORT SAMPLE

2. Protocol converters

Stop arguing about [protocols]! Modern protocols are a little better than old ones. There [are] only really two kinds of implementations to worry about: Client/Server (polled) and Pub/Sub (pushed). We have to deal with both kinds, and none of the protocols are going away any time soon.”

– Jonathan Wise, Chief Technology Architect, CESMII, during the CESMII Online Workshop on February 14, 2024

Protocol converter definition (according to IoT Analytics)

Industrial connectivity software that performs 2 key industrial connectivity functions:

OT-to-OT protocol conversion
OT-to-IT protocol conversion

Protocol converters help systems understand one another. Often, systems from various vendors leverage various protocols—a standardized set of rules and formats that govern how data is transmitted and exchanged between different devices, systems, or applications. Software is needed to translate one input protocol and produce a different output protocol. Protocol converters enable this translation between different protocols, allowing various operational systems (such as machines, sensors, and controllers) to communicate with each other.

Selection of popular OT and IT protocols

OT	IT
HART	HTTP
PROFIBUS	OPC-UA PubSub
PROFINET	MQTT
Modbus	AMQP
S7	CoAP
EtherNet/IP	DDS
OPC UA
IO-Link

Selection of challenges that protocol converters can help address

Integration complexity/interfacing with OT networks – Protocol converters enable seamless communication between varying networks, reducing the need for custom integrations and streamlining the connection of disparate networks.
Legacy system modernization – Many industrial operations rely on legacy systems that use outdated or proprietary protocols. Protocol converters allow these systems to interface with modern platforms and technologies by converting data into compatible formats.
Scaling solutions/devices – As organizations scale their operations and add new devices, protocol converters ensure that new and existing systems can communicate effectively.

Select protocol converter trend from the Industrial Connectivity Market Report 2024–2028

Protocol converters increasingly deployed at the edge. Edge-based industrial protocol converters facilitate instantaneous data transfer and task synchronization in automated systems. While they were often installed on desktops and centralized servers in the past, now they are being increasingly installed on edge devices on containers. This expands their capability and enables applications such as predictive maintenance, automated quality inspections, and real-time cloud services to be installed and carried out efficiently at the edge.

Example: Prosys’ OPC-UA Forge accesses operational data from OPC-UA servers and via Modbus, ADS (Beckhoff), and S7 (Siemens) protocols. It can run on a variety of hardware using containers.

DOWNLOAD REPORT SAMPLE

3. Message brokers

“Today, MQTT [broker] is used by many companies to connect data from OT machines and processes to IT systems to improve factory process efficiency, increase OEE, and decrease costs.”

– Ravi Subramanyan, director of industry solutions, HiveMQ

Message broker definition (according to IoT Analytics)

An intermediary service that allows producers (OT/IT systems) to publish messages to topics to which multiple consumers (OT/IT systems) can subscribe

Message brokers coordinate data messages across systems. In industrial setups where multiple systems must communicate, a tightly coupled setup can create rigidity. Message brokers enable a decoupled architecture, facilitating scalable communication between different systems, applications, or services, making them ideal for adoption in digital transformation projects. They act as intermediaries that route, transform, and manage messages, allowing systems to communicate without being directly connected or dependent on each other. This decoupling enhances the overall system architecture’s flexibility, scalability, and fault tolerance, making it easier to handle high volumes of data and integrate disparate systems.

MQTT leads the pack in popularity. By far, MQTT—listed above as a popular IT protocol—is the most adopted message broker system in industrial settings. MQTT is a lightweight, publish-subscribe messaging protocol designed for efficient, low-bandwidth communication. It can support different message formats, such as JSON or XML, and the payloads can carry a range of information, such as sensor data, commands, or settings changes. Receiving systems can subscribe to the data topics most relevant to them, and the interpretation of the data is up to the receiving application.

MQTT has 4 key features that make it stand apart from other, lesser-used protocols and make it ideal for adoption in digital transformation projects:

Connection and topic management – Oversees network connections, maintains sessions, handles topic subscriptions, and ensures message delivery
Quality of service (QoS) management – Implements different QoS levels, guaranteeing that messages are delivered as per broker-client agreements, from single attempts to guaranteed delivery
Message retention – Enables storing the most recent message for a topic to update new subscribers
Last will and testament feature – Provides a mechanism for clients to designate a message for the broker if it disconnects suddenly

One drawback of MQTT is that it lacks certain standardizations, which limit data interoperability across industrial applications. To address this, an additional specification, Sparkplug B, is added to the MQTT protocol. Sparkplug B standardizes MQTT message structures, adding time stamps, metrics, sequences, device data messages, and other elements to the message.

Selection of challenges that message brokers—specifically MQTT brokers—can help address

Data management – Message brokers organize data using topics, ensuring receivers subscribe to and access only the relevant data. They also ensure data integrity and reliability with features like message retention and simplify data handling by standardizing the exchange of data between different systems.
Scaling solutions/devices – Message brokers are designed to handle thousands of concurrent connections and high throughput, enabling systems to scale without performance degradation. Further, by decoupling publishers and subscribers, message brokers allow new devices and systems to be added without disrupting existing infrastructure.

Select message broker insights from the Industrial Connectivity Market Report 2024–2028

Sparkplug B continues to attract interest from vendors. The MQTT wave is making the Sparkplug B specification popular. Managed by the Eclipse Foundation and based on the MQTT 3.1.1 standard, Sparkplug B is an open specification for MQTT nodes to communicate within the MQTT infrastructure. MQTT is the exchange protocol, and Sparkplug defines the data sent. The MQTT + Sparkplug B combination is an alternate option to the OPC-UA standard in the opinion of several industrial software vendors.

For example, a robust set of vendors developing SCADA, MQTT brokers, and other applications (like historians, protocol converters, and DataOps solutions) have implemented Sparkplug B in their products. These companies include ABB, Schneider Electric, Wago, Inductive Automation, HiveMQ, and Canary Labs.

DOWNLOAD REPORT SAMPLE

Analyst opinion: The role of industrial connectivity for digital transformation

DataOps hold promise. Much like the fateful voyage of the Titanic, industrial digital transformation projects start with big celebrations and high expectations but face obstacles for a successful journey—the hidden icebergs. Technologies and tools that have recently emerged—such as DataOps and message brokers—promise to address these challenges.

The market is very nascent. Many of these tools are nascent, as many of the market players that are working to elevate the tools (e.g., Cognite, Litmus, Highbyte, or Cybus, among others) were founded in the last 10 years. For perspective, the DataOps market segment made up less than 0.1% of the total $89 billion industrial connectivity market in 2023 and less than 0.025% of the whole $269 billion enterprise IoT market. Larger industrial automation hardware and software vendors only recently started integrating DataOps solutions into their overall tech stack. One notable example is AspenTech DataWorks, which is marketed as an “industrial data management solution.” DataWorks heavily relies on technology developed by Inmation, a Germany-based company founded in 2013, which Aspentech acquired in 2022.

Growth prospects are strong. Nonetheless, companies are increasingly looking to adopt these tools to circumvent the project icebergs, and the ability of these tools to help steer digital transformation projects clear of connectivity obstacles is a major factor in the high projected CAGRs of each through 2028:

DataOps – 49%
Protocol converters – 12%
Message brokers – 28%

In turn, IoT Analytics expects the collective industrial connectivity software segment to be a key driver behind overall industrial connectivity market growth—5% CAGR until 2028.

Yes, DataOps can help steer away from disaster but much more is needed. These tools do not solve all industrial digital transformation challenges, as change management and skill gaps still remain. However, much like the International Ice Patrol helps ships navigate the iceberg-laden waters of the North Atlantic today, these tools promise to help manufacturers navigate dangerous points along their industrial digital transformation journey if properly adopted and used.

Disclosure

Companies mentioned in this article—along with their products—are used as examples to showcase a vibrant IoT startup landscape. No company paid or received preferential treatment in this article, and it is at the discretion of the analyst to select which examples are used. IoT Analytics makes efforts to vary the companies and products mentioned to help shine attention to the numerous IoT and related technology market players. It is worth noting that IoT Analytics may have commercial relationships with some companies mentioned in its articles, as some companies license IoT Analytics market research. However, for confidentiality, IoT Analytics cannot disclose individual relationships. Please contact compliance(at)iot-analytics.com for any questions or concerns on this front.

Avoiding digital transformation icebergs: Can industrial DataOps steer projects away from disaster?

Industrial digital transformation: An adventurous voyage with unforeseen challenges

Key industrial digital transformation challenges in 2024

The role of industrial connectivity