Applying the 4 Principles of Data Mesh to Deliver the Promise of Industry 4.0

Leading manufacturers have begun to embrace the four principles of a data mesh to evolve their architecture into one that supports delivering Industry 4.0 projects at scale.

Monica McDonnell
Monica McDonnell
3. März 2023 4 min Lesezeit
Data mesh and the promise of industry 4.0

Artificial Intelligence (AI) and Machine Learning (ML) can return significant value to factories, but there are still challenges to overcome. These challenges are not in the data science – the code usually works well. The primary challenges to delivering AI/ML projects at scale are rooted in architecture and data governance. Manufacturers need to find an efficient, and cost-effective way to find, understand, use & then reuse data across multiple projects.

Leading manufacturers have begun to embrace the four principles of a data mesh, even if it is not referred to as a data mesh. Their aim is to evolve their architecture into one that supports delivering Industry 4.0 projects at scale and avoiding the two most common modes of failure.

Initial failures were labelled ‘pilot purgatory’. Here the primary mode of failure was technology scaling:  what worked in the lab did not always scale to support the demands of the production environment.

Once the technology scaling issue was rectified, ‘use case purgatory’ emerged as a new failure mode. Here the challenge is resource scaling. Companies were delivering Industry 4.0 solutions using an Industry 3.0 architecture. In other words, creating a dedicated pipeline to feed an application which supports a single, or few use cases. In this purgatory, use cases are deployed, but at a high price. The development costs of a dedicated data pipeline coupled with support cost of maintaining isolated apps means this purgatory is not filled with failure, it is sparsely populated. Additional use cases are out of reach due to limited budget.

The clear path out of use case purgatory is a better use of resources – and in this case the key resources are data and data science skills. In a factory environment the amount and variety of data is large and growing.  However, it is also finite – and crucially multipurpose. For example, sensor data that comes from a machine can support maintenance use cases, provide insight into the quality of products produced and support process improvement. Machine data is both a large data set and needs very specific operator knowledge to provide context to measurements, alerts, and error codes. To gather and preparing this set of data multiple times is a waste of data science resources.  Why not collect, cleanse and pre-process it once for all three use cases, plus others?  

This is why the 4 principles of a data mesh should help make better use of limited resources. The first two principles describe a multipurpose approach to data cleansing.

Firstly, domain-driven ownership of data will ensure those people who really understand contextual data are given responsibility to prepare this for multiple use cases. As OT and IT have traditionally been separate in a manufacturing company, the ownership principle should be relatively easy to implement.

Secondly, data must be treated as a product. Creating data products is a new approach, but one that should be readily embraced given the use case purgatory experience. Data products can be as simple as data sets prepared for analytic purposes, or more complex such as the output of ML routines.

These two principles do not require specific technology, the main implication is organizational. Distributed groups must accept responsibility for ownership of a specific set of data – and ensuring this is available to all authorized users. This has added benefits in terms of reducing cost and increasing security by reducing data movements and redundancies.

Thirdly, a self-serve data infrastructure as a platform needs to be implemented. This is crucial to allowing a variety of teams to build on the work of others to create their own insights from data, as well as additional data products.  

And fourthly, the federated computational governance requires a blend of mesh-oriented governance practice and the ability to automate some tasks such as schema or lineage creation.  

The final two principles do imply some technical capabilities or tools. At Teradata, we have ongoing investment in technical capabilities to find, understand, use and re-use data – regardless of where it is stored. This is perfectly aligned with implementing a data mesh. Instead of having to move data at a high cost, we can implement an open data ecosystem and bring analytics to the data.  

This open ecosystem will also help manufacturers achieve the ideal of good, quick, and cheap in their Industry 4.0 analytics projects:

Good:  Increased use case deployment based on trusted & secure data 

Quick: Speed of use case deployment & use case execution

Cheap: Limit data movements and redundancies, improve compute efficiency

Manufacturing operations, supported by both IT and OT are not too far away from implementing the data mesh principles. To reap the full potential of AI and ML within a factory setting only small adjustments are required: Data governance practice to adapt ownership and introduce data products, coupled with highly scalable compute power able to operate efficiently in a diverse data ecosystem are the next steps.


Über Monica McDonnell

Monica McDonnell is a highly experienced consultant in the field of enterprise software, digital transformation and analytics. Her career has spanned Africa, the US and Europe with time spent on ERP and supply chain planning before focusing on delivering value from data. Monica advises on how to deliver business value by combining good data governance and advanced analytics technologies. Helping automotive companies understand how to release the full potential of Industry 4.0 technologies, and dramatically improve customer experience management as enabled by the connected vehicle is central to her role. Monica earned a BSc in Industrial Engineering from the University of Witwatersrand, and a MSc in Software Engineering from Oxford University. 

Zeige alle Beiträge von Monica McDonnell

Bleiben Sie auf dem Laufenden

Abonnieren Sie den Blog von Teradata, um wöchentliche Einblicke zu erhalten

Ich erkläre mich damit einverstanden, dass mir die Teradata Corporation als Anbieter dieser Website gelegentlich Marketingkommunikations-E-Mails mit Informationen über Produkte, Data Analytics und Einladungen zu Events und Webinaren zusendet. Ich nehme zur Kenntnis, dass ich mein Einverständnis jederzeit widerrufen kann, indem ich auf den Link zum Abbestellen klicke, der sich am Ende jeder von mir erhaltenen E-Mail befindet.

Der Schutz Ihrer Daten ist uns wichtig. Ihre persönlichen Daten werden im Einklang mit der globalen Teradata Datenschutzrichtlinie verarbeitet.