Industry: RETAIL

Efficient Data Management with Data Mesh Lab

Overcoming Data Silos
Grappling with the complexities of vast data landscape can be a chore. With data scattered across multiple departments maintaining their own data models, pipelines, and governance standards, our customer – a retail European tycoon – had to change their game.

The Data Mesh Approach: A New Way to Manage Data
To address these challenges a decentralized, domain-oriented architecture was introduced – aka the Data Mesh.

What was introduced:

  • Domain-Oriented Ownership: The company redefined its data ownership model by aligning data management with business domains.
  • Data as a Product Mindset: Domain teams were encouraged to adopt a product
    thinking mindset.
  • Self-Service Data Infrastructure: To empower domain teams to work independently, a robust self-service data infrastructure was implemented.
  • Federated Computational Governance: A federated approach to governance was introduced.

Project Challenges:

  • Unified Infrastructure for All Teams: Our approach was to start with a solution that meets 90% of the general requirements, while the remaining 10% were under constant analysis to be introduced for particular domain team, but also introducing value added to other Teams. No shortcuts here!
  • Infrastructure Management by Platform Team: It is unrealistic to expect a flawless release, and compromises were inevitable.
  • Security Challenges: Ensuring security within VPC subnets requires substantial effort in terms of infrastructure management, which the platform team had to handle.
  • Domain-Specific Language (DSL) for Mesh Infrastructure: The creation of a complete mesh infrastructure in just a few steps using a Domain-Specific Language (DSL) was some serious task to pull off!