Integration Master Data Management (Cloud)

Data Ingestion Flow (External to MDM):

1. External Source System: An external system (e.g., partner application, data provider) initiates an update or provides new reference data.
2. Web Application Firewall (WAF) / API Gateway / Message Broker: Inbound traffic traverses the Public Internet to a secure ingress point. For synchronous updates, an API Gateway authenticates, authorizes, and routes the request. For asynchronous updates, a cloud-native Message Broker (e.g., a managed queue or streaming service) receives the event/message, ensuring delivery. Both are protected by a Web Application Firewall (WAF).
3. Data Ingestion Workload: A dedicated ingestion service or microservice within the Private Subnet (Workloads) consumes the incoming data from the API Gateway or Message Broker. This service performs initial validation, data quality checks, and potentially light transformation.
4. MDM Platform: The validated data is submitted to the Master Data Management (MDM) platform. The MDM applies its mastering rules (e.g., matching, merging, survivorship) to integrate the new data with existing records, creating or updating the golden record.
5. Audit & Response: The MDM platform records the transaction for auditability and either sends an acknowledgment (for synchronous requests) or publishes a "golden record updated" event (for asynchronous flows).

Data Distribution Flow (MDM to External):

1. MDM Platform (Golden Record Update): Upon creation or update of a golden record for reference data, the MDM platform triggers a distribution event or makes the data available.
2. Data Distribution Workload / Message Broker:
   • Asynchronous Push: The MDM platform or a dedicated distribution service publishes the golden record update event to a cloud-native Message Broker (e.g., pub/sub topic). External consuming systems subscribe to these topics.
   • Synchronous Pull: A dedicated API endpoint exposed via an API Gateway provides read access to the golden records.
3. Egress Gateway / API Gateway: For asynchronous push, the Message Broker delivers the event. For synchronous pull, the external system queries the API Gateway. Outbound connections from the MDM environment, if direct to external systems, pass through a Managed NAT / Egress Gateway for filtering and control.
4. External Consuming System: The external system receives the updated reference data either via an API call or by consuming messages/events from the Message Broker, then updates its local data stores.

• Single authoritative source of truth vs the governance overhead of stewardship, data-quality rules, and survivorship configuration.
• Decoupled, resilient exchange via API Gateway and Message Broker vs added infrastructure, edge components, and operational complexity to run and secure.
• Dual synchronous/asynchronous interfaces vs maintaining versioned API contracts and event schemas in parallel.
• Strict mediated access with no direct DB connections vs slower onboarding for external systems that must integrate through managed interfaces.
• End-to-end observability and auditability vs the cost of centralized logging, tracing, and metrics across every integration hop.

An international hotel group runs a cloud-native MDM platform as the single source of truth for guest, property, and rate-plan reference data across its brands. Channel partners and OTAs push updated property and room-type details as events across the Public Internet into a WAF-protected Message Broker; an ingestion microservice in the private subnet validates and runs data-quality checks before submitting to the MDM, which applies matching, merging, and survivorship rules to master each golden record. On every change, the platform publishes a 'golden record updated' event to a pub/sub topic, so central reservation, loyalty, and property-management systems subscribe to synchronize their local stores, while a partner extranet pulls current rate plans synchronously through the API Gateway — each interaction authenticated, mediated, and logged for full auditability.

• MDM Style & Strategy: The choice of MDM style (e.g., Registry, Consolidation, Coexistence, Centralized) will dictate the complexity and types of interfaces required for integration.
• Data Governance & Quality: Implement robust data governance frameworks including data stewardship, data quality rules, and data validation processes within the MDM platform to maintain the integrity of reference data.
• API Design & Event Schema: Define clear, versioned API contracts (e.g., OpenAPI Specification) for synchronous access and standardized event schemas (e.g., AsyncAPI, Avro) for asynchronous distribution. Ensure data formats are consistent (e.g., JSON, XML).
• Error Handling & Idempotency: Implement comprehensive error handling, retry mechanisms with exponential backoff, and idempotency patterns for both ingestion and distribution flows to ensure data consistency and system resilience. Survivorship rules must be configured within the MDM for conflicting data updates.
• Observability: Implement end-to-end observability, including centralized logging, metrics (e.g., data volume, latency, error rates, mastering success rates), and distributed tracing across all integration components (API Gateway, Message Broker, ingestion/distribution workloads, MDM platform). This provides real-time visibility into data flow and quality.
• Security by Design: Direct database connections between external parties and the MDM database are strictly prohibited. All interactions must be mediated by secure, managed interfaces.
• Infrastructure as Code (IaC): Manage and deploy all MDM platform components, integration interfaces, and security configurations using IaC practices and CI/CD pipelines to ensure consistency and repeatability.