Librería Portfolio

As data management and integration continue to evolve rapidly, storing all your data in one place, such as a data warehouse, is no longer scalable. In the very near future, data will need to be distributed and available for several technological solutions. With this practical book, you'll learnhow to migrate your enterprise from a complex and tightly coupled data landscape to a more flexible architecture ready for the modern world of data consumption.

Executives, data architects, analytics teams, and compliance and governance staff will learn how to build a modern scalable data landscape using the Scaled Architecture, which you can introduce incrementally without a large upfront investment. Author Piethein Strengholt provides blueprints, principles, observations, best practices, and patterns to get you up to speed.

Examine data management trends, including technological developments, regulatory requirements, and privacy concerns
Go deep into the Scaled Architecture and learn how the pieces fit together
Explore data governance and data security, master data management, self-service data marketplaces, and the importance of metadata



Table of contents

Foreword
Preface
Who Is This Book For?
What Will I Learn?
Navigating Through This Book
Conventions Used in This Book
O'Reilly Online Learning
How to Contact Us
Acknowledgments
1. The Disruption of Data Management
Data Management
Analytics Is Fragmenting the Data Landscape
Speed of Software Delivery Is Changing
Networks Are Getting Faster
Privacy and Security Concerns Are a Top Priority
Operational and Transactional Systems Need to Be Integrated
Data Monetization Requires an Ecosystem-to-Ecosystem Architecture
Enterprises Are Saddled with Outdated Data Architectures
Enterprise Data Warehouse and Business Intelligence
Data Lake
Centralized View
Summary
2. Introducing the Scaled Architecture: Organizing Data at Scale
Universally Acknowledged Starting Points
Each Application Has an Application Database
Applications Are Specific and Have Unique Context
Golden Source
There's No Escape from the Data Integration Dilemma
Applications Play the Roles of Data Providers and Data Consumers
Key Theoretical Considerations
Object-Oriented Programming Principles
Domain-Driven Design
Business Architecture
Communication and Integration Patterns
Point-to-Point
Silos
Hub-Spoke Model
Scaled Architecture
Golden Sources and Domain Data Stores
Data Delivery Contracts and Data Sharing Agreements
Eliminating the Siloed Approach
Domain-Driven Design on an Enterprise Scale
Read-Optimized Data
Data Layer as a Holistic Picture
Metadata and the Target Operating Model
Summary
3. Managing Vast Amounts of Data: The Read-Only Data Stores Architecture
Introducing the RDS Architecture
Command and Query Responsibility Segregation
What Is CQRS?
CQRS at Scale
Read-Only Data Store Components and Services
Metadata
Data Quality
RDS Tiers
Data Ingestion
Integrating Commercial Off-the-Shelf Solutions
Extracting Data from External APIs and SaaSs
Historical Data Service
Design Variations
Data Replication
Access Layer
File Manipulation Service
Delivery Notification Service
De-Identification Service
Distributed Orchestration
Intelligent Consumption Services
Populating RDSs on Demand
RDS Direct Usage Considerations
Summary
4. Services and API Management: The API Architecture
Introducing the API Architecture
What Is Service-Oriented Architecture?
Enterprise Application Integration
Service Orchestration
Service Choreography
Public Services and Private Services
Service Models and Canonical Data Models
Similarities Between SOA and Enterprise Data Warehousing Architecture
Modern View on SOA
API Gateway
Responsibility Model
The New Role of the ESB
Service Contracts
Service Discovery
Microservices
The Role of the API Gateway Within Microservices
Functions
Service Mesh
Microservices Boundaries
Microservices Within the API Reference Architecture
Ecosystem Communication
API-Based Communication Channels
GraphQL
Backend for Frontend
Metadata
Using RDSs for Real-Time and Intensive Reads
Summary
5. Event and Response Management: The Streaming Architecture
Introducing the Streaming Architecture
The Asynchronous Event Model Makes the Difference
What Do Event-Driven Architectures Look Like?
Mediator Topology
Broker Topology
Event Processing Styles
A Gentle Introduction to Apache Kafka
Distributed Event Data
Apache Kafka Features
The Streaming Architecture
Event Producers
Event Consumers
Event Platform
Event Sourcing and Command Sourcing
Governance Model
Business Streams
Streaming Consumption Patterns
Event-Carried State Transfer
Playing the Role of an RDS
Using Streaming to Populate RDSs
Controls and Policies for Guiding the Domains
Streaming as the Operational Backbone
Guarantees and Consistency
Consistency Level
"At Least Once, Exactly Once, and at Most Onceö Processing
Message Order
Dead Letter Queue
Streaming Interoperability
Metadata for Governance and Self-Service Models
Summary
6. Connecting the Dots
Recap of the Architectures
RDS Architecture
API Architecture
Streaming Architecture
Strengthening Patterns
Enterprise Interoperability Standards
Stable Data Endpoints
Data Delivery Contracts
Accessible and Addressable Data
Crossing Network Principles
Enterprise Data Standards
Consumption-Optimization Principles
Discoverability of Metadata
Semantic Consistency
Supplyin