Modern Data Architecture
description.
Most companies today are now in the process of adopting AI as part of a digital transformation program that is reshaping their business. But these are challenges and obstacles in the way of progress, including:
- Data complexity caused by many more data sources, with data now stored in SaaS applications, on multiple clouds, on-premises and streaming in from the edge
- Business units are buying many different tools and data catalogs to help understand, govern, and provision data
- Multiple siloed analytical systems exist in the enterprise to support different analytical workloads on structured data, including data warehouses, data lakes, graph databases, data lakehouses, and streaming analytics, each of which offers support for a specific analytical workload. However, this approach forces companies to copy data several times for each analytical system and makes data architecture and data engineering more complex than it needs to be
- Data modelling seems to have disappeared
- Data engineering is now happening everywhere, and new technologies like Data Fabric and Modern Data Stacks have emerged, offering way more than ETL in AI-assisted workflows
Simultaneously, CEO’s now see data and AI as strategic and needed in every part of the business. They are demanding a way forward to speed up development so that AI-assistants and AI agents can be used in every business function. They want people to interact with AI-Assistants and to be able to ask complex questions that require analysis of structured AND unstructured data and that use multiple analytical techniques on shared data to provide richer answers for better decision-making and timely action-taking. They want data and AI to underpin the entire business. They want data to be high quality, secure and AI-ready. They want responsible, governed AI that they can trust and that understands their data. They are already reading that context is king and that somehow, they have to create trust in data and AI.
The question is, how do you make this possible? With so many competing data architectures, what does a modern data architecture look like? Is there a future for the data warehouse? Is data modelling dead? What else do I need in an architecture to support AI? How do you deal with structured AND unstructured data? How do you scale this to handle data being accessed by thousands of AI Agents as well as people? How do you store long-term memory for AI? How do you meet all requirements, accelerate development, prevent chaos, and provide a modern data architecture that is able to support the artificially intelligent enterprise? That's what this 2-day course is all about.
Why attend
You will learn:
- How data architecture has evolved and how you can modernize to support the AI-driven enterprise
- How to assess your existing environment, define future requirements, and design a new data architecture that modernizes your data warehouse, makes it possible to get rid of silos, and still support multiple analytical workloads like data science, streaming analytics, and graph analysis
- How to use a data catalog, data fabric and data observability to build resilient DataOps pipelines to create reusable data products published in a data marketplace that help shorten time to value by enabling new insights and AI to be delivered more rapidly
- How to use data catalogs to help build an enterprise ontology and context graph that understands your business, providing context to AI Agents, and that can synchronise multiple semantic layers to drive consistent context across the enterprise for AI Agents
Who should attend
CDOs, CIO’s, CAIO’s, IT Managers, CTOs, Business Analysts, data scientists, BI Managers, data warehousing professionals, enterprise architects, data architects, solution architects, BI Specialists, AI professionals, Data and AI strategists, Database administrators and IT consultants.
Prerequisites
This course assumes you understand basic data management principles and data architecture, plus a reasonable understanding of data cleansing, data integration, data catalogs, data lakes, and data governance.
outline.
What is Data Architecture?
This module looks at what data architect is, why companies need one, and what current issues are with respect to data architecture today.
- What is data architecture?
- Why do you need a data architecture?
- What are the differences between data architecture, solution architecture, and enterprise architecture?
- What are the main capabilities of a data architecture?
- Reference data architectures and their pros and cons
Data Warehouse, Data Lake and Data Lakehouse
This module looks at the evolution of analytical data architecture to understand how architecture has moved to accommodate different analytical workloads with different analytical data stores and engines.
- Data complexity and growth in data sources
- Centralised analytical data architectures and their pros and cons
- Classic data warehouse
- Data Lake
- Logical data warehouse
- Streaming data
- Introducing Data Lakehouse and its pros and cons
- What data storage, data processing, and data analytics technologies are used in data architecture?
- What are the pros and cons of cloud computing in a data architecture?
- How does it all fit together across all environments?
Data Mesh, Data Fabric and Data Catalogs
This module looks at Data Mesh and the use of data fabric software, and different data architecture options to incrementally build a high-quality, compliant data foundation for analytics and AI by creating reusable data products.
- Introducing Data Mesh, its principles and how it works
- What is a Data Product?
- Steps for creating Data Products
- Requirements to implement a Data Mesh or Data Lakehouse
- Key technologies needed: Data Fabric, Data Catalogs, Data Marketplace
- Multiple architecture options for decentralized creation of Data Products
- Centralised vs federated approaches
Next Generation Analytical Data Architecture
This module looks at how the emergence of open table formats, advances in SQL, universal data lake APIs, and federated query are changing data architecture to provide a data foundation for an AI-driven enterprise.
- The impact of Open table formats
- The OneTable initiative
- Merging of data warehouse, lakehouse, and streaming on multipurpose open tables
- Separation of storage from compute
- Multiple query and data engineering engines on shared data integration
- Universal administration of multiple engines
- Advances in SQL to support graph analytics
- Federated query across open tables on multiple clouds and on-premises storage
Data and AI Technology Options
This module looks at the technology options to consider for a modern data architecture for the AI-powered enterprise.
- Single vendor data and AI stacks
- Data fabric including data catalog, collaborative data engineering, data automation – using a data catalog and generative AI to auto generate pipelines, virtual views, APIs and architecture, DataOps, Data Observability, and Data virtualization for on-demand data integration
- End-to-end data and AI governance
- Modern data stack
- Best-of-breed tools
- AI Agents, MCP, Agentic Workflows, and Knowledge Graphs for AI Context
Data Engineering and Data Warehouse Modernization
This module focuses on what you need to think about with respect to data engineering and data warehouse modernization in a modern data architecture.
- Metadata and reuse
- Ensuring tools are integrated to share metadata
- Reuse of business terms, data models, data transformations, data quality rules, and data governance policies
- Data warehouse modernisation
- Moving away from siloed analytical systems
- Modernising ETL processing
- Introducing continuous change and change data capture (CDC) to lower data latency
- Data Warehouse automation
- Evolution of Data Warehouse to Lakehouse
- Migration to the cloud
- Virtual data marts
- The future of data modelling
- Modern data engineering, DataOps and Data Observability
- Supporting citizen data engineers, IT professional data engineers, and code in data engineering pipelines
- Establishing a universal approach to data ingestion
- Incorporating DataOps, CI/CD, data orchestration, and data observability into data pipelines
- Data Governance, Privacy, and Data Sharing
- Dealing with sensitive data to govern data privacy
- Establishing a data marketplace and data contracts to share data and analytical products
Assessing Your Existing Data Architecture and Defining Your Future Requirements
This module looks at how to assess your existing data architecture to understand how well it is serving your business needs from both an operational and analytical perspective. This includes documenting the current problems and identifying standalone projects that currently bypass your architecture.
- Assessing Your Existing Data Architecture
- Objectives of a data architecture assessment
- What is involved in understanding and assessing existing data architectures?
- Understanding and assessing how your business currently works
- Documenting and ranking business cases where data issues are impacting the ability to achieve goals
- SWOT and gap analysis
- Gauging where you are on a maturity model
- Defining Your Future Data Architecture Requirements
Designing an Innovative New Data Architecture for AI
In this module, we focus on how to design a modern data architecture and merge workloads to create a multi-purpose platform supporting multiple analytical workloads.
- Data architecture design principles
- Defining the key data capabilities needed in your data architecture
- Alignment of data capabilities needed to meet business strategy objectives, priorities, and outcomes
- Designing a data concept model for your business
- Designing your data architecture and data flows to support:
- Operational processing
- Analytical processing
- Knowledge management
- Merging your data warehouse, data lake, lakehouse, streaming, and graph to create a new multi-purpose hybrid analytical data platform
- Adding an enterprise ontology and knowledge graph to provide context for AI Agents
- Synchronising multiple semantic layers across the enterprise for consistent meaning and context
Getting Started
This module looks at what you have to do to get started with a new data architecture initiative.
- Creating an action plan to bring it to life
- Change vs rebuild?
- What order do you do this in?
- How do you minimize impact on the business while you re-architect?
- How do you deal with a backlog of change when you are also trying to re-architect?
- Pros and cons of build vs automating development
- What new skills are needed?
- Delivering new business value whilst re-architecting
- How do you involve business professionals in the re-architecting effort?
instructor.
Mike Ferguson
Mike Ferguson is the Managing Director of Intelligent Business Strategies Limited. As an independent IT industry analyst and consultant, he specializes in BI/Analytics and data management. With over 40 years of IT experience, Mike has consulted for dozens of companies on BI/analytics, data strategy, technology selection, data architecture and data management.
Mike is also conference chairman of Big Data LDN, the fastest-growing data and analytics conference in Europe and a member of the EDM Council CDMC Executive Advisory Board. He has spoken at events all over the world and written numerous articles.
Formerly he was a principal and co-founder of Codd and Date Europe Limited – the inventors of the Relational Model, a Chief Architect at Teradata on the Teradata DBMS.
He teaches popular master classes in Data Warehouse Modernization, Big Data Architecture & Technology, How to Govern Data Across a Distributed Data Landscape, Practical Guidelines for Implementing a Data Mesh (Data Catalog, Data Fabric, Data Products, Data Marketplace), Real-Time Analytics, Embedded Analytics, Intelligent Apps & AI Automation, Migrating your Data Warehouse to the Cloud, Modern Data Architecture and Data Virtualisation & the Logical Data Warehouse.
dates & price.
Course
Delivery Method
Dates
Location
Price
1117 CE Schiphol, Netherlands
pricing.
The fee for this 2-day course is EUR 1.450,00 (+VAT) per person.
We offer the following discounts:
- 10% discount for groups of 2 or more students from the same company registering at the same time.
- 20% discount for groups of 4 or more students from the same company registering at the same time.
