AI Architecture

AI systems are rarely one model. A production system combines a data layer, one or more models and an inference stack, retrieval and application patterns, agents and tool protocols, a control layer for routing and policy, operations and observability, and security and governance. The pages below start with the whole-system views and then follow each layer in turn.

Cloud providers and standards bodies publish architecture guidance for AI workloads. These are good starting points when you need a vendor-neutral checklist or a reference design on a specific cloud.

Models are only as useful as the data that grounds them. The data layer covers architecture patterns for pipelines and storage, lakehouses and medallion layers, vector and graph stores for retrieval, and memory for agents.

This layer covers the models themselves, how they are trained, and the serving stack that runs them: engines, platforms, quantization and formats, accelerators and privacy-preserving training.

Retrieval-augmented generation is the most common pattern for grounding models in enterprise data, and MCP is a common way to connect models to tools and data.

Agent systems add planning, tools, memory and coordination. These pages cover architectural patterns, frameworks, platforms, communication protocols and multi-agent designs.

A control layer sits between applications and models or agents to route traffic, enforce policy and limit what agents may do.

Running AI in production needs deployment, monitoring, evaluation and cost control, extended to cover non-deterministic model behaviour and agent traces.

Security and governance run through every layer: protecting applications and agents, managing AI security posture and governing models and data.

Where AI runs shapes the architecture: at the edge, on devices and on Kubernetes clusters.