7-Layer Reference Architecture
Enterprise AI agents require more than an LLM. This architecture separates user interface, orchestration, agents, decision logic, models, integration, and infrastructure into seven independent layers. Governance spans all layers as a cross-cutting concern.
Why Seven Layers
A monolithic AI system is not auditable, not scalable, and not maintainable. The 7-layer architecture separates responsibilities: each layer has a defined purpose and clear interfaces. Changing the model does not affect business logic. A new target system does not change the agent. A new compliance requirement does not change the infrastructure.
This architecture is the result of enterprise requirements: tenant isolation, Audit Trail, transparency for employee representation bodies, EU AI Act conformity, model-agnostic deployment. Standard LLM APIs deliver none of this.
1. Presentation Layer
The interface between system and user. No business logic, no decisions - only display and input.
- Chat UI: Web-based interface for end users (HR staff, accountants). PWA-capable, responsive.
- Dashboard: Status overview for agents, running workflows, open escalations. Role-based: case workers see their cases, managers see KPIs.
- Auditor Portal: Auditor access to Audit Trail, controls, and evidence. Read-only. For external auditors, employee representation bodies, and internal audit teams.
- REST API: Machine-readable interface for integration into existing systems. Versioned, documented, authenticated.
2. Orchestration Layer
Coordinates data flow between agents, systems, and users. Manages workflows, queues, and API routing.
- Workflow Engine: Open-source engines (Trigger.dev, Camunda) for complex, multi-step processes. Visual workflows, API integration, webhooks.
- API Gateway: Unified entry point with rate limiting, authentication, logging, monitoring.
- Queue System: Asynchronous processing for batch operations (month-end closing, bulk import).
- Event System: Real-time response to incoming documents, status changes, escalations.
3. Agent Layer
Specialized AI agents performing domain-specific tasks. Each agent has a defined scope and operates within the boundaries set by the Decision Layer.
Document Agents
Read, understand, and process documents with genuine language comprehension. Invoices, medical certificates, contracts, compliance documents, receipts. No template matching, no rigid rules - contextual understanding.
Workflow Agents
Orchestrate processes across systems. When a document needs to be read, a decision made, and an action triggered in a target system - the Workflow Agent coordinates the sequence.
Knowledge Agents
Deliver context-based answers from enterprise knowledge. Governance frameworks, company policies, collective bargaining agreements, compliance rules. Every answer includes the source reference and the rule version.
4. Decision Layer
Decomposes every business process into individual decision steps and defines for each step: human, rule engine, or AI. Every decision is documented - auditable for external auditors, employee representation bodies, and internal audit teams.
Rules Engine: Versioned, traceable business rule sets. Collective bargaining agreements, company policies, posting logic, compliance rules. Each rule has a version, an effective date, and a scope.
Confidence Routing: Automatic assessment of decision certainty. High confidence and low risk: autonomous decision. Low confidence or high risk: escalation to a human.
Human-in-the-Loop: Architecturally enforced human review for defined decision types. Bias risk, discrimination potential, employee representation requirements.
Audit Trail: Complete, immutable documentation of every decision. Input, model, assessment, rule, result, timestamp. Append-only.
Deep dive: Decision Layer in Detail · Three Types of AI Decisions
5. Model Layer
The LLM layer. Interchangeable, model-agnostic, decoupled from business logic.
Cloud LLMs
Claude (Anthropic), ChatGPT (OpenAI), Gemini (Google) - via EU regions of the respective cloud providers.
Open-Source / Open-Weight LLMs
Llama (Meta), Mistral, DeepSeek, gpt-oss (OpenAI, Apache 2.0) - fully self-hostable on your own hardware. gpt-oss-120B runs on a single H100 GPU, gpt-oss-20B on 16 GB consumer hardware.
Hybrid
Cloud LLMs for standard workloads, self-hosted LLMs for sensitive data. Automatic routing based on data classification.
Model choice is a trade-off between performance, cost, data residency, and latency. The Model Layer is interchangeable - switching models does not affect the business logic above.
Concrete hosting options, hardware requirements, and technology stack: AI Infrastructure in Detail
6. Integration Layer
Connection to existing enterprise systems. The agent does not replace systems - it extends them.
| System Category | Integration |
|---|---|
| ERP / Finance | SAP FI/CO, SAP S/4HANA, DATEV, Oracle Financials |
| HR / Payroll | SAP SuccessFactors, Workday, Personio |
| Collaboration | SharePoint, Microsoft Teams (via Microsoft Graph) |
| DMS / ECM | SharePoint, d.velop, ELO, nscale |
| Other | Any system with a REST or SOAP interface |
Agent logic is decoupled from the target system. Posting logic is separated from export. When the target system changes (e.g. from DATEV to SAP), the export layer changes - not the agent.
7. Infrastructure Layer
The deployment foundation. The entire architecture runs in the client's infrastructure - not at Gosign, not at a third-party provider.
- Cloud (EU): Azure, AWS, or GCP - EU regions exclusively. Managed Kubernetes, managed databases, LLM hosting. (US: US-region deployment available for organizations without EU data residency requirements.)
- Managed EU: Vercel EU + Supabase EU. Lightweight EU option without dedicated Kubernetes infrastructure.
- Self-Hosted: Own servers, own data center. Docker/Kubernetes, open-source LLMs on your own GPUs. Complete Cloud Act independence.
- Hybrid: Combination based on data classification. Sensitive workloads self-hosted, standard workloads in the cloud.
All layers above Infrastructure remain identical - regardless of deployment model.
Cloud regions, hardware sizing, technology stack: AI Infrastructure in Detail
Governance as Cross-Cutting Concern
Governance is not a single layer - it spans the entire architecture. Every layer produces governance data, and every layer is controlled by governance rules.
- Presentation: Role-based access, Auditor Portal
- Orchestration: Workflow logging, escalation documentation
- Agent: Agent decisions generate Audit Trail entries
- Decision Layer: Rules Engine, Confidence Routing, Human-in-the-Loop
- Model: Model version tracking, input hashing, reproducibility
- Integration: Interface logging, data flow documentation
- Infrastructure: Encryption, Row-Level Security, tenant isolation
EU AI Act · Cert-Ready by Design · Co-Determination · Data Residency
Runtime and Scaling
Production operation is not an afterthought - it is an architectural component. The 7-layer architecture is designed for operation under load.
- Container Orchestration: Kubernetes-based deployment. Each layer runs in its own containers, independently scalable.
- Horizontal Scaling: Agent Layer and Model Layer scale horizontally based on utilization. A new agent means more pods, not more architecture.
- Health Checks and Self-Healing: Liveness and readiness probes on all containers. Automatic restart on failure, automatic rerouting under overload.
- Monitoring and Alerting: Prometheus metrics across all layers. Grafana dashboards for latency, throughput, error rates, queue depth. Alerting on threshold violations.
- CI/CD: GitOps-based deployments. Infrastructure as Code (Terraform/Pulumi). Automated tests, blue-green or canary deployments.
Data Architecture
Data flows through all seven layers. The architecture defines where data originates, how it is stored, and who has access.
- Data Flow: Input (document, query) > Agent (analysis) > Decision Layer (decision) > Integration (target system export). Every step generates an Audit Trail entry.
- Vector Store: PostgreSQL with pgvector for semantic search (RAG). Enterprise knowledge is stored as embeddings - never transmitted to external services.
- Tenant Isolation: Row-Level Security (RLS) at the database level. Enforced architecturally, not through application logic alone. Each tenant is fully isolated.
- Encryption: At rest (AES-256) and in transit (TLS 1.3). Key management via the client's identity provider or Hardware Security Modules (HSM).
- Data Retention: Configurable per requirement. Regulatory retention periods and GDPR Art. 17 (right to erasure) are reconciled through anonymization rather than deletion.
- Backup and Recovery: Automated backups, point-in-time recovery. Recovery Point Objective (RPO) and Recovery Time Objective (RTO) are configured per tenant.
Interface Architecture
The architecture communicates through defined interfaces - internally between layers and externally with source and target systems.
- REST API: Versioned APIs (v1, v2) with OpenAPI documentation. Breaking changes only in new versions; legacy versions are operated in parallel.
- Event-Driven: Webhook-based event processing for real-time responses. Incoming document > event > agent processes. No polling, no batch delay.
- MCP (Model Context Protocol): Standardized protocol for tool integration in LLM agents. Agents access external tools via MCP - typed, documented, auditable.
- Batch Processing: For bulk operations (month-end closing, annual accounts, mass import). Queue-based with progress tracking and error handling.
- API Gateway: Central entry point. Authentication (SSO/OIDC), rate limiting, request logging, monitoring. Decouples internal architecture from external consumers.
Design Principles
Model-agnostic: No vendor lock-in to a single LLM. Models are interchangeable. Claude today, gpt-oss tomorrow, a model that does not exist yet the day after.
Infrastructure-agnostic: Same architecture on Azure, AWS, GCP, self-hosted, or hybrid. The infrastructure choice belongs to the client, not to the architecture.
System-agnostic: Agent logic is decoupled from the target system. Posting logic separated from export. A system migration changes the Integration Layer, not the agent.
Governance by Design: Audit Trail, RBAC, Decision Layer, and Human-in-the-Loop are architectural components - not optional features retrofitted after launch.
Cert-Ready by Design: Controls are first-class data objects with automatic evidence generation. ISO 27001, SOC 2, EU AI Act - the architecture produces the audit evidence. (US: Aligned with NIST AI RMF and SOC 2 Type II requirements.)
Source Code Access: Full access to source code, all prompts, and all rule sets. Configurations and rule sets remain with the client. Open-source stack where possible. After 12-18 months, the client operates the agents independently.
Decision Layer - Decision Flow
┌──────────┐ ┌──────────────┐ ┌────────────────┐
│ Input │───>│ AI Agent │───>│ Decision Layer │
│(document,│ │ analyzes, │ │ │
│ query) │ │ understands,│ │ Check rules │
└──────────┘ │ evaluates │ │ │
└──────────────┘ │ Assess │
│ confidence │
│ │
│ Route │
│ decision │
└───────┬────────┘
│
┌─────────────┴──────────────┐
│ │
┌────────▼────────┐ ┌──────────▼──────────┐
│ Autonomous │ │ Human-in-the-Loop │
│ │ │ │
│ High confidence │ │ Bias risk │
│ Low risk │ │ Low confidence │
│ No governance │ │ Governance │
│ constraint │ │ constraint active │
└────────┬────────┘ └──────────┬──────────┘
│ │
│ ┌──────────────┐ │
│ │ Human │ │
│ │ decides │◄──┘
│ └──────┬───────┘
│ │
┌────────▼────────────────▼────────┐
│ Audit Trail │
│ Input · Model · Rule · │
│ Assessment · Result · │
│ Timestamp │
└──────────────────────────────────┘
│
┌────────▼────────┐
│ Target System │
│ (ERP, HR, │
│ Payroll) │
└─────────────────┘Deep Dives
Implementation
AI Infrastructure
Concrete technologies, cloud regions, hardware sizing, full tech stack breakdown.
Infrastructure in Detail >Knowledge Resource
Blueprint 2026
Eleven in-depth articles on the infrastructure decisions that matter in 2026.
Read the Series >Agents
AI Agents Overview
Document Agents, Workflow Agents, Knowledge Agents - three agent types for enterprise processes.
Explore AI Agents >Governance
Governance Framework
EU AI Act, Cert-Ready, Co-Determination, Data Residency - all governance topics at a glance.
Governance Overview >Deep Dive in the Agent Briefing
Our article series for decision-makers implementing AI agents in the enterprise.
Frequently Asked Questions about the Reference Architecture
Why a dedicated architecture instead of standard LLM APIs?
Standard LLM APIs deliver language understanding - but no governance, no Audit Trail, no tenant isolation, no role-based access. The 7-layer architecture is the layer between LLM and enterprise system that provides exactly that. Without this layer, every LLM deployment remains a prototype.
Is the architecture model-agnostic?
Yes. The Model Layer is fully interchangeable. Currently supported: Claude, ChatGPT, Gemini, Llama, Mistral, DeepSeek, gpt-oss. New models are integrated without modifying any layer above. No vendor lock-in to a single LLM provider.
How does the architecture scale under production load?
Each layer scales independently. Agent Layer and Model Layer scale horizontally without changes to the Decision Layer or Integration Layer. Kubernetes-based deployment enables auto-scaling based on utilization.
How is tenant isolation enforced?
At the database level through Row-Level Security (RLS). Each tenant sees only their own data, rules, and Audit Trails. Isolation is enforced architecturally - not through application logic alone.
Can the architecture be introduced incrementally?
Yes. The layers are decoupled. A typical starting point: one agent (e.g. Document Agent) with Decision Layer, connected to an existing system. Additional agents, integrations, and governance capabilities are added step by step.
What distinguishes this page from the Infrastructure page?
This page describes the architecture pattern - which layers exist and why. The Infrastructure page describes the concrete implementation - which technologies, which cloud regions, which hardware. Architecture is the what, infrastructure is the how.
Architecture Assessment for Your Organization
We analyze your existing system landscape and show how the 7-layer architecture fits into your infrastructure.
Schedule a Conversation