Haystack Enterprise Platform Expansion: Control AI Agent Costs and Improve Quality

At NVIDIA GTC Berlin 2026, we announced a major expansion of Haystack Enterprise Platform to connect agent development with production execution, observability, and optimization.

‍

Teams can build and deploy agents, inspect their behavior and costs in production, and use those insights to improve models, context, tools, and workflows, all within the same platform. This connected optimization loop helps organizations improve agent quality and efficiency while retaining control over their models, data, and infrastructure.

One platform to build, run, improve, and govern AI agents

The platform brings together four connected areas. Development and runtime provide the foundation; production traces and cost data feed improvements back into the agent harness, with governance controls across the lifecycle.

1. Agent harness and context: Control how agents behave

An agent's behavior depends on more than its underlying model. The context it receives, the tools it can access, its memory, and the instructions and guardrails that shape its actions all influence the result.

Built on open-source Haystack, the platform lets teams design and test workflows, connect models with enterprise data and tools, and inspect the components behind agent behavior.

Its modular architecture makes it possible to swap models, retrieval strategies, and other components without rebuilding an application.

2. Runtime and services: Run agents reliably in production

Haystack Enterprise Platform allows teams to publish version-controlled agents through stable service endpoints. Applications can continue to call the same endpoint even as the underlying agent is updated.

Serverless execution reduces the need to provision and manage dedicated infrastructure for each workload, allowing execution resources to scale with demand.

3. Optimization loop: Improve quality and reduce costs

Choosing a model is only one part of managing AI costs. How an agent is designed and orchestrated also determines how much inference it requires, how efficiently it completes tasks, and whether it produces the required results.

Production traces and cost reports help teams find expensive or unreliable steps in an agent workflow.

Teams can inspect individual runs and spending across models, agents, services, and workspaces. For example:

• Which agents or models account for the most AI spending?
• Where are errors, unexpected behaviors, or unnecessary execution steps occurring?
• Could changes to context, retrieval, model selection, or tool usage improve efficiency?
• How can teams assess whether an optimization maintains the required quality?

The next stage connects that visibility with evaluations and optimization.

Planned capabilities will help teams test model and harness changes against cost and quality objectives.

Over time, the platform will recommend improvements based on production evidence, with human approval before deployment.

4. Control plane and gateway: Govern agents across the organization

Organizations operating multiple agents need consistent controls for access, usage, and interaction with enterprise systems.

‍

Haystack Enterprise Platform brings together shared governance capabilities, including role-based access controls, workspace management, and operational visibility.

Its control plane and gateway architecture provides a foundation for making agents, tools, and skills accessible through standard interfaces, with common controls for permissions, policies, and usage.

What's new in this expansion

The expanded platform includes these production capabilities:

• Production tracing: Inspect agent execution and investigate failures or unexpected behavior.
• Cost reporting: Track AI spend by model, agent, service, and workspace, including alongside individual traces.
• Production services: Publish version-controlled pipelines through stable endpoints.
• Serverless deployments: Scale execution with demand without managing dedicated infrastructure for each workload.
• Flexible deployment: Choose managed cloud, sovereign cloud, or self-hosted infrastructure.
• Evaluations and more automated optimization recommendations are planned, building on the production visibility available today.

Sovereign AI, with flexibility across models and infrastructure

Deployment choice is essential for organizations with strict data, security, or regulatory requirements.

Haystack Enterprise Platform supports deployment in public cloud, European sovereign cloud environments such as STACKIT, and self-hosted infrastructure.

Together with Haystack's open-source, model-agnostic architecture, these options preserve flexibility across models and infrastructure.

Specialized AI model services

Alongside the platform expansion, deepset introduced custom AI model services for government, defense, and regulated industries.

‍

Our Forward Deployed Engineers help organizations select, adapt, and post-train decision models and small language models for specific tasks. These can work alongside LLMs in Haystack Enterprise Platform, with customers owning the models developed for them.

Toward an operating system for sovereign AI agents

We're building Haystack Enterprise Platform toward an operating system for sovereign AI agents: a common foundation to build, run, improve, and govern agents.

The expansion connects development and production operations today, with more systematic evaluation and optimization ahead.

‍

Explore Haystack Enterprise Platform and Model Services to learn more about building and operating sovereign AI agents.