TLDR
Key Metrics:
Agent development is a constantly changing, iterative process. Teams update prompts, switch models or data sources, refine retrieval strategies, and update business logic as they learn from testing and real-world usage. But the applications and workflows that depend on your agents need stability.
So how do you empower your AI engineering teams to work fast and test flexibly without the fear of breaking things? That’s where Services comes into play.
Services: stable endpoints for evolving AI
In Haystack Enterprise Platform, a Service is a stable, managed endpoint connected to a chosen pipeline version powering your agent. Apps and other agents connect to the Service rather than directly to the underlying pipeline. When a new pipeline version is ready, teams can publish it without changing the endpoint used by its consumers.
With Services, teams can:
- Publish versions safely: Choose exactly which pipeline version is live, record what changed, and keep the last successful version serving if a new deployment fails.
- Separate environments and consumers: Create independent Services for testing, staging, production, or individual applications and integrations.
- Configure each runtime: Set replicas and idle timeouts for every Service, or use recommended system defaults.
- Operate with visibility: Review traces, performance, connection details, and publishing history at the Service level.
Services create a reliable interface between an organization’s AI capabilities and everything that consumes them.
The foundation for our AI Gateway
AI Gateways are often associated with one endpoint for accessing multiple models, supported by routing, fallbacks, and spend tracking. That remains an important part of the category, but enterprise requirements are expanding.
AI applications increasingly depend on more than models. They use tools, pipelines, data, MCP servers, and other agents. Before you can control access, cost, security, or routing, you need a stable object those controls can apply to. Services provide that foundation.
In Haystack Enterprise Platform, Gateway is the product area where teams expose and operate the AI capabilities they build. Services are the first foundational capability inside it.
Get started
If you're running pipelines in production, Services are the recommended way to publish them. You create and manage them in Gateway, and existing deployed pipelines can be moved over when you're ready.
See Gateway and Create a Service in the docs to publish your first one.
Curious about building AI Apps and Agents?
meet the author
Table of Contents
.png)
