deepset Accelerates Enterprise Adoption of Domain-Specific Sovereign AI with Llama Stack

We’re excited to announce the integration of deepset and Meta’s Llama Stack to accelerate the development and deployment of domain-specific AI across enterprise, public sector, and defense use cases.
‍

This collaboration integrates Llama Stack’s modular and flexible deployment stack with deepset’s open source Haystack framework and AI platform to build enterprise solutions using agents,  retrieval-augmented generation (RAG), and other leading architectures for LLM application development. Together, we offer a modular, end-to-end foundation for organizations to build, orchestrate, and scale their own sovereign AI solutions with full control and transparency.
‍

A Unified Stack for Building and Deploying High-Purpose AI Applications

Through this collaboration, deepset and Llama Stack are enabling teams to combine:

• Performant on-premise deployment of open source models like the Llama collection, delivered through Llama Stack’s secure, model-agnostic inference and deployment layers.
  
  
• A modular AI development environment (Haystack and deepset AI Platform) that includes standardized components and interfaces for GenAI agents and pipelines, including retrievers, generators, rankers, agents, tools, guardrails, and observability.
  
  
• Full control over infrastructure enables safe and private deployment across cloud, hybrid, or on-prem environments, supporting highly regulated industries and privacy-sensitive workloads in public sector and defense. With deepset/Haystack and Llama Stack, organizations gain full ownership of the AI stack, from inference to the application layer.
  
  

Together, the joint stack allows companies to prototype, iterate, and ship LLM-native applications including agentic workflow automations, decisioning copilots, chat applications, and analytics assistants with full confidence, traceability, and customization.
‍

Technical Integrations

‍The partnership is grounded in deep technical compatibility:

• Inference: deepset integrates Llama Stack’s inference engine, enabling you to run any LLM  including Llama, Mistral, OpenAI, Anthropic, Ollama and others  through standardized components and interfaces in Haystack and the deepset AI Platform.
  
  
• Agents & Tools: Haystack’s agentic AI orchestration framework enables domain-specific retrieval, reasoning, tool invocation, conditional routing, guardrails, and looping logic fully compatible with structured function calling and multi-tool chaining supported by Llama Stack.
  
  
• Modular Pipeline Composition:  The Haystack open source framework allows users to customize pipelines, agents, data indexes, and tools which can be combined into reusable supercomponents to support multiple use cases at scale.
  
  
• Safety & Evaluation: Integration with Langfuse, Weights & Biases, DeepEval, and RAGAS provides observability and robustness for production-grade apps.
  
  
• Memory & Composition: The deepset platform supports per-session and longterm conversational memory to bring valuable user context into complex workflows over time.
  ‍
• Deployment: deepset applications are Kubernetes-native, packaged as containerized microservices, and can be deployed via development teams or professional services in any customer-controlled environment. This aligns directly with Llama Stack’s focus on simplifying on-prem deployments in collaboration with joint ecosystem partners like Dell, Oracle, and NVIDIA.

Our Vision for On-Premise, Open, and Interoperable AI

‍
‍deepset provides an open and interoperable Gen AI ecosystem, where organizations can:

• Run their stack locally or in the cloud without vendor lock-in.
• Choose the right model for the job (open or proprietary).
• Extend functionality with custom tools, agents, and pipelines.
• Maintain observability and control across every stage of the AI lifecycle.
  
  

deepset’s ability to offer a custom AI orchestration framework and platform out-of-the-box with built-in tracing, data indexing, deployment, and UI, making it a natural match for Meta’s scalable and customizable LLM stack.
‍

“Collaborating with Meta strengthens our ability to offer enterprise teams in regulated environments a complete and open foundation for Gen AI platform and application development from inference to orchestration,” said Milos Rusic, Co-Founder & CEO at deepset. “Together, we’re making it easier to build reliable, safe, and highly customized AI systems that organizations can truly own.”

‍

Start Building Today

• How to Integrate Llama Stack + Haystack
• Build with Llama Stack and Haystack Agents
• Llama Stack documentation

‍

‍