Make Custom AI Agents Work the Way You Do

deepset is a leader in framework and platform technology that accelerates AI application development with large language models (LLMs). As the creator of the deepset AI Platform and the Haystack open-source framework, deepset powers custom AI solutions in production across industries and government, earning recognition as a Gartner Cool Vendor in AI Engineering.

‍

AI agents are LLM-powered applications that can reason, reflect, and act. They use tools, data, and thoughts to solve problems. Unlike other, simpler AI systems, AI agents are capable of critical introspection into their own thinking and decision-making processes, and can choose between different courses of action and iterate through the same step multiple times. AI agents were envisioned by humans long before the technical capabilities were available, but now, thanks to the reasoning capabilities of large language models (LLMs) that resemble human thinking, these AI systems are beginning to take on tasks previously performed by knowledge workers.

Compound AI systems integrate multiple components, like LLMs, retrieval mechanisms, and external tools, to handle complex business tasks with greater control and efficiency. This shift from standalone models enables dynamic, advanced capabilities. deepset provides a platform for building compound AI systems, empowering organizations to create custom solutions that combine modular AI tools for maximum speed, trust, and impact.

AI agents solve tasks by dynamically using the tools at their disposal. They have autonomy over how they go about solving the task and in what order and how often they use the tools, although the degree of autonomy can vary widely. In practice, it makes sense to limit the agent's autonomy, for example, by setting an upper limit on the number of times a tool can be used in a call, and by including deterministic workflow elements. Agents can also include their human operators in a human-in-the-loop setup. Interaction between users and AI agents typically takes place in natural language through a chat interface.

deepset combines the open-source Haystack framework with a customizable platform and enterprise-grade expertise, enabling businesses to quickly build and deploy tailored AI solutions using agents, RAG, and other advanced AI methods with expert support.

AI agents as a solution are still evolving, so we cannot yet speak of a typology of AI agents. Driven by large language models (LLMs), they represent one of the most complex applications of Gen AI today, and the tools for building and maintaining them are still being developed. However, we can already observe that AI agents, just like Retrieval Augmented Generation (RAG) systems, are best designed as modular Compound AI systems. Compound AI is a design paradigm that enables highly customized AI-enabled products.

‍

deepset serves diverse industries, including finance, legal, media and publishing, government, healthcare, retail, and consumer goods. Organizations use deepset to tackle domain-specific challenges like streamlining document processing, enhancing search, enabling intelligent chat systems, and building custom AI agents, delivering tailored solutions to meet their unique needs.

The utility of AI agents lies in their ability to solve real-world, highly labor-intensive tasks independently or with minimal human assistance. As such, they work best when they are customized for the specific task at hand. This means that virtually every AI agent in production is a custom AI agent, with special components, tailored business logic and security measures, different database access and user interfaces, and so on. Compound AI facilitates the creation of custom AI agents that grow with their use case, meaning they should be reviewed and updated regularly to ensure their continued performance in evolving environments.

deepset accelerates AI development with 10x faster speed, 40% greater efficiency, and over 50% cost savings. It also fosters collaboration between business and technical teams, offers platform-agnostic flexibility, and centralizes AI management in a single pane of glass for streamlined oversight and control.

At deepset, we offer several pre-built templates for AI agents that already include key agent workflows and logic, allowing users to get started on their development journey. However, our recommendation (and the way to get the most value out of your AI agent) is to iterate on this prototype and extend the pre-built agent with additional functionality, tools, and logic to customize it to your business and use case. That's because in an increasingly AI-driven world, customization will be the key to building products that differentiate you from your competitors. Pre-built agents can also be very valuable for ad hoc use cases that require quick solutions and little or no customization.

An AI agent is a complex technology that incorporates Gen AI models, other machine learning models, custom logic, and multiple integrations with different data sources and other applications or APIs, which it uses as “tools.” When the AI agent is activated (usually through a user input), it starts devising a strategy to fulfill the user’s request. There are multiple techniques for that, some of which include up-front planning, chain-of-thought, and human in the loop. 

To get the most out of agents, organizations should deploy them in areas where complex decisions need to be made. Typically, these are tasks that would be complex for humans, requiring reasoning and the use of multiple tools and data sources. It is a good practice to develop more on the deterministic side, and then experiment with giving the agent more autonomy as you gain a better understanding of its capabilities and limitations. Users can be involved in the agent's decision-making process. Because agents are still emerging, there is still a lot of research and experimentation to be done on how to interface with users. Agents are an ideal technology to run in the background because they can do a lot of work under the hood and it is not always necessary to interact with them in real time. Of course, continuous evaluation and monitoring remain critical components of successful and secure agents.

The difference between AI agents and AI assistants is that one describes a design/architecture paradigm and the other a practical application of that architecture. However, the line between the two can often be blurred. The term "AI agent" incorporates the notion of agency, meaning that the machine mimics human autonomous behavior in solving tasks that involve reasoning, planning, and the targeted use of tools to achieve a specific goal. However, when we talk about "AI assistants," we mean that an AI-powered technology helps us accomplish our tasks, such as managing our schedules, reviewing our code, or researching a topic. In reality, AI assistants are often implemented as AI agents, but AI agents can cover a wider range of use cases than just acting as AI assistants.