TLDR
Key Metrics:
AI Agents are often portrayed as the most advanced and powerful evolution of AI systems. This can create the perception that enterprises should inevitably move toward autonomous agents and away from deterministic AI workflows like retrieval augmented generation (RAG).
But what we observe when working on real-world use cases with our customers is that deterministic and agentic approaches exist on a spectrum, with neither inherently superior to the other. The most effective AI implementations strategically position themselves on this spectrum based on the requirements of their individual use case rather than following industry hype.
The field of AI is complex and evolving at breakneck speed. Yet our recommendation remains consistent, just as it has been over the years: Develop a strong understanding of all the technology options across this spectrum – their capabilities, cost, limitations, and risks – as well as your specific use case requirements. In this article, we’ll explore what this means in practice and how to find the right balance for your AI application.
Deterministic vs. agentic AI systems
Let's take a closer look at determinism and autonomy in AI systems. Note that although it may sound like a binary opposition, both deterministic and agentic AIs exist within the same space of Compound AI systems, which consist of modular, interacting components. That's why it's so natural to combine the two approaches.
Deterministic AI systems
Deterministic AI systems follow clearly defined, predictable processes set by their developers. They operate like assembly lines, with each component performing a specific function in a predetermined sequence.
A classic example is a basic RAG pipeline that consistently executes the same steps:
- It takes a user query
- Retrieves relevant documents from a database
- Ranks them by relevance
- Generates a response using an LLM based on this retrieved information.
The flow of information is unidirectional and fixed, making them highly reliable and efficient for consistent and predictable tasks.
Agentic AI systems
Agency in AI systems involves greater autonomy to dynamically determine the steps required to complete a task. Rather than strictly following predefined sequences, agentic systems can plan, adapt, and use appropriate tools based on the evolving context of a problem. They also move away from unidirectionality by introducing looping: an agent can traverse the same section of a path a number of times if it deems it necessary to do so.
However, this flexibility introduces complexity. By removing predefined pathways, agentic systems must navigate a vastly expanded decision space, which can lead to inefficiencies or unpredictable behavior if not carefully constrained.
Hybrid approaches
Real-world AI implementations often don't exist at either extreme but rather occupy positions along this spectrum. They strategically combine the deterministic with the autonomous approach to balance reliability with flexibility. Recognizing the power of this combination enables teams to design systems that appropriately match their specific requirements.
Adopting an outcome-based strategy
The development of effective AI systems begins with understanding desired outcomes. This process is inherently use case-driven rather than technology-driven. You need to first answer the question "What do my users need and expect from this tool?", before you can ask "What type of implementation will best deliver those outcomes?".
When adopting this approach, consider that starting from the user experience extends beyond just technical implementation choices. You'll need to account for accessibility, user interface design, acceptable latency thresholds, integration within existing workflows, and other factors that impact how users interact with your system.
Additionally, AI product development rarely follows a linear path – it's iterative by nature. This typically means starting with a barebones design that allows you to understand what's possible before gradually maturing your product and adding capabilities as you learn.
Iteration for the win
This iterative process often leads to discoveries about where your implementation should fall on the deterministic-agency spectrum.
Moving towards determinism
You might start with more agency only to discover your use case requires less flexibility than anticipated. For example, when implementing a customer segmentation tool, a team might initially design an agent that dynamically chooses analytical approaches based on the data. Through testing, they might find that a predefined workflow with clear analysis steps provides more consistent results and makes it easier to explain the segmentation logic to business stakeholders.
Moving towards agency
Conversely, you might start with a highly deterministic approach only to find that certain aspects benefit from more flexibility. A customer support AI might begin as a straightforward RAG pipeline that retrieves policies and generates responses. Over time, as you observe how users interact with the system, you might discover that complex inquiries spanning multiple policy areas require a more adaptive approach. Introducing agentic capabilities that can decompose these queries, research different knowledge domains, and synthesize comprehensive answers significantly improves customer satisfaction for these edge cases while maintaining the efficient deterministic flow for more standard requests.
Tips for finding the sweet spot between autonomy and regulation
When developing AI applications, a few key principles can guide your implementation decisions.
Start simple
Start with simple, deterministic foundations and add complexity only where it demonstrably improves outcomes. This helps keep costs low and avoids introducing unnecessary complexity and overhead.
Build modularly
Instead of building monolithic applications, use specialized components with single responsibilities. This makes your system easier to test, maintain, and modify. Modular design also enables you to apply different levels of determinism or agency to different parts of your workflow based on your specific requirements.
Understand Agents
To build successful agents, it's useful to understand the conditions under which they work best. While they can be extremely complex and powerful, agents benefit from well-defined tasks and tools, as well as meaningful, helpful error messages. What's more, the principle of modularity extends to the agent design itself, where it can be easier to manage a multi-agent setup where each agent is responsible for a specific task and wields a few well-defined tools, rather than having one agent manage everything.
Center users
The most effective and widely used AI systems don't necessarily use the most advanced or newsworthy approaches. They're the ones that thoroughly understand user needs and use the right tools and techniques to meet those needs efficiently.
Success in AI implementation comes not from following industry hype, but from methodically matching technological approaches to the actual problems you're solving. Ready to start exploring which technology best fits your needs? Schedule a consultation to receive an assessment of your use case and how best to implement it.
Curious about building AI Apps and Agents?
Table of Contents
