deepset & Agentic AI – The Workbench, Not the Final Product

(Translated from original article appearing in Tagespiegel Background: Original Article)

By Elena Metz & Bastian Hosan
‍
When an application for long-term care assistance is received in the Borken district, the hard work begins for the local case workers. That’s because applications involve a lot of paperwork: some run up to 300 pages. 100 pages is typical. They are often scanned as loose pages and unsorted. Whoever reviews them must first break them down into individual documents. They must separate bank statements from official notices. Everything must be labeled correctly. Only then does the actual review begin—and with it, the real work.

Andre Funke heads the Department of Organization, Digitization, and IT for the Borken district. He knows the numbers.

“Over the past four years, we’ve had a 25 percent increase in applications. Bottom line, that 25 percent increase actually always means a 25 percent increase in staffing needs if you don’t take countermeasures,” he says. The district deliberately chose the
long-term care application to see how things could be improved.

“We’re not starting with the easiest one to see if it works, but with one of the most comprehensive applications. If it works here, then it will work in many other areas as well.”

Enter deepset. In the Federal Ministry of Digital Affairs’ (BMDS) Agentic AI Hub, the company has stepped up to provide the residents of the Borken district in the Münsterland region with a solution. According to the company, this saves about 35 percent of the work steps. The company promises AI costs of one euro per application.Two to three of the ten people working on these projects in Borken could thus be freed up for other tasks.

“We think pilot projects are great, but going live is even better,” says Anna Gründler, who is responsible for strategy and business development at deepset, had already said at the kickoff. She noted that real benefits only come once the system is in operation. “We’re still at the pilot stage and want to integrate the solution into live operations,” says Funke.

A Workbench Instead of a Single Application

deepset took a different approach than the other companies in the hub. Those who processed housing subsidies, meeting minutes, or information requests brought a finished product with them. Deepset brought the platform for those products. “For us, the product wasn’t a specific process, but the AI platform,” says Gründler. The ministry, too, did not classify the company under specialized procedures but rather under AI infrastructure—as a bridge between language models and sensitive specialized systems.

Milos Rusic describes his product as a modular system. You can “build anything that is an AI application” on it, says the co-founder and CEO of Deepset. “You can think of it like a workbench where nails, a hammer, and a tape measure are ready to go. We also provide the craftsmen. We’re independent of administrative procedures and processes,” says Rusic.

“We spoke with the case workers on site and asked what takes a lot of time and can be automated effectively,” says Gründler. “We built small agents for every step.” In Borken, there are three agents and three AI workflows. One splits up and names the documents; one transfers them into a knowledge graph; others handle the completeness check—including automatic follow-up requests—as well as the asset review, which reveals suspicious account transactions from past years. At the critical points, a human remains involved to give final approval to a generated text before it is released. “Human in the Loop,” one of the much-praised AI principles in public administration.

Government Can Move Quickly

For deepset, it was less the technology and more the pace of the administration that was remarkable. “They were really quick to implement it; I was impressed by their openness and speed,” says Gründler. In general, the world of municipal
digitalization is new territory for deepset.

The pace was also unusual for the administration. “This wasn’t a traditional AI project with a lengthy familiarization and approach phase,” says Funke. “We were all pretty much thrown in at the deep end together and knew from
the outset that we only had three months.”

At the core of this platform is the open-source framework Haystack, which deepset is developing. The company derives its promise of autonomy from this. The models used can be freely selected; an agent can be operated using a U.S. model just as easily as with an open-source or European provider.

No Longer a Young Startup

And deepset isn't a traditional startup any more. Founded in 2018, the company has offices in Berlin and New York. By AI standards, it therefore has a long history. When Google unveiled the BERT language model in late 2018, the team realized that a major shift was on the horizon. Deepset built a German BERT model early on, released the open-source Haystack in 2020, and in 2021 provided the technology behind a project by the financial regulator BaFin.

Two weeks ago, Deepset announced a partnership with the Munich-based startup Celonis. Celonis specializes in process analysis; its platform identifies inefficient business processes and automates them using AI. Together with Deepset, the goal is to create an integrated, autonomous AI platform for “mission-critical operations” in the public sector, as well as in cybersecurity, defense, law enforcement, and critical infrastructure.

These scenarios are reminiscent of the U.S. analytics software company Palantir. “Europe’s quiet response to Palantir,” read the headline in Wirtschaftswoche. Rusic remains unfazed by such comparisons, viewing them “relatively neutrally.” “There are certain use cases in which data processing is supported by AI. We see this as a market segment where Celonis and we are a good fit,” he explains. “We see many problems in the public sector that require such solutions,” says Rusic.

Scaling Is the Real Challenge

The AI Hub project is now facing a difficult question – the very one thatFederal Minister for Digital Affairs Karsten Wildberger (CDU) called the “billion-dollar question” at the hub’s closing ceremony: How does a solution from a single municipality make its way to the many, many municipalities across Germany? Deepset is feeling the impact of this problem firsthand. “We’re holding discussions with each municipality individually, and they all have a different setup,” says Rusic. Sometimes a regional IT service provider is involved, other times a data center with its own specifications.

What the company wants is a single location where a solution is securely hosted and a municipality can access it as a service with a single click. With its dynamic procurement system, the ministry is planning exactly such a solution, but full-scale operation isn’t scheduled to begin until early 2027. Deepset is neither willing nor able to wait that long. The company is currently working with Borken to refine a more streamlined package, and neighboring municipalities are also in talks, so that multiple examples can demonstrate that it works.

Standardization – Done Correctly

And then there’s the question of standards.

Standardization has clear advantages, says Rusic. It’s about reusability, consistent training, and greater speed. Haystack is already mentioned in the planned Germany Stack as a building block for AI orchestration.

The AI functions developed in the hub are predominantly in-house developments by the
companies, and these companies are not obligated to make them available as open source,
according to the BMDS. However, the ministry is ensuring “appropriate interfaces as well as a
container-based architecture.”