AI Hallucinations: What DELEGATE-52 Shows for Businesses

Microsoft Research has published DELEGATE-52, a benchmark that tests 19 frontier LLMs in multi-stage document pipelines ^[1]. The result is uncomfortable: even the strongest models corrupt around 25 % of content when a document passes through them multiple times. If your agent stack is already passing content along today, AI hallucinations are no longer a rare anomaly but an expected outcome – and that changes the evaluation of every AI roadmap.

For executives and those responsible for AI, this means: pipeline length and handoff points become the risk variable, not the model alone. This article puts the study in context, connects it with independent findings on Context Rot ^[2] and vibe coding risks ^[3][4], and identifies five concrete measures you can start today.

How DELEGATE-52 Tests LLM Pipelines

The benchmark does not measure how well a model answers a question, but rather what happens when a document is passed through an LLM multiple times – a realistic setup for agents that summarize, translate, restructure, and output content. Microsoft calls this Round-Trip-Relay: content in, content out, content back in, up to twenty times.

The second dimension is domain breadth: the pipeline runs across 52 domains – Finance, Legal, Medical, Marketing, Code, and academic literature. This makes the results robust against the objection that “maybe this only applies to code.”

On average, approximately 170 documents are tested per domain, totaling 8,840. For evaluation, the authors combine Faithfulness (does the content remain accurate?) and Completeness (is information lost?) into a single pipeline score.

Frontier Models Lose a Quarter of Their Information

Table 1 is the core statement of the paper. It shows 19 LLMs – from open-source Llama 3 and Mixtral to closed frontier models – and their information loss per pipeline step.

Three observations are critical for executives:

• Frontier is not safe. GPT-4o, Claude 3.7 Sonnet, and Gemini 2.5 Pro all show approximately 25 % information loss after 20 hops. The gap compared to smaller models is real, but it does not change the order of magnitude.
• The error is cumulative. Per step, the best model loses around 1.3 %. That sounds harmless – until you multiply it twenty times.
• Domain matters, but it won't save you. Even in “simple” domains like marketing copy, the top models do not fall below 15 % loss.

“If your workflow contains more than three LLM handoffs, the probability of hallucination is no longer a model question, but a pipeline question.”

Paraphrased reading of Table 1, DELEGATE-52

Three Amplifiers for AI Hallucinations

The most compelling half of the study shows what amplifies the error. Microsoft tests this in two additional setups that closely mirror real-world pipelines.

Amplifier 1: Distractor Content

As soon as irrelevant context enters the prompt – a second paragraph, a second dataset, a footnote – information loss increases significantly. This is exactly what happens in any RAG setup when the retriever pulls too much.

Amplifier 2: Context Rot

Chroma Research independently showed in 2025 that LLMs do not scale on a “longer = better” basis: as the input token count grows, response quality declines even within the declared context window ^[2]. DELEGATE-52 measures precisely this effect within the pipeline. Both findings together provide methodological confirmation that Context Rot is not a tool bug, but an architectural property.

Amplifier 3: Multimodality

As soon as images are introduced, the effect is amplified further. Table 8 demonstrates this for image-editing pipelines – a realistic setup for anyone processing product images or reports with LLM agents.

The same mechanism can also be found in code generation: CodeRabbit and Veracode documented in 2025 that vibe coding pipelines – that is, rapid, multi-stage code generation without diff review – produce security vulnerabilities and silent hallucinations at significantly higher frequencies than monolithic generations ^[3][4].

What Executives Should Do Now

DELEGATE-52 does not change whether you use AI, but how you operationally safeguard it. Gartner estimates that for 2025, over 60% of AI pilot projects in German B2B companies are launched without formal risk management ^[5]. That is precisely the lever. Five concrete measures you can start today:

1. Pipeline-Length Audit. Count the LLM hops in every production agent stack. Anything over three hops belongs on a watchlist. Anything over five belongs in a diff-review pipeline.
2. Retrieval Instead of Round-Trip. Wherever possible: not “LLM rewrites what LLM wrote,” but “LLM responds with source reference.” RAG with source citation dramatically reduces round-trip distance.
3. Mandatory Diff Checks at Every Handoff. Every output between two LLMs gets a diff check against the original – either by a second model, through embedding similarity, or through deterministic regex checks on numbers and proper nouns.
4. Measure Domain Benchmarks Yourself. DELEGATE-52 is adaptable as a method. Task your developers with creating a mini-benchmark from 50 internal documents, run them through your stack ten times, and measure the drift. This costs less than a single workshop day and will change your model selection.
5. AI Risk Management as a Board-Level KPI. Glean and Adlib reported double-digit hallucination rates in production enterprise setups in 2025 ^[6]. This figure belongs in the same table as conversion rate and MRR – not in a separate tech report that nobody reads.

Conclusion

DELEGATE-52 turns AI hallucinations into a measurable system property. They are not a model bug that disappears with the next release, but an architectural effect of round-trip pipelines – an effect that scales with distractor content, long context, and multimodality. For every AI roadmap, this means: the question “which model?” is secondary to “how many hops and what handoff checks?”

“Trust but verify” has become operationalizable with DELEGATE-52. Anyone running AI pipelines in production should now test them like critical infrastructure – with domain benchmarks, pipeline-length limits, and diff checks at every handoff. That is the difference between a pilot that scales quickly and one that silently burns.

Sources

1. Laban, P., Schnabel, T., Rashkin, H., Neville, J. (2026) – LLMs Corrupt Your Documents When You Delegate. arXiv:2604.15597. Microsoft Research. arxiv.org
2. Chroma Research (2025) – Context Rot: How Increasing Input Tokens Impacts LLM Performance. research.trychroma.com
3. CodeRabbit (2025) – Vibe Coding Risk Report. coderabbit.ai
4. Veracode (2025) – AI-Generated Code Vulnerabilities. veracode.com
5. Gartner (2025) – Top Strategic Predictions: AI Hallucination Risk Management. gartner.com
6. Glean & Adlib (2025) – Enterprise surveys on LLM hallucinations in production workflows. glean.com