New: our AI Act guide for SMEs is now available Download โ†’
VULYX VULYX
Free AI diagnostic โ†’
LLM reliability Research note Completed

EXP-011Measuring hallucinations in a document assistant

Jun 20269 min read
AT A GLANCE
The questionMeasuring hallucinations in a document assistant What we did[Protocol summarized in one sentence.] What we learned150-question evaluation protocol published

Context and hypothesis

[Why this question matters for an SME or mid-sized company, and what we expected to find before starting.]

Protocol: data, models, metrics

DATA
[Dataset]
TECHNOLOGIES TESTED
RAG ยท LLM-as-a-judge ยท Expert annotation
METRICS
[Metrics]

Results

FIG. 1 ยท [CHART TITLE]SAMPLE DATA
Baseline48%
Variant A64%
Variant B72%
Human88%

Limitations

[What this experiment doesn't allow us to conclude: sample size, possible biases, untested conditions.]

What this means for your business

[Translating the result into a concrete decision for a business leader.]

Resources

RELATED SERVICE ยท GENERATIVE AI & AGENTSApply this result to my business โ†’
RELATED EXPERIMENTS
EXP-014 ยท Autonomous agents Can an agent close a level-1 support ticket on its own? EXP-013 ยท Frugal AI Small models vs. large models: where's the real sweet spot? EXP-012 ยท Vision & multimodality Detecting a surface defect with 50 training images