Self-consistency
Sample the same prompt repeatedly and measure agreement. An optional LLM judge handles the uncertain lexical band.
Post-hoc hallucination detection for LLM outputs
LLM said it. Now verify it.
halluciguard does not change how you call a model. It evaluates the response afterward — consistency sampling, claim-level grounding, and citation verification — and fuses the signals into one trust score.
0signals
0providers
0trust score
MITlicense
01Why halluciguard
Three detectors, one verdict
Each signal catches a different failure mode. Together they cover what a single check always misses.
Claim-level grounding
Extract atomic claims, retrieve evidence sentences, and check entailment in one batched call with per-claim fallback.
Citation verification
Match citation-like spans to provided sources and flag numeric mismatches. Synchronous, no provider call needed.
Fused trust score
Weighted fusion into overall.score in [0, 1] with low / moderate / high levels — a triage signal you can route on.
Provider-agnostic
One small GuardProvider interface. Built in: OpenAI chat, OpenAI Responses, OpenAI-compatible gateways, Anthropic.
Resilience built in
withResilience wraps any provider with exponential-backoff retries and per-call timeouts that actually fire.
02Signals
How the score is built
Grounding carries the most weight; consistency supports it; unverified citations multiply the score down.
| Signal | Question it answers | Default weight | Share |
|---|---|---|---|
grounding |
Is each atomic claim supported by the supplied context? | 0.7 when present | |
consistency |
Do repeated samples agree semantically? | 0.3 when present | |
citations |
Do cited sources exist and do numbers match? | ×0.85 penalty when unverified |
Levels: score ≥ 0.8 → high, ≥ 0.5 → moderate, otherwise low. Defaults are a starting point — calibrate per domain before production.
03Install
One package, one provider SDK
The detection logic is self-contained. Install only the provider SDK you actually use.
1
Core library
Zero required dependencies beyond your provider SDK.
npm install halluciguard
2
Provider SDK — pick yours
Each provider is an optional peer dependency.
npm install openainpm install @anthropic-ai/sdk3
Verify
Type-check your integration and run your first evaluation.
npx tsc --noEmit04Live demo
Watch a response get graded
A simulated run of the real pipeline: claims extracted, evidence checked, samples compared, citations verified — then one score.
Model response
Supplied context
Analysis
idle
0.00
05How it works
Post-hoc by design
halluciguard never touches your generation path. You call the model exactly as before, hand the response to evaluate, and get a structured trust report back.
Every detector fails safe: batched entailment falls back to per-claim judging, then to lexical thresholds. Missing context throws early instead of producing a misleading score.
- checkConsistency — repeated sampling + optional LLM judge
- checkGrounding — claims, evidence, batched entailment
- verifyCitations — synchronous, provider-free
- evaluate — runs the signals and fuses the score
06Documentation
Read the module docs
Every subsystem has its own deep-dive, in English and Chinese.
HallucGuard & evaluation flow
The facade, the four methods, evaluate defaults, and the result shape.
Detectors
Consistency sampling, grounding's four steps, and citation matching rules.
Scoring
Weights, level thresholds, the citation penalty, and calibration guidance.
Providers & resilience
The GuardProvider interface, built-in providers, and withResilience.
Testing & calibration
Mock-provider suite, the pre-production calibration loop, known limits.
Text & similarity utilities
Tokenizer, bilingual sentence splitting, similarity, and JSON extraction.
Design notes
Principles, the detection algorithms, and the comparison landscape.
07FAQ
Frequently asked questions
Straight answers about what halluciguard detects and how to use the score.
What is halluciguard?
halluciguard is an open-source TypeScript library for post-hoc hallucination detection. It does not change how you call a model — it evaluates the response afterward with consistency sampling, claim-level grounding, and citation verification, then fuses the signals into a trust score.
Which providers are supported?
Built in: OpenAI chat.completions, the OpenAI Responses API, OpenAI-compatible gateways (via baseURL), and Anthropic. Anything else can be added by implementing the small GuardProvider interface — one complete method that returns raw text.
How is the trust score computed?
Grounding weighs 0.7 and consistency 0.3 when both are present (a single signal gets weight 1). Unverified citations multiply the result by 0.85. The score is clamped to [0, 1] and mapped to levels: ≥ 0.8 high, ≥ 0.5 moderate, otherwise low.
What does grounding actually check?
Four steps: extract atomic claims from the response, retrieve the most similar evidence sentence from your context for each, check entailment in one batched provider call (with per-claim then lexical fallback), and average the claim scores. grounded is true only when every claim is supported.
Does it work with Chinese text?
Yes. The tokenizer emits character unigrams and bigrams for Han text so short Chinese answers still produce a useful lexical signal, and sentence splitting handles both English and Chinese punctuation, including citation forms like 来源：WHO 2024.
Is halluciguard production ready? What is the license?
halluciguard is in beta: detectors are unit-tested with mock providers, and you should calibrate thresholds with your own model, domain, and risk tolerance before production use. It is MIT licensed and free to use.
Fluent is not the same as true.
Score every response before your users do.