Design notes

An open-source hallucination detection and fact-checking library for LLM outputs. Make LLM output trustworthy.

1. Background and problem

The best-known defect of LLMs is hallucination — fluent output that is actually fabricated.

Current pain points:

• In RAG apps, LLMs may invent citations that do not exist in the source documents.
• In high-risk scenarios like support, healthcare, or legal, hallucinations can cause real losses.
• Developers lack a lightweight tool to detect and control hallucination.

Existing option	Problem
Patronus AI / Galileo	SaaS products, closed source, vendor lock-in
SelfCheckGPT	Reference implementation of a paper, not a production tool
G-Eval / FActScore	Academic code, not directly integrable
LangChain + custom	Heavy DIY, no standard approach

Opportunity: the open-source community has no widely adopted hallucination detection tool.

2. Target users

• RAG application developers (the core audience)
• AI support / Q&A system developers
• Teams needing trust scores for AI output
• AI product QA / monitoring scenarios

3. Core design principles

Principle	Meaning
Post-hoc detection	Never changes the user's LLM call flow; detection happens after output
Multi-signal fusion	No single detection method; several signals are fused into one score
Configurable thresholds	Users decide the acceptable hallucination level per business scenario
Lightweight integration	One function call, no framework lock-in

4. API design

4.1 Self-consistency check

import { HallucGuard } from 'halluciguard'

const guard = new HallucGuard({
  provider: new OpenAIProvider({ apiKey: '...' }),
  model: 'gpt-4o-mini',
})

const result = await guard.checkConsistency({
  prompt: 'What is the capital of France?',
  samples: 3,             // sample 3 times
  temperature: 0.7,       // randomness is required to detect contradiction
})

// { consistent: true, confidence: 0.95,
//   responses: ['Paris', 'Paris', 'The capital of France is Paris'] }

4.2 Grounding check

const result = await guard.checkGrounding({
  context: sourceDocument,
  response: modelOutput,
})

// {
//   grounded: false,
//   score: 0.65,
//   claims: [
//     { text: '...', grounded: true,  evidence: '...' },
//     { text: '...', grounded: false, evidence: null },  // hallucination
//   ],
// }

4.3 Citation verification

const result = await guard.verifyCitations({
  response: modelOutput,
  sources: [
    { id: 'paper1', content: 'Smith et al. (2023) report 95% accuracy...' },
  ],
})
// Flags citations whose numbers do not match the matched source.

4.4 One-call evaluation

const report = await guard.evaluate({
  prompt, context, response: modelOutput,
  options: { consistency: true, grounding: true, samples: 3 },
})
// { overall: { score: 0.78, level: 'moderate' }, consistency: {...}, grounding: {...} }

5. Architecture

src/
├── index.ts               # unified exports
├── core/
│   ├── types.ts           # type definitions
│   └── guard.ts           # HallucGuard main class
├── detectors/
│   ├── consistency.ts     # self-consistency check
│   ├── grounding.ts       # grounding (claims + coverage)
│   └── citation.ts        # citation verification
├── extractors/
│   └── claims.ts          # claim extraction
├── scoring/
│   └── aggregator.ts      # multi-signal fusion
├── providers/
│   ├── base.ts            # abstract base
│   ├── openai.ts
│   └── anthropic.ts
└── utils/
    ├── similarity.ts      # text similarity
    └── text.ts            # text processing

6. Detection algorithms

6.1 Self-consistency

Principle: if an LLM answers the same question consistently across samples, it likely "knows"; if every sample differs, it is probably fabricating.

1. Sample N times with the same prompt at non-zero temperature (default 3–5).
2. Compute a cheap lexical confidence locally; inside the uncertain band, optionally blend an LLM-as-judge semantic estimate.
3. Score = agreeing pairs / total pairs; with the judge, the average of lexical and judge confidence.

6.2 Grounding

Principle: split the output into atomic claims and check each against the source document.

1. Claim extraction — LLM splits output into independent factual claims.
2. Evidence retrieval — find the most relevant passage per claim.
3. Entailment — batched LLM judging by default; falls back to per-claim on parse failure.
4. Score = grounded claims / total claims.

6.3 Citation verification

1. Extract citation markers (paper names, source names, data points).
2. Search the provided sources for a match.
3. For matched citations, verify that the numbers agree.

7. Dependencies

dependencies:     (all detection logic is self-contained)
peerDependencies: openai (optional) | @anthropic-ai/sdk (optional)
devDependencies:  tsup, typescript, vitest

No LangChain / LlamaIndex or other heavy frameworks.

8. Comparison

Feature	halluciguard	Patronus AI	SelfCheckGPT	GPTCache
Open source	Yes	No (SaaS)	Paper code only	Yes
Self-consistency	Yes	Yes	Yes	No
Grounding	Yes	Yes	No	No
Citation verification	Yes	No	No	No
Fused score	Yes	Yes	No	No
Lightweight integration	One function	SDK integration	DIY	Partial
Multi-provider	Yes	No	No	No
TypeScript-native	Yes	No	No	No

9. Release plan

• v0.1.0 — self-consistency + claim extraction + basic grounding
• v0.2.0 — citation verification + fused scoring + Anthropic provider
• v0.3.0 — embedding acceleration (local embeddings) + caching
• v1.0.0 — stable API, complete docs, production ready