title	RAG EU AI Act
emoji	⚖️
colorFrom	blue
colorTo	indigo
sdk	gradio
sdk_version	4.44.1
app_file	app.py
pinned	false
license	mit
short_description	Offline, citation-enforced RAG over Regulation (EU) 2024/1689

RAG · EU AI Act

Production-shaped retrieval-augmented generation over Regulation (EU) 2024/1689 (the EU AI Act): structure-aware ingestion with data lineage, dense retrieval, enforced citations, a FastAPI service, a Gradio demo, and a reproducible evaluation harness.

Status: runs offline by default. The demo backend uses bundled EU AI Act excerpts, deterministic embeddings, in-memory cosine search, and an extractive answerer, so CI and the Hugging Face Space need no Qdrant instance, model download, network, GPU, or Anthropic key. Set RAG_BACKEND=live for the full Qdrant + Claude path.

See it answer (offline, no key)

$ uv run rag query "What obligations apply to providers of high-risk AI systems?"

Article 16: Article 16 sets obligations for providers of high-risk AI systems.
Providers shall ensure that their high-risk AI systems comply with the
requirements set out in Section 2 before placing them on the market or putting
them into service [1].

demo-extractive · 0.29 ms · grounded=True
  [1] Article 16: Article 16 sets obligations for providers of high-risk AI systems…

The [1] marker resolves to a specific retrieved passage; if the answer cited a passage that wasn't retrieved, the claim is dropped and grounded=false is shown instead of being hidden. See docs/runbook.md for a full demo walkthrough (and how the screencast above is recorded).

Reviewer fast path

If you are reviewing this as an AI, data, or software engineering portfolio project:

1. Run make check to see the engineering gate: ruff, formatting, strict mypy, and hermetic tests.
2. Run make eval to see retrieval quality, citation grounding, and latency metrics without any secret or external service.
3. Run uv run rag query "What obligations apply to providers of high-risk AI systems?" to inspect one answer with resolved citations.
4. Read docs/portfolio-review.md for the AI/DE/SWE signal map and docs/demo-script.md for a short walkthrough.

Why this exists

RAG demos are easy; RAG you'd put in front of a regulator is not. This project optimises for the things that separate the two: provenance you can audit, answers that cannot silently hallucinate, an evaluation loop, and a codebase that is testable without live infrastructure.

Why this isn't a generic "chat with a PDF":

• It is structure-aware, not flat: ingestion segments on the regulation's own units (articles, annexes) so a citation names Article 6 or Annex III, not "page 12, chunk 4".
• Citations are enforced, not decorative: a marker that doesn't resolve to a retrieved passage is dropped and the answer is flagged ungrounded.
• It is measured: hit-rate, attribution-precision, and abstention on a versioned question set with bootstrap confidence intervals, not a vibe check.
• It is clear about limits: misses, the dense-vs-hybrid trade-off, and the demo backend's inability to abstain are documented in EVAL.md, not hidden.

Highlights

• Hybrid retrieval: dense embeddings fused with BM25 lexical scoring via reciprocal rank fusion. On the 28-question eval set this lifts article hit-rate@5 from 0.786 (dense-only) to 0.893 (hybrid) by recovering discriminative-term queries the dense signal misses (audited both ways: see EVAL.md). Toggle with retrieval_mode: dense | hybrid.
• Enforced grounding: the model must cite [n] passages; unsupported claims are dropped and surfaced as is_grounded=false, never hidden.
• Measured the headline claim, not just asserted it: an attribution-precision metric checks each citation actually supports its claim (not just resolves), and an abstention metric checks the system declines unanswerable questions. Both are reported with 95% bootstrap CIs.
• Offline demo backend: RAG_BACKEND=demo runs from bundled article excerpts with deterministic hashing embeddings and in-memory cosine retrieval.
• Live backend: RAG_BACKEND=live wires sentence-transformers, Qdrant, and Claude through the same protocol interfaces.
• Structure-aware chunking: ingestion segments on the regulation's article and annex headings before windowing, so a chunk never straddles two matched units and each carries its unit_type, so every citation can name its governing article or annex. (Segmentation is unit-tested on fixtures; the heading regex is not yet re-audited against the full live EUR-Lex text.)
• Data lineage: every live index build writes a manifest with source hash, chunking parameters, and embedding model.
• Hermetic CI: lint, strict types, and end-to-end tests run with no secrets, no model downloads, and no network.

Architecture

See docs/architecture.md for more detail and docs/decisions.md for the decision log.

flowchart LR
    Q[Question] --> R[Retriever]
    C[EU AI Act chunks] --> E[Embedder]
    E --> S[Vector store]
    R --> S
    S --> A[Answerer]
    A --> G[Citation enforcement]
    G --> O[Grounded answer + sources]

    subgraph Demo backend
        DE[HashingEmbedder]
        DS[InMemoryStore]
        DA[DemoAnswerer]
    end

    subgraph Live backend
        LE[SentenceTransformer]
        LS[Qdrant]
        LA[Claude]
    end

Loading

Quickstart

# 1. Install core + dev tools, then run the test gate.
make install
make check

# 2. Run deterministic offline eval. No Qdrant, model download, or key needed.
make eval

# 3. Run the local UI in demo mode.
make install EXTRAS="--extra ui"
make ui

Non-secret defaults live in config.yaml. Secrets stay in .env or RAG_* environment variables.

Live backend

make install EXTRAS="--extra ml --extra ui"
export RAG_BACKEND=live
export RAG_ANTHROPIC_API_KEY=sk-...
docker run -p 6333:6333 qdrant/qdrant
uv run rag ingest
uv run rag query "What practices are prohibited under the EU AI Act?"
uv run rag serve

API

curl -s localhost:8000/query \
  -H 'content-type: application/json' \
  -d '{"question":"What obligations apply to high-risk AI systems?"}' | jq

Returns the answer, resolved citations, retrieved passages with scores, model, latency, and is_grounded.

Evaluation

make eval runs the dependency-free baseline against eval/dataset.jsonl. It is a smoke / regression set: 28 answerable questions over representative articles plus 6 unanswerable probes, run on every push so retrieval, grounding, and refusal can't silently regress. It is deliberately small and offline: a guardrail, not a leaderboard.

Metric	Value	Note
Questions	28 + 6	28 answerable (18 baseline + 10 hard edge cases: Article 5 boundaries, cross-article, out-of-corpus probes) + 6 unanswerable abstention probes
Article hit-rate@5	0.893, 95% CI [0.786, 1.0] (28q, hybrid) · 0.962 excl. probes	demo backend, hybrid (the config.yaml default); 3 misses: 2 intentional out-of-corpus probes + 1 cross-article hard case. The CI is a 95% bootstrap interval: at N=28 the point estimate alone overstates precision. Hybrid lifts the 28q set from 0.786 (dense) → 0.893, and the original 18q from 0.944 → 1.000 (see EVAL.md)
Grounded-rate	1.000	every cited [n] resolves to a retrieved passage
Attribution-precision	1.000 (proxy)	fraction of cited claims the cited passage actually supports, not just resolves to; lexical-containment proxy, 1.0 by construction on the extractive demo. Closes the grounded-rate loophole; see EVAL.md
Abstention accuracy	0.000 (demo)	fraction of the 6 unanswerable probes the system correctly declines. The extractive demo backend reports 0.0 because it always answers from its nearest chunk. Abstention is a live-path property of the generative answerer. See EVAL.md
Latency p50 / p95 (ms)	0.4 / 0.5	offline determinism artifact, not a perf claim: the demo path is pure-Python hashing + in-memory cosine; real latency lives in the live embed + Qdrant + Claude path
RAGAS faithfulness / relevancy / context precision / recall	n/a	requires --extra eval + ANTHROPIC_API_KEY; run uv run python eval/run_eval.py

Metric history across runs is tracked in EVAL.md. Each run also writes a full per-question JSON report under eval/results/, and CI uploads it as a build artifact.

Project layout

config.yaml              # non-secret runtime defaults
requirements.txt         # Hugging Face Space install entry
app.py                   # Gradio demo
eval/                    # offline dataset + evaluation harness
data/fixtures/           # bundled demo corpus excerpts
src/rag_eu_ai_act/
├── config.py            # validated settings (config.yaml + RAG_ env prefix)
├── demo.py              # offline backend: hashing, in-memory store, answerer
├── factory.py           # backend-aware composition root
├── pipeline.py          # retrieve -> generate -> enforce citations
├── ingest/              # fetch, chunk, manifest
├── index/               # sentence-transformers + Qdrant live adapters
├── retrieve/            # dense + BM25 retrievers, RRF hybrid fusion
├── generate/            # Claude answerer + citation enforcement
└── api/                 # FastAPI app

Data & compliance

The corpus is public EUR-Lex text (CELEX 32024R1689). The demo backend bundles short article excerpts for deterministic offline behavior; the live ingestion path fetches the full text on demand and writes reproducible manifests. See data/README.md.

Regulatory timeline

The Act applies in phases. Getting these right matters for anyone reasoning about scope, so the milestones (per the Commission timeline):

Date	Milestone
2 Feb 2025	Prohibited-practice bans (Article 5) and AI-literacy duties apply
2 Aug 2025	GPAI model obligations + governance provisions begin
2 Aug 2026	Commission enforcement/penalty powers fully activate (fines up to 3% of global turnover or €15M); GPAI Code of Practice in force
2 Dec 2027	Stand-alone high-risk systems (Annex III) obligations apply (deferred from the original August-2026 reading)
2 Aug 2028	High-risk systems embedded in regulated products (Annex I)

Note the common misconception: August 2026 is the GPAI/penalty milestone, not the date stand-alone high-risk obligations land; those were moved to December 2027.

Roadmap

• Hybrid retrieval (BM25 + dense, reciprocal rank fusion): closed the eval miss, see EVAL.md. Live-backend BM25 reuses the same fusion.
• Cross-encoder reranking
• Deploy the Gradio demo to a public Hugging Face Space (sync workflow ready; needs the HF_TOKEN secret; see docs/runbook.md)
• Record and embed the CLI demo GIF (recipe in docs/runbook.md)
• Expand the eval corpus beyond 6 articles
• Multilingual corpus (DE/FR) via parallel EUR-Lex versions

Portfolio

This repository is part of a small portfolio of production-shaped AI / data / software engineering projects (see github.com/dkling-it):

1. RAG over the EU AI Act (this repo): production-shaped RAG with hybrid search (dense + BM25/RRF) and an evaluation harness, over EU regulatory text.
2. Agentic workflow: a multi-step agent with tool use and a human-in-the-loop approval checkpoint.
3. ML platform: medallion data lake → feature store → serving → drift monitoring.

License

MIT. See LICENSE.