🔍 FusionRAG-Ex

An explainability-first Retrieval-Augmented Generation framework with multi-mode importance scoring.

What does this UI do?

FusionRAG-Ex lets you ask natural-language questions against a document collection. It retrieves the most relevant documents, generates an answer with a language model, and explains which parts of the retrieved context drove that answer using perturbation-based explainability. Three distinct scoring modes let you compare what the generator, the retriever, and their fusion each consider most important.

Key Features

• ExplainRAG tab — ask a question, see the generated answer, retrieved documents, and a highlighted context showing the most influential units for the selected importance mode.
• Comparative Analysis tab — runs all three importance-scoring modes in a single perturbation pass and renders side-by-side highlighted context panels, a ranked unit table with full score breakdowns, a counterfactual re-ranking probe, and a colour-coded importance map.
• Retriever Explorer tab — run the same query through Dense, BM25, and Hybrid retrievers simultaneously and inspect documents, relevance scores, and overlap.
• Three importance modes — Perturbation Dissimilarity (PD): pure generator-side signal; Retrieval-Weighted (RW): PD gated by the retrieval relevance of each unit’s source document; Confidence-Retrieval Fusion: blends response dissimilarity and generation confidence drop, then gates by retrieval relevance. The α slider controls the blend.
• Reliable highlighting — context panels only highlight units that are statistically above the mean score (mean × 1.5 floor). For RW and Fusion the single highest-scored unit is always surfaced; for PD the top 10% are shown.
• Retrieval Score Threshold — prevents the system from generating an answer when no retrieved document clears the minimum relevance score, reducing hallucination on out-of-distribution questions.
• Explanation levels — Word, Phrase (noun-chunks), Sentence, Paragraph — controls how finely the context is split for perturbation.
• Multi-dataset support — hot-swap between HotpotQA and TriviaQA (or any custom JSONL corpus) at runtime without restarting the server.
• Ground-truth evaluation — automatically resolves the question against the active dataset to show gold answer, gold-relevant documents, and retriever recall.
• Counterfactual diagnosis — if the answer is wrong, the system determines whether the failure is a retriever failure (gold evidence not retrieved) or a rank effect (evidence retrieved but ranked too low), and shows the answer that would have been produced if the gold document had been ranked first.

How to use it

Why is it useful?

RAG systems are often black boxes — you don’t know why a particular answer was generated or whether the right documents were retrieved. FusionRAG-Ex surfaces that reasoning at the unit level across three complementary lenses, helping you debug retrieval failures, compare retrieval strategies, identify hallucinations, and understand which parts of the knowledge base actually drive the model’s outputs.