Out-of-Domain Finetuning to Bootstrap Hallucination Detection

Explores using out-of-domain data to bootstrap hallucination detection in LLMs. By pre-finetuning a BART-based NLI model on Wikipedia summaries (USB dataset) before finetuning on the Factual Inconsistency Benchmark (FIB) for news summaries, PR AUC improved from 0.69 to 0.85 — a 23% gain. Recall at threshold 0.8 jumped 25x (0.02 to 0.50) and precision rose 35%. The key insight: even though USB pre-finetuning alone didn't improve FIB performance, it primed the model to learn more effectively from FIB data, demonstrating that transfer learning via sequential finetuning on related open-source datasets can reduce the need for costly in-domain labeled data.