Interrogating the Machine: A Human-AI Workflow for Structured Extraction From Historical Sources

Author Information

Robin Cherix, University of Applied Sciences and Arts Western Switzerland, Switzerland
Melinda Fleury, University of Geneva, Switzerland
Elena Mugellini, University of Applied Sciences and Arts Western Switzerland, Switzerland

Abstract

Text-analysis technology has evolved rapidly with the advent of Large Language Models (LLMs). As these tools reshape how we process language, the humanities face a pressing question: can LLMs meaningfully support—rather than replace—the historian's craft of extracting information from primary sources? Extracting precise data such as events, characters, and locations from historical texts remains a labor-intensive process that resists scaling. While computational approaches such as named entity recognition have long been applied to textual sources, they typically deliver outputs without offering historians the means to interrogate or refine the extraction process itself. We argue that a human-AI collaboration enables historians to process more sources, more systematically, while retaining interpretive control. Our approach automatically extracts events, characters, and locations from historical texts. Historians then review, validate, correct, and enrich the proposed data. They can also interact with the LLM through a chatbot interface to query the reasoning behind each extraction, preserving transparency and scholarly agency. To evaluate this approach, we invited historians to test our system, which implements Mistral Large 2.1, on a 16th-century historical source. Elements extracted by the LLM closely matched those identified manually by historians, demonstrating the tool's reliability. Qualitative feedback further indicated that this human-AI collaboration empowered historians to work at greater scale while maintaining a clear understanding of what was extracted and why, notably through the ability to request explanations from the model.

Save as a Favourite

• Category: Humanities - Cyberspace/Technology

Posted by James Alexander Gordon

All Posts