How to test, evaluate, and roll out AI tools in newsrooms: lessons from Reuters

Artificial Intelligence is rapidly transforming journalism, offering new opportunities but also raising critical questions about trust, editorial integrity, and responsible adoption. For newsrooms, rigorous evaluation of AI tools is essential to ensure accuracy, fairness, and transparency. This workshop provides a hands-on framework for journalists and editors to systematically assess AI systems, drawing on Reuters’ experience and adaptable for diverse newsroom contexts. Participants will leave equipped to integrate robust evaluation into daily operations, safeguarding journalistic values while enabling innovation.
Areas to explore:
> Key criteria for evaluating newsroom AI: accuracy, bias, explainability, editorial alignment
> Building and applying an evaluation matrix for real-world use cases
> Integrating evaluation workflows into newsroom routines
> Governance and risk management for AI adoption
> Iterative testing and feedback: how to refine tools before full rollout
Reuters case study. Multi-layered AI tool and prompt evaluation framework – from proof of concept to production, balancing innovation with editorial standards. The framework includes performance metrics, editorial checks, explainability, and governance, applied iteratively throughout the AI lifecycle.
Audience engagement. After the case study, participants will be invited to share their own newsroom tools or workflows. The group will collaboratively discuss how to apply the evaluation framework to these examples, fostering practical learning and peer feedback.
Session deliverables:
> One-page evaluation checklist
> Scoring template for AI tools
> Workflow guide for iterative testing and feedback
> Guidance on governance and risk management
Organised in association with Reuters.