Building reliable conversational AI systems remains a complex challenge. Despite rapid progress in large language models, developers and researchers still face fragmented datasets, inconsistent evaluation practices, and limited control over system behavior. These challenges make it difficult not only to ensure quality and trustworthiness, but also to reproduce results across different contexts.
To address this, the ELOQUENCE project introduces SDialog, an open-source toolkit designed to bring structure, transparency, and reproducibility to the full dialogue system pipeline.
Recently presented at EACL 2026 in Rabat, SDialog highlights how conversational AI can move from experimental setups toward more robust and scientifically grounded methodologies.
From fragmented workflows to reproducible pipelines
SDialog tackles some of the most pressing issues in dialogue system development by providing a unified framework that supports the entire lifecycle of conversational AI.
With SDialog, users can:
- Simulate multi-agent conversations with persona-driven LLMs
- Standardize datasets using a flexible JSON schema
- Evaluate dialogues with metrics and LLM-as-judge methods
- Steer and interpret model behavior with built-in controls
- Plug into tools like Hugging Face, OpenAI, or Ollama
By connecting these components into a coherent workflow, SDialog enables more consistent experimentation and easier comparison of results, key requirements for advancing trustworthy AI.
Toward more trustworthy and controllable AI systems
At its core, SDialog aligns closely with ELOQUENCE’s mission: improving the trustworthiness, explainability, and evaluation of conversational AI systems, particularly in multilingual and real-world settings.
By enabling reproducible experiments and better evaluation practices, SDialog helps researchers and practitioners move beyond trial-and-error development toward more controlled and interpretable AI systems.
A step forward for the AI community
As conversational AI continues to expand into critical domains, the need for reliable and transparent development tools becomes increasingly important. SDialog represents a step in that direction, supporting a shift from ad-hoc experimentation to reproducible, scalable, and trustworthy AI development.
Stay tuned for more updates as SDialog continues to evolve within the ELOQUENCE ecosystem.
