GAUGE benchmark evaluates whether frontier AI models recognise uncertainty and act on it responsibly

LONDON , UNITED KINGDOM, July 17, 2026 /EINPresswire.com/ -- Eros Innovation today announced that Eros GenAI researchers Arjun Thilak R. and Ramkumar M. V. have been named among the four Grand Prize winners of the Google DeepMind × Kaggle “Measuring Progress Toward AGI: Cognitive Abilities” Hackathon.

Their winning benchmark, GAUGE: Measuring the Gap Between What LLMs Know and What They Do About It, introduces a multi-turn evaluation protocol designed to assess whether large language models not only recognise uncertainty but also respond appropriately by abstaining when an answer may be unreliable.

The benchmark addresses an important limitation in current AI systems: a model may accurately express uncertainty yet continue to act with unjustified confidence. In high-stakes environments such as government, education, healthcare, and enterprise systems, the ability to abstain and escalate a decision to a human operator can be as important as raw model accuracy.

GAUGE evaluates three connected dimensions:

- How accurately a model assesses its own confidence
- Whether it changes its behaviour when uncertain
- Whether confidence and decision-making are meaningfully connected

The Eros GenAI researchers evaluated eight leading models across four model families using a three-stage metacognitive testing protocol spanning mathematics, logic and factual knowledge.

Ridhima Lulla, Co-Founder and Co-President of Eros Innovation, said: “This recognition is an important milestone for Eros GenAI and demonstrates the calibre of original research emerging from our team. As AI systems become more powerful and increasingly autonomous, trust will depend not only on what a model knows, but on whether it understands the limits of that knowledge and behaves responsibly when uncertain. GAUGE directly supports our vision of building sovereign, culturally intelligent and accountable AI systems in which human oversight, safety and responsible action are designed into the architecture from the beginning.”

Eros Innovation intends to explore the integration of the GAUGE methodology into its wider model-evaluation and AI assurance framework, supporting the assessment of global, open-weight and Eros-developed models across reliability, uncertainty, safety and human-escalation behaviour.

The company also plans to extend this research into Cultural Intelligence, rights-aware AI, education, citizen services, and other sovereign AI applications.

Arjun Thilak R., AI Research Scientist at Eros GenAI, said: “A model that knows it is uncertain but continues to act with confidence creates a serious risk for human operators. GAUGE was designed to measure the relationship between self-awareness and responsible action, rather than evaluating accuracy alone.”

Ramkumar M. V., AI Research Scientist at Eros GenAI, said: “GAUGE moves beyond conventional single-answer testing by examining how a model assesses uncertainty across multiple turns and whether that self-assessment changes its behaviour. This is especially important for real-world AI systems, where a reliable model must know when to proceed, when to abstain, and when to seek human oversight.”

The achievement supports Eros Innovation's broader ambition to build a global Sovereign Cultural Intelligence platform that combines advanced model orchestration, Cultural AI, rights and identity infrastructure, trustworthy agents, and country-specific intelligence systems.

About Eros Innovation

Eros Innovation is a Sovereign AI and Cultural Intelligence company headquartered in the Isle of Man, with operations across the United Kingdom, India, and the Middle East. The Company is building trusted AI infrastructure for governments, enterprises and consumers through proprietary foundation models, multimodal AI platforms, Digital Public Infrastructure and intelligent applications spanning culture, education, healthcare and the creator economy.

At the core of the platform is the Eros Large Cultural Model (LCM) family, including Eros LCM-Vision, Eros LCVM-Voice, and Eros LCMM-Music, developed in collaboration with the Indian Institute of Technology Madras. Unlike conventional large language models trained primarily on publicly available internet data, the Eros models are built on a proprietary corpus of approximately 1.5 trillion rights-cleared, ethically sourced cultural tokens, derived from more than 11,000 films, 100,000 characters, music, dialogue, mythology, and cultural knowledge accumulated over four decades. The corpus has been independently valued at US$1.7 billion by OxValue.AI, a University of Oxford spin-out.

The Eros AI platform powers a portfolio of businesses including Eros Universe, Eros Brahmand, Eros AI Vidya, Eros Music World, Eros Wellness, Eros Experiences, and sovereign AI solutions for governments and enterprises. Built on a common AI infrastructure stack, these platforms enable the creation, protection, monetisation and responsible deployment of cultural and domain-specific intelligence at scale.

Eros Innovation's mission is to become the world's trusted Cultural Intelligence Infrastructure, building sovereign, ethical AI systems that preserve, enhance, and responsibly commercialise culture while advancing innovation across education, healthcare, tourism, media, digital identity and strategic national infrastructure.

Piyush Bhatia
Eros Innovation
email us here

Legal Disclaimer:

EIN Presswire provides this news content "as is" without warranty of any kind. We do not accept any responsibility or liability for the accuracy, content, images, videos, licenses, completeness, legality, or reliability of the information contained in this article. If you have any complaints or copyright issues related to this article, kindly contact the author above.