Evaluating LLMs Beyond Accuracy: Pragmatic Reasoning

Amir Feizpour is the founder, CEO, and Chief Scientist at Aggregate Intellect building a generative business brain for service and science based companies. Amir has built and grown a global community of 5000+ AI practitioners and researchers gathered around topics in AI research, engineering, product development, and responsibility. Prior to this, Amir was an NLP Product Lead at Royal Bank of Canada. Amir held a research position at University of Oxford conducting experiments on quantum computing resulting in high profile publications and patents. Amir holds a PhD in Physics from University of Toronto. Amir also serves the AI ecosystem as an advisor at MaRS Discovery District, works with several startups as fractional chief AI officer, and engages with a wide range of community audiences (

Tara Azin is a PhD candidate in Cognitive Science at Carleton University, where she is a member of the Language and Logic Lab (LOLA). Her research focuses on how large language models handle pragmatic reasoning and implicit meaning in language.