Roland Memisevic, Senior Director at Qualcomm
Live fitness coaching as a testbed for real-world conversational AI
Despite the immense recent progress in multi-modal language models, building AI systems that can have a real-world conversation about what is happening in front of the camera remains an extremely ambitious open problem. One reason is that existing models are trained to respond to prompts, instead of having a free-flowing live conversation. Another is that a natural real-world interaction can branch into many unforeseen directions at any point in time, making it hard to collect training data with sufficient coverage for models to generalize. In this talk I will argue that live fitness coaching is not only a highly useful task to solve. It also provides a perfect testbed for building AI models that see, hear and interact with a user in real time: on the one hand it requires a model to see and hear events as they happen, so it can respond appropriately. On the other hand, it is a highly constrained experience, much like a real-world “game”, consisting of a sequence of timed exercises and a strict definition of what counts as a correct or less correct action. As such, fitness coaching comes with clear expectations and guardrails, within which otherwise free-form interactions can be defined and collected as training data. I will discuss in detail our work in recent years towards building real-world streaming models that can coach you in real time. This includes the collection of large-scale training data and the exploration of efficient multi-modal streaming architectures. I will also discuss open problems, such as challenges in state tracking and multi-modal fusion.
Katie Mills, Football Research Consultant at FIFA & Research Fellow at Sheffield Hallam University
Advancing Football Through Research on a Global Stage
At FIFA, research and innovation are at the heart of the game’s evolution. The FIFA Research Programme fosters collaboration between academia, industry and football stakeholders to develop and validate innovations that enhance the game’s integrity, performance, and accessibility. Our work provides the empirical foundation to tackle some of football’s biggest technological challenges – from improving match officiating with semi-automated offside technologies, to enabling deeper tactical and performance analysis through advanced tracking systems. We’re also exploring alternative, lower-cost solutions to make these tools available across all levels of the game. By sharing what we learn and working openly with the global research community, we’re helping make football more accessible, data-driven, and ready for the future.