CSE faculty Dr. Aakur is part of a multi-university collaborative project awarded a $1M National Science Foundation (NSF) grant to carry on fundamental research for the project entitled “Collaborative Research: RI:Medium: Understanding Events from Streaming Video - Joint Deep and Graph Representations, Commonsense Priors, and Predictive Learning". 

This multi-university collaborative project totaling $1M includes The University of South Florida, Florida State University, and Oklahoma State University. Dr. Aakur will work with Professor Sudeep Sarkar and Professor Anuj Srivastava in this four-year collaboration and lead the effort at OSU.

The goal of this proposed study is to formulate a computer vision-based event understanding algorithm that operates in a self-supervised, streaming fashion. The algorithm will predict and detect old and new events, learn to build hierarchical event representations, all in the context of a prior knowledge-base that is updated over time. The intent is to generate interpretations of an event that go beyond what is seen, rather than just recognition. This research pushes the frontier of computer vision by coupling the self-supervised learning process with prior knowledge, moving the field towards open-world algorithms, and needing little or no supervision.