Learning Intuitively: AI Predicts Human Behaviour from Videos

Abhinav Raj, Writer
@uxconnections

A Columbia University study unveiled a new computer vision technique that imparts intuitive sense to machines—allowing AI to leverage ‘higher-level associations’ to predict movements.

Life is unpredictable—but not human movements. At least for AI.

Engineers at Columbia University are leveraging predictive machine learning (a technique that involves estimating current and historical data) to “make better predictions about human behaviour” by analysing movies, sports games and popular TV shows like “The Office”.

The study, published in arXiv was presented virtually at the International Conference on Computer Vision and Pattern Recognition on June 24.

“Our algorithm is a step toward machines being able to make better predictions about human behaviour, and thus better coordinate their actions with ours,” said Carl Vondrick, assistant professor of computer science at Columbia, who has spearheaded the research. Vondrick believes that there is space for diversified applications of the algorithm in the market.

“Our results open a number of possibilities for human-robot collaboration, autonomous vehicles, and assistive technology,” he stated.

Image: Ahmad Odeh on Unsplash

The predictive computer vision technique can predict actions several minutes into the future, ranging from handshaking, fist-bumping, high-fiving or at times, even non-actions like “ignoring” by employing higher-level associations between people, animals as well as objects. Although there have been past attempts in predictive machine learning to foresee actions—the uncertainty gradient remained too high for most machine learning models to make accurate predictions by extrapolating using the available data.  

PhD students Didac Suris and Ruoshi Liu, however, have modelled their AI such that it employs unconventional geometries to predict human behaviour—allowing for the organisation of high-level concepts.

The PhD students’ attempt becomes the first to approach the long-range prediction conundrum by proposing a predictive model in the hyperbolic geometry—as opposed to its Euclidean counterpart that most modern machine learning algorithms incorporate and obey.

The algorithm introduces abstraction into its reasoning to account for uncertainty in its prediction. “Not everything in the future is predictable,” says Suris, the co-lead author of ‘Learning the Predictability of the Future’.

“When a person cannot foresee exactly what will happen, they play it safe and predict at a higher level of abstraction. Our algorithm is the first to learn this capability to reason abstractly about future events.”

Columbia Engineering’s effort in developing predictive modelling in ML may unfold a generation of machines adept at anticipating human behaviour—allowing for greater understanding between humans and machines and subsequently fostering trust. Researchers believe that machines that employ predictive reasoning can perform diversified roles consolidating security and safety.

Predictive machine learning, in that way, is the future.