Working from first principles in Bayesian probability theory and Shannon’s theory of communication, two Southern California researchers have developed a mathematical theory of surprise – and how the brain perceives novelty, importance, or noteworthiness. Pierre Baldi of UC Irvine and Laurent Itti of the University of Southern California developed the theory working with agents in a digital environment, and confirmed their findings with eye-tracking experiments using human subjects viewing dynamic stimuli in a variety of contexts.

The theory was so successful that Baldi and Itti were recently awarded a $600,000 NSF grant to test its validity further.

Itti hails from a computational neuroscience lab which seeks not only to model the human brain (specifically, how it delegates attention), but develop mathematically optimal algorithms with problem-solving applications in “automatic target detection in cluttered natural scenes, video compression, autonomous robotic nagivation on land or under water, or animation of virtual agents”. The result are algorithms that give a better bang-for-your-bit on certain attention tasks than the human brain does. Applied to vision compression, the attention delegation model was able to cut filesize in half by preserving only the information in the video judged to be salient.

Bayesian probability theory may be used in a context-independent way to judge the extent to which an incoming piece of information forces a rational agent to change his or her beliefs. Applied to attention studies, Bayesian models showed superior performance to artificially constructed models of salience or computations of Shannon entropy.

The ambitious goal of the project is to develop a model that naturally breaks down a pure information stream (like the sequence of bits comprising a video file) into “feature channels” which isolate salient features in the information such as color or shape. This has obvious applications in machine vision and other areas.

The researchers state: “At the foundation of our model is a simple theory which describes a principled approach to computing surprise in data streams. While surprise is not a new concept it had lacked a formal definition, broad enough to capture the intuitive meaning of the term, yet quantitative and computable… Beyond vision, computable surprise could guide the development of data mining, as it can in principle be applied to any type of data, including visual, auditory or text.”

In news articles the theory is portrayed as a model of human brain activity, but in actuality it goes beyond being a model. By working from first principles, the theory offers a recipe for attention-delegation that surpasses the capabilities of the human brain. Early applications might be the integration of “computable surprise” algorithms into a heads-up-display for soldiers on the field. Scanning a cluttered scene more rapidly than a human would be capable, such a system could alert the soldier to potential threats in advance of their noticing independently. Further applications would be in advanced Artificial Intelligence capable of formulating plans and accomplishing real-world goals.