Kernelizing feature matching

In my Ph.D. work, Arnulf Graf, Barbara Caputo and I came up with an easy extension to the Support Vector framework that would allow people to perform feature matching inside of the Kernel evaluation. Although our initial proof needed to be revised, this publication was the first to connect local feature methods with Kernel methods and to demonstrate significant gains in performance (Wallraven et al., 2003)!

What information is available at a glance from a scene? Can we create computer vision algorithms that capture the “gist” of a scene?

Humans are able to get the “gist” of a scene already after 150ms. Using perceptual experiments, we have shown that humans can estimate the horizon of a scene with high accuracy. First computational experiments indicate that this might be done using a global, frequency spectrum analysis (Herdtweck and Wallraven, 2010, 2013).

What information determines aesthetic judgments? Can a computer become an art critic?

With the recent advances in image processing, we ask how far state-of-the-art image measures can be used to model the aesthetic experience of observers looking at a painting. In a series of papers, we have shown that these measures already capture aspects of aesthetic judgments (e.g., Wallraven et al., 2009, Rigau et al., 2010) - the computer, however, will need much more training to become an art expert!

In another project, we have shown that although art is seen as more aesthetic than ordinary photographs, they are actually not remembered better. Several computational features were used to try to model memorability of either category, but also here we found that low-level features have limited power for prediction (Wallraven et al., 2015).

Evaluation of algorithms

In a visual ”Turing test”, we found that people were unable to tell the difference between computer graphics objects that were inserted into the scene and real objects, thus validating the approach.

Similarly, we were able to evaluate a sophisticated image interpolation technique using perceptual experiments to show that it produced the least perceptual artefacts.

In addition, several algorithms on creation of bas-reliefs were evaluated to see which methods would produce the most appealing effect.

Publications

1. C. Wallraven, B. Caputo, and A.B.A. Graf. Recognition with Local Features: the Kernel Recipe. In ICCV 2003, volume 2, pages 257–264. IEEE Press, 2003.
2. C. Herdtweck and C. Wallraven: Beyond the horizon: perceptual and computational estimates of horizon position. Applied Perception, Graphics, and Visualization (2010)