Tuesday, April 08, 2008

Recognition by Association via Learning Per-exemplar Distances

Tomasz Malisiewicz, Alexei A. Efros. Recognition by Association via Learning Per-exemplar Distances. In CVPR, June 2008.


We pose the recognition problem as data association. In this setting, a novel object is explained solely in terms of a small set of exemplar objects to which it is visually similar. Inspired by the work of Frome et al., we learn separate distance functions for each exemplar; however, our distances are interpretable on an absolute scale and can be thresholded to detect the presence of an object. Our exemplars are represented as image regions and the learned distances capture the relative importance of shape, color, texture, and position features for that region. We use the distance functions to detect and segment objects in novel images by associating the bottom-up segments obtained from multiple image segmentations with the exemplar regions. We evaluate the detection and segmentation performance of our algorithm on real-world outdoor scenes from the LabelMe dataset and also show some promising qualitative image parsing results.



  1. Hi,
    I just read your CVPR2008 paper and I got a few questions that I'd like to ask you. Do not take them as critics, I am a PhD student just like you and I hate paper bashing. Plus your results look great, which is something that should be respected.

    Here we go:

    1) Could you give me a hint at the approximate time that is required to compute the distance weights? That must be awfully long, right?

    2) Similarly to your paper, I am currently working on an approach that aims at matching candidate object parts to one or several model parts. (The method that I am investigating is way different than yours though. The meat of my work is the localisation of interesting parts in the image and its representation, and I use very simple distance functions between my representations). This is for BMVC2008 (deadline in two weeks). I am having trouble finding ways to compare my results to others, as I guess you must have as well since no one uses the same LabelMe dataset. So:
    a) As a paper author, I was wondering if you had any reproaches from a reviewer about the lack of comparison with existing methods?
    b) In the future, will you consider releasing the subset of the LabelMe dataset that you used so that others can compare their results with yours?

    PS: I really like the idea of a research blog. I have thought about starting one for some time now because I feel like it could be a great tool for personal scientific conversations outside of conferences.

  2. Hi Regis,

    Finding the distance weights, or learning a distance function per-exemplars, is not that slow. The bottleneck is the distance computations. Given 13,000 exemplars spanning 171 unique labels whose distance functions I want to learn and 30,000 negative segments, it takes less than 6 hours to learn the weights. Since each learning problem is independent I never compute the entire (13,000 x 43,000 x 14) elementary-distance matrix.

    This is a total of 13,000x15 different numbers that are learned.

    In terms of using LabelMe, a problem is that it is a 'growing' dataset. However you can still use another algorithm (maybe get code from the author) and run it on your subset of LabelMe.

  3. jon huang8:51 PM

    Hey Tomasz - congrats on cvpr!