Attention: A Machine Learning Perspective

    Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review


    We review a statistical machine learning model of top-down task driven attention based on the notion of ‘gist’. In this framework we consider the task to be represented as a classification problem with two sets of features — a gist of coarse grained global features and a larger set of low-level local features. Attention is modeled as the choice process over the low-level features given the gist. The model takes its departure in a classical information theoretic framework for experimental design. This approach requires the evaluation over marginalized and conditional distributions. By implementing the classifier within a Gaussian Discrete mixture it is straightforward to marginalize and condition, hence, we obtained a relatively simple expression for the feature dependent information gain — the top-down saliency. As the top-down attention mechanism is modeled as a simple classification problem, we can evaluate the strategy simply by estimating error rates on a test data set. We illustrate the attention mechanism on a simple simulated visual domain in which the choice is over nine patches in which a binary pattern has to be classified. The performance of the classifier equipped with the attention mechanism is almost as good as one that has access to all low-level features and clearly improving over a simple ‘random attention’ alternative.
    Original languageEnglish
    Title of host publication2012 3rd International Workshop on Cognitive Information Processing (CIP)
    Number of pages6
    Publication date2012
    ISBN (Print)978-1-4673-1877-8
    Publication statusPublished - 2012
    Event3rd International Workshop on Cognitive Information Processing (CIP) - Baiona, Spain
    Duration: 28 May 201230 May 2012


    Workshop3rd International Workshop on Cognitive Information Processing (CIP)
    Internet address


    Dive into the research topics of 'Attention: A Machine Learning Perspective'. Together they form a unique fingerprint.

    Cite this