Multiplicative updates for the LASSO

Morten Mørup; Line Katrine Harder Clemmensen

doi:10.1109/MLSP.2007.4414278

Multiplicative updates for the LASSO

Morten Mørup, Line Katrine Harder Clemmensen

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

668 Downloads (Pure)

Abstract

Multiplicative updates have proven useful for non-negativity constrained optimization. Presently, we demonstrate how multiplicative updates also can be used for unconstrained optimization. This is for instance useful when estimating the least absolute shrinkage and selection operator (LASSO), i.e. least squares minimization with $L_1$-norm regularization, since the multiplicative updates (MU) can efficiently exploit the structure of the problem traditionally solved using quadratic programming (QP). We derive an algorithm based on MU for the LASSO and compare the performance to Matlabs standard QP solver as well as the basis pursuit denoising algorithm (BP) which can be obtained from www.sparselab.stanford.edu. The algorithms were tested on three benchmark bio-informatic datasets: A small scale data set where the number of observations is larger than the number of variables estimated ($M

Original language	English
Title of host publication	2007 IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING : MLSP2007
Publisher	IEEE
Publication date	2007
Pages	33-38
ISBN (Print)	978-1-4244-1565-6
DOIs	https://doi.org/10.1109/MLSP.2007.4414278
Publication status	Published - 2007
Event	2007 17th IEEE Workshop on Machine Learning for Signal Processing - Thessaloniki, Greece Duration: 27 Aug 2007 → 29 Aug 2007 Conference number: 17 https://ieeexplore.ieee.org/xpl/conhome/4414264/proceeding

Conference

Conference	2007 17th IEEE Workshop on Machine Learning for Signal Processing
Number	17
Country/Territory	Greece
City	Thessaloniki
Period	27/08/2007 → 29/08/2007
Internet address	https://ieeexplore.ieee.org/xpl/conhome/4414264/proceeding

Bibliographical note

Copyright: 2007 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE

Keywords

Basis Pursuit Denoising (BPD)
Least Absolute Shrinkage and Selection Operator (LASSO)
Multiplicative Updates

Access to Document

10.1109/MLSP.2007.4414278

Imm5254Submitted manuscript, 405 KB
MørupFinal published version, 470 KB

OpenUrl availability

Full text

Cite this

@inproceedings{fa500ad9f9b34994af8c0c2f197a20c5,

title = "Multiplicative updates for the LASSO",

abstract = "Multiplicative updates have proven useful for non-negativity constrained optimization. Presently, we demonstrate how multiplicative updates also can be used for unconstrained optimization. This is for instance useful when estimating the least absolute shrinkage and selection operator (LASSO), i.e. least squares minimization with $L_1$-norm regularization, since the multiplicative updates (MU) can efficiently exploit the structure of the problem traditionally solved using quadratic programming (QP). We derive an algorithm based on MU for the LASSO and compare the performance to Matlabs standard QP solver as well as the basis pursuit denoising algorithm (BP) which can be obtained from www.sparselab.stanford.edu. The algorithms were tested on three benchmark bio-informatic datasets: A small scale data set where the number of observations is larger than the number of variables estimated ($M",

keywords = "Basis Pursuit Denoising (BPD), Least Absolute Shrinkage and Selection Operator (LASSO), Multiplicative Updates",

author = "Morten M{\o}rup and Clemmensen, {Line Katrine Harder}",

note = "Copyright: 2007 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE; 2007 17th IEEE Workshop on Machine Learning for Signal Processing, MLSP 2007 ; Conference date: 27-08-2007 Through 29-08-2007",

year = "2007",

doi = "10.1109/MLSP.2007.4414278",

language = "English",

isbn = "978-1-4244-1565-6",

pages = "33--38",

booktitle = "2007 IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING",

publisher = "IEEE",

address = "United States",

url = "https://ieeexplore.ieee.org/xpl/conhome/4414264/proceeding",

}

TY - GEN

T1 - Multiplicative updates for the LASSO

AU - Mørup, Morten

AU - Clemmensen, Line Katrine Harder

N1 - Conference code: 17

PY - 2007

Y1 - 2007

N2 - Multiplicative updates have proven useful for non-negativity constrained optimization. Presently, we demonstrate how multiplicative updates also can be used for unconstrained optimization. This is for instance useful when estimating the least absolute shrinkage and selection operator (LASSO), i.e. least squares minimization with $L_1$-norm regularization, since the multiplicative updates (MU) can efficiently exploit the structure of the problem traditionally solved using quadratic programming (QP). We derive an algorithm based on MU for the LASSO and compare the performance to Matlabs standard QP solver as well as the basis pursuit denoising algorithm (BP) which can be obtained from www.sparselab.stanford.edu. The algorithms were tested on three benchmark bio-informatic datasets: A small scale data set where the number of observations is larger than the number of variables estimated ($M

AB - Multiplicative updates have proven useful for non-negativity constrained optimization. Presently, we demonstrate how multiplicative updates also can be used for unconstrained optimization. This is for instance useful when estimating the least absolute shrinkage and selection operator (LASSO), i.e. least squares minimization with $L_1$-norm regularization, since the multiplicative updates (MU) can efficiently exploit the structure of the problem traditionally solved using quadratic programming (QP). We derive an algorithm based on MU for the LASSO and compare the performance to Matlabs standard QP solver as well as the basis pursuit denoising algorithm (BP) which can be obtained from www.sparselab.stanford.edu. The algorithms were tested on three benchmark bio-informatic datasets: A small scale data set where the number of observations is larger than the number of variables estimated ($M

KW - Basis Pursuit Denoising (BPD)

KW - Least Absolute Shrinkage and Selection Operator (LASSO)

KW - Multiplicative Updates

U2 - 10.1109/MLSP.2007.4414278

DO - 10.1109/MLSP.2007.4414278

M3 - Article in proceedings

SN - 978-1-4244-1565-6

SP - 33

EP - 38

BT - 2007 IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING

PB - IEEE

T2 - 2007 17th IEEE Workshop on Machine Learning for Signal Processing

Y2 - 27 August 2007 through 29 August 2007

ER -

Multiplicative updates for the LASSO

Abstract

Conference

Bibliographical note

Keywords

Access to Document

OpenUrl availability

Fingerprint

Cite this