Pr. Philippe Mahey
Pr. Philippe Mahey

Affiliation

 

LIMOS, ISIMA, France 

 

Tutorial: Optimization methods for data analysis : some old and new algorithms

Abstract

We present an introduction to optimization techniques for computational statistics, data analysis and machine learning in the context of the celebrated 'Big Data' environment. After briefly recalling some of the main challenges of modern Data Analysis, we focus on the necessity to prepare and clean the huge amount of data collected before processing it which is exactly where specially taylored optimization algorithms may be useful. The basic data analysis task is thus to identify a mapping between the data elements and a small set of outputs (clusters, labels,...), the latter being used for further analysis of the data set or for predicting the influence of future data elements. The huge dimension of the original data set is coupled with additional difficulties like noisy perturbations, overfitting, missing data or labels, and online learning. 

We survey some elementary optimization algorithms from accelerated gradient method 'a la Nesterov' to coordinate-descent algorithms and Augmented Lagrangians. In the last section, we focus on monotone operator splitting techniques which have become very popular in the last decade for that purpose.From the classical ADMM (Alternate Direction Method of Multipliers) introduced in the 70's for decomposing nonlinear variational inequalities to recent Proximal Decomposition and block-iterative fixed-point methods, we show the main features and performance limits of these new trends for data analysis and machine learning.