Student Projects

  • We offer a wide variety of projects in the areas of Machine Learning, Optimization and NLP. The list below is not complete but serves as an overview.
  • Students who are interested to do a project at the MLO lab are encouraged to have a look at our

    where we describe what you can expect from us (and what we expect from you).

  • Please send us a transcript of your marks at EPFL when applying for a project.
  • We are only able to supervise a limited number of projects each semester. Priority will be given to Master Thesis Projects (full time).

Available MSc, BSc and PhD Semester Projects

  • Distributed Machine Learning using PyTorch, TensorFlow and other frameworks
    (Practical project)
    The goal in this project is to build and evaluate robust, open-source, system-adaptive and correct implementations of distributed optimization schemes for machine learning, as part of our MLbench open source project. One direction is also to expand it to HPC, such as CSCS. Basic experience with open source projects will be required.
  • Restart strategies for SGD
    (Theoretical project)
    The goal in this project is evaluate and prove convergence guarantees for different strategies to restart stochastic gradient descent algorithms. Theoretical guarantees can be developed in the convex setting and evaluated for deep learning. Contact: Aymeric Dieuleveut
  • Adaptive choices of parameters for Least Squares regression and stochastic gradient descent in high dimension.
    (Theoretical project)
    Least squares regression is one of the simplest yet most important problem in ML. Though the behavior of the empirical risk minimizer is well understood in finite (small) dimension, features in modern datasets involve live in high dimensional spaces. Suitable methods to solve the problem are typically first order stochastic methods. The optimal number of steps, and/or the optimal step size are well understood from the theoretical point of view, and typically depend on the optimal function, and the decay of the Hessian matrix eigenvalues (and both these quantities are typically unknown). Implementation of  “optimal’’ algorithms then relies on using cross validation, which is not convenient. Deriving data dependent really stopping rules, averaging strategies or step size calibration is thus a crucial challenge. Contact: Aymeric Dieuleveut
  • Adaptive Importance Sampling to Speed up SGD and Related ML Algorithms
    (Theory and/or practical project)
    Using more important training examples more often during training is a promising direction for speeding up SGD. As the importances of the samples are changing, finding good adaptive distributions is the main challenge here. Importance sampling can be explored for coordinate descent (CD) and stochastic gradient descent (SGD) type of algorithms. (Related papers: Safe Bounds, Adaptive Sampling, DL applicationContact: Sebastian Stich
  • Text Steganography using Neural Networks
    (Practical project)
    See paper here. As a next step, we want to explore extensions to a joint adversarial training setting for steganography.
  • Computational Aspects of Learning Text Sequence Representations
    (Algorithm design and analysis), building on top of sent2vec (PyTorch available also)
  • Distributed Methods in Machine Learning
    (Theory and implementation)
    Study the convergence of optimization algorithms (like CD, SGD, SVRG) in a distributed setting. Contact: Sebastian Stich

    • Distributed SVRG: design of a new algorithm & convergence analysis. (Related paper: k-SVRG).
    • Decentralized Local SGD (Related paper: local SGD)
    • Decentralized SGD with Communication Compression (Related paper: Choco SGD)
    • Adaptive schemes for communication compression (Related papers: sparsified SGD, Normalized Gradients)
    • Basis Pursuit, explore new ideas for communication efficient SGD
  • Quasi-Newton Methods for Machine Learning
    (Theory and implementation)
    Quasi-Newton methods can substantially accelerate the convergence of first-order optimization algorithms. Contact: Sebastian Stich
  • Acceleration and Momentum for Deep Learning
    “Momentum” is indispensable for fast convergence of SGD. In this project, we try to have a new look at different momentum-schemes. Contact: Sebastian Stich
  • GAN training
    (Implementation and Theory)
    GANS often show impressive results, however, they are notoriously hard to train. In this project we investigate these issues in theory and practice (mostly on toy problems).
    (Related paper: GAN) Contact: Sebastian Stich
  • Derivative-free Methods for Machine Learning
    (Theory and implementation)
    The (theoretical) performance of derivative-free methods on machine learning problems (for instance non-convex problems) is not well understood. We want to explore extensions of the to more general settings. Contact: Sebastian Stich

    • Theory for Gradient Based Methods (Related paper: Gradient-Free)
    • Theory for Gradient Free Methods
    • Variance Reduced Methods (Related paper: VR-DFO)
  • Closing the Generalisation Gap of Adaptive Gradient Methods for Deep Learning
    (Practical project), please contact Tao for more details.
  • Model-Parallel Training of Deep Learning
    (Practical project), please contact Tao for more details.

Contact us for more details!

Interdisciplinary Projects

  • A Machine-Learning Platform for Emerging Viral Diseases in Africa

Background: In the wake of the recent Ebola and Zika epidemics, it is becoming clear that infectious disease surveillance in Africa should include highly sensitive screening tools with the capacity to identify unrecognized viral species, unexpected variants or emerging pathogens with epidemic potential. However, population-level, high-resolution, virome-wide genetic sequencing is financially and practically impossible in countries with limited resources.

Aims: This project aims to develop an intelligent diagnostic model able to short-list the most probable viral causes of a patient’s’ fever based on simple and inexpensive clinical and laboratory tests.

1) Based on our detailed clinical dataset of 2500 pediatric and adult febrile patients in Africa, implement and evaluate different machine-learning models for predicting the diagnosis and possible causal viruses of fevers in Africa.

2) Create a model based on simple diagnostic source data and check if it is competitive with the virus results from existing high-resolution genetic sequencing results. We will also study the evolution over time using data from new cohorts of patients.

3) Integrate the resulting system into an easy-to-use mobile app (existing template) with a centralised database for real-world use in developing countries.

Contacts: Mary-Anne Hartley, UNIL&CHUV, and Martin Jaggi, EPFL

  • Dream Prediction from EEG data

The goal of this project is to apply machine learning to predict when (and what!) a sleeper is dreaming based on electroencephalogram (EEG) time series. We have access to a large dataset consisting of 500 Hz signals from 256 electrodes over the 2 minutes before awakening, labeled by whether the sleeper remembered a dream or not. We will propose to use a convolutional neural network (CNN) to classify these EEG time series.

Completed Thesis Projects

Master Theses (internal at EPFL):

Andreas Hug, 2018, Unsupervised Learning of Embeddings for Detecting Lexical Entailment
Jean-Baptiste Cordonnier, 2018, Convex Optimization using Sparsified Stochastic Gradient Descent with Memory
Lie He, 2018, COLA: Communication-Efficient Decentralized Linear Learning
Wang Zhengchao,  2017,  Network Optimization for Smart Grids
Marina Mannari,  2017,  Faster Coordinate Descent for Machine Learning through Limited Precision Operations
Matilde Gargiani,  2017,  Hessian-CoCoA: a general parallel and distributed framework for non-strongly convex regularizers

Anastasia Koloskova,  2017,  Coordinate Descent using LSH
Vasu Sharma,  2017,  CNNs for Unsupervised Text Representation Learning
Pooja Kulkarni,  2017,  Variable metric Coordinate Descent
Tao Lin,  2017,  Adversarial Training for Text
Tina Fang,  2017,  Generating Steganographic Text with LSTMs
Valentin Thomas,  2017,  Model-parallel back-propagation
Anahita Talebi Amiri,  2017,  Lasso – Distributed and Pair-Wise Features

Semester Projects:
William Borgeaud,  2017,  Adaptive Sampling in Stochastic Coordinate Descent
Castellón Arevalo Joel,  2017,  Complexity analysis for AdaRBFGS: a primitive for methods between first and second order
Alberto Chiappa,  2017,  Asynchronous updates for Stochastic Gradient Descent
Ahmed Kulovic,  2017,  Mortality Prediction from Twitter
Arno Schneuwly,  2017,  Mortality Prediction from Twitter
Sina Fakheri,  2017,  A Machine-learning Mobile App to support prognosis of Ebola Virus Diseases in an evolving environment
Remy Sun,  2017,  A Convolutional Dictionary Method to Acquire Sentence Embeddings
Hakan Gökcesu,  2016,  Distributed SGD with Fault Tolerance
He Lie,  2017,  Distributed TensorFlow implementation of sparse CoCoA
Oberle Jeremia,  2016,  A Machine-Learning Prediction Tool for the Triage and Clinical Management of Ebola Virus disease
Akhilesh Gotmare,  2016,  ADMM for Model-Parallel Training of Neural Networks

Francesco Locatello: Greedy Optimization and Applications to Structured Tensor Factorizations,
Master thesis, ETH, September 2016

Dmytro Perekrestenko: Faster Optimization through Adaptive Importance Sampling,
Master thesis (jointly supervised with Volkan Cevher), EPFL, August 2016

Elias Sprengel: Audio Based Bird Species Identification using Deep Learning Techniques,
Master thesis (jointly supervised with Yannic Kilcher), ETH, August 2016

Jonathan Rosenthal: Deep Learning for Go
Bachelor thesis (jointly supervised with Yannic Kilcher and Thomas Hofmann), ETH, June 2016

Maurice Gonzenbach: Sentiment Classification and Medical Health Record Analysis using Convolutional Neural Networks,
Master thesis (jointly supervised with Valeria De Luca), ETH, May 2016

Jan Deriu: Sentiment Analysis using Deep Convolutional Neural Networks with Distant Supervision,
Master thesis (jointly supervised with Aurelien Lucchi), ETH, April 2016

Pascal Kaiser: Learning city structures from online maps,
Master thesis (jointly supervised with Aurelien Lucchi and Jan Dirk Wegner), ETH, March 2016

Adrian Kündig: Prediction of Cerebral Autoregulation in Intensive Care Patients,
Master thesis (jointly supervised with Valeria De Luca), ETH, January 2016

Bettina Messmer: Automatic Analysis of Large Text Corpora,
Master thesis (jointly supervised with Aurelien Lucchi), ETH, January 2016

Tribhuvanesh Orekondy: HADES: Hierarchical Approximate Decoding for Structured Prediction,
Master thesis (jointly supervised with Aurelien Lucchi), ETH, September 2015

Jakob Olbrich: Screening Rules for Convex Problems,
Master thesis (jointly supervised with Bernd Gärtner), ETH, September 2015

Sandro Felicioni: Latent Multi-Cause Model for User Profile Inference,
Master thesis (jointly supervised with Thomas Hofmann, and 1plusX), ETH, September 2015

Ruben Wolff: Distributed Structured Prediction for 3D Image Segmentation,
Master thesis (jointly supervised with Aurelien Lucchi), ETH, September 2015

Simone Forte: Distributed Optimization for Non-Strongly Convex Regularizers,
Master thesis (jointly supervised with Matthias Seeger, Amazon Berlin, and Virginia Smith, UC Berkeley), ETH, September 2015

Xiaoran Chen: Classification of stroke types with SNP and phenotype datasets,
Semester project (jointly supervised with Roqueiro Damian and Xiao He), ETH, June 2015

Yannic Kilcher: Towards efficient second-order optimization for big data,
Master thesis (jointly supervised with Aurelien Lucchi and Brian McWilliams), ETH, May 2015

Matthias Hüser: Forecasting intracranial hypertension using time series and waveform features,
Master thesis (jointly supervised with Valeria De Luca), ETH, April 2015

Lei Zhong: Adaptive Probabilities in Stochastic Optimization Algorithms,
Master thesis, ETH, April 2015

Maurice Gonzenbach: Prediction of Epileptic Seizures using EEG Data,
Semester project (jointly supervised with Valeria De Luca), ETH, Feb 2015

Julia Wysling: Screening Rules for the Support Vector Machine and the Minimum Enclosing Ball,
Bachelor’s thesis (jointly supervised with Bernd Gärtner), ETH, Feb 2015

Tribhuvanesh Orekondy: dissolvestruct – A distributed implementation of Structured SVMs using Spark,
Semester project, ETH, August 2014

Michel Verlinden: Sublinear time algorithms for Support Vector Machines,
Semester project, ETH, July 2011

Clément Maria: An Exponential Lower Bound on the Complexity of Regularization Paths,
Internship project (jointly supervised with Bernd Gärtner), ETH, August 2010

Dave Meyer: Implementierung von geometrischen Algorithmen für Support-Vektor-Maschinen,
Diploma thesis, ETH, August 2009

Gabriel Katz: Tropical Convexity, Halfspace Arrangements and Optimization,
Master’s thesis (jointly supervised with Uli Wagner), ETH, September 2008