Course Overview
General Info: 2nd year course, CentraleSupélec, November 2019  January 2020
Lecture hours: Tuesday (8:3012:00), Friday (13:45  17:00)
Instructors: Fragkiskos Malliaros and Maria Vakalopoulou
Office hours: Right after class (or send us an email and we will find a good time to meet)
TAs: Mohamed El Amine Seddik, Yunshi Huang, Yoann Pradat, Jun Zhu
Piazza: piazza.com/centralesupelec/winter2020/2el1730/home
Machine learning is the scientific field that provides computers the ability to learn without being explicitly programmed (definition by Wikipedia). Machine learning lies at the heart of many realworld applications, including recommender systems, web search, computer vision, autonomous cars and automatic language translation.
The course will provide an overview of fundamental topics as well as important trends in machine learning, including algorithms for supervised and unsupervised learning, dimensionality reduction methods and their applications. A substantial lab section will involve group projects on a data science competition and will provide the students the ability to apply the course theory to realworld problems.
Schedule and Lectures
The topics of the lectures are subject to change (the following schedule outlines the topics that will be covered in the course). The slides for each lecture will be posted in
piazza just before the start of the class.
The due dates of the assignments/project are subject to change.
Lecture 
Date 
Topic 
Material 
Assignments/Project 
1  November 26  Introduction; Model selection and evaluation  Lecture 1  
2  November 29  Dimensionality reduction  Lecture 2  
3  December 3  Linear and logistic regression  Lecture 3  Assignment 1 out 
4  December 6  Probabilistic classifiers and linear discriminant analysis  Lecture 4 

5  December 13  Nonparametric learning and nearest neighbor methods  Lecture 5  Project proposal due on December 13 Assignment 2 out 
6  December 17  Support Vector Machines  Lecture 6  Assignment 1 due on December 17 
7  December 20  Treebased methods and ensemble learning  Lecture 7  
8  January 7  Neural networks  Lecture 8  
9  January 10  Introduction to deep learning Guest lecture by Dr. Stergios Christodoulidis (Institut Gustave Roussy)  Lecture 9  
10  January 14  Introduction to reinforcement learning Guest lecture by Dr. Nikolaos Tziortziotis (Tradelab R&D)  Lecture 10  Assignment 2 due on January 14 
11  January 17  Unsupervised learning: clustering  Lecture 11  
12  January 20  Exams   Project final report due on January 26 
[November 26] Lecture 1: Introduction; Model selection and evaluation
Introduction to machine learning, administrivia, course structure and overview of the topics that will be covered in the course. Overfitting and generalization. Biasvariance tradeoff. Training, validation and test sets. Crossvalidation. Evaluation of supervised learning algorithms. Basic concepts in optimization.
Reading:
Additional:
 M. Kuhn and K. Johnson. An Introduction to Feature Selection. Applied Predictive Modeling, pages 487519, 2013. [For the part of the lecture on feature selection].
 Concentration of the empirical risk (also here), lecture notes by Dimitris Papailiopoulos (UWMadison).
 Model evaluation, model selection, and algorithm selection in machine learning: Part I (Basics), Part II (Bootstrapping and uncertainties), and Part III (Crossvalidation and hyperparameter tuning). Interesting blog post by Sebastian Raschka, 2016.
 Convex Optimization: Algorithms and Complexity (Sections 1.1, 1.2, 1.3)
 Convex optimization and gradient descent, lecture notes by Nisheeth Vishnoi, EPFL (Sections 1.1, 1.2, 1.3)
[November 29] Lecture 2: Dimensionality reduction
Dimemensionality reduction techniques. Singular Value Decomposition (SVD). Principal Component Analysis (PCA). Multidimensional Scaling (MDS) and nonlinear dimensionality reduction.
Reading:
Additional:
 SVD and Low Rank Matrix Approximations, lecture notes by Tim Roughgarden and Gregory Valiant (Stanford University)
 Understanding Machine Learning: From Theory to Algorithms (Section 23.1)
 J. B. Tenenbaum, V. De Silva, and J. C. Langford. A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science, 290:5500, pp. 23192323, 2000
 S. T. Roweis and L. K. Saul. Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science, 290:5500, pp. 23232326, 2000
 M. Belkin and P. Niyogi. Laplacian eigenmaps and spectral techniques for embedding and clustering. In NIPS, 2001
[December 3] Lecture 3: Linear and logistic regression
Supervised learning models. Linear regression. Regularization. Linear classification models. Logistic regression. Maximum likelihood estimation.
Reading:
Additional:
[December 6] Lecture 4: Probabilistic classifiers and linear discriminant analysis
Bayes rule. Naive Bayes classifier. Maximum a posteriori estimation. Linear discriminant analysis (LDA).
Reading:
Additional:
[December 13] Lecture 5: Nonparametric learning and nearest neighbor methods
Introduction to nonparametric learning methods. Distance and similarity metrics. Nearest neighbor algorithms.
Reading:
Additional:
[December 17] Lecture 6: Support Vector Machines
Maximum margin classifier. Linear SVMs. Primal and dual optimization problems. Nonlinearly separable data and the kernel trick. Regularization and the nonseparable case.
Reading:
Additional:
[December 20] Lecture 7: Treebased methods and ensemble learning
Decision trees. Ensemble learning. Bagging and Boosting. The AdaBoost algorithm.
Reading:
Additional:
[January 7] Lecture 8: Neural networks
Introduction to neural networks. The perceptron algorithm. Multilayer perceptron. Backpropagation. Applications.
Reading:
[January 19] Lecture 9: Introduction to deep learning
Deep learning, CNNs
Reading:
[January 14] Lecture 10: Introduction to reinforcement learning
Intelligence agents, dynamic programming, Monte Carlo methods, temporal difference learning
Reading:
[January 17] Lecture 11: Unsupervised learning: clustering
Introduction to unsupervised learning methods. Data clustering. Hierarchical clustering. kmeans clustering. Spectral clustering.
Reading:
Additional:
Course Structure and Objectives
Structure
Each section of the course is divided into 1h30' lecture and 1h30' lab. The labs will include handson assignments (using Python) and will provide the students the opportunity to deal with ML tasks in practice.
Learning objectives
The course aims to introduce students to the field of machine learning by:
 Covering a wide range of topics, methodologies and related applications.
 Giving the students the opportunity to obtain handson experience on dealing with.
We expect that by the end of the course, the students will be able to:
 Identify problems that can be solved using machine learning methodologies.
 Given a problem, identify and apply the most appropriate algorithm(s).
 Implement some of those algorithms from scratch.
 Evaluate and compare machine learning algorithms for a
particular task.
 Deal with realworld data challenges.
Prerequisites
There is no official prerequisite for this course. However, the students are expected to:
 Have basic knowledge of probability theory and linear algebra.
 Be familiar with at least one programming language (e.g., Python or any language of their preference).
Reading material
There is no single requiered textbook for the course. We will recommend specific chapters from the following books:
 Shai ShalevShwartz and Shai BenDavid. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014.
 Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, 2011.
 Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Second Edition, Springer, 2017.
 Jure Leskovec, Anand Rajaraman, and Jeff Ullman. Mining of Massive Datasets. Cambridge University Press, 2014.
Evaluation
The evaluation of the course will be based on the following:
 Two assignments: the assignments will include theoretical questions as well handson practical questions that will familiarize the students with basic machine learning tasks.
 Project: The students are expected to form groups of 34 people, propose a topic for their project, and submit a final project report. Please, read the project section for more details.
 Final exam: Final exam in the material covered in the course.
The grading will be as follows:
Assignment 1 (individually):  10% 
Assignment 2 (groups of 34 students):  20% 
Project (groups of 34 students):  30% 
Final exam:  40% 
Academic integrity
All of your work must be your own. Don't copy another student's assignment, in part or in total, and submit it as your own work. Acknowledge and cite source material in your papers or assignments.
Project
Details about the project of the course have been posted on piazza.
Resources
Datasets
Software tools
Related conferences
Please find below a list of conferences related to the contents of the course (mostly in the field of machine learning and data mining. We provide the DBLP website of each venue where you can access the proceedings (papers, tutorials, etc).
Check out the website of each conference (e.g.,
KDD 2016 ) for more information.