Foundations of Machine Learning

Course Overview


Lecture room: Batiment Eiffel, CentraleSupélec

Instructor: Fragkiskos Malliaros
Email: fragkiskos.me [at] gmail.com
Office hours: Right after class (or send me an email and we will find a good time to meet)

TA: Sylvain Lannuzel
Email: sylvain.s.lannuzel [at] gmail.com

Edunao: https://centralesupelec.edunao.com/course/view.php?id=1095


Machine learning is the scientific field that provides computers the ability to learn without being explicitly programmed (definition by Wikipedia). Machine learning lies at the heart of many real-world applications, including recommender systems, web search, computer vision, autonomous cars and automatic language translation.

The course will provide an overview of fundamental topics as well as important trends in machine learning, including algorithms for supervised and unsupervised learning, dimensionality reduction methods and their applications. A substantial lab section will involve group projects on a data science competition and will provide the students the ability to apply the course theory to real-world problems.





Schedule and Lectures

The topics of the lectures are subject to change (the following schedule outlines the topics that will be covered in the course). The slides for each lecture will be posted in edunao just before the start of the class. The due dates of the assignments/project are subject to change.

Date Topic Material Assignments/Project
1October 8Introduction; Model selection and evaluationLecture 1
2October 15Dimensionality reductionLecture 2Assignment 1 out
3October 22Linear and logistic regressionLecture 3
4October 29Probabilistic classifiers and linear discriminant analysisLecture 4Project proposal due on November 1
Assignment 2 out
5November 5Non-parametric learning and nearest neighbor methodsLecture 5Assignment 1 due on November 8
6November 12Tree-based methods and ensemble learningLecture 6
7November 19Support Vector MachinesLecture 7
8November 26Unsupervised learning: clusteringLecture 8Assignment 2 due on November 29
9December 10Project presentations (TBD)Project final report due



[October 8] Lecture 1: Introduction; Model selection and evaluation

Introduction to machine learning, administrivia, course structure and overview of the topics that will be covered in the course. Overfitting and generalization. Bias-variance tradeoff. Training, validation and test sets. Cross-validation. Evalution of supervised learning algorithms.

Reading:
Additional:

[October 15] Lecture 2: Dimensionality reduction

Dimemensionality reduction techniques. Singular Value Decomposition (SVD). Principal Component Analysis (PCA). Multidimensional Scaling (MDS) and nonlinear dimensionality reduction.

Reading:
Additional:

[October 22] Lecture 3: Linear and logistic regression

Supervised learning models. Linear regression. Linear classification models. Logistic regression. Maximum likelihood estimation.

Reading: Additional:

[October 29] Lecture 4: Probabilistic classifiers and linear discriminant analysis

Bayes rule. Naive Bayes classifier. Maximum a posterioti estimation. Linear discriminant analysis (LDA).

Reading: Additional:

[November 5] Lecture 5: Non-parametric learning and nearest neighbor methods

Introduction to non-parametric learning methods. Distance and similarity metrics. Nearest neighbor algorithms.

Reading: Additional:

[November 12] Lecture 6: Tree-based methods and ensemble learning

Decision trees. Ensemble learning. Bagging and Boosting. The AdaBoost algorithm.

Reading: Additional:

[November 19] Lecture 7: Support Vector Machines

Maximum margin classifier. Linear SVMs. Primal and dual optimization problems. Non-linearly separable data and the kernel trick. Regularization and the non-separable case.

Reading:
Additional:

[November 26] Lecture 8: Unsupervised learning: clustering

Introduction to unsupervised learning methods. Data clustering. Hierarchical clustering. k-means clustering. Spectral clustering.

Reading:
Additional:

Project presentations

Presentation of the projects of each team.



Course Structure and Objectives

Structure
Each section of the course is divided into 1h30' lecture and 1h30' lab. The labs will include hands-on assignments (using Python) and will provide the students the opportunity to deal with ML tasks in practice.


Learning objectives

The course aims to introduce students to the field of machine learning by:
  • Covering a wide range of topics, methodologies and related applications.
  • Giving the students the opportunity to obtain hands-on experience on dealing with real-world problems.
We expect that by the end of the course, the students will be able to:
  • Identify problems that can be solved using machine learning methodologies.
  • Given a problem, identify and apply the most appropriate algorithm(s).
  • Implement some of those algorithms from scratch.
  • Evaluate and compare machine learning algorithms for a particular task.
  • Deal with real-world data challenges.



Prerequisites

There is no official prerequisite for this course. However, the students are expected to:

  • Have basic knowledge of probability theory and linear algebra.
  • Be familiar with at least one programming language (e.g., Python or any language of their preference).



Reading material

There is no single required textbook for the course. We will recommend specific chapters from the following books:



Evaluation

The evaluation of the course will be based on the following:

  1. Two assignments: the assignments will include theoretical questions as well hands-on practical questions and will familiarize the students with basic machine learning tasks.
  2. Project: The students are expected to form groups of 3-4 people, propose a topic for their project, and submit a final project report (it would also be interesting to organize a poster session at the end of the quarter). Please, read the project section for more details.

The grading will be as follows:

Assignment 1 (individual): 15%
Assignment 2 (groups of 3-4 students): 35%
Project (groups of 3-4 students): 50%


Academic integrity

All of your work must be your own. Don't copy another student's assignment, in part or in total, and submit it as your own work. Acknowledge and cite source material in your papers or assignments.



Project

Details about the project of the course will be posted soon on edunao.




Resources

Datasets


Software tools


Related conferences
Please find below a list of conferences related to the contents of the course (mostly in the field of machine learning and data mining. We provide the DBLP website of each venue where you can access the proceedings (papers, tutorials, etc).

Check out the website of each conference (e.g., KDD 2020) for more information.