Introduction

What's ML? | How DL Differs from Classic ML? | About This Courses | FAQ

Slides Notation

Scientific Python 101

This lab guides you through the setup of scientific Python environment and provides useful references for self-reading.

Notebook

Linear Algebra

Span & Linear Dependence | Norms | Eigendecomposition | Singular Value Decomposition | Traces | Determinant

Video Slides

Data Exploration & PCA

This lab guides you through the process of Exploratory Data Analysis (EDA) and discuss how you can leverage the Principle Component Analysis (PCA) to visualize and understand high-dimensional data.

Notebook

Probability & Information Theory

Random Variables & Probability Distributions | Multivariate & Derived Random Variables | Bayes’ Rule & Statistics | Principal Components Analysis | Information Theory | Decision Trees & Random Forest

Video Slides

Decision Trees & Random Forest

In this lab, we will apply the Decision Tree and Random Forest algorithms to the classification and dimension reduction problems using the Wine dataset.

Notebook

Numerical Optimization

Numerical Computation | Optimization Problems | Unconstrained Optimization | Stochastic Gradient Descent | Perceptron | Adaline | Constrained Optimization | Linear & Polynomial Regression | Duality

Video Slides

Perceptron & Adaline

In this lab, we will guide you through the implementation of Perceptron and Adaline, two of the first machine learning algorithms for the classification problem. We will also discuss how to train these models using the optimization techniques.

Notebook

Regression

This lab guides you through the linear and polynomial regression using the Housing dataset. We will also extend the Decision Tree and Random Forest classifiers to solve the regression problem.

Notebook

Learning Theory & Regularization

Point Estimation | Bias & Variance | Consistency | Decomposing Generalization Error | Weight Decay | Validation

Video Slides

Regularization

In this lab, we will guide you through some common regularization techniques such as weight decay, sparse weight, and validation.

Notebook

Probabilistic Models

Maximum Likelihood Estimation | Maximum A Posteriori Estimation | Bayesian Estimation

Video Slides

Logistic Regression & Metrics

In this lab, we will guide you through the practice of Logistic Regression. We will also introduce some common evaluation metrics other than the "accuracy" that we have been used so far.

Notebook

Non-Parametric Methods & SVMs

KNNs | Parzen Windows | Local Models | Support Vector Classification (SVC) | Nonlinear SVC | Kernel Trick

Video Slides

SVMs & Scikit-Learn Pipelines

In this lab, we will classify nonlinearly separable data using the KNN and SVM classifiers. We will show how to pack multiple data preprocessing steps into a single Pipeline in Scikit-learn to simplify the training workflow.

Notebook

Cross Validation & Ensembling

CV | How Many Folds? | Voting | Bagging | Boosting | Why AdaBoost Works?

Video Slides

CV & Ensembling

In this lab, we will guide you through the cross validation technique for hyperparameter selection. We will also practice and compare some ensemble learning techniques.

Notebook