{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Machine Learning Concepts\n", "\n", "[🎥 Introducing machine-learning concepts](https://inria.github.io/scikit-learn-mooc/ml_concepts/slides.html)\n", "\n", "[✅ Quiz Intro.01](https://inria.github.io/scikit-learn-mooc/ml_concepts/quiz_intro_01.html)\n", "\n", "# The predictive modeling pipeline\n", "\n", "[Module overview](https://inria.github.io/scikit-learn-mooc/predictive_modeling_pipeline/predictive_modeling_module_intro.html)\n", "\n", "### Tabular data exploration\n", "\n", "* [First look at our dataset](notebooks/01_tabular_data_exploration.ipynb)\n", "* [📝 Exercise M1.01](notebooks/01_tabular_data_exploration_ex_01.ipynb)\n", "* [📃 Solution for Exercise M1.01](notebooks/01_tabular_data_exploration_sol_01.ipynb)\n", "* [✅ Quiz M1.01](https://inria.github.io/scikit-learn-mooc/predictive_modeling_pipeline/01_tabular_data_exploration_quiz_m1_01.html)\n", "\n", "### Fitting a scikit-learn model on numerical data\n", "\n", "* [First model with scikit-learn](notebooks/02_numerical_pipeline_introduction.ipynb)\n", "* [📝 Exercise M1.02](notebooks/02_numerical_pipeline_ex_00.ipynb)\n", "* [📃 Solution for Exercise M1.02](notebooks/02_numerical_pipeline_sol_00.ipynb)\n", "* [Working with numerical data](notebooks/02_numerical_pipeline_hands_on.ipynb)\n", "* [📝 Exercise M1.03](notebooks/02_numerical_pipeline_ex_01.ipynb)\n", "* [📃 Solution for Exercise M1.03](notebooks/02_numerical_pipeline_sol_01.ipynb)\n", "* [Preprocessing for numerical features](notebooks/02_numerical_pipeline_scaling.ipynb)\n", "* [🎥 Validation of a model](https://inria.github.io/scikit-learn-mooc/predictive_modeling_pipeline/02_numerical_pipeline_video_cross_validation.html)\n", "* [Model evaluation using cross-validation](notebooks/02_numerical_pipeline_cross_validation.ipynb)\n", "* [✅ Quiz M1.02](https://inria.github.io/scikit-learn-mooc/predictive_modeling_pipeline/02_numerical_pipeline_quiz_m1_02.html)\n", "\n", "### Handling categorical data\n", "\n", "* [Encoding of categorical variables](notebooks/03_categorical_pipeline.ipynb)\n", "* [📝 Exercise M1.04](notebooks/03_categorical_pipeline_ex_01.ipynb)\n", "* [📃 Solution for Exercise M1.04](notebooks/03_categorical_pipeline_sol_01.ipynb)\n", "* [Using numerical and categorical variables together](notebooks/03_categorical_pipeline_column_transformer.ipynb)\n", "* [📝 Exercise M1.05](notebooks/03_categorical_pipeline_ex_02.ipynb)\n", "* [📃 Solution for Exercise M1.05](notebooks/03_categorical_pipeline_sol_02.ipynb)\n", "* [🎥 Visualizing scikit-learn pipelines in Jupyter](https://inria.github.io/scikit-learn-mooc/predictive_modeling_pipeline/03_categorical_pipeline_visualization_video.html)\n", "* [Visualizing scikit-learn pipelines in Jupyter](notebooks/03_categorical_pipeline_visualization.ipynb)\n", "* [✅ Quiz M1.03](https://inria.github.io/scikit-learn-mooc/predictive_modeling_pipeline/03_categorical_pipeline_quiz_m1_03.html)\n", "\n", "[🏁 Wrap-up quiz 1](https://inria.github.io/scikit-learn-mooc/predictive_modeling_pipeline/wrap_up_quiz.html)\n", "\n", "[Main take-away](https://inria.github.io/scikit-learn-mooc/predictive_modeling_pipeline/predictive_modeling_module_take_away.html)\n", "\n", "# Selecting the best model\n", "\n", "[Module overview](https://inria.github.io/scikit-learn-mooc/overfit/overfit_module_intro.html)\n", "\n", "### Overfitting and underfitting\n", "\n", "* [🎥 Overfitting and Underfitting](https://inria.github.io/scikit-learn-mooc/overfit/overfitting_vs_under_fitting_slides.html)\n", "* [Cross-validation framework](notebooks/cross_validation_train_test.ipynb)\n", "* [✅ Quiz M2.01](https://inria.github.io/scikit-learn-mooc/overfit/overfitting_vs_under_fitting_quiz_m2_01.html)\n", "\n", "### Validation and learning curves\n", "\n", "* [🎥 Comparing train and test errors](https://inria.github.io/scikit-learn-mooc/overfit/learning_validation_curves_slides.html)\n", "* [Overfit-generalization-underfit](notebooks/cross_validation_validation_curve.ipynb)\n", "* [Effect of the sample size in cross-validation](notebooks/cross_validation_learning_curve.ipynb)\n", "* [📝 Exercise M2.01](notebooks/cross_validation_ex_01.ipynb)\n", "* [📃 Solution for Exercise M2.01](notebooks/cross_validation_sol_01.ipynb)\n", "* [✅ Quiz M2.02](https://inria.github.io/scikit-learn-mooc/overfit/learning_validation_curves_quiz_m2_02.html)\n", "\n", "### Bias versus variance trade-off\n", "\n", "* [🎥 Bias versus Variance](https://inria.github.io/scikit-learn-mooc/overfit/bias_vs_variance_slides.html)\n", "* [✅ Quiz M2.03](https://inria.github.io/scikit-learn-mooc/overfit/bias_vs_variance_quiz_m2_03.html)\n", "\n", "[🏁 Wrap-up quiz 2](https://inria.github.io/scikit-learn-mooc/overfit/overfit_wrap_up_quiz.html)\n", "\n", "[Main take-away](https://inria.github.io/scikit-learn-mooc/overfit/overfit_take_away.html)\n", "\n", "# Hyperparameter tuning\n", "\n", "[Module overview](https://inria.github.io/scikit-learn-mooc/tuning/parameter_tuning_module_intro.html)\n", "\n", "### Manual tuning\n", "\n", "* [Set and get hyperparameters in scikit-learn](notebooks/parameter_tuning_manual.ipynb)\n", "* [📝 Exercise M3.01](notebooks/parameter_tuning_ex_02.ipynb)\n", "* [📃 Solution for Exercise M3.01](notebooks/parameter_tuning_sol_02.ipynb)\n", "* [✅ Quiz M3.01](https://inria.github.io/scikit-learn-mooc/tuning/parameter_tuning_manual_quiz_m3_01.html)\n", "\n", "### Automated tuning\n", "\n", "* [Hyperparameter tuning by grid-search](notebooks/parameter_tuning_grid_search.ipynb)\n", "* [Hyperparameter tuning by randomized-search](notebooks/parameter_tuning_randomized_search.ipynb)\n", "* [🎥 Analysis of hyperparameter search results](https://inria.github.io/scikit-learn-mooc/tuning/parameter_tuning_parallel_plot_video.html)\n", "* [Analysis of hyperparameter search results](notebooks/parameter_tuning_parallel_plot.ipynb)\n", "* [Evaluation and hyperparameter tuning](notebooks/parameter_tuning_nested.ipynb)\n", "* [📝 Exercise M3.02](notebooks/parameter_tuning_ex_03.ipynb)\n", "* [📃 Solution for Exercise M3.02](notebooks/parameter_tuning_sol_03.ipynb)\n", "* [✅ Quiz M3.02](https://inria.github.io/scikit-learn-mooc/tuning/parameter_tuning_automated_quiz_m3_02.html)\n", "\n", "[🏁 Wrap-up quiz 3](https://inria.github.io/scikit-learn-mooc/tuning/parameter_tuning_wrap_up_quiz.html)\n", "\n", "[Main take-away](https://inria.github.io/scikit-learn-mooc/tuning/parameter_tuning_module_take_away.html)\n", "\n", "# Linear models\n", "\n", "[Module overview](https://inria.github.io/scikit-learn-mooc/linear_models/linear_models_module_intro.html)\n", "\n", "### Intuitions on linear models\n", "\n", "* [🎥 Intuitions on linear models](https://inria.github.io/scikit-learn-mooc/linear_models/linear_models_slides.html)\n", "* [Linear regression without scikit-learn](notebooks/linear_regression_without_sklearn.ipynb)\n", "* [📝 Exercise M4.01](notebooks/linear_models_ex_01.ipynb)\n", "* [📃 Solution for Exercise M4.01](notebooks/linear_models_sol_01.ipynb)\n", "* [Linear regression using scikit-learn](notebooks/linear_regression_in_sklearn.ipynb)\n", "* [Linear models for classification](notebooks/logistic_regression.ipynb)\n", "* [✅ Quiz M4.01](https://inria.github.io/scikit-learn-mooc/linear_models/linear_models_quiz_m4_01.html)\n", "\n", "### Non-linear feature engineering for linear models\n", "\n", "* [Non-linear feature engineering for Linear Regression](notebooks/linear_regression_non_linear_link.ipynb)\n", "* [📝 Exercise M4.02](notebooks/linear_models_ex_02.ipynb)\n", "* [📃 Solution for Exercise M4.02](notebooks/linear_models_sol_02.ipynb)\n", "* [Non-linear feature engineering for Logistic Regression](notebooks/linear_models_feature_engineering_classification.ipynb)\n", "* [📝 Exercise M4.03](notebooks/linear_models_ex_03.ipynb)\n", "* [📃 Solution for Exercise M4.03](notebooks/linear_models_sol_03.ipynb)\n", "* [✅ Quiz M4.02](https://inria.github.io/scikit-learn-mooc/linear_models/linear_models_quiz_m4_02.html)\n", "\n", "### Regularization in linear model\n", "\n", "* [🎥 Intuitions on regularized linear models](https://inria.github.io/scikit-learn-mooc/linear_models/regularized_linear_models_slides.html)\n", "* [Regularization of linear regression model](notebooks/linear_models_regularization.ipynb)\n", "* [📝 Exercise M4.04](notebooks/linear_models_ex_04.ipynb)\n", "* [📃 Solution for Exercise M4.04](notebooks/linear_models_sol_04.ipynb)\n", "* [✅ Quiz M4.03](https://inria.github.io/scikit-learn-mooc/linear_models/linear_models_quiz_m4_03.html)\n", "\n", "[🏁 Wrap-up quiz 4](https://inria.github.io/scikit-learn-mooc/linear_models/linear_models_wrap_up_quiz.html)\n", "\n", "[Main take-away](https://inria.github.io/scikit-learn-mooc/linear_models/linear_models_module_take_away.html)\n", "\n", "# Decision tree models\n", "\n", "[Module overview](https://inria.github.io/scikit-learn-mooc/trees/trees_module_intro.html)\n", "\n", "### Intuitions on tree-based models\n", "\n", "* [🎥 Intuitions on tree-based models](https://inria.github.io/scikit-learn-mooc/trees/slides.html)\n", "* [✅ Quiz M5.01](https://inria.github.io/scikit-learn-mooc/trees/trees_quiz_m5_01.html)\n", "\n", "### Decision tree in classification\n", "\n", "* [Build a classification decision tree](notebooks/trees_classification.ipynb)\n", "* [📝 Exercise M5.01](notebooks/trees_ex_01.ipynb)\n", "* [📃 Solution for Exercise M5.01](notebooks/trees_sol_01.ipynb)\n", "* [✅ Quiz M5.02](https://inria.github.io/scikit-learn-mooc/trees/trees_quiz_m5_02.html)\n", "\n", "### Decision tree in regression\n", "\n", "* [Decision tree for regression](notebooks/trees_regression.ipynb)\n", "* [📝 Exercise M5.02](notebooks/trees_ex_02.ipynb)\n", "* [📃 Solution for Exercise M5.02](notebooks/trees_sol_02.ipynb)\n", "* [✅ Quiz M5.03](https://inria.github.io/scikit-learn-mooc/trees/trees_quiz_m5_03.html)\n", "\n", "### Hyperparameters of decision tree\n", "\n", "* [Importance of decision tree hyperparameters on generalization](notebooks/trees_hyperparameters.ipynb)\n", "* [✅ Quiz M5.04](https://inria.github.io/scikit-learn-mooc/trees/trees_quiz_m5_04.html)\n", "\n", "[🏁 Wrap-up quiz 5](https://inria.github.io/scikit-learn-mooc/trees/trees_wrap_up_quiz.html)\n", "\n", "[Main take-away](https://inria.github.io/scikit-learn-mooc/trees/trees_module_take_away.html)\n", "\n", "# Ensemble of models\n", "\n", "[Module overview](https://inria.github.io/scikit-learn-mooc/ensemble/ensemble_module_intro.html)\n", "\n", "### Ensemble method using bootstrapping\n", "\n", "* [🎥 Intuitions on ensemble models: bagging](https://inria.github.io/scikit-learn-mooc/ensemble/bagging_slides.html)\n", "* [Introductory example to ensemble models](notebooks/ensemble_introduction.ipynb)\n", "* [Bagging](notebooks/ensemble_bagging.ipynb)\n", "* [📝 Exercise M6.01](notebooks/ensemble_ex_01.ipynb)\n", "* [📃 Solution for Exercise M6.01](notebooks/ensemble_sol_01.ipynb)\n", "* [Random forests](notebooks/ensemble_random_forest.ipynb)\n", "* [📝 Exercise M6.02](notebooks/ensemble_ex_02.ipynb)\n", "* [📃 Solution for Exercise M6.02](notebooks/ensemble_sol_02.ipynb)\n", "* [✅ Quiz M6.01](https://inria.github.io/scikit-learn-mooc/ensemble/ensemble_quiz_m6_01.html)\n", "\n", "### Ensemble based on boosting\n", "\n", "* [🎥 Intuitions on ensemble models: boosting](https://inria.github.io/scikit-learn-mooc/ensemble/boosting_slides.html)\n", "* [Adaptive Boosting (AdaBoost)](notebooks/ensemble_adaboost.ipynb)\n", "* [Gradient-boosting decision tree (GBDT)](notebooks/ensemble_gradient_boosting.ipynb)\n", "* [📝 Exercise M6.03](notebooks/ensemble_ex_03.ipynb)\n", "* [📃 Solution for Exercise M6.03](notebooks/ensemble_sol_03.ipynb)\n", "* [Speeding-up gradient-boosting](notebooks/ensemble_hist_gradient_boosting.ipynb)\n", "* [✅ Quiz M6.02](https://inria.github.io/scikit-learn-mooc/ensemble/ensemble_quiz_m6_02.html)\n", "\n", "### Hyperparameter tuning with ensemble methods\n", "\n", "* [Hyperparameter tuning](notebooks/ensemble_hyperparameters.ipynb)\n", "* [📝 Exercise M6.04](notebooks/ensemble_ex_04.ipynb)\n", "* [📃 Solution for Exercise M6.04](notebooks/ensemble_sol_04.ipynb)\n", "* [✅ Quiz M6.03](https://inria.github.io/scikit-learn-mooc/ensemble/ensemble_quiz_m6_03.html)\n", "\n", "[🏁 Wrap-up quiz 6](https://inria.github.io/scikit-learn-mooc/ensemble/ensemble_wrap_up_quiz.html)\n", "\n", "[Main take-away](https://inria.github.io/scikit-learn-mooc/ensemble/ensemble_module_take_away.html)\n", "\n", "# Evaluating model performance\n", "\n", "[Module overview](https://inria.github.io/scikit-learn-mooc/evaluation/evaluation_module_intro.html)\n", "\n", "### Comparing a model with simple baselines\n", "\n", "* [Comparing model performance with a simple baseline](notebooks/cross_validation_baseline.ipynb)\n", "* [📝 Exercise M7.01](notebooks/cross_validation_ex_02.ipynb)\n", "* [📃 Solution for Exercise M7.01](notebooks/cross_validation_sol_02.ipynb)\n", "* [✅ Quiz M7.01](https://inria.github.io/scikit-learn-mooc/evaluation/evaluation_quiz_m7_01.html)\n", "\n", "### Choice of cross-validation\n", "\n", "* [Stratification](notebooks/cross_validation_stratification.ipynb)\n", "* [Sample grouping](notebooks/cross_validation_grouping.ipynb)\n", "* [Non i.i.d. data](notebooks/cross_validation_time.ipynb)\n", "* [✅ Quiz M7.02](https://inria.github.io/scikit-learn-mooc/evaluation/evaluation_quiz_m7_02.html)\n", "\n", "### Nested cross-validation\n", "\n", "* [Nested cross-validation](notebooks/cross_validation_nested.ipynb)\n", "* [✅ Quiz M7.03](https://inria.github.io/scikit-learn-mooc/evaluation/evaluation_quiz_m7_03.html)\n", "\n", "### Classification metrics\n", "\n", "* [Classification](notebooks/metrics_classification.ipynb)\n", "* [📝 Exercise M7.02](notebooks/metrics_ex_01.ipynb)\n", "* [📃 Solution for Exercise M7.02](notebooks/metrics_sol_01.ipynb)\n", "* [✅ Quiz M7.04](https://inria.github.io/scikit-learn-mooc/evaluation/evaluation_quiz_m7_04.html)\n", "\n", "### Regression metrics\n", "\n", "* [Regression](notebooks/metrics_regression.ipynb)\n", "* [📝 Exercise M7.03](notebooks/metrics_ex_02.ipynb)\n", "* [📃 Solution for Exercise M7.03](notebooks/metrics_sol_02.ipynb)\n", "* [✅ Quiz M7.05](https://inria.github.io/scikit-learn-mooc/evaluation/evaluation_quiz_m7_05.html)\n", "\n", "[🏁 Wrap-up quiz 7](https://inria.github.io/scikit-learn-mooc/evaluation/evaluation_wrap_up_quiz.html)\n", "\n", "[Main take-away](https://inria.github.io/scikit-learn-mooc/evaluation/evaluation_module_take_away.html)\n", "\n", "# Concluding remarks\n", "\n", "[🎥 Concluding remarks](https://inria.github.io/scikit-learn-mooc/concluding_remarks_video.html)\n", "\n", "[Concluding remarks](https://inria.github.io/scikit-learn-mooc/concluding_remarks.html)" ] } ], "metadata": {}, "nbformat": 4, "nbformat_minor": 5 }