## Homework The goal of this homework is to get familiar with MLflow, the tool for experiment tracking and model management. ## Q1. Install MLflow To get started with MLflow you'll need to install the MLflow Python package. For this we recommend creating a separate Python environment, for example, you can use [conda environments](https://docs.conda.io/projects/conda/en/latest/user-guide/getting-started.html#managing-envs), and then install the package there with `pip` or `conda`. Once you installed the package, run the command `mlflow --version` and check the output. What's the version that you have? ## Q2. Download and preprocess the data We'll use the Green Taxi Trip Records dataset to predict the duration of each trip. Download the data for January, February and March 2023 in parquet format from [here](https://www1.nyc.gov/site/tlc/about/tlc-trip-record-data.page). Use the script `preprocess_data.py` located in the folder [`homework`](homework) to preprocess the data. The script will: * load the data from the folder `` (the folder where you have downloaded the data), * fit a `DictVectorizer` on the training set (January 2023 data), * save the preprocessed datasets and the `DictVectorizer` to disk. Your task is to download the datasets and then execute this command: ``` python preprocess_data.py --raw_data_path --dest_path ./output ``` Tip: go to `02-experiment-tracking/homework/` folder before executing the command and change the value of `` to the location where you saved the data. How many files were saved to `OUTPUT_FOLDER`? * 1 * 3 * 4 * 7 ## Q3. Train a model with autolog We will train a `RandomForestRegressor` (from Scikit-Learn) on the taxi dataset. We have prepared the training script `train.py` for this exercise, which can be also found in the folder [`homework`](homework). The script will: * load the datasets produced by the previous step, * train the model on the training set, * calculate the RMSE score on the validation set. Your task is to modify the script to enable **autologging** with MLflow, execute the script and then launch the MLflow UI to check that the experiment run was properly tracked. Tip 1: don't forget to wrap the training code with a `with mlflow.start_run():` statement as we showed in the videos. Tip 2: don't modify the hyperparameters of the model to make sure that the training will finish quickly. What is the value of the `min_samples_split` parameter: * 2 * 4 * 8 * 10 ## Q4. Launch the tracking server locally Now we want to manage the entire lifecycle of our ML model. In this step, you'll need to launch a tracking server. This way we will also have access to the model registry. Your task is to: * launch the tracking server on your local machine, * select a SQLite db for the backend store and a folder called `artifacts` for the artifacts store. You should keep the tracking server running to work on the next two exercises that use the server. In addition to `backend-store-uri`, what else do you need to pass to properly configure the server? * `default-artifact-root` * `serve-artifacts` * `artifacts-only` * `artifacts-destination` ## Q5. Tune model hyperparameters Now let's try to reduce the validation error by tuning the hyperparameters of the `RandomForestRegressor` using `hyperopt`. We have prepared the script `hpo.py` for this exercise. Your task is to modify the script `hpo.py` and make sure that the validation RMSE is logged to the tracking server for each run of the hyperparameter optimization (you will need to add a few lines of code to the `objective` function) and run the script without passing any parameters. After that, open UI and explore the runs from the experiment called `random-forest-hyperopt` to answer the question below. Note: Don't use autologging for this exercise. The idea is to just log the information that you need to answer the question below, including: * the list of hyperparameters that are passed to the `objective` function during the optimization, * the RMSE obtained on the validation set (February 2023 data). What's the best validation RMSE that you got? * 4.817 * 5.335 * 5.818 * 6.336 ## Q6. Promote the best model to the model registry The results from the hyperparameter optimization are quite good. So, we can assume that we are ready to test some of these models in production. In this exercise, you'll promote the best model to the model registry. We have prepared a script called `register_model.py`, which will check the results from the previous step and select the top 5 runs. After that, it will calculate the RMSE of those models on the test set (March 2023 data) and save the results to a new experiment called `random-forest-best-models`. Your task is to update the script `register_model.py` so that it selects the model with the lowest RMSE on the test set and registers it to the model registry. Tip 1: you can use the method `search_runs` from the `MlflowClient` to get the model with the lowest RMSE, Tip 2: to register the model you can use the method `mlflow.register_model` and you will need to pass the right `model_uri` in the form of a string that looks like this: `"runs://model"`, and the name of the model (make sure to choose a good one!). What is the test RMSE of the best model? * 5.060 * 5.567 * 6.061 * 6.568 ## Submit the results * Submit your results here: https://courses.datatalks.club/mlops-zoomcamp-2025/homework/hw2 * If your answer doesn't match options exactly, select the closest one