{ "cells": [ { "cell_type": "markdown", "id": "02c22456", "metadata": {}, "source": [ "# Logging Visualizations with MLflow\n", "\n", "In this part of the guide, we emphasize the **importance of logging visualizations with MLflow**. Retaining visualizations alongside trained models enhances model interpretability, auditing, and provenance, ensuring a robust and transparent ML model development workflow.\n", "\n", "#### What Are We Doing?\n", "- **Storing Visual Artifacts:** We are logging various plots as visual artifacts in MLflow, ensuring that they are always accessible and aligned with the corresponding model and run data.\n", "- **Enhancing Model Interpretability:** These visualizations aid in understanding and explaining model behavior, contributing to improved model transparency and accountability.\n", "\n", "#### How Does It Apply to MLflow?\n", "- **Integrated Visualization Logging:** MLflow seamlessly integrates facilities for logging and accessing visual artifacts, enhancing the ease and efficiency of handling visual context and insights.\n", "- **Convenient Access:** Logged figures are displayable within the Runs view pane in the MLflow UI, ensuring quick and easy access for analysis and review.\n", "\n", "#### Caution\n", "While MLflow offers simplicity and convenience for logging visualizations, it's crucial to ensure the consistency and relevance of the visual artifacts with the corresponding model data, maintaining the integrity and comprehensiveness of the model information.\n", "\n", "#### Why Is Consistent Logging Important?\n", "- **Auditing and Provenance:** Consistent and comprehensive logging of visualizations is pivotal for auditing purposes, ensuring that every model is accompanied by relevant visual insights for thorough analysis and review.\n", "- **Enhanced Model Understanding:** Proper visual context enhances the understanding of model behavior, aiding in effective model evaluation, and validation.\n", "\n", "In conclusion, MLflow's capabilities for visualization logging play an invaluable role in ensuring a comprehensive, transparent, and efficient ML model development workflow, reinforcing model interpretability, auditing, and provenance.\n" ] }, { "cell_type": "markdown", "id": "63e20328", "metadata": {}, "source": [ "### Generating Synthetic Apple Sales Data\n", "\n", "In this next section, we dive into **generating synthetic data for apple sales demand prediction** using the `generate_apple_sales_data_with_promo_adjustment` function. This function simulates a variety of features relevant to apple sales, providing a rich dataset for exploration and modeling.\n", "\n", "#### What Are We Doing?\n", "- **Simulating Realistic Data:** Generating a dataset with features like date, average temperature, rainfall, weekend flag, and more, simulating realistic scenarios for apple sales.\n", "- **Incorporating Various Effects:** The function incorporates effects like promotional adjustments, seasonality, and competitor pricing, contributing to the 'demand' target variable.\n", "\n", "#### How Does It Apply to Data Generation?\n", "- **Comprehensive Dataset:** The synthetic dataset provides a comprehensive set of features and interactions, ideal for exploring diverse aspects and dimensions for demand prediction.\n", "- **Freedom and Flexibility:** The synthetic nature allows for unconstrained exploration and analysis, devoid of real-world data sensitivities and constraints.\n", "\n", "#### Caution\n", "While synthetic data offers numerous advantages for exploration and learning, it's crucial to acknowledge its limitations in capturing real-world complexities and nuances.\n", "\n", "#### Why Is Acknowledging Limitations Important?\n", "- **Real-World Complexities:** Synthetic data may not capture all the intricate patterns and anomalies present in real-world data, potentially leading to over-simplified models and insights.\n", "- **Transferability to Real-World Scenarios:** Ensuring that insights and models derived from synthetic data are transferable to real-world scenarios requires careful consideration and validation.\n", "\n", "In conclusion, the `generate_apple_sales_data_with_promo_adjustment` function offers a robust tool for generating a comprehensive synthetic dataset for apple sales demand prediction, facilitating extensive exploration, and analysis while acknowledging the limitations of synthetic data.\n" ] }, { "cell_type": "code", "execution_count": 2, "id": "7b060ff8", "metadata": {}, "outputs": [], "source": [ "import math\n", "import pathlib\n", "from datetime import datetime, timedelta\n", "\n", "import matplotlib.pylab as plt\n", "import numpy as np\n", "import pandas as pd\n", "import seaborn as sns\n", "from scipy import stats\n", "from sklearn.linear_model import Ridge\n", "from sklearn.metrics import (\n", " mean_absolute_error,\n", " mean_squared_error,\n", " mean_squared_log_error,\n", " median_absolute_error,\n", " r2_score,\n", ")\n", "from sklearn.model_selection import train_test_split\n", "\n", "import mlflow\n", "\n", "\n", "def generate_apple_sales_data_with_promo_adjustment(\n", " base_demand: int = 1000,\n", " n_rows: int = 5000,\n", " competitor_price_effect: float = -50.0,\n", "):\n", " \"\"\"\n", " Generates a synthetic dataset for predicting apple sales demand with multiple\n", " influencing factors.\n", "\n", " This function creates a pandas DataFrame with features relevant to apple sales.\n", " The features include date, average_temperature, rainfall, weekend flag, holiday flag,\n", " promotional flag, price_per_kg, competitor's price, marketing intensity, stock availability,\n", " and the previous day's demand. The target variable, 'demand', is generated based on a\n", " combination of these features with some added noise.\n", "\n", " Args:\n", " base_demand (int, optional): Base demand for apples. Defaults to 1000.\n", " n_rows (int, optional): Number of rows (days) of data to generate. Defaults to 5000.\n", " competitor_price_effect (float, optional): Effect of competitor's price being lower\n", " on our sales. Defaults to -50.\n", "\n", " Returns:\n", " pd.DataFrame: DataFrame with features and target variable for apple sales prediction.\n", "\n", " Example:\n", " >>> df = generate_apple_sales_data_with_promo_adjustment(base_demand=1200, n_rows=6000)\n", " >>> df.head()\n", " \"\"\"\n", "\n", " # Set seed for reproducibility\n", " np.random.seed(9999)\n", "\n", " # Create date range\n", " dates = [datetime.now() - timedelta(days=i) for i in range(n_rows)]\n", " dates.reverse()\n", "\n", " # Generate features\n", " df = pd.DataFrame({\n", " \"date\": dates,\n", " \"average_temperature\": np.random.uniform(10, 35, n_rows),\n", " \"rainfall\": np.random.exponential(5, n_rows),\n", " \"weekend\": [(date.weekday() >= 5) * 1 for date in dates],\n", " \"holiday\": np.random.choice([0, 1], n_rows, p=[0.97, 0.03]),\n", " \"price_per_kg\": np.random.uniform(0.5, 3, n_rows),\n", " \"month\": [date.month for date in dates],\n", " })\n", "\n", " # Introduce inflation over time (years)\n", " df[\"inflation_multiplier\"] = 1 + (df[\"date\"].dt.year - df[\"date\"].dt.year.min()) * 0.03\n", "\n", " # Incorporate seasonality due to apple harvests\n", " df[\"harvest_effect\"] = np.sin(2 * np.pi * (df[\"month\"] - 3) / 12) + np.sin(\n", " 2 * np.pi * (df[\"month\"] - 9) / 12\n", " )\n", "\n", " # Modify the price_per_kg based on harvest effect\n", " df[\"price_per_kg\"] = df[\"price_per_kg\"] - df[\"harvest_effect\"] * 0.5\n", "\n", " # Adjust promo periods to coincide with periods lagging peak harvest by 1 month\n", " peak_months = [4, 10] # months following the peak availability\n", " df[\"promo\"] = np.where(\n", " df[\"month\"].isin(peak_months),\n", " 1,\n", " np.random.choice([0, 1], n_rows, p=[0.85, 0.15]),\n", " )\n", "\n", " # Generate target variable based on features\n", " base_price_effect = -df[\"price_per_kg\"] * 50\n", " seasonality_effect = df[\"harvest_effect\"] * 50\n", " promo_effect = df[\"promo\"] * 200\n", "\n", " df[\"demand\"] = (\n", " base_demand\n", " + base_price_effect\n", " + seasonality_effect\n", " + promo_effect\n", " + df[\"weekend\"] * 300\n", " + np.random.normal(0, 50, n_rows)\n", " ) * df[\"inflation_multiplier\"] # adding random noise\n", "\n", " # Add previous day's demand\n", " df[\"previous_days_demand\"] = df[\"demand\"].shift(1)\n", " df[\"previous_days_demand\"].fillna(method=\"bfill\", inplace=True) # fill the first row\n", "\n", " # Introduce competitor pricing\n", " df[\"competitor_price_per_kg\"] = np.random.uniform(0.5, 3, n_rows)\n", " df[\"competitor_price_effect\"] = (\n", " df[\"competitor_price_per_kg\"] < df[\"price_per_kg\"]\n", " ) * competitor_price_effect\n", "\n", " # Stock availability based on past sales price (3 days lag with logarithmic decay)\n", " log_decay = -np.log(df[\"price_per_kg\"].shift(3) + 1) + 2\n", " df[\"stock_available\"] = np.clip(log_decay, 0.7, 1)\n", "\n", " # Marketing intensity based on stock availability\n", " # Identify where stock is above threshold\n", " high_stock_indices = df[df[\"stock_available\"] > 0.95].index\n", "\n", " # For each high stock day, increase marketing intensity for the next week\n", " for idx in high_stock_indices:\n", " df.loc[idx : min(idx + 7, n_rows - 1), \"marketing_intensity\"] = np.random.uniform(0.7, 1)\n", "\n", " # If the marketing_intensity column already has values, this will preserve them;\n", " # if not, it sets default values\n", " fill_values = pd.Series(np.random.uniform(0, 0.5, n_rows), index=df.index)\n", " df[\"marketing_intensity\"].fillna(fill_values, inplace=True)\n", "\n", " # Adjust demand with new factors\n", " df[\"demand\"] = df[\"demand\"] + df[\"competitor_price_effect\"] + df[\"marketing_intensity\"]\n", "\n", " # Drop temporary columns\n", " df.drop(\n", " columns=[\n", " \"inflation_multiplier\",\n", " \"harvest_effect\",\n", " \"month\",\n", " \"competitor_price_effect\",\n", " \"stock_available\",\n", " ],\n", " inplace=True,\n", " )\n", "\n", " return df" ] }, { "cell_type": "markdown", "id": "883a47ee", "metadata": {}, "source": [ "### Generating Apple Sales Data\n", "\n", "In this cell, we call the `generate_apple_sales_data_with_promo_adjustment` function to generate a dataset of apple sales. \n", "\n", "#### Parameters Used:\n", "- `base_demand`: Set to 1000, representing the baseline demand for apples.\n", "- `n_rows`: Set to 10,000, determining the number of rows or data points in the generated dataset.\n", "- `competitor_price_effect`: Set to -25.0, representing the impact on our sales when the competitor's price is lower.\n", "\n", "By running this cell, we obtain a dataset `my_data`, which holds the synthetic apple sales data with the aforementioned configurations. This dataset will be used for further exploration and analysis in subsequent steps of this notebook.\n", "\n", "You can see the data in the cell after the generation cell." ] }, { "cell_type": "code", "execution_count": 3, "id": "e687b85c", "metadata": {}, "outputs": [], "source": [ "my_data = generate_apple_sales_data_with_promo_adjustment(\n", " base_demand=1000, n_rows=10_000, competitor_price_effect=-25.0\n", ")" ] }, { "cell_type": "code", "execution_count": 4, "id": "d585dbbe", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
dateaverage_temperaturerainfallweekendholidayprice_per_kgpromodemandprevious_days_demandcompetitor_price_per_kgmarketing_intensity
01996-05-11 13:10:40.68999930.5847271.831006101.57838711301.6473521326.3242660.7557250.323086
11996-05-12 13:10:40.68999915.4650690.761303101.96512501143.9726381326.3242660.9139340.030371
21996-05-13 13:10:40.68999810.7865251.427338001.4976230890.3192481168.9422672.8792620.354226
31996-05-14 13:10:40.68999723.6481543.737435001.9529360811.206168889.9650210.8260150.953000
41996-05-15 13:10:40.68999713.8613915.598549002.0599930822.279469835.2531681.1301450.953000
....................................
99952023-09-22 13:10:40.68289523.3588687.061220001.55682911981.1958842089.6444540.5605070.889971
99962023-09-23 13:10:40.68289514.8590480.868655101.63291802180.6981382005.3059132.4607660.884467
99972023-09-24 13:10:40.68289417.94103513.739986100.82772312675.0936712179.8136711.3219220.884467
99982023-09-25 13:10:40.68289314.5338621.610512000.58917201703.2872852674.2092042.6040950.812706
99992023-09-26 13:10:40.68288913.0485495.287508001.79412211971.0292661702.4745791.2616350.750458
\n", "

10000 rows × 11 columns

\n", "
" ], "text/plain": [ " date average_temperature rainfall weekend \n", "0 1996-05-11 13:10:40.689999 30.584727 1.831006 1 \\\n", "1 1996-05-12 13:10:40.689999 15.465069 0.761303 1 \n", "2 1996-05-13 13:10:40.689998 10.786525 1.427338 0 \n", "3 1996-05-14 13:10:40.689997 23.648154 3.737435 0 \n", "4 1996-05-15 13:10:40.689997 13.861391 5.598549 0 \n", "... ... ... ... ... \n", "9995 2023-09-22 13:10:40.682895 23.358868 7.061220 0 \n", "9996 2023-09-23 13:10:40.682895 14.859048 0.868655 1 \n", "9997 2023-09-24 13:10:40.682894 17.941035 13.739986 1 \n", "9998 2023-09-25 13:10:40.682893 14.533862 1.610512 0 \n", "9999 2023-09-26 13:10:40.682889 13.048549 5.287508 0 \n", "\n", " holiday price_per_kg promo demand previous_days_demand \n", "0 0 1.578387 1 1301.647352 1326.324266 \\\n", "1 0 1.965125 0 1143.972638 1326.324266 \n", "2 0 1.497623 0 890.319248 1168.942267 \n", "3 0 1.952936 0 811.206168 889.965021 \n", "4 0 2.059993 0 822.279469 835.253168 \n", "... ... ... ... ... ... \n", "9995 0 1.556829 1 1981.195884 2089.644454 \n", "9996 0 1.632918 0 2180.698138 2005.305913 \n", "9997 0 0.827723 1 2675.093671 2179.813671 \n", "9998 0 0.589172 0 1703.287285 2674.209204 \n", "9999 0 1.794122 1 1971.029266 1702.474579 \n", "\n", " competitor_price_per_kg marketing_intensity \n", "0 0.755725 0.323086 \n", "1 0.913934 0.030371 \n", "2 2.879262 0.354226 \n", "3 0.826015 0.953000 \n", "4 1.130145 0.953000 \n", "... ... ... \n", "9995 0.560507 0.889971 \n", "9996 2.460766 0.884467 \n", "9997 1.321922 0.884467 \n", "9998 2.604095 0.812706 \n", "9999 1.261635 0.750458 \n", "\n", "[10000 rows x 11 columns]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "my_data" ] }, { "cell_type": "markdown", "id": "8708900f", "metadata": {}, "source": [ "### Time Series Visualization of Demand\n", "\n", "In this section, we're creating a time series plot to visualize the demand data alongside its rolling average. \n", "\n", "#### Why is this Important?\n", "Visualizing time series data is crucial for identifying patterns, understanding variability, and making more informed decisions. By plotting the rolling average alongside, we can smooth out short-term fluctuations and highlight longer-term trends or cycles. This visual aid is essential for understanding the data and making more accurate and informed predictions and decisions.\n", "\n", "#### Structure of the Code:\n", "- **Input Verification**: The code first ensures the data is a pandas DataFrame.\n", "- **Date Conversion**: It converts the 'date' column to a datetime format for accurate plotting.\n", "- **Rolling Average Calculation**: It calculates the rolling average of the 'demand' with a specified window size (`window_size`), defaulting to 7 days.\n", "- **Plotting**: It plots both the original demand data and the calculated rolling average on the same plot for comparison. The original demand data is plotted with low alpha to appear \"ghostly,\" ensuring the rolling average stands out.\n", "- **Labels and Legend**: Adequate labels and legends are added for clarity.\n", "\n", "#### Why Return a Figure?\n", "We return the figure object (`fig`) instead of rendering it directly so that each iteration of a model training event can consume the figure as a logged artifact to MLflow. This approach allows us to persist the state of the data visualization with precisely the state of the data that was used for training. MLflow can store this figure object, enabling easy retrieval and rendering within the MLflow UI, ensuring that the visualization is always accessible and paired with the relevant model and data information.\n" ] }, { "cell_type": "code", "execution_count": 5, "id": "69d8b5c5", "metadata": {}, "outputs": [], "source": [ "def plot_time_series_demand(data, window_size=7, style=\"seaborn\", plot_size=(16, 12)):\n", " if not isinstance(data, pd.DataFrame):\n", " raise TypeError(\"df must be a pandas DataFrame.\")\n", "\n", " df = data.copy()\n", "\n", " df[\"date\"] = pd.to_datetime(df[\"date\"])\n", "\n", " # Calculate the rolling average\n", " df[\"rolling_avg\"] = df[\"demand\"].rolling(window=window_size).mean()\n", "\n", " with plt.style.context(style=style):\n", " fig, ax = plt.subplots(figsize=plot_size)\n", " # Plot the original time series data with low alpha (transparency)\n", " ax.plot(df[\"date\"], df[\"demand\"], \"b-o\", label=\"Original Demand\", alpha=0.15)\n", " # Plot the rolling average\n", " ax.plot(\n", " df[\"date\"],\n", " df[\"rolling_avg\"],\n", " \"r\",\n", " label=f\"{window_size}-Day Rolling Average\",\n", " )\n", "\n", " # Set labels and title\n", " ax.set_title(\n", " f\"Time Series Plot of Demand with {window_size} day Rolling Average\",\n", " fontsize=14,\n", " )\n", " ax.set_xlabel(\"Date\", fontsize=12)\n", " ax.set_ylabel(\"Demand\", fontsize=12)\n", "\n", " # Add legend to explain the lines\n", " ax.legend()\n", " plt.tight_layout()\n", "\n", " plt.close(fig)\n", " return fig" ] }, { "cell_type": "markdown", "id": "6e2f3aa6", "metadata": {}, "source": [ "### Visualizing Demand on Weekends vs. Weekdays with Box Plots\n", "\n", "In this section, we're utilizing box plots to visualize the distribution of demand on weekends versus weekdays. This visualization assists in understanding the variability and central tendency of demand based on the day of the week.\n", "\n", "#### Why is this Important?\n", "Understanding how demand differs between weekends and weekdays is crucial for making informed decisions regarding inventory, staffing, and other operational aspects. It helps identify the periods of higher demand, allowing for better resource allocation and planning.\n", "\n", "#### Structure of the Code:\n", "- **Box Plot**: The code uses Seaborn to create a box plot that shows the distribution of demand on weekends (1) and weekdays (0). The box plot provides insights into the median, quartiles, and possible outliers in the demand data for both categories.\n", "- **Adding Individual Data Points**: To provide more context, individual data points are overlayed on the box plot as a strip plot. They are jittered for better visualization and color-coded based on the day type.\n", "- **Styling**: The plot is styled for clarity, and unnecessary legends are removed to enhance readability.\n", "\n", "#### Why Return a Figure?\n", "As with the time series plot, this function also returns the figure object (`fig`) instead of displaying it directly. " ] }, { "cell_type": "code", "execution_count": 6, "id": "863ed386", "metadata": {}, "outputs": [], "source": [ "def plot_box_weekend(df, style=\"seaborn\", plot_size=(10, 8)):\n", " with plt.style.context(style=style):\n", " fig, ax = plt.subplots(figsize=plot_size)\n", " sns.boxplot(data=df, x=\"weekend\", y=\"demand\", ax=ax, color=\"lightgray\")\n", " sns.stripplot(\n", " data=df,\n", " x=\"weekend\",\n", " y=\"demand\",\n", " ax=ax,\n", " hue=\"weekend\",\n", " palette={0: \"blue\", 1: \"green\"},\n", " alpha=0.15,\n", " jitter=0.3,\n", " size=5,\n", " )\n", "\n", " ax.set_title(\"Box Plot of Demand on Weekends vs. Weekdays\", fontsize=14)\n", " ax.set_xlabel(\"Weekend (0: No, 1: Yes)\", fontsize=12)\n", " ax.set_ylabel(\"Demand\", fontsize=12)\n", " for i in ax.get_xticklabels() + ax.get_yticklabels():\n", " i.set_fontsize(10)\n", " ax.legend_.remove()\n", " plt.tight_layout()\n", " plt.close(fig)\n", " return fig" ] }, { "cell_type": "markdown", "id": "23f8b6b5", "metadata": {}, "source": [ "### Exploring the Relationship Between Demand and Price per Kg\n", "\n", "In this visualization, we're creating a scatter plot to investigate the relationship between the `demand` and `price_per_kg`. Understanding this relationship is crucial for pricing strategy and demand forecasting.\n", "\n", "#### Why is this Important?\n", "- **Insight into Pricing Strategy:** This visualization helps reveal how demand varies with the price per kg, providing valuable insights for setting prices to optimize sales and revenue.\n", "- **Understanding Demand Elasticity:** It aids in understanding the elasticity of demand concerning price, helping in making informed and data-driven decisions for promotions and discounts.\n", "\n", "#### Structure of the Code:\n", "- **Scatter Plot:** The code generates a scatter plot, where each point's position is determined by the `price_per_kg` and `demand`, and the color indicates whether the day is a weekend or a weekday. This color-coding helps in quickly identifying patterns specific to weekends or weekdays.\n", "- **Transparency and Jitter:** Points are plotted with transparency (`alpha=0.15`) to handle overplotting, allowing the visualization of the density of points.\n", "- **Regression Line:** For each subgroup (weekend and weekday), a separate regression line is fitted and plotted on the same axes. These lines provide a clear visual indication of the trend of demand concerning the price per kg for each group." ] }, { "cell_type": "code", "execution_count": 7, "id": "6fccaa4b", "metadata": {}, "outputs": [], "source": [ "def plot_scatter_demand_price(df, style=\"seaborn\", plot_size=(10, 8)):\n", " with plt.style.context(style=style):\n", " fig, ax = plt.subplots(figsize=plot_size)\n", " # Scatter plot with jitter, transparency, and color-coded based on weekend\n", " sns.scatterplot(\n", " data=df,\n", " x=\"price_per_kg\",\n", " y=\"demand\",\n", " hue=\"weekend\",\n", " palette={0: \"blue\", 1: \"green\"},\n", " alpha=0.15,\n", " ax=ax,\n", " )\n", " # Fit a simple regression line for each subgroup\n", " sns.regplot(\n", " data=df[df[\"weekend\"] == 0],\n", " x=\"price_per_kg\",\n", " y=\"demand\",\n", " scatter=False,\n", " color=\"blue\",\n", " ax=ax,\n", " )\n", " sns.regplot(\n", " data=df[df[\"weekend\"] == 1],\n", " x=\"price_per_kg\",\n", " y=\"demand\",\n", " scatter=False,\n", " color=\"green\",\n", " ax=ax,\n", " )\n", "\n", " ax.set_title(\"Scatter Plot of Demand vs Price per kg with Regression Line\", fontsize=14)\n", " ax.set_xlabel(\"Price per kg\", fontsize=12)\n", " ax.set_ylabel(\"Demand\", fontsize=12)\n", " for i in ax.get_xticklabels() + ax.get_yticklabels():\n", " i.set_fontsize(10)\n", " plt.tight_layout()\n", " plt.close(fig)\n", " return fig" ] }, { "cell_type": "markdown", "id": "bd9a1a6c", "metadata": {}, "source": [ "### Visualizing Demand Density: Weekday vs. Weekend\n", "\n", "This visualization allows us to observe the distribution of `demand` separately for weekdays and weekends. \n", "\n", "#### Why is this Important?\n", "- **Demand Distribution Insight:** Understanding the distribution of demand on weekdays versus weekends can inform inventory management and staffing needs. \n", "- **Informing Business Strategy:** This insight is vital for making data-driven decisions regarding promotions, discounts, and other strategies that might be more effective on specific days.\n", "\n", "#### Structure of the Code:\n", "- **Density Plot:** The code generates a density plot for `demand`, separated into weekdays and weekends.\n", "- **Color-Coded Groups:** The two groups (weekday and weekend) are color-coded (blue and green respectively), making it easy to distinguish between them.\n", "- **Transparency and Filling:** The areas under the density curves are filled with a light, transparent color (`alpha=0.15`) for easy visualization while avoiding visual clutter.\n", "\n", "#### What are the Visual Elements?\n", "- **Two Density Curves:** The plot comprises two density curves, one for weekdays and another for weekends. These curves provide a clear visual representation of the distribution of demand for each group.\n", "- **Legend:** A legend is added to help identify which curve corresponds to which group (weekday or weekend).\n" ] }, { "cell_type": "code", "execution_count": 8, "id": "2466319e", "metadata": {}, "outputs": [], "source": [ "def plot_density_weekday_weekend(df, style=\"seaborn\", plot_size=(10, 8)):\n", " with plt.style.context(style=style):\n", " fig, ax = plt.subplots(figsize=plot_size)\n", "\n", " # Plot density for weekdays\n", " sns.kdeplot(\n", " df[df[\"weekend\"] == 0][\"demand\"],\n", " color=\"blue\",\n", " label=\"Weekday\",\n", " ax=ax,\n", " fill=True,\n", " alpha=0.15,\n", " )\n", "\n", " # Plot density for weekends\n", " sns.kdeplot(\n", " df[df[\"weekend\"] == 1][\"demand\"],\n", " color=\"green\",\n", " label=\"Weekend\",\n", " ax=ax,\n", " fill=True,\n", " alpha=0.15,\n", " )\n", "\n", " ax.set_title(\"Density Plot of Demand by Weekday/Weekend\", fontsize=14)\n", " ax.set_xlabel(\"Demand\", fontsize=12)\n", " ax.legend(fontsize=12)\n", " for i in ax.get_xticklabels() + ax.get_yticklabels():\n", " i.set_fontsize(10)\n", "\n", " plt.tight_layout()\n", " plt.close(fig)\n", " return fig" ] }, { "cell_type": "markdown", "id": "8da46df8", "metadata": {}, "source": [ "### Visualization of Model Coefficients\n", "\n", "In this section, we're utilizing a bar plot to visualize the coefficients of the features from the trained model. \n", "\n", "#### Why is this Important?\n", "Understanding the magnitude and direction of the coefficients is essential for interpreting the model. It helps in identifying the most significant features that influence the prediction. This insight is crucial for feature selection, engineering, and ultimately improving the model performance. \n", "\n", "#### Structure of the Code:\n", "- **Context Setting**: The code initiates by setting the plot style to 'seaborn' for aesthetic enhancement.\n", "- **Figure Initialization**: It creates a figure and axes for plotting.\n", "- **Bar Plot**: It uses a horizontal bar plot (`barh`) for visualizing each feature's coefficient. The y-axis represents the feature names, and the x-axis represents the coefficient values. This visualization makes it easy to compare the coefficients, providing insight into their relative importance and impact on the target variable.\n", "- **Title and Labels**: It sets an appropriate title (\"Coefficient Plot\") and labels for the x (\"Coefficient Value\") and y (\"Features\") axes to ensure clarity and understandability.\n", "\n", "By visualizing the coefficients, we can gain a deeper understanding of the model, making it easier to explain the model's predictions and make more informed decisions regarding feature importance and impact.\n" ] }, { "cell_type": "code", "execution_count": 9, "id": "c5c74f96", "metadata": {}, "outputs": [], "source": [ "def plot_coefficients(model, feature_names, style=\"seaborn\", plot_size=(10, 8)):\n", " with plt.style.context(style=style):\n", " fig, ax = plt.subplots(figsize=plot_size)\n", " ax.barh(feature_names, model.coef_)\n", " ax.set_title(\"Coefficient Plot\", fontsize=14)\n", " ax.set_xlabel(\"Coefficient Value\", fontsize=12)\n", " ax.set_ylabel(\"Features\", fontsize=12)\n", " plt.tight_layout()\n", " plt.close(fig)\n", " return fig" ] }, { "cell_type": "markdown", "id": "dfb82643", "metadata": {}, "source": [ "### Visualization of Residuals\n", "\n", "In this section, we're creating a plot to visualize the residuals of the model, which are the differences between the observed and predicted values.\n", "\n", "#### Why is this Important?\n", "A residual plot is a fundamental diagnostic tool in regression analysis used to investigate the unpredictability in the relationship between the predictor variable and the response variable. It helps in identifying non-linearity, heteroscedasticity, and outliers. This plot assists in validating the assumption that the errors are normally distributed and have constant variance, crucial for the reliability of the regression model's predictions.\n", "\n", "#### Structure of the Code:\n", "- **Residual Calculation**: The code begins by calculating the residuals as the difference between the actual (`y_test`) and predicted (`y_pred`) values.\n", "- **Context Setting**: The code sets the plot style to 'seaborn' for a visually appealing plot.\n", "- **Figure Initialization**: It creates a figure and axes for plotting.\n", "- **Residual Plotting**: It utilizes the `residplot` from Seaborn to create the residual plot, with a lowess (locally weighted scatterplot smoothing) line to highlight the trend in the residuals.\n", "- **Zero Line**: It adds a dashed line at zero to serve as a reference for observing the residuals. Residuals above the line indicate under-prediction, while those below indicate over-prediction.\n", "- **Title and Labels**: It sets an appropriate title (\"Residual Plot\") and labels for the x (\"Predicted values\") and y (\"Residuals\") axes to ensure clarity and understandability.\n", "\n", "By examining the residual plot, we can make better-informed decisions on the model's adequacy and the possible need for further refinement or additional complexity.\n" ] }, { "cell_type": "code", "execution_count": 10, "id": "4d9e2bb9", "metadata": {}, "outputs": [], "source": [ "def plot_residuals(y_test, y_pred, style=\"seaborn\", plot_size=(10, 8)):\n", " residuals = y_test - y_pred\n", "\n", " with plt.style.context(style=style):\n", " fig, ax = plt.subplots(figsize=plot_size)\n", " sns.residplot(\n", " x=y_pred,\n", " y=residuals,\n", " lowess=True,\n", " ax=ax,\n", " line_kws={\"color\": \"red\", \"lw\": 1},\n", " )\n", "\n", " ax.axhline(y=0, color=\"black\", linestyle=\"--\")\n", " ax.set_title(\"Residual Plot\", fontsize=14)\n", " ax.set_xlabel(\"Predicted values\", fontsize=12)\n", " ax.set_ylabel(\"Residuals\", fontsize=12)\n", "\n", " for label in ax.get_xticklabels() + ax.get_yticklabels():\n", " label.set_fontsize(10)\n", "\n", " plt.tight_layout()\n", "\n", " plt.close(fig)\n", " return fig" ] }, { "cell_type": "markdown", "id": "3e460595", "metadata": {}, "source": [ "### Visualization of Prediction Errors\n", "\n", "In this section, we're creating a plot to visualize the prediction errors, showcasing the discrepancies between the actual and predicted values from our model.\n", "\n", "#### Why is this Important?\n", "Understanding the prediction errors is crucial for assessing the performance of a model. A prediction error plot provides insight into the error distribution and helps identify trends, biases, or outliers. This visualization is a critical component for model evaluation, helping in identifying areas where the model may need improvement, and ensuring it generalizes well to new data.\n", "\n", "#### Structure of the Code:\n", "- **Context Setting**: The code sets the plot style to 'seaborn' for a clean and attractive plot.\n", "- **Figure Initialization**: It initializes a figure and axes for plotting.\n", "- **Scatter Plot**: The code plots the predicted values against the errors (actual values - predicted values). Each point on the plot represents a specific observation, and its position on the y-axis indicates the magnitude and direction of the error (above zero for under-prediction and below zero for over-prediction).\n", "- **Zero Line**: A red dashed line at y=0 is plotted as a reference, helping in easily identifying the errors. Points above this line are under-predictions, and points below are over-predictions.\n", "- **Title and Labels**: It adds a title (\"Prediction Error Plot\") and labels for the x (\"Predicted Values\") and y (\"Errors\") axes for better clarity and understanding.\n", "\n", "By analyzing the prediction error plot, practitioners can gain valuable insights into the model's performance, helping in the further refinement and enhancement of the model for better and more reliable predictions.\n" ] }, { "cell_type": "code", "execution_count": 11, "id": "3367e6f2", "metadata": {}, "outputs": [], "source": [ "def plot_prediction_error(y_test, y_pred, style=\"seaborn\", plot_size=(10, 8)):\n", " with plt.style.context(style=style):\n", " fig, ax = plt.subplots(figsize=plot_size)\n", " ax.scatter(y_pred, y_test - y_pred)\n", " ax.axhline(y=0, color=\"red\", linestyle=\"--\")\n", " ax.set_title(\"Prediction Error Plot\", fontsize=14)\n", " ax.set_xlabel(\"Predicted Values\", fontsize=12)\n", " ax.set_ylabel(\"Errors\", fontsize=12)\n", " plt.tight_layout()\n", " plt.close(fig)\n", " return fig" ] }, { "cell_type": "markdown", "id": "473196a0", "metadata": {}, "source": [ "### Visualization of Quantile-Quantile Plot (QQ Plot)\n", "\n", "In this section, we will generate a QQ plot to visualize the distribution of the residuals from our model predictions.\n", "\n", "#### Why is this Important?\n", "A QQ plot is essential for assessing if the residuals from the model follow a normal distribution, a fundamental assumption in linear regression models. If the points in the QQ plot do not follow the line closely and show a pattern, this indicates that the residuals may not be normally distributed, which could imply issues with the model such as heteroscedasticity or non-linearity.\n", "\n", "#### Structure of the Code:\n", "- **Residual Calculation**: The code first calculates the residuals by subtracting the predicted values from the actual test values.\n", "- **Context Setting**: The plot style is set to 'seaborn' for aesthetic appeal.\n", "- **Figure Initialization**: A figure and axes are initialized for plotting.\n", "- **QQ Plot Generation**: The `stats.probplot` function is used to generate the QQ plot. It plots the quantiles of the residuals against the quantiles of a normal distribution.\n", "- **Title Addition**: A title (\"QQ Plot\") is added to the plot for clarity.\n", "\n", "By closely analyzing the QQ plot, we can ensure our model's residuals meet the normality assumption. If not, it may be beneficial to explore other model types or transformations to improve the model's performance and reliability.\n" ] }, { "cell_type": "code", "execution_count": 12, "id": "cf1c1104", "metadata": {}, "outputs": [], "source": [ "def plot_qq(y_test, y_pred, style=\"seaborn\", plot_size=(10, 8)):\n", " residuals = y_test - y_pred\n", " with plt.style.context(style=style):\n", " fig, ax = plt.subplots(figsize=plot_size)\n", " stats.probplot(residuals, dist=\"norm\", plot=ax)\n", " ax.set_title(\"QQ Plot\", fontsize=14)\n", " plt.tight_layout()\n", " plt.close(fig)\n", " return fig" ] }, { "cell_type": "markdown", "id": "7f24826b", "metadata": {}, "source": [ "### Feature Correlation Matrix\n", "\n", "In this section, we're generating a **feature correlation matrix** to visualize the relationships between different features in the dataset.\n", "\n", "**NOTE:** Unlike the other plots in this notebook, we're saving a local copy of the plot to disk to show an alternative logging mechanism for arbitrary files, the ``log_artifact()`` API. Within the main model training and logging section below, you will see how this plot is added to the MLflow run.\n", "\n", "#### Why is this Important?\n", "Understanding the correlation between different features is essential for:\n", "- Identifying multicollinearity, which can affect model performance and interpretability.\n", "- Gaining insights into relationships between variables, which can inform feature engineering and selection.\n", "- Uncovering potential causality or interaction between different features, which can inform domain understanding and further analysis.\n", "\n", "#### Structure of the Code:\n", "- **Correlation Calculation**: The code first calculates the correlation matrix for the provided DataFrame.\n", "- **Masking**: A mask is created for the upper triangle of the correlation matrix, as the matrix is symmetrical, and we don't need to visualize the duplicate information.\n", "- **Heatmap Generation**: A heatmap is generated to visualize the correlation coefficients. The color gradient and annotations provide clear insights into the relationships between variables.\n", "- **Title Addition**: A title is added for clear identification of the plot.\n", "\n", "By analyzing the correlation matrix, we can make more informed decisions about feature selection and understand the relationships within our dataset better.\n" ] }, { "cell_type": "code", "execution_count": 13, "id": "4ac26cfa", "metadata": {}, "outputs": [], "source": [ "def plot_correlation_matrix_and_save(\n", " df, style=\"seaborn\", plot_size=(10, 8), path=\"/tmp/corr_plot.png\"\n", "):\n", " with plt.style.context(style=style):\n", " fig, ax = plt.subplots(figsize=plot_size)\n", "\n", " # Calculate the correlation matrix\n", " corr = df.corr()\n", "\n", " # Generate a mask for the upper triangle\n", " mask = np.triu(np.ones_like(corr, dtype=bool))\n", "\n", " # Draw the heatmap with the mask and correct aspect ratio\n", " sns.heatmap(\n", " corr,\n", " mask=mask,\n", " cmap=\"coolwarm\",\n", " vmax=0.3,\n", " center=0,\n", " square=True,\n", " linewidths=0.5,\n", " annot=True,\n", " fmt=\".2f\",\n", " )\n", "\n", " ax.set_title(\"Feature Correlation Matrix\", fontsize=14)\n", " plt.tight_layout()\n", "\n", " plt.close(fig)\n", " # convert to filesystem path spec for os compatibility\n", " save_path = pathlib.Path(path)\n", " fig.savefig(save_path)" ] }, { "cell_type": "markdown", "id": "9f56ee17", "metadata": {}, "source": [ "### Detailed Overview of Main Execution for Model Training and Visualization\n", "\n", "This section delves deeper into the comprehensive workflow executed for model training, prediction, error calculation, and visualization. The significance of each step and the reason for specific choices are thoroughly discussed.\n", "\n", "#### The Benefits of Structured Execution\n", "Executing all crucial steps of model training and evaluation in a structured manner is fundamental. It provides a framework that ensures every aspect of the modeling process is considered, offering a more reliable and robust model. This streamlined execution aids in avoiding overlooked errors or biases and guarantees that the model is evaluated on all necessary fronts.\n", "\n", "#### Importance of Logging Visualizations to MLflow\n", "Logging visualizations to MLflow offers several key benefits:\n", "\n", "- **Permanence**: Unlike the ephemeral state of notebooks where cells can be run out of order leading to potential misinterpretation, logging plots to MLflow ensures that the visualizations are stored permanently with the specific run. This permanence assures that the visual context of the model training and evaluation is preserved, eliminating confusion and ensuring clarity in interpretation.\n", "\n", "- **Provenance**: By logging visualizations, the exact state and relationships in the data at the time of model training are captured. This practice is crucial for models trained a significant time ago. It offers a reliable reference point to understand the model's behavior and the data characteristics at the time of training, ensuring that insights and interpretations remain valid and reliable over time.\n", "\n", "- **Accessibility**: Storing visualizations in MLflow makes them easily accessible to all team members or stakeholders involved. This centralized storage of visualizations enhances collaboration, allowing diverse team members to easily view, analyze, and interpret the visualizations, leading to more informed and collective decision-making.\n", "\n", "#### Detailed Structure of the Code:\n", "\n", "1. **Setting up MLflow**: \n", " - The tracking URI for MLflow is defined.\n", " - An experiment named \"Visualizations Demo\" is set up, under which all runs and logs will be stored.\n", "\n", "2. **Data Preparation**: \n", " - `X` and `y` are defined as features and target variables, respectively.\n", " - The dataset is split into training and testing sets to ensure the model's performance is evaluated on unseen data.\n", "\n", "3. **Initial Plot Generation**: \n", " - Initial plots including time series, box plot, scatter plot, and density plot are generated.\n", " - These plots offer a preliminary insight into the data and its characteristics.\n", "\n", "4. **Model Definition and Training**: \n", " - A Ridge regression model is defined with an `alpha` of 1.0.\n", " - The model is trained on the training data, learning the relationships and patterns in the data.\n", "\n", "5. **Prediction and Error Calculation**: \n", " - The trained model is used to make predictions on the test data.\n", " - Various error metrics including MSE, RMSE, MAE, R2, MSLE, and MedAE are calculated to evaluate the model's performance.\n", "\n", "6. **Additional Plot Generation**: \n", " - Additional plots including residuals plot, coefficients plot, prediction error plot, and QQ plot are generated.\n", " - These plots offer further insight into the model's performance, residuals behavior, and the distribution of errors.\n", "\n", "7. **Logging to MLflow**: \n", " - The trained model, calculated metrics, defined parameter (`alpha`), and all the generated plots are logged to MLflow.\n", " - This logging ensures that all information and visualizations related to the model are stored in a centralized, accessible location.\n", "\n", "#### Conclusion:\n", "By executing this comprehensive and structured code, we ensure that every aspect of model training, evaluation, and interpretation is covered. The practice of logging all relevant information and visualizations to MLflow further enhances the reliability, accessibility, and interpretability of the model and its performance, contributing to more informed and reliable model deployment and utilization.\n" ] }, { "cell_type": "code", "execution_count": 14, "id": "8ee72cf2", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023/09/26 13:10:41 INFO mlflow.tracking.fluent: Experiment with name 'Visualizations Demo' does not exist. Creating a new experiment.\n", "/Users/benjamin.wilson/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/mlflow/models/signature.py:333: UserWarning: Hint: Inferred schema contains integer column(s). Integer columns in Python cannot represent missing values. If your input data contains missing values at inference time, it will be encoded as floats and will cause a schema enforcement error. The best way to avoid this problem is to infer the model schema based on a realistic data sample (training dataset) that includes missing values. Alternatively, you can declare integer columns as doubles (float64) whenever these columns may have missing values. See `Handling Integers With Missing Values `_ for more details.\n", " input_schema = _infer_schema(input_ex)\n", "/Users/benjamin.wilson/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.\n", " warnings.warn(\"Setuptools is replacing distutils.\")\n" ] } ], "source": [ "mlflow.set_tracking_uri(\"http://127.0.0.1:8080\")\n", "\n", "mlflow.set_experiment(\"Visualizations Demo\")\n", "\n", "X = my_data.drop(columns=[\"demand\", \"date\"])\n", "y = my_data[\"demand\"]\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)\n", "\n", "fig1 = plot_time_series_demand(my_data, window_size=28)\n", "fig2 = plot_box_weekend(my_data)\n", "fig3 = plot_scatter_demand_price(my_data)\n", "fig4 = plot_density_weekday_weekend(my_data)\n", "\n", "# Execute the correlation plot, saving the plot to a local temporary directory\n", "plot_correlation_matrix_and_save(my_data)\n", "\n", "# Define our Ridge model\n", "model = Ridge(alpha=1.0)\n", "\n", "# Train the model\n", "model.fit(X_train, y_train)\n", "\n", "# Make predictions\n", "y_pred = model.predict(X_test)\n", "\n", "# Calculate error metrics\n", "mse = mean_squared_error(y_test, y_pred)\n", "rmse = math.sqrt(mse)\n", "mae = mean_absolute_error(y_test, y_pred)\n", "r2 = r2_score(y_test, y_pred)\n", "msle = mean_squared_log_error(y_test, y_pred)\n", "medae = median_absolute_error(y_test, y_pred)\n", "\n", "# Generate prediction-dependent plots\n", "fig5 = plot_residuals(y_test, y_pred)\n", "fig6 = plot_coefficients(model, X_test.columns)\n", "fig7 = plot_prediction_error(y_test, y_pred)\n", "fig8 = plot_qq(y_test, y_pred)\n", "\n", "# Start an MLflow run for logging metrics, parameters, the model, and our figures\n", "with mlflow.start_run() as run:\n", " # Log the model\n", " mlflow.sklearn.log_model(sk_model=model, input_example=X_test, name=\"model\")\n", "\n", " # Log the metrics\n", " mlflow.log_metrics({\n", " \"mse\": mse,\n", " \"rmse\": rmse,\n", " \"mae\": mae,\n", " \"r2\": r2,\n", " \"msle\": msle,\n", " \"medae\": medae,\n", " })\n", "\n", " # Log the hyperparameter\n", " mlflow.log_param(\"alpha\", 1.0)\n", "\n", " # Log plots\n", " mlflow.log_figure(fig1, \"time_series_demand.png\")\n", " mlflow.log_figure(fig2, \"box_weekend.png\")\n", " mlflow.log_figure(fig3, \"scatter_demand_price.png\")\n", " mlflow.log_figure(fig4, \"density_weekday_weekend.png\")\n", " mlflow.log_figure(fig5, \"residuals_plot.png\")\n", " mlflow.log_figure(fig6, \"coefficients_plot.png\")\n", " mlflow.log_figure(fig7, \"prediction_errors.png\")\n", " mlflow.log_figure(fig8, \"qq_plot.png\")\n", "\n", " # Log the saved correlation matrix plot by referring to the local file system location\n", " mlflow.log_artifact(\"/tmp/corr_plot.png\")" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.13" } }, "nbformat": 4, "nbformat_minor": 5 }