{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "title: Lab 2 - Machine Learning para Séries Temporais (notebook)\n", "jupyter: python3\n", "execute:\n", " enabled: false\n", "---\n", "\n", "Versão enxuta e executável do Lab 2.\n", "\n", "Dados: **produção de veículos no Brasil** (Banco Central, SGS 1373).\n", "\n", "Para todos os exemplos, vamos utilizar o esquema de validação cruzada temporal com 4 janelas de teste de 12 meses cada, *expanding*. Ou seja, a cada janela o modelo treina com tudo que viu até ali, sem embaralhar os dados. Assim a comparação entre modelos é justa, pois todos preveem os mesmos 48 pontos (4 janelas × 12 meses) e o mesmo protocolo de treino/teste." ] }, { "cell_type": "code", "metadata": {}, "source": [ "import warnings\n", "warnings.filterwarnings(\"ignore\")\n", "\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "\n", "from scipy.fft import rfft, rfftfreq\n", "from sklearn.ensemble import GradientBoostingRegressor\n", "from sklearn.metrics import mean_absolute_error, mean_squared_error\n", "from statsmodels.tsa.statespace.sarimax import SARIMAX\n", "from statsforecast import StatsForecast\n", "from statsforecast.models import AutoARIMA, AutoETS, SeasonalNaive" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Obs*: tem mais pacotes usados mais adiante, mas deixei para importar na hora de cada exemplo. Tome cuidado pois alguns podem ser difíceis de instalar (torch, pycatch22, por exemplo)\n", "\n", "## 1. Dados - SGS 1373 (produção de veículos, mensal)" ] }, { "cell_type": "code", "metadata": {}, "source": [ "url = \"https://api.bcb.gov.br/dados/serie/bcdata.sgs.1373/dados?formato=csv&dataInicial=01/01/2012\"\n", "df = pd.read_csv(url, sep=\";\", decimal=\",\")\n", "df[\"data\"] = pd.to_datetime(df[\"data\"], format=\"%d/%m/%Y\")\n", "df = df.set_index(\"data\").sort_index().rename(columns={\"valor\": \"y\"})\n", "df = df.asfreq(\"MS\")\n", "df[\"y\"] = df[\"y\"].interpolate()\n", "\n", "print(f\"{len(df)} meses, de {df.index.min().date()} a {df.index.max().date()}\")\n", "df.tail(3)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": {}, "source": [ "plt.figure(figsize=(11, 4))\n", "plt.plot(df.index, df[\"y\"])\n", "plt.title(\"Producao de veiculos no Brasil - SGS 1373\")\n", "plt.ylabel(\"Unidades/mes\")\n", "plt.grid(alpha=0.3)\n", "plt.show()" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. Feature engineering\n", "\n", "Lags (curtos + sazonais), janelas móveis (sempre com `shift(1)` para não vazar\n", "o presente), calendário cíclico e uma dummy para o choque da COVID\n", "(fábricas paradas em 2020)." ] }, { "cell_type": "code", "metadata": {}, "source": [ "def construir_features(s: pd.Series) -> pd.DataFrame:\n", " \"\"\"Lags + rolling + calendario + dummy COVID. Tudo causal.\"\"\"\n", " X = pd.DataFrame(index=s.index)\n", "\n", " for lag in [1, 2, 3, 6, 12, 24]:\n", " X[f\"lag_{lag}\"] = s.shift(lag)\n", "\n", " for win in [3, 6, 12]:\n", " X[f\"roll_mean_{win}\"] = s.shift(1).rolling(win).mean()\n", " X[f\"roll_std_{win}\"] = s.shift(1).rolling(win).std()\n", "\n", " mes = s.index.month\n", " X[\"mes_sin\"] = np.sin(2 * np.pi * mes / 12)\n", " X[\"mes_cos\"] = np.cos(2 * np.pi * mes / 12)\n", "\n", " X[\"covid\"] = ((s.index >= \"2020-03-01\") & (s.index <= \"2020-06-01\")).astype(int)\n", " return X\n", "\n", "X_full = construir_features(df[\"y\"])\n", "X_full.head(13)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### FFT: periodograma\n", "\n", "A Transformada de Fourier revela quais ciclos dominam a série. Tiramos a\n", "tendência antes para a FFT enxergar melhor o ciclo." ] }, { "cell_type": "code", "metadata": {}, "source": [ "y = df[\"y\"].values.astype(float)\n", "y_detrend = y - np.polyval(np.polyfit(np.arange(len(y)), y, 1), np.arange(len(y)))\n", "\n", "amp = np.abs(rfft(y_detrend - y_detrend.mean()))\n", "freq = rfftfreq(len(y_detrend), d=1)\n", "periodo = np.divide(1, freq, where=freq > 0)\n", "\n", "mask = (periodo >= 2) & (periodo <= 48)\n", "top = periodo[mask][np.argmax(amp[mask])]\n", "\n", "plt.figure(figsize=(10, 4))\n", "plt.plot(periodo[mask], amp[mask])\n", "plt.axvline(top, color=\"green\", ls=\"--\", label=f\"Pico ~{top:.1f} meses\")\n", "plt.xlabel(\"Periodo (meses por ciclo)\")\n", "plt.ylabel(\"Amplitude\")\n", "plt.title(\"Periodograma\")\n", "plt.legend()\n", "plt.grid(alpha=0.3)\n", "plt.show()" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. ML \"tabular\": features manuais e automáticas\n", "\n", "Aqui juntamos o que antes estava em dois pontos separados: o **pipeline de\n", "ML tabular** com GradientBoosting, comparando **três jeitos de montar as\n", "features** sob a **mesma** validação cruzada (4 janelas × 12 meses,\n", "*expanding*):\n", "\n", "1. **Manual** - lags, rolling e calendário.\n", "2. **tsfresh** - centenas de features automáticas (configurável).\n", "3. **catch22** - 22 features canônicas, validadas em larga escala.\n", "\n", "Padrão de uso em previsão: para cada mês $t$, pega-se a janela das últimas\n", "$L$ observações e calculam-se as features sobre ela (autocorrelações,\n", "entropia, estatísticas de picos, etc.)." ] }, { "cell_type": "code", "metadata": {}, "source": [ "# helper: mesma CV expanding 4x12 com GradientBoosting sobre QUALQUER matriz X\n", "def cv_gb(X, y, n_windows=4, h=12):\n", " n = len(X)\n", " starts = [n - (n_windows - i) * h for i in range(n_windows)]\n", " out = []\n", " for s0 in starts:\n", " # obs: daria para fazer um grid search para esses hiperparâmetros!\n", " m = GradientBoostingRegressor(\n", " n_estimators=500, \n", " max_depth=4, \n", " learning_rate=0.03, \n", " random_state=42\n", " ).fit(X.iloc[:s0], y.iloc[:s0])\n", " out.append(pd.Series(m.predict(X.iloc[s0:s0+h]), index=y.index[s0:s0+h]))\n", " return pd.concat(out)\n", "\n", "y_train = df[\"y\"].iloc[:-48].values\n", "def mase_sn(y_true, y_pred, m=12):\n", " den = np.mean(np.abs(y_train[m:] - y_train[:-m]))\n", " return mean_absolute_error(y_true, y_pred) / den\n", "\n", "L = 24 # janela de 24 meses por observação\n", "idx_comum = df.index[L:] # mesmas datas para os 3 conjuntos\n", "y_comum = df[\"y\"].reindex(idx_comum)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": {}, "source": [ "# --- tsfresh: features automáticas sobre janelas móveis ---\n", "from tsfresh import extract_features\n", "from tsfresh.feature_extraction import ComprehensiveFCParameters\n", "from tsfresh.utilities.dataframe_functions import impute\n", "\n", "long = []\n", "for i in range(L, len(df)):\n", " janela = df[\"y\"].iloc[i-L:i].to_numpy()\n", " long.append(pd.DataFrame({\"id\": i, \"time\": np.arange(L), \"value\": janela}))\n", "long = pd.concat(long, ignore_index=True)\n", "\n", "X_tsf = extract_features(\n", " long, column_id=\"id\", column_sort=\"time\", column_value=\"value\",\n", " default_fc_parameters=ComprehensiveFCParameters(),\n", " disable_progressbar=True, n_jobs=0,\n", ")\n", "X_tsf = impute(X_tsf) # remove NaN/inf gerados em janelas degeneradas\n", "X_tsf.index = idx_comum\n", "print(\"tsfresh:\", X_tsf.shape[1], \"features\")\n", "X_tsf.head(3)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": {}, "source": [ "# --- catch22: 22 features canônicas sobre as mesmas janelas ---\n", "import pycatch22\n", "\n", "linhas = []\n", "for i in range(L, len(df)):\n", " res = pycatch22.catch22_all(df[\"y\"].iloc[i-L:i].tolist())\n", " linhas.append(res[\"values\"])\n", "X_c22 = pd.DataFrame(linhas, columns=res[\"names\"], index=idx_comum)\n", "print(\"catch22:\", X_c22.shape[1], \"features\")\n", "X_c22.head(3)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": {}, "source": [ "# --- comparação: manual vs tsfresh vs catch22, todos no MESMO GradientBoosting/CV ---\n", "X_man = construir_features(df[\"y\"]).reindex(idx_comum)\n", "\n", "X_all = pd.concat([X_man, X_tsf, X_c22], axis=1)\n", "resultados = {\n", " \"manual (lags+rolling+cal)\": cv_gb(X_man, y_comum),\n", " \"tsfresh (ComprehensiveFC)\": cv_gb(X_tsf, y_comum),\n", " \"catch22\": cv_gb(X_c22, y_comum),\n", " \"todos juntos\": cv_gb(X_all, y_comum),\n", "}\n", "\n", "y_obs = y_comum.iloc[-48:]\n", "cmp_feats = (\n", " pd.Series({\n", " nome: mase_sn(y_obs, pred.iloc[-48:]) for nome, pred in resultados.items()\n", " }, name=\"MASE\")\n", " .sort_values().round(3)\n", ")\n", "cmp_feats" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Leitura: features automáticas **não vencem por mágica**. Numa série curta e\n", "bem-comportada, o conjunto manual (lags sazonais + rolling) costuma empatar\n", "ou ganhar do tsfresh/catch22 - que brilham mais em **classificação de séries**\n", "ou quando você tem **muitas séries** e não sabe quais features importam.\n", "Ainda assim, são um ótimo *baseline* automático e um gerador de ideias.\n", "\n", "### Importância das features (os 3 conjuntos)\n", "\n", "Quais variáveis cada conjunto realmente usa? Ajustamos o GradientBoosting na\n", "janela alinhada inteira e olhamos as 10 features mais importantes de cada um." ] }, { "cell_type": "code", "metadata": {}, "source": [ "fig, axes = plt.subplots(2, 2, figsize=(12, 10))\n", "for ax, (nome, X) in zip(\n", " axes.flat, \n", " {\"manual\": X_man, \"tsfresh\": X_tsf, \"catch22\": X_c22, \"todos\": X_all}.items()\n", "):\n", " gb = GradientBoostingRegressor(\n", " n_estimators=500, max_depth=4, learning_rate=0.03, random_state=42\n", " ).fit(X, y_comum)\n", " (pd.Series(gb.feature_importances_, index=X.columns)\n", " .sort_values().tail(10).plot.barh(ax=ax))\n", " ax.set_title(f\"Top features: {nome}\")\n", "plt.tight_layout()\n", "plt.show()" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "O `cv_b` abaixo (GradientBoosting com features manuais) alimenta a comparação\n", "final da seção 6 - reaproveitamos o resultado **já calculado** na célula de\n", "comparação acima, sem re-treinar:" ] }, { "cell_type": "code", "metadata": {}, "source": [ "cv_b = (\n", " resultados[\"manual (lags+rolling+cal)\"]\n", " .rename(\"boost_manual\").rename_axis(\"ds\").reset_index()\n", ")\n", "cv_b.head(3)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Frameworks modernos\n", "\n", "Muitas vezes não é necessário escrever a CV à mão. Os dois frameworks\n", "dominantes fazem **o mesmo esquema** (4 janelas × 12, *expanding*), só com\n", "nomes de método diferentes: no MLForecast é `cross_validation`; no DARTS é\n", "`historical_forecasts` (o mesmo que usamos no N-BEATS). O LightGBM do DARTS\n", "abaixo já entra na comparação final como `lgbm_darts`." ] }, { "cell_type": "code", "metadata": {}, "source": [ "# MLForecast (Nixtla) (pip install mlforecast lightgbm)\n", "from mlforecast import MLForecast\n", "from mlforecast.lag_transforms import RollingMean\n", "from lightgbm import LGBMRegressor\n", "\n", "sf_df = (\n", " df\n", " .reset_index()\n", " .rename(columns={\"data\": \"ds\"})\n", " .assign(unique_id=\"veiculos\")[[\"unique_id\", \"ds\", \"y\"]]\n", ")\n", "\n", "mlf = MLForecast(\n", " models={\"lgbm_mlf\": LGBMRegressor(n_estimators=300, learning_rate=0.05)},\n", " freq=\"MS\", lags=[1, 2, 3, 12, 24],\n", " lag_transforms={1: [RollingMean(3), RollingMean(12)]},\n", " date_features=[\"month\", \"quarter\"],\n", ")\n", "\n", "cv_mlf = mlf.cross_validation(df=sf_df, h=12, n_windows=4)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": {}, "source": [ "# DARTS (pip install u8darts[all])\n", "from darts import TimeSeries\n", "from darts.models import LightGBMModel\n", "\n", "serie = TimeSeries.from_dataframe(df.reset_index(), \"data\", \"y\", freq=\"MS\")\n", "n = len(df)\n", "\n", "lgbm_darts = LightGBMModel(\n", " lags=[-1, -2, -3, -12, -24], output_chunk_length=12, verbose=-1\n", ")\n", "\n", "# mesma CV dos outros: re-treina e prevê 12 meses a cada janela\n", "hf_lgbm = lgbm_darts.historical_forecasts(\n", " serie,\n", " start=serie.time_index[n - 48], # últimos 48 meses = 4 janelas de 12\n", " forecast_horizon=12,\n", " stride=12, # janelas sem sobreposição\n", " retrain=True,\n", " last_points_only=False,\n", " verbose=False,\n", ")\n", "\n", "cv_lgbm = (\n", " pd.concat([h.to_series() for h in hf_lgbm])\n", " .rename(\"lgbm_darts\").rename_axis(\"ds\").reset_index()\n", ")\n", "\n", "cv_lgbm.head(3)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. modelos com a ordem na arquitetura\n", "\n", "ARIMA, ETS e SeasonalNaive via `statsforecast` (AutoARIMA/AutoETS escolhem\n", "ordens/forma por AICc). Já vimos esses antes." ] }, { "cell_type": "code", "metadata": {}, "source": [ "sf_df = (\n", " df\n", " .reset_index()\n", " .rename(columns={\"data\": \"ds\"})\n", " .assign(unique_id=\"veiculos\")[[\"unique_id\", \"ds\", \"y\"]]\n", ")\n", "\n", "sf = StatsForecast(\n", " models=[\n", " AutoARIMA(season_length=12, alias=\"arima\"),\n", " AutoETS(season_length=12, alias=\"ets\"),\n", " SeasonalNaive(season_length=12, alias=\"snaive\"),\n", " ],\n", " freq=\"MS\", n_jobs=1,\n", ")\n", "\n", "cv_sf = sf.cross_validation(df=sf_df, h=12, step_size=12, n_windows=4).reset_index()\n", "cv_sf.head(3)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ARIMA-Boost (híbrido)\n", "\n", "O **AutoARIMA** (algoritmo de Hyndman, via `statsforecast`) escolhe as ordens\n", "por AICc e captura tendência+sazonalidade; o boosting aprende nos **resíduos**\n", "que sobram. Implementação na mão para conseguir os resíduos in-sample." ] }, { "cell_type": "code", "metadata": {}, "source": [ "from statsforecast.models import AutoARIMA\n", "\n", "def cv_arima_boost(s, n_windows=4, h=12):\n", " # pega as features manuais\n", " X_all = construir_features(s)\n", " n = len(s)\n", " test_starts = [n - (n_windows - i) * h for i in range(n_windows)]\n", " rows = []\n", " for i, start in enumerate(test_starts):\n", " s_tr = s.iloc[:start]\n", "\n", " # AutoARIMA (Hyndman) escolhe (p,d,q)(P,D,Q) por AICc a cada janela\n", " arima = AutoARIMA(season_length=12)\n", " arima.fit(s_tr.to_numpy())\n", " prev_arima = arima.predict(h=h)[\"mean\"]\n", " fitted = arima.predict_in_sample()[\"fitted\"]\n", "\n", " resid = pd.Series(s_tr.to_numpy() - fitted,\n", " index=s_tr.index, name=\"r\")\n", " X_tr = X_all.loc[resid.index].dropna()\n", " r_tr = resid.loc[X_tr.index]\n", "\n", " boost = GradientBoostingRegressor(\n", " n_estimators=200,\n", " max_depth=3,\n", " learning_rate=0.05,\n", " random_state=42\n", " ).fit(X_tr, r_tr)\n", "\n", " X_te = X_all.iloc[start:start+h]\n", "\n", " # previsão final = arima + boost nos resíduos\n", " pred_sum = np.asarray(prev_arima) + boost.predict(X_te)\n", " rows.append(pd.DataFrame({\n", " \"ds\": X_te.index, \"y\": s.iloc[start:start+h].values,\n", " \"arima_boost\": pred_sum, \"window\": i,\n", " }))\n", " return pd.concat(rows, ignore_index=True)\n", "\n", "cv_h = cv_arima_boost(df[\"y\"])\n", "cv_h.head(3)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### N-BEATS (DARTS)\n", "\n", "N-BEATS treinado do zero (sem pré-treino). Para a comparação ser justa, ele\n", "precisa ser avaliado **no mesmo protocolo** dos outros modelos: 4 janelas de\n", "12 meses, *expanding*. O `historical_forecasts` do DARTS faz exatamente isso -\n", "re-treina e prevê 12 meses a cada janela (`stride=12` → janelas sem\n", "sobreposição, igual ao `step_size=12`)." ] }, { "cell_type": "code", "metadata": {}, "source": [ "from darts.models import NBEATSModel\n", "\n", "n = len(df)\n", "nbeats = NBEATSModel(\n", " input_chunk_length=24, \n", " output_chunk_length=12,\n", " n_epochs=100, \n", " random_state=42,\n", ")\n", "\n", "hf = nbeats.historical_forecasts(\n", " serie,\n", " start=serie.time_index[n - 48], # últimos 48 meses = 4 janelas de 12\n", " forecast_horizon=12,\n", " stride=12,\n", " retrain=True, # re-treina a cada janela\n", " last_points_only=False, # devolve as 4 janelas inteiras\n", " verbose=False,\n", ")\n", "\n", "cv_nb = (\n", " pd.concat([h.to_series() for h in hf])\n", " .rename(\"nbeats\").rename_axis(\"ds\").reset_index()\n", ")\n", "cv_nb.head(3)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### LSTM (DARTS)\n", "\n", "Rede recorrente, também do zero e no mesmo protocolo. Diferença prática: o\n", "LSTM é **sensível à escala**, então normalizamos a série com `Scaler` antes\n", "de treinar e desfazemos a transformação na previsão." ] }, { "cell_type": "code", "metadata": {}, "source": [ "from darts.models import RNNModel\n", "from darts.dataprocessing.transformers import Scaler\n", "\n", "sc = Scaler() # LSTM exige normalizar a serie\n", "serie_s = sc.fit_transform(serie)\n", "\n", "lstm = RNNModel(\n", " model=\"LSTM\",\n", " input_chunk_length=24,\n", " training_length=36,\n", " n_epochs=200,\n", " random_state=42,\n", ")\n", "hf_lstm = lstm.historical_forecasts(\n", " serie_s,\n", " start=serie_s.time_index[n - 48],\n", " forecast_horizon=12,\n", " stride=12,\n", " retrain=True,\n", " last_points_only=False,\n", " verbose=False,\n", ")\n", "cv_lstm = (\n", " pd.concat([sc.inverse_transform(h).to_series() for h in hf_lstm])\n", " .rename(\"lstm\").rename_axis(\"ds\").reset_index()\n", ")\n", "cv_lstm.head(3)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. Foundation model: Chronos (zero-shot)\n", "\n", "Modelos pré-treinados preveem *zero-shot* (sem treinar). O **Chronos**\n", "(Amazon) está **integrado no DARTS** (`Chronos2Model`), então usamos a mesma\n", "API `historical_forecasts` dos outros modelos - o código fica uniforme. O\n", "`fit` aqui **não treina** (o modelo já vem pré-treinado); ele só registra a\n", "série, e `retrain=False` garante o uso *zero-shot* a cada janela." ] }, { "cell_type": "code", "metadata": {}, "source": [ "# Chronos via DARTS - pip install \"u8darts[all]\"\n", "from darts import concatenate\n", "from darts.models import Chronos2Model\n", "\n", "n, H = len(df), 12\n", "chronos = Chronos2Model(input_chunk_length=96, output_chunk_length=H)\n", "chronos.fit(serie[:n - 48]) # nao treina (pre-treinado); so registra\n", "\n", "hf_ch = chronos.historical_forecasts(\n", " serie,\n", " start=serie.time_index[n - 48], # ultimos 48 meses = 4 janelas de 12\n", " forecast_horizon=H,\n", " stride=H, # janelas sem sobreposicao\n", " retrain=False, # zero-shot: usa o modelo como esta\n", " last_points_only=False,\n", " verbose=False,\n", ")\n", "cv_chronos = (\n", " concatenate(hf_ch).to_series()\n", " .rename(\"chronos\").rename_axis(\"ds\").reset_index()\n", ")\n", "cv_chronos.head(3)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Zero-shot vs. fine-tuning.** O código acima é *zero-shot*: o modelo chega\n", "congelado e prevê direto. Para apertar mais, dá para fazer *fine-tuning* -\n", "continuar o treino do modelo pré-treinado por algumas épocas na sua própria\n", "série. Ajuda quando o seu domínio difere do que o Chronos viu no pré-treino,\n", "ao custo de GPU. Script oficial em `scripts/training` no\n", "[repo do Chronos](https://github.com/amazon-science/chronos-forecasting).\n", "\n", "## 6. Comparação dos modelos\n", "\n", "Agora **todos** os modelos têm previsões nos **mesmos 48 pontos** (4 janelas ×\n", "12 meses, *expanding*, sem embaralhar), alinhadas por data. Só assim o MASE é\n", "comparável entre eles." ] }, { "cell_type": "code", "metadata": {}, "source": [ "def mase(y_true, y_pred, y_train, m=12):\n", " naive = np.mean(np.abs(y_train[m:] - y_train[:-m]))\n", " return mean_absolute_error(y_true, y_pred) / naive\n", "\n", "def metricas(y_true, y_pred, y_train):\n", " return {\n", " \"MAE\": mean_absolute_error(y_true, y_pred),\n", " \"RMSE\": np.sqrt(mean_squared_error(y_true, y_pred)),\n", " \"MASE\": mase(y_true, y_pred, y_train),\n", " }\n", "\n", "\n", "# cada peça já é a previsão da MESMA CV, indexada por ds\n", "eval_df = (\n", " cv_sf[[\"ds\", \"y\", \"arima\", \"ets\", \"snaive\"]]\n", " .merge(cv_b[[\"ds\", \"boost_manual\"]], on=\"ds\")\n", " .merge(cv_h[[\"ds\", \"arima_boost\"]], on=\"ds\")\n", " .merge(cv_mlf[[\"ds\", \"lgbm_mlf\"]], on=\"ds\")\n", " .merge(cv_lgbm[[\"ds\", \"lgbm_darts\"]], on=\"ds\")\n", " .merge(cv_nb[[\"ds\", \"nbeats\"]], on=\"ds\")\n", " .merge(cv_lstm[[\"ds\", \"lstm\"]], on=\"ds\")\n", " .merge(cv_chronos[[\"ds\", \"chronos\"]], on=\"ds\")\n", ")\n", "assert len(eval_df) == 48, f\"esperava 48 pontos, obtive {len(eval_df)}\"\n", "\n", "y_train = df[\"y\"].iloc[:-48].values\n", "modelos = [\"snaive\", \"ets\", \"arima\", \"arima_boost\", \"boost_manual\",\n", " \"lgbm_mlf\", \"lgbm_darts\", \"nbeats\", \"lstm\", \"chronos\"]\n", "resumo = pd.DataFrame({\n", " nome: metricas(eval_df[\"y\"], eval_df[nome], y_train)\n", " for nome in modelos\n", "}).T.sort_values(\"MASE\")\n", "resumo.round(2)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": {}, "source": [ "import plotly.graph_objects as go\n", "\n", "fig = go.Figure()\n", "fig.add_trace(go.Scatter(\n", " x=df.index[-60:], y=df[\"y\"].iloc[-60:],\n", " mode=\"lines\", name=\"Observado\", line=dict(color=\"gray\")))\n", "for nome in modelos:\n", " fig.add_trace(go.Scatter(\n", " x=eval_df[\"ds\"], y=eval_df[nome], mode=\"lines\", name=nome))\n", "fig.update_layout(\n", " title=\"Cross-validation em 4 janelas - todos os modelos\",\n", " template=\"plotly_white\", width=950, height=460,\n", " legend_title_text=\"Modelos\")\n", "fig.update_xaxes(showgrid=True, gridcolor=\"rgba(0,0,0,0.1)\")\n", "fig.update_yaxes(showgrid=True, gridcolor=\"rgba(0,0,0,0.1)\")\n", "fig.show()" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Obs: Não é que LightGBM seja \"ruim\". Ele perde **neste cenário específico**, por vários motivos: poucos dados, menos covariáveis que nosso modelo manual, e sem tuning." ] } ], "metadata": { "kernelspec": { "name": "python3", "language": "python", "display_name": "Python 3 (ipykernel)", "path": "C:\\Users\\jtrec\\AppData\\Local\\Programs\\Python\\Python313\\share\\jupyter\\kernels\\python3" } }, "nbformat": 4, "nbformat_minor": 4 }