{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ ">### 🚩 *Create a free WhyLabs account to get more value out of whylogs!*
\n", ">*Did you know you can store, visualize, and monitor whylogs profiles with the [WhyLabs Observability Platform](https://whylabs.ai/whylogs-free-signup?utm_source=whylogs-Github&utm_medium=whylogs-example&utm_campaign=Feature_Stores_and_whylogs)? Sign up for a [free WhyLabs account](https://whylabs.ai/whylogs-free-signup?utm_source=whylogs-Github&utm_medium=whylogs-example&utm_campaign=Feature_Stores_and_whylogs) to leverage the power of whylogs and WhyLabs together!*" ] }, { "cell_type": "markdown", "metadata": { "id": "W5HhG0NE8Gkf" }, "source": [ "# Logging data from Feature Stores with Feast and whylogs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/whylabs/whylogs/blob/mainline/python/examples/integrations/Feature_Stores_and_whylogs.ipynb)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> This is a `whylogs v1` example. For the analog example in `v0`, please refer to [this example](https://github.com/whylabs/whylogs/blob/maintenance/0.7.x/examples/feast_whylogs_example/feast_whylogs.ipynb)" ] }, { "cell_type": "markdown", "metadata": { "id": "oFQUD_anA1ie" }, "source": [ "## Context\n", "\n", "In this walkthrough, we'll see how you can use Feast and whylogs together at different parts of your ML pipeline - We'll use Feast to set up an online feature store, and then use it to enrich our serving data with additional features. After assembling our feature vector, we'll proceed to log it with whylogs. As the requests for prediction arrives, the logged input features will be statistically profiled. We will explore these profiles to see what kind of insights we can have.\n", "\n", "\n", "\n", "To do so, we'll use a sample dataset of daily taxi rides in NYC, extracted from [here](https://www1.nyc.gov/site/tlc/about/tlc-trip-record-data.page). Our final goal could be a prediction requested at the start of a given ride. This prediction could be whether the customer will give a high tip to the driver, or maybe whether the customer will give him a good review. As an input to the prediction model, in addition to the ride information (like number of passengers, day of the week, or trip distance), we might be interested in enriching our feature vector with information about the driver, like the driver's average speed, average rating or avg trips in the last 24 hours, with the hopes of improving the model's performances.\n", "\n", "![alt text](https://whylabs-public.s3.us-west-2.amazonaws.com/whylogs_examples/feast_integration/images/context.jpg \"Title\")\n", "\n", "The info about the specific ride will be known at inference time. However, the driver statistics might be available to us in a different data source, updated at specific time intervals. We will join these information to assemble a single feature vector by using Feast to set up an online feature store. Feast will materialize the features into the online store from a data source file. This data source will have driver statistic's, according the each driver's ID updated in an hourly basis.\n", "\n", "We will simulate a production pipeline, where requests for predictions will be made at different timestamps. We'll then log the feature vectors for each request into daily profiles for a period of 7 days. We'll then see how we can compare the obtained profiles for possible data issues or drifts we might have between days." ] }, { "cell_type": "markdown", "metadata": { "id": "Qw5QyJ3qBi5f" }, "source": [ "## Changes in Data\n", "\n", "Let's consider some scenarios in which logging and visualizing features would be helpful.\n", "\n", "### Data Freshness\n", "\n", "In this example, we have updated information about drivers in an hourly basis. Let's simulate a scenario in which this frequency gets affected by some reason, and for a particular period we have new information accessible only in 2-hour cycles.\n", "\n", "### Changes in Customer Behavior\n", "\n", "Let's consider a scenario where people's behavior changes: maybe people are riding less. For example, when covid started, the number of rides certainly plummeted. We could also have a change in the criterias people use to rate a driver. For example, now the given rates, or reviews, for each driver could be affected by specific services provided, like the presence of alcohol and/or physical barriers to ensure social distancing. " ] }, { "cell_type": "markdown", "metadata": { "id": "bz21zNiWBtcO" }, "source": [ "## The Feature Repository" ] }, { "cell_type": "markdown", "metadata": { "id": "NoHSMmLqGGIm" }, "source": [ "First of all, let's install the required packages for this tutorial:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Note: you may need to restart the kernel to use updated packages.\n", "%pip install --upgrade pip -qq" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%pip install feast==0.22.4 Pygments 'whylogs[viz]' -qqU" ] }, { "cell_type": "markdown", "metadata": { "id": "1wsv4K6IBziV" }, "source": [ "### Boilerplate - Registering feature definitions and deploying your feature store" ] }, { "cell_type": "markdown", "metadata": { "id": "K8RgLWexB73l" }, "source": [ "In order to deploy our feature store, we need to create a feature repository. In [Feast's quickstart example](https://docs.feast.dev/getting-started/quickstart), this is traditionally done with a `feast init` command. This example is based on the quickstart example, but with some changes in the python and configuration files.\n", "\n", "For this reason, let's quickly create a folder with the required files to create a feature repository adapted to our use case." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "5LfOoBnaC1gw" }, "outputs": [], "source": [ "%%sh\n", "mkdir feature_repo\n", "mkdir feature_repo/data\n", "mkdir feature_repo/whylogs_output\n", "touch feature_repo/__init__.py" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Writing our feature definition in the `example.py` inside our `feature_repo` folder:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "hXp3XAesD3xl", "outputId": "4f8c9d49-fa74-4220-cb9b-caadad338561" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Overwriting feature_repo/example.py\n" ] } ], "source": [ "%%writefile feature_repo/example.py\n", "# This is an example feature definition file\n", "\n", "from datetime import timedelta\n", "\n", "from feast import Entity, FeatureView, Field, FeatureService, FileSource, ValueType\n", "from feast.types import Float32, Int64\n", "\n", "# Read data from parquet files. Parquet is convenient for local development mode. For\n", "# production, you can use your favorite DWH, such as BigQuery. See Feast documentation\n", "# for more info.\n", "driver_hourly_stats = FileSource(\n", " path=\"data/driver_stats.parquet\",\n", " timestamp_field=\"event_timestamp\",\n", " created_timestamp_column=\"created\",\n", ")\n", "\n", "# Define an entity for the driver. You can think of entity as a primary key used to\n", "# fetch features.\n", "# Entity has a name used for later reference (in a feature view, eg)\n", "# and join_key to identify physical field name used in storages\n", "driver = Entity(name=\"driver\", value_type=ValueType.INT64, join_keys=[\"driver_id\"], description=\"driver id\",)\n", "\n", "# Our parquet files contain sample data that includes a driver_id column, timestamps and\n", "# three feature column. Here we define a Feature View that will allow us to serve this\n", "# data to our model online.\n", "driver_hourly_stats_view = FeatureView(\n", " name=\"driver_hourly_stats\",\n", " entities=[\"driver\"], # reference entity by name\n", " ttl=timedelta(seconds=86400 * 1),\n", " schema=[\n", " Field(name=\"rate_1m\", dtype=Int64),\n", " Field(name=\"avg_daily_trips\", dtype=Int64),\n", " Field(name=\"avg_speed\", dtype=Float32),\n", "\n", " ],\n", " online=True,\n", " source=driver_hourly_stats,\n", " tags={},\n", ")\n", "\n", "driver_stats_fs = FeatureService(\n", " name=\"driver_activity\",\n", " features=[driver_hourly_stats_view]\n", ")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Writing the `feature_store.yaml` configuration file:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "Iiq0DUK2EBde", "outputId": "5133b995-8d44-40fd-8795-4af9cd1e4596" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Overwriting feature_repo/feature_store.yaml\n" ] } ], "source": [ "%%writefile feature_repo/feature_store.yaml\n", "project: feature_repo\n", "registry: data/registry.db\n", "provider: local\n", "online_store:\n", " path: data/online_store.db" ] }, { "cell_type": "markdown", "metadata": { "id": "l5JrYDD_FDXK" }, "source": [ "### Downloading the Data Source" ] }, { "cell_type": "markdown", "metadata": { "id": "oq-X_vAzFT4E" }, "source": [ "Let's first navigate to our feature repository folder:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "cTb0fwR9Fdad", "outputId": "de993da7-011e-44e1-9836-95115f799bf9" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/felip/Documents/Projects-WhyLabs/whylogs2/python/examples/integrations/feature_repo\n" ] } ], "source": [ "%cd feature_repo" ] }, { "cell_type": "markdown", "metadata": { "id": "o7ltooeMFg6d" }, "source": [ "Make sure you're on the right folder. You should see an empty data folder (we'll populate it with our data source later), the `example.py` python script, which contains our feature definitions, and the `feature_store.yaml` configuration file." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "3ueZRWSyFmgY", "outputId": "1e1429e8-3591-4e7e-8917-1cd2c29f721f" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ ".:\n", "\u001b[0m\u001b[01;32m__init__.py\u001b[0m* \u001b[34;42mdata\u001b[0m/ \u001b[01;32mexample.py\u001b[0m* \u001b[01;32mfeature_store.yaml\u001b[0m* \u001b[34;42mwhylogs_output\u001b[0m/\n", "\n", "./data:\n", "\u001b[01;32mdriver_stats.parquet\u001b[0m* \u001b[01;32mregistry.db\u001b[0m*\n", "\n", "./whylogs_output:\n" ] } ], "source": [ "%ls -R" ] }, { "cell_type": "markdown", "metadata": { "id": "ci3PSbC5Fred" }, "source": [ "Now, let's download our data source and store it locally in our feature repository:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'0.22.4'" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import feast\n", "\n", "feast.__version__" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "VUTlby8YFvAH", "outputId": "056268fa-6815-428a-c016-b6658f6417bc" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Loading data from https://whylabs-public.s3.us-west-2.amazonaws.com/whylogs_examples/feast_integration/driver_stats.parquet\n", "Saving file source locally\n" ] } ], "source": [ "import pandas as pd\n", "path = f\"https://whylabs-public.s3.us-west-2.amazonaws.com/whylogs_examples/feast_integration/driver_stats.parquet\"\n", "print(f\"Loading data from {path}\")\n", "driver_stats = pd.read_parquet(path)\n", "print(f\"Saving file source locally\")\n", "\n", "driver_stats.to_parquet(\"data/driver_stats.parquet\")" ] }, { "cell_type": "markdown", "metadata": { "id": "W8fsPXZGFy6v" }, "source": [ "In the data source, we have driver's statistics on an hourly basis, such as the average trips done on the last 24 hours, average rating on the last month and average driving speed. You can see more information on how this data was created at the end of this notebook, in the Appendix." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 206 }, "id": "C7CrqYIMF1bG", "outputId": "fc588944-ce4d-41fa-b2e2-fee0a78b63a7" }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
indexevent_timestampdriver_idcreatedavg_daily_tripsrate_1mavg_speed
00.02020-02-10 00:00:0010012022-02-16 16:17:56.44677425316.87
10.02020-02-10 00:00:0010022022-02-16 16:17:56.44677435120.21
21.02020-02-10 01:00:0010012022-02-16 16:17:56.44677419420.77
31.02020-02-10 01:00:0010022022-02-16 16:17:56.44677429319.20
42.02020-02-10 02:00:0010012022-02-16 16:17:56.44677431317.41
\n", "
" ], "text/plain": [ " index event_timestamp driver_id created \\\n", "0 0.0 2020-02-10 00:00:00 1001 2022-02-16 16:17:56.446774 \n", "1 0.0 2020-02-10 00:00:00 1002 2022-02-16 16:17:56.446774 \n", "2 1.0 2020-02-10 01:00:00 1001 2022-02-16 16:17:56.446774 \n", "3 1.0 2020-02-10 01:00:00 1002 2022-02-16 16:17:56.446774 \n", "4 2.0 2020-02-10 02:00:00 1001 2022-02-16 16:17:56.446774 \n", "\n", " avg_daily_trips rate_1m avg_speed \n", "0 25 3 16.87 \n", "1 35 1 20.21 \n", "2 19 4 20.77 \n", "3 29 3 19.20 \n", "4 31 3 17.41 " ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "driver_stats.head()" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "index float64\n", "event_timestamp datetime64[ns]\n", "driver_id int64\n", "created datetime64[ns]\n", "avg_daily_trips int64\n", "rate_1m int64\n", "avg_speed float64\n", "dtype: object" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "driver_stats.dtypes" ] }, { "cell_type": "markdown", "metadata": { "id": "BzjZA_3mGdiO" }, "source": [ "### Deploying the Feature Store" ] }, { "cell_type": "markdown", "metadata": { "id": "KvjqWLdMF9Uf" }, "source": [ "Now, we will scan the python files in our feature repository for feature views/entity definitions, register the objects and deploy the infrastructure with the `feast apply command`." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "eum0PevxG4Oe", "outputId": "a9aa75eb-1af0-40df-c85f-2f759e09f529" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/felip/Documents/Projects-WhyLabs/whylogs2/python/.venv/lib/python3.8/site-packages/feast/entity.py:110: DeprecationWarning: The `value_type` parameter is being deprecated. Instead, the type of an entity should be specified as a Field in the schema of a feature view. Feast 0.24 and onwards will not support the `value_type` parameter. The `entities` parameter of feature views should also be changed to a List[Entity] instead of a List[str]; if this is not done, entity columns will be mistakenly interpreted as feature columns.\n", " warnings.warn(\n", "/mnt/c/Users/felip/Documents/Projects-WhyLabs/whylogs2/python/.venv/lib/python3.8/site-packages/feast/feature_view.py:180: DeprecationWarning: The `entities` parameter should be a list of `Entity` objects. Feast 0.24 and onwards will not support passing in a list of strings to define entities.\n", " warnings.warn(\n", "Created entity \u001b[1m\u001b[32mdriver\u001b[0m\n", "Created feature view \u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m\n", "Created feature service \u001b[1m\u001b[32mdriver_activity\u001b[0m\n", "\n", "Created sqlite table \u001b[1m\u001b[32mfeature_repo_driver_hourly_stats\u001b[0m\n", "\n" ] } ], "source": [ "!feast apply" ] }, { "cell_type": "markdown", "metadata": { "id": "375FJo5AXD4a" }, "source": [ "Let's also load our rides dataframe. In it we, have features about rides made during 10-Feb to 16-Feb (2020), such as the number of passengers, trip distance and pickup date and time. " ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 223 }, "id": "EFceJHojXGgZ", "outputId": "1e0b0028-4058-4bd9-90ee-a5972f68c04c" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Loading data from https://whylabs-public.s3.us-west-2.amazonaws.com/whylogs_examples/nyc_taxi_rides_feb_2020_changed.parquet\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
pickup_weekdaypassenger_counttrip_distancePULocationIDtpep_pickup_datetimepickup_date
22589701.01.202492020-02-10 00:23:212020-02-10
10830105.019.031322020-02-10 01:19:012020-02-10
19672906.00.38682020-02-10 01:29:232020-02-10
23949501.02.902632020-02-10 02:44:202020-02-10
7201406.016.052332020-02-10 04:12:222020-02-10
\n", "
" ], "text/plain": [ " pickup_weekday passenger_count trip_distance PULocationID \\\n", "225897 0 1.0 1.20 249 \n", "108301 0 5.0 19.03 132 \n", "196729 0 6.0 0.38 68 \n", "239495 0 1.0 2.90 263 \n", "72014 0 6.0 16.05 233 \n", "\n", " tpep_pickup_datetime pickup_date \n", "225897 2020-02-10 00:23:21 2020-02-10 \n", "108301 2020-02-10 01:19:01 2020-02-10 \n", "196729 2020-02-10 01:29:23 2020-02-10 \n", "239495 2020-02-10 02:44:20 2020-02-10 \n", "72014 2020-02-10 04:12:22 2020-02-10 " ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "\n", "path = f\"https://whylabs-public.s3.us-west-2.amazonaws.com/whylogs_examples/nyc_taxi_rides_feb_2020_changed.parquet\"\n", "print(f\"Loading data from {path}\")\n", "rides_df = pd.read_parquet(path)\n", "\n", "rides_df.head()" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "rides_df['passenger_count'] = rides_df['passenger_count'].fillna(0).astype('int64')" ] }, { "cell_type": "markdown", "metadata": { "id": "S3sEq_qpXQUj" }, "source": [ "## Additional Transformations" ] }, { "cell_type": "markdown", "metadata": { "id": "ismQ0mWDXacC" }, "source": [ "The real dataset doesn't contain information regarding the taxi driver that conducted the ride. Since our goal is to enrich the dataset with driver features from an external data source, we will create a `driver_id` column. For simplicity, let's consider that this dataset contains ride information of only 2 drivers (IDs `1001` and `1002`)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "id": "wiRJFzh0Xcv5" }, "outputs": [], "source": [ "import numpy as np\n", "rides_df['driver_id'] = np.random.randint(1001, 1003, rides_df.shape[0])" ] }, { "cell_type": "markdown", "metadata": { "id": "MdVylBImXh5C" }, "source": [ "## Features: Load, Fetch and Log" ] }, { "cell_type": "markdown", "metadata": { "id": "J4mc__uVXlkz" }, "source": [ "We will iterate on `rides_df`, where each row represents a point in time in which we will request a prediction. For each request, we will:\n", "\n", "- Materialize latest features into our online feature store\n", "- Get features from the online feature store\n", "- Join the features from the online store (driver features) with ride features\n", "- Log features with whylogs into a profile\n", "\n", "We'll consider that the __materialization__ job is run hourly. To simulate that, we will call __materialize__ for the last rounded hour, based on the request's timestamp `tpep_pickup_datetime`.\n", "\n", "We will iterate through all the requests on the dataset, generate profiles for daily batches of data, and then write the profiles to disk in a binary file for each of the seven days:" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 00:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 01:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 97.08it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 01:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 02:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 101.23it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 03:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 04:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 117.07it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 04:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 05:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 135.54it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 05:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 06:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 141.93it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 06:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 07:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 154.83it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 07:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 08:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 159.83it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 08:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 09:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 123.47it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 09:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 10:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 95.67it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 10:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 11:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.11it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 11:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 12:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 78.79it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 12:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 13:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 127.50it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 13:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 14:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 108.23it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 14:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 15:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 131.21it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 15:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 16:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 114.58it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 16:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 17:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 100.24it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 17:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 18:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 92.09it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 18:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 19:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 133.12it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 19:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 20:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 102.93it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 20:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 21:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 96.86it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 21:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 22:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 168.66it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 22:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 23:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 88.05it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Starting logger for day 2020-02-11 00:00:00....\n", "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 23:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 00:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 158.66it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 00:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 01:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 157.59it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 04:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 05:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 149.97it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 05:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 06:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "0it [00:00, ?it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 06:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 07:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 148.77it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 07:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 08:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "0it [00:00, ?it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 08:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 09:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 96.71it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 09:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 10:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "0it [00:00, ?it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 10:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 11:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 100.11it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 11:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 12:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "0it [00:00, ?it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 12:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 13:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.29it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 13:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 14:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "0it [00:00, ?it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 14:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 15:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 108.01it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 15:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 16:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "0it [00:00, ?it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 16:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 17:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 94.35it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 17:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 18:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "0it [00:00, ?it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 18:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 19:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 110.94it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 19:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 20:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "0it [00:00, ?it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 20:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 21:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 119.28it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 21:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 22:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "0it [00:00, ?it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 22:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 23:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 98.93it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Starting logger for day 2020-02-12 00:00:00....\n", "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 23:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 00:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "0it [00:00, ?it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 00:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 01:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 111.18it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 03:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 04:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 142.89it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 06:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 07:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 83.19it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 07:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 08:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 103.58it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 08:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 09:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 139.98it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 09:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 10:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 105.49it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 10:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 11:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 124.73it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 11:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 12:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 128.29it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 12:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 13:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 105.62it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 13:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 14:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 103.11it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 14:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 15:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 123.69it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 15:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 16:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 84.25it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 16:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 17:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 127.68it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 17:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 18:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 111.89it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 18:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 19:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 108.54it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 19:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 20:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 92.51it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 20:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 21:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 101.55it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 21:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 22:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 93.28it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 22:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 23:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 118.25it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Starting logger for day 2020-02-13 00:00:00....\n", "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 01:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 02:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 122.30it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 03:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 04:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 124.17it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 04:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 05:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 141.48it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 05:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 06:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 120.50it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 06:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 07:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 125.49it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 07:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 08:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 90.89it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 08:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 09:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 133.53it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 09:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 10:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 96.20it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 10:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 11:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 81.72it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 11:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 12:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 88.51it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 12:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 13:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 112.87it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 13:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 14:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 126.05it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 14:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 15:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.44it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 15:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 16:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 140.34it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 16:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 17:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 104.42it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 17:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 18:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 148.64it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 18:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 19:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 131.96it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 19:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 20:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 129.22it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 20:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 21:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 108.97it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 21:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 22:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 111.21it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 22:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 23:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 117.89it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Starting logger for day 2020-02-14 00:00:00....\n", "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 23:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 00:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 127.16it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 00:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 01:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 100.66it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 01:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 02:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 138.51it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 05:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 06:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.10it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 06:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 07:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 146.52it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 07:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 08:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 133.95it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 08:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 09:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 92.94it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 09:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 10:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 123.47it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 10:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 11:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 117.00it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 11:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 12:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 104.38it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 12:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 13:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 144.85it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 13:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 14:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 50.76it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 14:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 15:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 80.35it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 15:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 16:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.47it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 16:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 17:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 93.62it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 17:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 18:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 97.90it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 18:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 19:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 128.72it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 19:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 20:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 68.97it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 20:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 21:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 103.13it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 21:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 22:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 134.36it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 22:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 23:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 100.09it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Starting logger for day 2020-02-15 00:00:00....\n", "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 23:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 00:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.28it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 00:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 01:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 134.45it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 01:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 02:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 121.67it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 02:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 03:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 82.45it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 06:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 07:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 131.26it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 07:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 08:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 100.35it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 08:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 09:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 106.04it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 09:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 10:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 124.26it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 10:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 11:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 121.92it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 11:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 12:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 124.89it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 12:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 13:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 107.71it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 13:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 14:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.57it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 14:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 15:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 108.74it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 15:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 16:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 115.71it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 16:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 17:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 111.62it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 17:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 18:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 97.34it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 18:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 19:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 93.39it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 19:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 20:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 101.70it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 21:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 22:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 102.01it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 22:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 23:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 93.55it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Starting logger for day 2020-02-16 00:00:00....\n", "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 23:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 00:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 115.39it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 00:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 01:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 120.52it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 01:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 02:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.76it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 04:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 05:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 125.81it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 07:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 08:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 86.17it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 08:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 09:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 107.42it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 09:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 10:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 123.10it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 10:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 11:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 121.85it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 11:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 12:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 131.44it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 12:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 13:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 141.69it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 13:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 14:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 99.13it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 14:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 15:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 98.62it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 15:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 16:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 149.06it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 16:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 17:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 112.62it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 17:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 18:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 92.72it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 18:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 19:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 102.09it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 19:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 20:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 131.01it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 20:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 21:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 136.77it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 21:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 22:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 93.03it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 22:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 23:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n", "\n", "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 135.85it/s]\n" ] } ], "source": [ "from datetime import datetime, timedelta\n", "from pprint import pprint\n", "from feast import FeatureStore\n", "import os\n", "import whylogs as why\n", "\n", "store = FeatureStore(repo_path=\".\")\n", "\n", "prev_time = datetime(2020, 2, 10, 00, 00)\n", "target_time = datetime(2020, 2, 10, 1, 00)\n", "store.materialize(start_date=prev_time,end_date=target_time)\n", "\n", "# Initializing logger for the first day\n", "day_to_log = datetime(2020, 2, 10)\n", "\n", "profile = None\n", "for index,row in rides_df.iterrows():\n", "\n", " request_timestamp = row['tpep_pickup_datetime']\n", "\n", " # If new request is from the next day, close logger, save profile in-memory and start logger for the next day\n", " if request_timestamp.day > day_to_log.day:\n", " # let's write our profiles to whylogs_output folder\n", " why.write(profile,\"whylogs_output\",\"profile_{}_{}_{}.bin\".format(day_to_log.day,day_to_log.month,day_to_log.year))\n", " day_to_log = request_timestamp.replace(hour=0, minute=0, second=0, microsecond=0)\n", " print(\"Starting logger for day {}....\".format(day_to_log))\n", " profile = None\n", " if request_timestamp>target_time + timedelta(hours=1):\n", " target_time = datetime(request_timestamp.year,request_timestamp.month,request_timestamp.day,request_timestamp.hour)\n", " prev_time = target_time - timedelta(hours=1)\n", " store.materialize(start_date=prev_time,end_date=target_time)\n", "\n", " driver_feature_vector = store.get_online_features(\n", " features=[\n", " \"driver_hourly_stats:rate_1m\",\n", " \"driver_hourly_stats:avg_daily_trips\",\n", " \"driver_hourly_stats:avg_speed\"\n", " ],\n", " entity_rows=[{\"driver_id\": row['driver_id']},],\n", " ).to_dict()\n", "\n", " # Get features from both ride and driver\n", " assembled_feature_vector = {\n", " \"pickup_weekday\": row[\"pickup_weekday\"],\n", " \"passenger_count\": row[\"passenger_count\"],\n", " \"trip_distance\": row[\"trip_distance\"],\n", " \"PULocationID\": row[\"PULocationID\"],\n", " \"driver_avg_daily_trips\": driver_feature_vector[\"avg_daily_trips\"][0],\n", " \"driver_rate_1m\": driver_feature_vector[\"rate_1m\"][0],\n", " \"driver_avg_speed\": driver_feature_vector[\"avg_speed\"][0],\n", "\n", " }\n", "\n", " # Now that we have the complete set of features, model prediction could go here.\n", "\n", " # The first time data is logged to a profile, we call log(). For subsequent data to be logged in the same profile, let's use track(), until the daily batch is finished.\n", " if not profile:\n", " profile = why.log(row=assembled_feature_vector).profile()\n", " else:\n", " profile.track(assembled_feature_vector)\n", " \n", "why.write(profile,\"whylogs_output\",\"profile_{}_{}_{}.bin\".format(day_to_log.day,day_to_log.month,day_to_log.year))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's confirm that the profiles for each day was indeed written to the disk:" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "profile_10_2_2020.bin\n", "profile_11_2_2020.bin\n", "profile_12_2_2020.bin\n", "profile_13_2_2020.bin\n", "profile_14_2_2020.bin\n", "profile_15_2_2020.bin\n", "profile_16_2_2020.bin\n" ] } ], "source": [ "%%sh\n", "ls whylogs_output" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can rehydrate each of those profiles to check some of the metrics provided in the profile. Let's take the first day as our reference profile:" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
cardinality/estcardinality/lower_1cardinality/upper_1counts/ncounts/nulldistribution/maxdistribution/meandistribution/mediandistribution/mindistribution/n...distribution/stddevfrequent_items/frequent_stringsints/maxints/mintypetypes/booleantypes/fractionaltypes/integraltypes/objecttypes/string
column
PULocationID39.00000439.039.001951980264.000000161.663265161.0041.0098...64.712997[FrequentItem(value='161.000000', est=6, upper...264.041.0SummaryType.COLUMN009800
driver_avg_daily_trips17.00000117.017.00084998044.00000029.42857130.009.0098...7.729979[FrequentItem(value='36.000000', est=14, upper...44.09.0SummaryType.COLUMN009800
driver_avg_speed35.00000335.035.00175098031.13999921.18255120.9813.2198...3.748177NaNNaNNaNSummaryType.COLUMN098000
driver_rate_1m4.0000004.04.0002009804.0000002.5408163.001.0098...0.801653[FrequentItem(value='3.000000', est=45, upper=...4.01.0SummaryType.COLUMN009800
passenger_count6.0000006.06.0003009806.0000001.4081631.000.0098...1.199972[FrequentItem(value='1.000000', est=77, upper=...6.00.0SummaryType.COLUMN009800
pickup_weekday1.0000001.01.0000509800.0000000.0000000.000.0098...0.000000[FrequentItem(value='0.000000', est=98, upper=...0.00.0SummaryType.COLUMN009800
trip_distance83.00001783.083.00416198020.2200002.7915311.620.2498...3.606351NaNNaNNaNSummaryType.COLUMN098000
\n", "

7 rows × 28 columns

\n", "
" ], "text/plain": [ " cardinality/est cardinality/lower_1 \\\n", "column \n", "PULocationID 39.000004 39.0 \n", "driver_avg_daily_trips 17.000001 17.0 \n", "driver_avg_speed 35.000003 35.0 \n", "driver_rate_1m 4.000000 4.0 \n", "passenger_count 6.000000 6.0 \n", "pickup_weekday 1.000000 1.0 \n", "trip_distance 83.000017 83.0 \n", "\n", " cardinality/upper_1 counts/n counts/null \\\n", "column \n", "PULocationID 39.001951 98 0 \n", "driver_avg_daily_trips 17.000849 98 0 \n", "driver_avg_speed 35.001750 98 0 \n", "driver_rate_1m 4.000200 98 0 \n", "passenger_count 6.000300 98 0 \n", "pickup_weekday 1.000050 98 0 \n", "trip_distance 83.004161 98 0 \n", "\n", " distribution/max distribution/mean \\\n", "column \n", "PULocationID 264.000000 161.663265 \n", "driver_avg_daily_trips 44.000000 29.428571 \n", "driver_avg_speed 31.139999 21.182551 \n", "driver_rate_1m 4.000000 2.540816 \n", "passenger_count 6.000000 1.408163 \n", "pickup_weekday 0.000000 0.000000 \n", "trip_distance 20.220000 2.791531 \n", "\n", " distribution/median distribution/min distribution/n \\\n", "column \n", "PULocationID 161.00 41.00 98 \n", "driver_avg_daily_trips 30.00 9.00 98 \n", "driver_avg_speed 20.98 13.21 98 \n", "driver_rate_1m 3.00 1.00 98 \n", "passenger_count 1.00 0.00 98 \n", "pickup_weekday 0.00 0.00 98 \n", "trip_distance 1.62 0.24 98 \n", "\n", " ... distribution/stddev \\\n", "column ... \n", "PULocationID ... 64.712997 \n", "driver_avg_daily_trips ... 7.729979 \n", "driver_avg_speed ... 3.748177 \n", "driver_rate_1m ... 0.801653 \n", "passenger_count ... 1.199972 \n", "pickup_weekday ... 0.000000 \n", "trip_distance ... 3.606351 \n", "\n", " frequent_items/frequent_strings \\\n", "column \n", "PULocationID [FrequentItem(value='161.000000', est=6, upper... \n", "driver_avg_daily_trips [FrequentItem(value='36.000000', est=14, upper... \n", "driver_avg_speed NaN \n", "driver_rate_1m [FrequentItem(value='3.000000', est=45, upper=... \n", "passenger_count [FrequentItem(value='1.000000', est=77, upper=... \n", "pickup_weekday [FrequentItem(value='0.000000', est=98, upper=... \n", "trip_distance NaN \n", "\n", " ints/max ints/min type types/boolean \\\n", "column \n", "PULocationID 264.0 41.0 SummaryType.COLUMN 0 \n", "driver_avg_daily_trips 44.0 9.0 SummaryType.COLUMN 0 \n", "driver_avg_speed NaN NaN SummaryType.COLUMN 0 \n", "driver_rate_1m 4.0 1.0 SummaryType.COLUMN 0 \n", "passenger_count 6.0 0.0 SummaryType.COLUMN 0 \n", "pickup_weekday 0.0 0.0 SummaryType.COLUMN 0 \n", "trip_distance NaN NaN SummaryType.COLUMN 0 \n", "\n", " types/fractional types/integral types/object \\\n", "column \n", "PULocationID 0 98 0 \n", "driver_avg_daily_trips 0 98 0 \n", "driver_avg_speed 98 0 0 \n", "driver_rate_1m 0 98 0 \n", "passenger_count 0 98 0 \n", "pickup_weekday 0 98 0 \n", "trip_distance 98 0 0 \n", "\n", " types/string \n", "column \n", "PULocationID 0 \n", "driver_avg_daily_trips 0 \n", "driver_avg_speed 0 \n", "driver_rate_1m 0 \n", "passenger_count 0 \n", "pickup_weekday 0 \n", "trip_distance 0 \n", "\n", "[7 rows x 28 columns]" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "reference_profile = why.read(os.path.join(\"whylogs_output\",\"profile_10_2_2020.bin\"))\n", "# we generate a profile view, and then call to_pandas() to have a dataframe with the metrics to be inspected\n", "reference_metrics = reference_profile.view().to_pandas()\n", "reference_metrics" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you want to know more about inspecting profiles and metrics contained in them, check the example on [Inspecting Profiles](https://whylogs.readthedocs.io/en/stable/examples/basic/Inspecting_Profiles.html) !" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Injecting data issues and comparing profiles" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let's add some data error issues into the dataset and see how we could visually inspect this with some of whylog's functionalites. Some of the changes applied are shown as following:\n", "\n", "- Feb 10: No changes\n", "\n", "- Feb 11: (Data update error) New driver features are available only in 2 hour cycles. Simulating a scenario in which the sampling frequency is affected due to changes upstream.\n", "- Feb 16: (Feature drift) Based on the considerations made on section Changes in Data, we will: a) Reduce the number of passengers (passenger_count) and b) increase the standard deviation of rate_1m's distribution. For more information of how that was done, please see Appendix - Changing the Dataset." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We already have our reference profile, so let's load from disk two other profiles that contain the data update and feature drift issues, respectively." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "target_profile_1 = why.read(os.path.join(\"whylogs_output\",\"profile_11_2_2020.bin\"))\n", "target_profile_2 = why.read(os.path.join(\"whylogs_output\",\"profile_16_2_2020.bin\"))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Data update" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The data update issue is a subtle one, since we still have data available with the expected shape and values. The only difference is that the values are being updated less often. Ideally, this could be checked elsewhere in our pipeline, but with information available in our assembled feature vector, we could get signals of this issues indirectly by inspecting the cardinality of the features collected from the driver source.\n", "\n", "Let's check the cardinality of the average speed for our reference profile, which is a float variable:\n" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Cardinality for driver average speed for Reference dataset: 35.000002955397264\n" ] } ], "source": [ "card = reference_metrics.loc['driver_avg_speed']['cardinality/est']\n", "print(\"Cardinality for driver average speed for Reference dataset:\",card)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For the same frequency update, we can expect cardinality estimates around the value seen in our baseline.\n", "\n", "Let's now compare the cardinality estimations in the other two profiles:" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Cardinality for driver average speed for profile #1:\n", "24.00000137090692\n", "Cardinality for driver average speed for profile #2:\n", "35.000002955397264\n" ] } ], "source": [ "profile_1_metrics = target_profile_1.view().to_pandas() \n", "profile_2_metrics = target_profile_2.view().to_pandas() \n", "\n", "print(\"Cardinality for driver average speed for profile #1:\")\n", "print(profile_1_metrics.loc['driver_avg_speed']['cardinality/est'])\n", "print(\"Cardinality for driver average speed for profile #2:\")\n", "print(profile_2_metrics.loc['driver_avg_speed']['cardinality/est'])" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "We can see there's a significant difference for the profile that is updated less frequently.\n", "\n", "You could automate this type of assertion by using Constraints in order to do data validation in your data. If you want to know more, please see the example on [Building Metric Constraints](https://whylogs.readthedocs.io/en/stable/examples/advanced/Metric_Constraints.html)!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Feature Drift" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For February 16, we have a change in the average daily trips and also in the driver's monthly rating.\n", "\n", "We can compare both profiles in order to detect data drifts and generate a report for every feature in the profiles by using the `NotebookProfileVisualizer`" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [], "source": [ "from whylogs.viz import NotebookProfileVisualizer\n", "\n", "visualization = NotebookProfileVisualizer()\n", "visualization.set_profiles(target_profile_view=target_profile_2.view(), reference_profile_view=reference_profile.view())" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "visualization.summary_drift_report()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The report warns us of 3 possible drifts in the following features:\n", "\n", "- driver_avg_daily_trips\n", "- driver_rate_1m\n", "- pickup_weekday\n", "\n", "Indeed, we artificially changed the first two features, so we might expect to see a drift alert for those. The third one is also a drift, but probably not a relevant one, since it's pretty obvious that a feature that reflects the day of the week will be different for daily batches (unless both of them are from the same day of the week!)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can inspect these features further by using `distribution_chart()` for the driver's rating and `double_histogram()` for the daily trips:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Note: Even though both features are integers, each feature can be better visualized with different types of visualizations. That happens because integers can be viewed as number, properly, or as a sort of encoding categorical variables. Since the driver's rating has few different possible number, and it wouldn't make sense to group different numbers in a single bin, this feature is better visualized by treating them as categorical variables, and therefore using the `distribution_chart()` visualization." ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "visualization.distribution_chart(feature_name=\"driver_rate_1m\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There's a lot of ratings that don't even show in the reference profile, so it really likes like there's a significant drift here." ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "visualization.double_histogram(feature_name=\"driver_avg_daily_trips\")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Likewise, these histograms almost don't overlap, so it's pretty clear that these distributions are different." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `NotebookProfileVisualizer` has a bunch of other types of features and visualization. If you like to know more, be sure to check the example on the [Notebook Profile Visualizer](https://whylogs.readthedocs.io/en/stable/examples/basic/Notebook_Profile_Visualizer.html)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Appendix - Changing the Dataset" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This section is not really a part of the demonstration. It's just to show the changes made in the dataset that originated the `driver_stats_changed.parquet` file that will be used in the beginning of the notebook." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Driver Statistics\n", "\n", "The NYC taxi datasets provides only information about rides, but in this example we want to show an example of using an online feature store to enrich ride information with driver statistics. So, we'll fabricate some driver statistics and link them with the rides dataset (__nyc_taxi_rides_feb_2020.parquet__) through the `Driver_ID` key." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "\n", "\n", "dstats = pd.DataFrame(\n", " {'event_timestamp': pd.date_range('2020-02-10', '2020-02-17', freq='1H', closed='left')}\n", " )\n", "dstats['driver_id'] = '1001'\n", "\n", "dstats2 = pd.DataFrame(\n", " {'event_timestamp': pd.date_range('2020-02-10', '2020-02-17', freq='1H', closed='left')}\n", " )\n", "dstats2['driver_id'] = '1002'\n", "\n", "dstats_tot = pd.concat([dstats, dstats2])" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "dstats_tot = dstats_tot.sort_values(by=[\"event_timestamp\",\"driver_id\"])" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [], "source": [ "import datetime\n", "dstats_tot['created'] = datetime.datetime.now()" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "\n", "mu, sigma = 30, 6 # mean and standard deviation\n", "s = np.random.normal(mu, sigma, len(dstats_tot))\n", "daily_trips = np.round(s)\n", "daily_trips = [int(x) for x in daily_trips]\n", "dstats_tot['avg_daily_trips'] = daily_trips" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [], "source": [ "from scipy.stats import truncnorm\n", "\n", "def get_truncated_normal(mean=3, sd=0.75, low=1, upp=11):\n", " return truncnorm(\n", " (low - mean) / sd, (upp - mean) / sd, loc=mean, scale=sd)\n", "\n", "X = get_truncated_normal()\n", "\n", "dstats_tot['rate_1m'] = [int(x) for x in X.rvs(len(dstats_tot))]" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "\n", "mu, sigma = 20, 4 # mean and standard deviation\n", "s = np.random.normal(mu, sigma, len(dstats_tot))\n", "avg_speed = np.round(s,2)\n", "avg_speed\n", "dstats_tot['avg_speed'] = avg_speed" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Adding changes - Stats Update Frequency" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [], "source": [ "dstats_tot = dstats_tot.reset_index()\n", "cond = (dstats_tot['event_timestamp'].dt.day==11) & (dstats_tot['event_timestamp'].dt.month==2) & ((dstats_tot['event_timestamp'].dt.hour%2)!=0)\n", "df2 = dstats_tot.loc[cond]\n", "dstats_tot = dstats_tot[~dstats_tot.isin(df2)].dropna()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Adding changes - Rate_1m\n", "\n", "We're assuming that this change in customer's behaviour would not change the mean of the distribution, but would have an increased standard deviation, making the rates be more spreaded, increasing the frequency of extreme ratings (positive or negative)." ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "cond = (dstats_tot['event_timestamp'].dt.day==14) & (dstats_tot['event_timestamp'].dt.month==2)\n", "size = len(dstats_tot.loc[cond])\n", "\n", "X = get_truncated_normal(mean=3, sd=2, low=1, upp=11)\n", "rate_1m = [int(x) for x in X.rvs(size)]\n", "dstats_tot.loc[cond, 'rate_1m'] = rate_1m\n", "\n", "import numpy as np\n", "cond = (dstats_tot['event_timestamp'].dt.day==15) & (dstats_tot['event_timestamp'].dt.month==2)\n", "size = len(dstats_tot.loc[cond])\n", "\n", "X = get_truncated_normal(mean=3, sd=3, low=1, upp=11)\n", "rate_1m = [int(x) for x in X.rvs(size)]\n", "dstats_tot.loc[cond, 'rate_1m'] = rate_1m\n", "\n", "import numpy as np\n", "cond = (dstats_tot['event_timestamp'].dt.day==16) & (dstats_tot['event_timestamp'].dt.month==2)\n", "size = len(dstats_tot.loc[cond])\n", "\n", "X = get_truncated_normal(mean=3, sd=4, low=1, upp=11)\n", "rate_1m = [int(x) for x in X.rvs(size)]\n", "dstats_tot.loc[cond, 'rate_1m'] = rate_1m" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Adding Changes - Avg Daily Trips" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [], "source": [ "cond = (dstats_tot['event_timestamp'].dt.day==14) & (dstats_tot['event_timestamp'].dt.month==2)\n", "size = len(dstats_tot.loc[cond])\n", "\n", "mu, sigma = 24, 6 # mean and standard deviation\n", "s = np.random.normal(mu, sigma, size)\n", "daily_trips = np.round(s)\n", "daily_trips = [int(x) if x>0 else 0 for x in daily_trips]\n", "# daily_trips\n", "\n", "dstats_tot.loc[cond, 'avg_daily_trips'] = daily_trips\n", "\n", "cond = (dstats_tot['event_timestamp'].dt.day==15) & (dstats_tot['event_timestamp'].dt.month==2)\n", "size = len(dstats_tot.loc[cond])\n", "\n", "mu, sigma = 12, 6 # mean and standard deviation\n", "s = np.random.normal(mu, sigma, size)\n", "daily_trips = np.round(s)\n", "daily_trips = [int(x) if x>0 else 0 for x in daily_trips]\n", "# daily_trips\n", "\n", "dstats_tot.loc[cond, 'avg_daily_trips'] = daily_trips\n", "\n", "cond = (dstats_tot['event_timestamp'].dt.day==16) & (dstats_tot['event_timestamp'].dt.month==2)\n", "size = len(dstats_tot.loc[cond])\n", "\n", "mu, sigma = 3, 6 # mean and standard deviation\n", "s = np.random.normal(mu, sigma, size)\n", "daily_trips = np.round(s)\n", "daily_trips = [int(x) if x>0 else 0 for x in daily_trips]\n", "# daily_trips\n", "\n", "dstats_tot.loc[cond, 'avg_daily_trips'] = daily_trips\n", "\n", "dstats_tot = dstats_tot.astype({'driver_id': 'int64','avg_daily_trips':'int64','rate_1m':'int64'})\n", "# dstats_tot.to_parquet(\"driver_stats.parquet\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Rides Dataset" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `nyc_taxi_rides_feb_2020.parquet` was extracted from [the TLC trip record data](https://www1.nyc.gov/site/tlc/about/tlc-trip-record-data.page). We randomly sampled the data and selected a few chosen features, in order to reduce the dataset for this demonstration.\n", "\n", "In addition, one features was created: The day of the week, based from `tpep_pickup_datetime`.\n", "\n", "The original features are described in this [data dictionary](https://www1.nyc.gov/assets/tlc/downloads/pdf/data_dictionary_trip_records_yellow.pdf)." ] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "Feast_v1.ipynb", "provenance": [] }, "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.10" }, "vscode": { "interpreter": { "hash": "5dd5901cadfd4b29c2aaf95ecd29c0c3b10829ad94dcfe59437dbee391154aea" } } }, "nbformat": 4, "nbformat_minor": 0 }