{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "introduction" }, "source": [ "# Working with Tigris in Pixeltable\n", "\n", "This tutorial demonstrates how to configure Pixeltable to use [Tigris](https://tigrisdata.com) for storage. This lets you store unlimited amounts of images in Tigris' global data plane, allowing your images to load fast everywhere.\n", "\n", "## Prerequisites\n", "\n", "- A Tigris account, bucket, and access keypair ([https://storage.new](https://storage.new))\n", "\n", "## Important notes\n", "\n", "- Tigris usage may incur costs based on your plan.\n", "- Be mindful of sensitive data and consider security measures when integrating with external services.\n", "\n", "First you need to install required libraries and enter a Tigris access keypair obtained via the Tigris Admin Console." ] }, { "cell_type": "markdown", "metadata": { "id": "mPfFuWBgZJ5q" }, "source": [ "## Set up environment\n", "\n", "First, let's install Pixeltable:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "PrJK4OcmZJ5q", "outputId": "19dd3989-7837-403b-d8ec-d24bfcdb39ad" }, "outputs": [], "source": [ "%pip install -qU pixeltable datasets" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Configure authentication\n", "\n", "These steps will have you enter in your Tigris credentials:" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [], "source": [ "import os\n", "from getpass import getpass\n", "\n", "os.environ['AWS_ACCESS_KEY_ID'] = getpass('Tigris access key ID')\n", "os.environ['AWS_SECRET_ACCESS_KEY'] = getpass('Tigris secret access key')\n", "bucket_name = getpass('Tigris bucket name')\n", "\n", "os.environ['AWS_ENDPOINT_URL_S3'] = 'https://t3.storage.dev'\n", "os.environ['AWS_REGION'] = 'auto'\n", "os.environ['PIXELTABLE_INPUT_MEDIA_DEST'] = f's3://{bucket_name}/input/'\n", "os.environ['PIXELTABLE_OUTPUT_MEDIA_DEST'] = f's3://{bucket_name}/output/'" ] }, { "cell_type": "markdown", "metadata": { "id": "-hdc6e78ZJ5r" }, "source": [ "## Create a table for images\n", "\n", "Now let's create a table that will contain images from the [XeIaso/botw-screenshots-captioned](https://huggingface.co/datasets/XeIaso/botw-screenshots-captioned) dataset:" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Created directory 'tigris'.\n", "Created table 'screenshots'.\n", "Inserting rows into `screenshots`: 100 rows [00:01, 51.72 rows/s]\n", "Inserting rows into `screenshots`: 100 rows [00:01, 55.57 rows/s]\n", "Inserting rows into `screenshots`: 100 rows [00:01, 52.74 rows/s]\n", "Inserting rows into `screenshots`: 100 rows [00:02, 33.96 rows/s]\n", "Inserting rows into `screenshots`: 100 rows [00:02, 42.64 rows/s]\n", "Inserting rows into `screenshots`: 100 rows [00:02, 39.65 rows/s]\n", "Inserting rows into `screenshots`: 100 rows [00:02, 47.36 rows/s]\n", "Inserting rows into `screenshots`: 28 rows [00:00, 6786.12 rows/s]\n", "Inserted 728 rows with 0 errors.\n" ] } ], "source": [ "import pixeltable as pxt\n", "from datasets import load_dataset\n", "\n", "# Create directory for this demo\n", "pxt.drop_dir('tigris', force=True)\n", "pxt.create_dir('tigris', if_exists='replace')\n", "\n", "\n", "# Load the dataset\n", "ds = load_dataset('XeIaso/botw-screenshots-captioned')\n", "# Import it to pixeltable with the name screenshots\n", "pxt.drop_table('tigris/screenshots', force=True)\n", "screenshots = pxt.create_table(\n", " 'tigris/screenshots', source=ds, if_exists='replace'\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once the import is done, you can create thumbnails with a [computed column](https://docs.pixeltable.com/tutorials/computed-columns):" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Added 728 column values with 0 errors.\n" ] }, { "data": { "text/plain": [ "728 rows updated, 728 values computed." ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Add a computed column for thumbnails\n", "# Uses output_media_dest by default, or specify a custom destination\n", "screenshots.add_computed_column(\n", " thumbnail=screenshots.image.resize((256, 256)),\n", " destination=f's3://{bucket_name}/botw-screenshots/thumbnails/',\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And then inspect that with the `collect` method:" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
imagelabeltextthumbnail
\n", " \n", "
F1C11A22FAEE3B82F21B330E1B786A39in_BOTW masterpiece, best quality, 8k, cinematic screenshot from The Legend of Zelda: Breath of the Wild, close-up of the Roc banner, detailed intricate design, golden accents, dynamic lighting, cinematic, fantastical, fantasy.
\n", " \n", "
" ], "text/plain": [ " image \\\n", "0 \n", " \n", " \n", " image\n", " image_url\n", " thumbnail\n", " thumbnail_url\n", " \n", " \n", " \n", " \n", "
\n", " \n", "
\n", " s3://pxt-test/input/8e2c31257bff46d1be81fadb991e271d/29/2964/8e2c31257bff46d1be81fadb991e271d_0_1_29648cb12dd44da5ba0c6e7b4c58a5d0\n", "
\n", " \n", "
\n", " s3://pxt-test/botw-screenshots/thumbnails/8e2c31257bff46d1be81fadb991e271d/43/43a7/8e2c31257bff46d1be81fadb991e271d_9_9_43a7e608a36b47d5aa4f0073bc0b0754.jpeg\n", " \n", " \n", "" ], "text/plain": [ " image \\\n", "0 \n", " \n", " \n", " image\n", " image_url\n", " image_presigned\n", " thumbnail\n", " thumbnail_url\n", " thumbnail_presigned\n", " \n", " \n", " \n", " \n", "
\n", " \n", "
\n", " s3://pxt-test/input/8e2c31257bff46d1be81fadb991e271d/29/2964/8e2c31257bff46d1be81fadb991e271d_0_1_29648cb12dd44da5ba0c6e7b4c58a5d0\n", " https://t3.storage.dev/pxt-test/input/8e2c31257bff46d1be81fadb991e271d/29/2964/8e2c31257bff46d1be81fadb991e271d_0_1_29648cb12dd44da5ba0c6e7b4c58a5d0?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=tid_iVdmvOytflyeziMuWhEbCCYZoFQYHSSFlRMwsvZlT_LgNjNflW%2F20251217%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20251217T192201Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=8990e18d85d5f82831089c458a7f20fa2a1d7dd29e3e3efdaac52ae8d19f255b\n", "
\n", " \n", "
\n", " s3://pxt-test/botw-screenshots/thumbnails/8e2c31257bff46d1be81fadb991e271d/43/43a7/8e2c31257bff46d1be81fadb991e271d_9_9_43a7e608a36b47d5aa4f0073bc0b0754.jpeg\n", " https://t3.storage.dev/pxt-test/botw-screenshots/thumbnails/8e2c31257bff46d1be81fadb991e271d/43/43a7/8e2c31257bff46d1be81fadb991e271d_9_9_43a7e608a36b47d5aa4f0073bc0b0754.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=tid_iVdmvOytflyeziMuWhEbCCYZoFQYHSSFlRMwsvZlT_LgNjNflW%2F20251217%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20251217T192201Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=7c86a3a5389d94f1980e914c3c94d328eadd63c57d900c9b394b9f47f275410d\n", " \n", " \n", "" ], "text/plain": [ " image \\\n", "0