{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "983yjns496tx" }, "source": [ "[![Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/pixeltable/pixeltable/blob/release/docs/notebooks/integrations/working-with-replicate.ipynb)  \n", "[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/notebooks/integrations/working-with-replicate.ipynb)  \n", "\"Download\n", "\n", "# Working with Replicate in Pixeltable\n", "\n", "Pixeltable's Replicate integration enables you to access Replicate's models via the Replicate API.\n", "\n", "### Prerequisites\n", "\n", "- A Replicate account with an API token.\n", "\n", "### Important Notes\n", "\n", "- Replicate usage may incur costs based on your Replicate plan.\n", "- Be mindful of sensitive data and consider security measures when integrating with external services.\n", "\n", "First you'll need to install required libraries and enter a Replicate API token." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "9pckrD01ik-e", "outputId": "060b8b32-48a6-48a0-e720-4eacf94d83ef" }, "outputs": [], "source": [ "%pip install -qU pixeltable replicate" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "AQ6_Py7_7d0r", "outputId": "f82cfe36-be9e-4d43-f13e-9f6f5b680e8e" }, "outputs": [], "source": [ "import os\n", "import getpass\n", "\n", "if 'REPLICATE_API_TOKEN' not in os.environ:\n", " os.environ['REPLICATE_API_TOKEN'] = getpass.getpass('Replicate API Token:')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now let's create a Pixeltable directory to hold the tables for our demo." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "brtjK-88tTSS", "outputId": "55d08c91-438a-4c3e-c217-3cea72faca11" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Connected to Pixeltable database at: postgresql+psycopg://postgres:@/pixeltable?host=/Users/asiegel/.pixeltable/pgdata\n", "Created directory `replicate_demo`.\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pixeltable as pxt\n", "\n", "# Remove the `replicate_demo` directory and its contents, if it exists\n", "pxt.drop_dir('replicate_demo', force=True)\n", "pxt.create_dir('replicate_demo')" ] }, { "cell_type": "markdown", "metadata": { "id": "0kmjJoDq9Oqe" }, "source": [ "## Chat Completions\n", "\n", "Create a Table: In Pixeltable, create a table with columns to represent your input data and the columns where you want to store the results from Replicate." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "5ti10tXu5m3X", "outputId": "30848066-1e9b-4efd-aad7-b2271a031ec3" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Created table `chat`.\n", "Added 0 column values with 0 errors.\n" ] } ], "source": [ "from pixeltable.functions.replicate import run\n", "\n", "# Create a table in Pixeltable and pick a model hosted on Replicate with some parameters\n", "\n", "t = pxt.create_table('replicate_demo.chat', {'prompt': pxt.String})\n", "\n", "input = {\n", " 'system_prompt': 'You are a helpful assistant.',\n", " 'prompt': t.prompt,\n", " # These parameters are optional and can be used to tune model behavior:\n", " 'max_tokens': 300,\n", " 'top_p': 0.9,\n", " 'temperature': 0.8\n", "}\n", "t.add_computed_column(output=run(input, ref='meta/meta-llama-3-8b-instruct'))" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "mePjoku95iUn", "outputId": "07c3f8f0-5301-44ba-ba33-25d10c3c26fa" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Added 0 column values with 0 errors.\n" ] } ], "source": [ "# Parse the response into a new column\n", "t.add_computed_column(response=pxt.functions.string.join('', t.output))" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 599 }, "id": "IkMM7OYb5rQ_", "outputId": "8e94af3e-485c-49f2-d7ba-b5490ec83af9" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Computing cells: 100%|████████████████████████████████████████████| 4/4 [00:07<00:00, 1.89s/ cells]\n", "Inserting rows into `chat`: 1 rows [00:00, 171.89 rows/s]\n", "Computing cells: 100%|████████████████████████████████████████████| 4/4 [00:07<00:00, 1.89s/ cells]\n", "Inserted 1 row with 0 errors.\n" ] }, { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
promptresponse
What foods are rich in selenium?Selenium is an essential mineral that plays a crucial role in various bodily functions, such as protecting cells from damage, supporting immune function, and acting as an antioxidant. Here are some foods that are rich in selenium:\n", "\n", "1. Brazil nuts: Brazil nuts are one of the richest sources of selenium, with a single nut providing about 60-90 micrograms (mcg) of selenium. This is equivalent to the daily recommended intake of selenium.\n", "2. Fish: Fatty fish like tuna, mackerel, and sardines ar ...... e of selenium, with a 3-ounce serving providing about 30 mcg of selenium.\n", "5. Chicken: Chicken breast is also a good source of selenium, with a 3-ounce serving providing about 25 mcg of selenium.\n", "6. Eggs: Eggs are a good source of selenium, with a large egg providing about 20 mcg of selenium.\n", "7. Wheat germ: Wheat germ is a good source of selenium, with a 1/4 cup serving providing about 20 mcg of selenium.\n", "8. Brown rice: Brown rice is a good source of selenium, with a 1/2 cup serving providing
" ], "text/plain": [ " prompt \\\n", "0 What foods are rich in selenium? \n", "\n", " response \n", "0 \\n\\nSelenium is an essential mineral that play... " ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Start a conversation\n", "t.insert(prompt='What foods are rich in selenium?')\n", "t.select(t.prompt, t.response).show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Image Generation\n", "\n", "Here's an example that shows how to use Replicate's image generation models with Pixeltable. We'll use the FLUX Schnell model." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Created table `images`.\n", "Added 0 column values with 0 errors.\n" ] } ], "source": [ "t = pxt.create_table('replicate_demo.images', {'prompt': pxt.String})\n", "\n", "input = {\n", " 'prompt': t.prompt,\n", " 'go_fast': True,\n", " 'megapixels': '1'\n", "}\n", "t.add_computed_column(output=run(input, ref='black-forest-labs/flux-schnell'))" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Computing cells: 100%|████████████████████████████████████████████| 2/2 [00:00<00:00, 2.17 cells/s]\n", "Inserting rows into `images`: 1 rows [00:00, 198.61 rows/s]\n", "Computing cells: 100%|████████████████████████████████████████████| 2/2 [00:00<00:00, 2.14 cells/s]\n", "Inserted 1 row with 0 errors.\n" ] }, { "data": { "text/plain": [ "UpdateStatus(num_rows=1, num_computed_values=2, num_excs=0, updated_cols=[], cols_with_excs=[])" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "t.insert(prompt='Draw a pencil sketch of a friendly dinosaur playing tennis in a cornfield.')" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
promptoutput
Draw a pencil sketch of a friendly dinosaur playing tennis in a cornfield.["https://replicate.delivery/yhqm/DH2e9QRelTr0KEhcfciu9GHPINCt1rJ9kAShds91nCG7grYnA/out-0.webp"]
" ], "text/plain": [ " prompt \\\n", "0 Draw a pencil sketch of a friendly dinosaur pl... \n", "\n", " output \n", "0 [https://replicate.delivery/yhqm/DH2e9QRelTr0K... " ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "t.select(t.prompt, t.output).collect()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see that Replicate returns our image as an array containing a single URL. To turn it into an actual image, we cast the string to type `pxt.Image` in a new computed column:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Computing cells: 100%|████████████████████████████████████████████| 1/1 [00:00<00:00, 93.29 cells/s]\n", "Added 1 column value with 0 errors.\n" ] }, { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
image
\n", " \n", "
" ], "text/plain": [ " image\n", "0