{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "983yjns496tx" }, "source": [ "# Working with Ollama in Pixeltable\n", "\n", "Ollama is a popular platform for local serving of LLMs. In this tutorial, we'll show how to integrate Ollama models into a Pixeltable workflow.\n", "\n", "## Install Ollama\n", "\n", "You'll need to have an Ollama server instance to query. There are several ways to do this.\n", "\n", "### Running on a local machine\n", "\n", "If you're running this notebook on your own machine, running Windows, Mac OS, or Linux, you can install Ollama at: https://ollama.com/download\n", "\n", "### Running on Google Colab\n", "\n", "- OR, if you're running on Colab, you can install Ollama by uncommenting and running the following code." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# To install Ollama on colab, uncomment and run the following\n", "# three lines (this will also work on a local Linux machine\n", "# if you don't already have Ollama installed).\n", "\n", "# !curl -fsSL https://ollama.com/install.sh | sh\n", "# import subprocess\n", "# ollama_process = subprocess.Popen(['ollama', 'serve'], stderr=subprocess.PIPE)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Running on a remote Ollama server\n", "\n", "- OR, if you have access to an Ollama server running remotely, you can uncomment and run the following line, replacing the default URL with the URL of your remote Ollama instance." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# To run the notebook against an instance of Ollama running on a\n", "# remote server, uncomment the following line and specify the URL.\n", "\n", "# os.environs['OLLAMA_HOST'] = 'https://127.0.0.1:11434'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once you've completed the installation, run the following commands to verify that it's been successfully installed. This may result in an LLM being downloaded, so it may take some time." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%pip install -qU ollama" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'The capital of Missouri is Jefferson City. Jefferson City was originally named after the French explorer Pierre-Jacques Houget and the American statesman Thomas Jefferson, who lived in this city from 1764 to 1805. It became the seat of government for most of Jefferson County when it was established in 1836. In more recent times, the name has changed several times due to various political changes and legal changes.'" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import ollama\n", "\n", "ollama.pull('qwen2.5:0.5b')\n", "ollama.generate('qwen2.5:0.5b', 'What is the capital of Missouri?')[\n", " 'response'\n", "]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Install Pixeltable" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let's install Pixeltable and create a table for the demo." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "9pckrD01ik-e", "outputId": "060b8b32-48a6-48a0-e720-4eacf94d83ef" }, "outputs": [], "source": [ "%pip install -qU pixeltable" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "5ti10tXu5m3X", "outputId": "30848066-1e9b-4efd-aad7-b2271a031ec3" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Connected to Pixeltable database at: postgresql+psycopg://postgres:@/pixeltable?host=/Users/asiegel/.pixeltable/pgdata\n", "Created directory 'ollama_demo'.\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pixeltable as pxt\n", "from pixeltable.functions.ollama import chat\n", "\n", "pxt.drop_dir('ollama_demo', force=True)\n", "pxt.create_dir('ollama_demo')" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Created table 'chat'.\n", "Added 0 column values with 0 errors in 0.01 s\n" ] }, { "data": { "text/plain": [ "No rows affected." ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "t = pxt.create_table('ollama_demo/chat', {'input': pxt.String})\n", "\n", "messages = [{'role': 'user', 'content': t.input}]\n", "\n", "# Add a computed column that runs the model to generate responses\n", "t.add_computed_column(\n", " output=chat(\n", " messages=messages,\n", " model='qwen2.5:0.5b',\n", " # These parameters are optional and can be used to tune model behavior:\n", " options={'max_tokens': 300, 'top_p': 0.9, 'temperature': 0.5},\n", " )\n", ")" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Added 0 column values with 0 errors in 0.01 s\n" ] }, { "data": { "text/plain": [ "No rows affected." ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Extract the message content into a separate column\n", "t.add_computed_column(response=t.output.message.content)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can insert our input prompts into the table now. As always, Pixeltable automatically updates the computed columns by calling the relevant Ollama endpoint." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 599 }, "id": "IkMM7OYb5rQ_", "outputId": "8e94af3e-485c-49f2-d7ba-b5490ec83af9" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Inserted 1 row with 0 errors in 1.28 s (0.78 rows/s)\n" ] }, { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
inputresponse
What are the most popular services for LLM inference?LLM inference is a type of artificial intelligence that can generate human-like text based on specific input data. In order to find the most popular services for LLM inference, we need to consider several factors such as the availability of resources, the quality of the models, and the popularity among users.\n", "\n", "One common service for LLM inference is Hugging Face's transformers library, which provides a wide range of pre-trained language models including BERT, RoBERTa, and GPT-2. This library ...... Qwen model, Hugging Face's transformers library, and many more. The Alibaba Cloud LLM platform provides a wide range of pre-trained models for various tasks, including language generation, text classification, and more.\n", "\n", "Overall, both Hugging Face's transformers library and the Alibaba Cloud LLM platform offer popular services for LLM inference. However, it is essential to consider the specific use case and requirements when choosing a service, as each has its own strengths and limitations.
" ], "text/plain": [ " input \\\n", "0 What are the most popular services for LLM inf... \n", "\n", " response \n", "0 LLM inference is a type of artificial intellig... " ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Start a conversation\n", "t.insert(input='What are the most popular services for LLM inference?')\n", "t.select(t.input, t.response).show()" ] }, { "cell_type": "markdown", "metadata": { "id": "lTtQcjKQAlis" }, "source": [ "### Learn More\n", "\n", "To learn more about advanced techniques like RAG operations in Pixeltable, check out the [RAG Operations in Pixeltable](https://docs.pixeltable.com/howto/use-cases/rag-operations) tutorial.\n", "\n", "If you have any questions, don't hesitate to reach out." ] } ], "metadata": { "colab": { "provenance": [] }, "kernelspec": { "display_name": "pxt", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.19" } }, "nbformat": 4, "nbformat_minor": 4 }