{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ollama + OpenAI + Python\n",
    "\n",
    "## 1. Specify the model name\n",
    "\n",
    "If you pulled in a different model than \"phi3:mini\", change the value in the cell below.\n",
    "That variable will be used in code throughout the notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "MODEL_NAME = \"phi3:mini\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Setup the Open AI client\n",
    "\n",
    "Typically the OpenAI client is used with OpenAI.com or Azure OpenAI to interact with large language models.\n",
    "However, it can also be used with Ollama, since Ollama provides an OpenAI-compatible endpoint at \"http://localhost:11434/v1\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install openai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import openai\n",
    "\n",
    "client = openai.OpenAI(\n",
    "    base_url=\"http://localhost:11434/v1\",\n",
    "    api_key=\"nokeyneeded\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Generate a chat completion\n",
    "\n",
    "Now we can use the OpenAI SDK to generate a response for a conversation. This request should generate a haiku about cats:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = client.chat.completions.create(\n",
    "    model=MODEL_NAME,\n",
    "    temperature=0.7,\n",
    "    n=1,\n",
    "    messages=[\n",
    "        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
    "        {\"role\": \"user\", \"content\": \"Write a haiku about a hungry cat\"},\n",
    "    ],\n",
    ")\n",
    "\n",
    "print(\"Response:\")\n",
    "print(response.choices[0].message.content)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Prompt engineering\n",
    "\n",
    "The first message sent to the language model is called the \"system message\" or \"system prompt\", and it sets the overall instructions for the model.\n",
    "You can provide your own custom system prompt to guide a language model to generate output in a different way.\n",
    "Modify the `SYSTEM_MESSAGE` below to answer like your favorite famous movie/TV character, or get inspiration for other system prompts from [Awesome ChatGPT Prompts](https://github.com/f/awesome-chatgpt-prompts?tab=readme-ov-file#prompts).\n",
    "\n",
    "Once you've customized the system message, provide the first user question in the `USER_MESSAGE`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SYSTEM_MESSAGE = \"\"\"\n",
    "I want you to act like Elmo from Sesame Street.\n",
    "I want you to respond and answer like Elmo using the tone, manner and vocabulary that Elmo would use.\n",
    "Do not write any explanations. Only answer like Elmo.\n",
    "You must know all of the knowledge of Elmo, and nothing more.\n",
    "\"\"\"\n",
    "\n",
    "USER_MESSAGE = \"\"\"\n",
    "Hi Elmo, how are you doing today?\n",
    "\"\"\"\n",
    "\n",
    "response = client.chat.completions.create(\n",
    "    model=MODEL_NAME,\n",
    "    temperature=0.7,\n",
    "    n=1,\n",
    "    messages=[\n",
    "        {\"role\": \"system\", \"content\": SYSTEM_MESSAGE},\n",
    "        {\"role\": \"user\", \"content\": USER_MESSAGE},\n",
    "    ],\n",
    ")\n",
    "\n",
    "print(\"Response:\")\n",
    "print(response.choices[0].message.content)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Few shot examples\n",
    "\n",
    "Another way to guide a language model is to provide \"few shots\", a sequence of example question/answers that demonstrate how it should respond.\n",
    "\n",
    "The example below tries to get a language model to act like a teaching assistant by providing a few examples of questions and answers that a TA might give, and then prompts the model with a question that a student might ask.\n",
    "\n",
    "Try it first, and then modify the `SYSTEM_MESSAGE`, `EXAMPLES`, and `USER_MESSAGE` for a new scenario."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SYSTEM_MESSAGE = \"\"\"\n",
    "You are a helpful assistant that helps students with their homework.\n",
    "Instead of providing the full answer, you respond with a hint or a clue.\n",
    "\"\"\"\n",
    "\n",
    "EXAMPLES = [\n",
    "    (\n",
    "        \"What is the capital of France?\",\n",
    "        \"Can you remember the name of the city that is known for the Eiffel Tower?\"\n",
    "    ),\n",
    "    (\n",
    "        \"What is the square root of 144?\",\n",
    "        \"What number multiplied by itself equals 144?\"\n",
    "    ),\n",
    "    (   \"What is the atomic number of oxygen?\",\n",
    "        \"How many protons does an oxygen atom have?\"\n",
    "    ),\n",
    "]\n",
    "\n",
    "USER_MESSAGE = \"What is the largest planet in our solar system?\"\n",
    "\n",
    "\n",
    "response = client.chat.completions.create(\n",
    "    model=MODEL_NAME,\n",
    "    temperature=0.7,\n",
    "    n=1,\n",
    "    messages=[\n",
    "        {\"role\": \"system\", \"content\": SYSTEM_MESSAGE},\n",
    "        {\"role\": \"user\", \"content\": EXAMPLES[0][0]},\n",
    "        {\"role\": \"assistant\", \"content\": EXAMPLES[0][1]},\n",
    "        {\"role\": \"user\", \"content\": EXAMPLES[1][0]},\n",
    "        {\"role\": \"assistant\", \"content\": EXAMPLES[1][1]},\n",
    "        {\"role\": \"user\", \"content\": EXAMPLES[2][0]},\n",
    "        {\"role\": \"assistant\", \"content\": EXAMPLES[2][1]},\n",
    "        {\"role\": \"user\", \"content\": USER_MESSAGE},\n",
    "    ],\n",
    ")\n",
    "\n",
    "\n",
    "print(\"Response:\")\n",
    "print(response.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Retrieval Augmented Generation\n",
    "\n",
    "RAG (Retrieval Augmented Generation) is a technique to get a language model to answer questions accurately for a particular domain, by first retrieving relevant information from a knowledge source and then generating a response based on that information.\n",
    "\n",
    "We have provided a local CSV file with data about hybrid cars. The code below reads the CSV file, searches for matches to the user question, and then generates a response based on the information found. Note that this will take longer than any of the previous examples, as it sends more data to the model. If you notice the answer is still not grounded in the data, you can try system engineering or try other models. Generally, RAG is more effective with either larger models or with fine-tuned versions of SLMs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import csv\n",
    "\n",
    "SYSTEM_MESSAGE = \"\"\"\n",
    "You are a helpful assistant that answers questions about cars based off a hybrid car data set.\n",
    "You must use the data set to answer the questions, you should not provide any information that is not in the provided sources.\n",
    "\"\"\"\n",
    "\n",
    "USER_MESSAGE = \"how fast is a prius?\"\n",
    "\n",
    "# Open the CSV and store in a list\n",
    "with open(\"hybrid.csv\", \"r\") as file:\n",
    "    reader = csv.reader(file)\n",
    "    rows = list(reader)\n",
    "\n",
    "# Normalize the user question to replace punctuation and make lowercase\n",
    "normalized_message = USER_MESSAGE.lower().replace(\"?\", \"\").replace(\"(\", \" \").replace(\")\", \" \")\n",
    "\n",
    "# Search the CSV for user question using very naive search\n",
    "words = normalized_message.split()\n",
    "matches = []\n",
    "for row in rows[1:]:\n",
    "    # if the word matches any word in row, add the row to the matches\n",
    "    if any(word in row[0].lower().split() for word in words) or any(word in row[5].lower().split() for word in words):\n",
    "        matches.append(row)\n",
    "\n",
    "# Format as a markdown table, since language models understand markdown\n",
    "matches_table = \" | \".join(rows[0]) + \"\\n\" + \" | \".join(\" --- \" for _ in range(len(rows[0]))) + \"\\n\"\n",
    "matches_table += \"\\n\".join(\" | \".join(row) for row in matches)\n",
    "print(f\"Found {len(matches)} matches:\")\n",
    "print(matches_table)\n",
    "\n",
    "# Now we can use the matches to generate a response\n",
    "response = client.chat.completions.create(\n",
    "    model=MODEL_NAME,\n",
    "    temperature=0.7,\n",
    "    n=1,\n",
    "    messages=[\n",
    "        {\"role\": \"system\", \"content\": SYSTEM_MESSAGE},\n",
    "        {\"role\": \"user\", \"content\": USER_MESSAGE + \"\\nSources: \" + matches_table},\n",
    "    ],\n",
    ")\n",
    "\n",
    "print(\"Response:\")\n",
    "print(response.choices[0].message.content)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}