{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {},
      "outputs": [],
      "source": [
        "import jupyter_black # type: ignore\n",
        "\n",
        "from baml_client.sync_client import b\n",
        "from baml_py import Collector # type: ignore\n",
        "from dotenv import load_dotenv # type: ignore\n",
        "from notebooks._utils import print_used_model\n",
        "\n",
        "jupyter_black.load()\n",
        "load_dotenv()\n",
        "\n",
        "c = Collector()\n",
        "b = b.with_options(collector=c)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# How to use LLM Clients effectively\n",
        "\n",
        "### 1. How to conveniently route any of your LLM calls to any LLM provider at runtime\n",
        "\n",
        "Let's say you have explicitly stated the provider and model you use with a function:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "class LLMClientConfig_ExtractedCity {\n",
            "    city string @description(#\"Unabbreviated city name without the state\"#)\n",
            "}\n",
            "function LLMClientConfig_ExtractCity(text: string) -> LLMClientConfig_ExtractedCity {\n",
            "  client \"openai/gpt-4.1-nano\"\n",
            "  prompt #\"\n",
            "    Extract: \n",
            "    \"{{ text }}\". \n",
            "    {{ ctx.output_format }}\n",
            "  \"#\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "# This command shows file contents\n",
        "!awk '/^class LLMClientConfig_ExtractedCity/,/^}/' ../baml_src/notebooks/01_baml_llm_client_config.baml\n",
        "!awk '/^function LLMClientConfig_ExtractCity/,/^}/' ../baml_src/notebooks/01_baml_llm_client_config.baml"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "This is what the prompt will look like:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[system]\n",
            "Extract: \n",
            "\"I'm John from Philly\". \n",
            "Answer in JSON using this schema:\n",
            "{\n",
            "  // Unabbreviated city name without the state\n",
            "  city: string,\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "from baml_agents import display_prompt\n",
        "\n",
        "request = b.request.LLMClientConfig_ExtractCity(\"I'm John from Philly\")\n",
        "display_prompt(request)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "This is changing the client used to call the LLM::"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Used model: gpt-4.1-nano-2025-04-14\n",
            "Philadelphia\n"
          ]
        }
      ],
      "source": [
        "answer = b.LLMClientConfig_ExtractCity(\"I'm John from Philly\")\n",
        "print_used_model(c.last)\n",
        "print(answer.city)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "You can easily replace it:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Used model: gpt-4.1-mini-2025-04-14\n",
            "Philadelphia\n"
          ]
        }
      ],
      "source": [
        "from baml_agents import with_model\n",
        "\n",
        "b = with_model(b, \"gpt-4.1-mini\")\n",
        "\n",
        "city = b.LLMClientConfig_ExtractCity(\"I'm John from Philly\")\n",
        "print_used_model(c.last)\n",
        "print(answer.city)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "You can also replace with a predefined client. Let's say you have:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {},
      "outputs": [],
      "source": [
        "# This command shows file contents\n",
        "!awk '/^client<llm> GPT41/,/^}/' ../baml_src/notebooks/01_baml_llm_client_config.baml"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Used model: gpt-4.1-2025-04-14\n",
            "Philadelphia\n"
          ]
        }
      ],
      "source": [
        "from baml_agents import with_baml_client\n",
        "\n",
        "b = with_baml_client(\n",
        "    b,\n",
        "    provider=\"openai\",\n",
        "    options={\"model\": \"gpt-4.1\"},\n",
        ")\n",
        "\n",
        "city = b.LLMClientConfig_ExtractCity(\"I'm John from Philly\")\n",
        "print_used_model(c.last)\n",
        "print(answer.city)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 2. What to do if `baml` doesn't support an LLM tracing integration you need (e.g. Langfuse, LangSmith, etc.)\n",
        "\n",
        "#### Option 1:\n",
        "- Download LLM Proxy such as [OpenRouter](https://openrouter.ai/), [LiteLLM](https://www.litellm.ai/), etc.\n",
        "- Configure it to use the tracing integration you need\n",
        "- Start it, it will become available with some URL, such as `http://localhost:8080`\n",
        "- Set the environment variable `OPENAI_API_BASE` to that URL\n",
        "\n",
        "You'll then be able to switch with just a single line of code change:\n",
        "\n",
        "#### Option 2 (Advanced):\n",
        "- Use [baml Modular API](https://docs.boundaryml.com/guide/baml-advanced/modular-api) to directly get the request and response objects. You would then proceed to do integration yourself.\n",
        "\n",
        "### 3. What to do if `baml` doesn't support the LLM provider you need (e.g. IBM WatsonX AI, etc.)\n",
        "\n",
        "#### Option 1:\n",
        "- Download LLM Proxy such as [OpenRouter](https://openrouter.ai/), [LiteLLM](https://www.litellm.ai/), etc.\n",
        "- Configure it to use the models you need\n",
        "- Start it, it will become available with some URL, such as `http://localhost:8080`\n",
        "- Set the environment variable `OPENAI_API_BASE` to that URL\n",
        "\n",
        "You'll then be able to switch with just a single line of code change:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {},
      "outputs": [],
      "source": [
        "b = with_model(b, \"gpt-4\")\n",
        "b = with_model(b, \"claude-3-opus-20240229\")\n",
        "b = with_model(b, \"claude-3-opus-20240229-with-thinking-enabled\")\n",
        "b = with_model(b, \"claude-3-opus-20240229-another-api-key\")\n",
        "b = with_model(b, \"TinyLlama/TinyLlama-1.1B-Chat-v1.0\")\n",
        "b = with_model(b, \"NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO\")\n",
        "b = with_model(b, \"WizardLM/WizardCoder-Python-13B-V1.0\")\n",
        "b = with_model(b, \"my-localhost-model\")\n",
        "# etc."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n",
        "#### Option 2 (Advanced):\n",
        "- Use [baml Modular API](https://docs.boundaryml.com/guide/baml-advanced/modular-api) to directly get the request and response objects. You would then proceed to do integration yourself."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 3. What to do I can't get `baml` to find the environment variables (e.g. OPENAI_API_KEY, etc.) or I don't want to use them\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Use the [Client Registry](https://docs.boundaryml.com/ref/baml_client/client-registry) to register your client with the API key and other parameters you need. You can also take a look at `with_model()` implementation details to use as an example."
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": ".venv",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.13.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}