{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"source": [
"[](https://colab.research.google.com/drive/1XrYHDFVni94NPPXBWdKgv85tBWeDZqJF?usp=sharing)"
],
"metadata": {
"id": "lpHbAWDeKaZK"
}
},
{
"cell_type": "markdown",
"source": [
"\n",
"\n",
"# Monitoring CrewAI agents with Portkey\n",
"\n",
"[**CrewAI**](https://github.com/joaomdmoura/crewAI) is an AI agent framework that lets you create and orchestrate role-playing, autonomous AI agents. You can can create agents, assign them goals and have them create and complete tasks to achieve these goals.\n",
"\n",
"One of the key challenges of building AI agents is the lack of visibility into key performance metrics such as:\n",
"\n",
"- **Cost Analysis**: How much does it cost to run CrewAI agents for your project.\n",
"- **Token Usage**: How many tokens are being consumed for each agent run.\n",
"- **Latency**: How long is the agent taking to complete a set of tasks.\n",
"\n",
"
\n",
"\n",
"**Portkey** is an open source [**AI Gateway**](https://github.com/Portkey-AI/gateway) that helps you manage access to 250+ LLMs through a unified API while providing visibility into\n",
"\n",
"✅ cost \n",
"✅ performance \n",
"✅ accuracy metrics\n",
"\n",
"This notebook demonstrates how you can bring visibility and flexbility to CrewAI agents using Portkey's AI Gateway."
],
"metadata": {
"id": "XwdzPG6zsi7a"
}
},
{
"cell_type": "markdown",
"source": [
"## Setup\n",
"First, let's install both CrewAI and Portkey AI SDKs in our notebook."
],
"metadata": {
"id": "B3Sr_1XA5UUr"
}
},
{
"cell_type": "code",
"source": [
"!pip install -qU crewai portkey-ai"
],
"metadata": {
"id": "W7deHW8-jpWj",
"collapsed": true
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"To monitor your agents, connect the LLMs to the Portkey AI Gateway through the `ChatOpenAI` interface.\n",
"\n",
"- Set the `base_url` as `PORTKEY_GATEWAY_URL`\n",
"- Add `default_headers` needed by Portkey using the `createHeaders` helper method.\n",
"\n",
"To start, get your Portkey API key by signing up [here](https://app.portkey.ai/). (Click the profile icon on the bottom left, then click on \"Copy API Key\")"
],
"metadata": {
"id": "4kjzWLX_i_0f"
}
},
{
"cell_type": "code",
"source": [
"from crewai import LLM\n",
"from portkey_ai import createHeaders, PORTKEY_GATEWAY_URL\n",
"\n",
"\n",
"gpt_llm = LLM(\n",
" model=\"gpt-4o\",\n",
" base_url=PORTKEY_GATEWAY_URL,\n",
" api_key=\"dummy\", # We are using Virtual key\n",
" extra_headers=createHeaders(\n",
" api_key=\"YOUR_PORTKEY_API_KEY\",\n",
" virtual_key=\"YOUR_VIRTUAL_KEY\", # Enter your OpenAI Virtual key from Portkey\n",
" )\n",
")\n",
"\n",
"\n",
"anthropic_llm = LLM(\n",
" model=\"claude-3-5-sonnet-latest\",\n",
" base_url=PORTKEY_GATEWAY_URL,\n",
" api_key=\"dummy\", # We are using Virtual key\n",
" extra_headers=createHeaders(\n",
" api_key=\"YOUR_PORTKEY_API_KEY\",\n",
" virtual_key=\"YOUR_VIRTUAL_KEY\", # Enter your Anthroipc Virtual key from Portkey\n",
" config=\"YOUR_PORTKEY_CONFIG_ID\", # All your model parmaeters and routing strategy\n",
" trace_id=\"llm2\"\n",
" )\n",
")\n",
"\n"
],
"metadata": {
"id": "AK1lavI4siki"
},
"execution_count": 4,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
""
],
"metadata": {
"id": "c0s-PloTsi4y"
}
},
{
"cell_type": "markdown",
"source": [
"# Simple CrewAI Agent"
],
"metadata": {
"id": "wFbFoBRu7_yA"
}
},
{
"cell_type": "code",
"source": [
"from crewai import Agent, Task, Crew, Process\n",
"\n",
"# Define your agents with roles and goals\n",
"\n",
"coder = Agent(\n",
" role='Software develoepr',\n",
" goal='Write clear - concise code on demand',\n",
" backstory='An expert coder with a keen eye for software trends.',\n",
" llm = gpt_llm\n",
")\n",
"\n",
"\n",
"# Create tasks for your agents\n",
"task1 = Task(\n",
" description=\"Define the HTML for making a simple website with heading- Hello World! Portkey is working! .\",\n",
" expected_output=\"A clear and concise HTML code\",\n",
" agent=coder\n",
")\n",
"\n",
"# Instantiate your crew with a sequential process\n",
"crew = Crew(\n",
" agents=[coder],\n",
" tasks=[task1],\n",
")\n",
"\n",
"# Get your crew to work!\n",
"result = crew.kickoff()\n",
"print(\"######################\")\n",
"print(result)\n",
"\n",
"\n"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "PbEjaDv3RnhN",
"outputId": "8d63f55d-73c0-4ad4-d87a-216da5c516e4"
},
"execution_count": 5,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"WARNING:opentelemetry.trace:Overriding of current TracerProvider is not allowed\n"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"######################\n",
"```html\n",
"\n",
"\n",
"