RubyLLM The Ruby AI framework

Build AI features the Ruby way: models, tools, agents, sandboxed code, evaluations, and Rails conventions, designed together, with one consistent API.

[Website](https://rubyllm.com/) · [Getting Started](https://rubyllm.com/getting-started/) · [What's New in 2.1](https://rubyllm.com/whats-new-in-2-1/) Battle tested at [Chat with Work](https://chatwithwork.com) - *Fully private work AI* [![Gem Version](https://badge.fury.io/rb/ruby_llm.svg)](https://badge.fury.io/rb/ruby_llm) [![Ruby Style Guide](https://img.shields.io/badge/code_style-rubocop-brightgreen.svg)](https://github.com/rubocop/rubocop) [![Gem Downloads](https://img.shields.io/gem/dt/ruby_llm)](https://rubygems.org/gems/ruby_llm) [![codecov](https://codecov.io/gh/crmne/ruby_llm/branch/main/graph/badge.svg)](https://codecov.io/gh/crmne/ruby_llm) crmne%2Fruby_llm | Trendshift
https://github.com/user-attachments/assets/519fcb20-ab8c-4af3-9185-0953b3ba7b81 > [!NOTE] > Using RubyLLM? [Share your story](https://tally.so/r/3Na02p)! Takes 5 minutes. --- ## 23 providers. One API. Build with the models you want. Move between hosted and local providers without rewriting your application, or connect an OpenAI-compatible endpoint. [Browse models and pricing](https://rubyllm.com/available-models/) · [Connect a provider](https://rubyllm.com/configuration-providers/) ## A complete AI framework for Ruby More than a client for model APIs. RubyLLM gives your application the conventions, the runtime, and every piece in between: agents that run code safely, workflows, evaluations, observability, and the data they need, designed to work together. * **Code execution (main):** Let the model run Ruby, Python, JavaScript, and shell in a sandbox on your servers with [`with_code_execution`](https://rubyllm.com/next/code-execution/) * **Skills (main):** Teach the model your workflows with folders of instructions and scripts in the Agent Skills format with [`with_skills`](https://rubyllm.com/next/skills/) * **MCP:** Connect agents to your systems with server classes in `app/mcp`, per-user inputs, tool filtering, and OAuth with [`RubyLLM::MCP`](https://rubyllm.com/mcp/) * **Agents:** Reusable assistants with `RubyLLM::Agent` * **The agentic loop:** Drive it yourself with `ask_later`, `step`, and `complete?` * **Evaluations (main):** Reusable datasets, model criteria, and Ruby assertions with [`RubyLLM::Evaluation`](https://rubyllm.com/next/evaluations/) * **Chat:** Conversational AI with `RubyLLM.chat` * **Vision:** Analyze images and videos * **Audio:** Transcribe speech with `RubyLLM.transcribe` and generate it with `RubyLLM.speak` * **Documents:** Ask questions about PDFs, text files, and other supported formats * **OCR:** Turn documents into markdown with `RubyLLM.ocr` * **Image generation:** Create images with `RubyLLM.paint` * **Video generation:** Create videos with `RubyLLM.animate` * **Embeddings:** Generate embeddings with `RubyLLM.embed` * **Reranking:** Order retrieval candidates by relevance with `RubyLLM.rerank` * **Moderation:** Content flags, categories, and scores with `RubyLLM.moderate` * **Judgments:** Probabilities, choices, and scores with `RubyLLM::Judge` and `RubyLLM.judge` * **Tools:** Let AI call your Ruby methods * **Tool approval:** Park a run until a human approves with `requires_approval` * **Provider tools:** Web search, code execution, and MCP connectors with `with_provider_tools` * **Prompt templates:** ERB prompts in `app/prompts`, rendered with `RubyLLM.render_prompt` * **Workflows:** Correlate multi-agent runs in your telemetry with `RubyLLM.workflow` * **Structured output:** Define a Ruby schema and read the result with `response.parsed` * **Streaming:** Real-time responses with blocks * **Rails:** Active Record persistence, Active Storage attachments, Hotwire streaming, and generators * **Files:** Upload once and reuse across chats with `RubyLLM.upload` * **Prompt caching:** Turn on the provider's cache with `with_caching` and `cache_until_here` * **Fallbacks and cancellation:** Retry on backup models with `with_fallbacks`, stop a run with `cancel` * **Cost tracking:** A per-attempt usage ledger behind `chat.tokens` and `chat.cost` * **Async:** Fiber-based concurrency * **Model registry:** Browse capabilities, limits, and pricing across providers * **Extended thinking:** Control, view, and persist model deliberation * **Citations:** Normalized source citations from documents, search, and grounding * **Batches:** Provider-side batch processing with provider-specific discounts via `RubyLLM.batch` * **Compaction:** Let providers condense long conversations with `with_compaction` * **Token counting:** Count a request before you send it with `count_tokens` * **Providers:** OpenAI, Azure, xAI, Anthropic, Gemini, VertexAI, Bedrock, Cohere, DeepSeek, Hetzner, Meta, MiniMax, Mistral, Moonshot AI, Ollama, Ollama Cloud, OpenRouter, Perplexity, GPUStack, ElevenLabs, Deepgram, TypeSafe, Z.ai, and any OpenAI-compatible API ## Evaluate your agents Keep questions and expected answers in a dataset. RubyLLM checks correctness by default. Run the same evaluation from Ruby, RSpec, Minitest, or Rake. Evaluations are available on `main`. Follow the [development installation guide](https://rubyllm.com/next/getting-started/) to try them. `app/agents/support_agent.rb` ```ruby class SupportAgent < RubyLLM::Agent instructions "Accept returns of unopened items within 30 days." end ``` `app/evals/support_evaluation.rb` ```ruby class SupportEvaluation < RubyLLM::Evaluation def perform(input) SupportAgent.new.ask(input) end end ``` `app/evals/support_evaluation.yml` ```yaml cases: - name: within_return_window inputs: Can I return an unopened item after 14 days? expected_output: Yes, unopened items can be returned within 30 days. - name: past_return_window inputs: Can I return an unopened item after 45 days? expected_output: No, the 30-day return window has passed. ``` Run the evaluation: ```ruby report = SupportEvaluation.run report.passed? report.cost.total ``` The model evaluator checks meaning, so answers can use different wording. Declare your own criteria, add Ruby assertions, or choose a configured evaluator agent or judgment model. [Read the evaluation guides](https://rubyllm.com/next/evaluations/). ## Feels at home in Rails Save conversations with Active Record and stream replies with Hotwire. The generators give you a working chat UI. Watch the two-minute demo. https://github.com/user-attachments/assets/65422091-9338-47da-a303-92b918bd1345 ```bash # Install Rails Integration bin/rails generate ruby_llm:install bin/rails db:migrate bin/rails ruby_llm:load_models # Add Chat UI (optional) bin/rails generate ruby_llm:chat_ui ``` ```ruby class Chat < ApplicationRecord acts_as_chat end chat = Chat.create! model: "gpt-6-luna" chat.ask "What's in this file?", with: "report.pdf" ``` Start your Rails server and visit `http://localhost:3000/chats` to try the chat interface. See the [Rails guide](https://rubyllm.com/rails/) for persistence, streaming, and background jobs. ## Start with one line. Add files, tools, and agents These examples use **2.1.1**. Follow [Getting Started](https://rubyllm.com/getting-started/) to install it and configure the providers you want to try. For 1.x, use the [1.x docs](https://rubyllm.com/v1/). ```ruby # Just ask RubyLLM.chat.ask "What's the best way to learn Ruby?" ``` ```ruby # Ask about files with a model that supports their input types chat = RubyLLM.chat(model: "gemini-3.7-flash") chat.ask "What's in this image?", with: "ruby_conf.jpg" chat.ask "What's happening in this video?", with: "video.mp4" chat.ask "Describe this meeting", with: "meeting.wav" chat.ask "Summarize this document", with: "contract.pdf" chat.ask "Explain this code", with: "app.rb" ``` ```ruby # Multiple files at once chat.ask "Analyze these files", with: ["diagram.png", "report.pdf", "notes.txt"] ``` ```ruby # Stream responses chat.ask "Tell me a story about Ruby" do |chunk| print chunk.content end ``` ```ruby # Generate images image = RubyLLM.paint "a sunset over mountains in watercolor style" image.save "sunset.png" ``` ```ruby # Generate videos video = RubyLLM.animate "a paper boat sailing down a rainy gutter" video.save "paper_boat.mp4" ``` ```ruby # Create embeddings embedding = RubyLLM.embed "Ruby is elegant and expressive" embedding.vectors ``` ```ruby # Rank search results documents = ["Reset your password in Settings.", "Invoices arrive by email."] ranked = RubyLLM.rerank("How do I reset my password?", documents, model: "rerank-v3.5") ranked.results.first.document ``` ```ruby # Transcribe audio to text transcript = RubyLLM.transcribe "meeting.wav" puts transcript.text ``` ```ruby # Turn text into speech speech = RubyLLM.speak "Hello, welcome to RubyLLM!" speech.save "welcome.mp3" ``` ```ruby # Extract document text as markdown document = RubyLLM.ocr "contract.pdf" puts document.markdown ``` ```ruby # Check whether a moderation model flags content RubyLLM.moderate("Some user-generated content").flagged? ``` ```ruby # Ask a typed question about your data class Urgency < RubyLLM::Judge probability :urgent, "Does this need attention today?" end Urgency.judge("Please refund my duplicate charge today.").urgent.probability ``` ```ruby # Let AI use your code class Weather < RubyLLM::Tool description "Get current weather" def execute(latitude:, longitude:) url = "https://api.open-meteo.com/v1/forecast?latitude=#{latitude}&longitude=#{longitude}¤t=temperature_2m,wind_speed_10m" JSON.parse(Faraday.get(url).body) end end chat.with_tools(Weather).ask "What's the weather in Berlin?" ``` ```ruby # Define an agent with instructions + tools class WeatherAssistant < RubyLLM::Agent model "gpt-6-luna" instructions "Be concise and always use tools for weather." tools Weather end WeatherAssistant.new.ask "What's the weather in Berlin?" ``` ```ruby # Get structured output class ProductSchema < Schematist::Schema string :name number :price array :features do string end end response = chat.with_schema(ProductSchema).ask "Analyze this product", with: "product.txt" response.parsed ``` ## Installation Install RubyLLM 2.1: ```bash bundle add ruby_llm --version 2.1.1 ``` Configure a provider in your script, or in `config/initializers/ruby_llm.rb` in Rails: ```ruby require 'ruby_llm' RubyLLM.configure do |config| config.openai_api_key = ENV.fetch('OPENAI_API_KEY') end ``` Configure the other providers used by the examples as needed: Gemini for files, xAI for video, Mistral for OCR, and Cohere for reranking. [Getting Started](https://rubyllm.com/getting-started/) shows each setup beside its example. Upgrading an existing app? Follow the [upgrade guide](https://rubyllm.com/upgrading/). ## AI coding assistants Give your coding assistant the RubyLLM API and documentation that match your application. From your application directory, install the skill packaged with your gem: ```bash gem exec skill add "$(bundle show ruby_llm)" --skill rubyllm --agents coding ``` `--agents coding` installs it for the coding assistants your project uses, not into your app's `app/skills`. See [AI Coding Assistants](https://rubyllm.com/ai-coding-assistants/) for setup and updates. ## Documentation [Guides](https://rubyllm.com/getting-started/) · [API reference](https://rubyllm.com/api/) · [Models](https://rubyllm.com/available-models/) · [Upgrading](https://rubyllm.com/upgrading/) · [1.x docs](https://rubyllm.com/v1/) ## Contributing See [CONTRIBUTING.md](CONTRIBUTING.md). ## License Released under the MIT License.