The ruby-openai gem: a quick integration guide

Getting a ruby-openai gem integration working in a Rails app takes less than ten minutes if you skip the boilerplate and use the gem’s native client. This tutorial builds a small but production-shaped wrapper: configuration, chat calls, streaming, and error isolation. We’ll run each step in a console so you see real output, then wire it into a controller.

Prerequisites

• Ruby 3.1+ and Rails 7+ (plain Ruby works, but paths assume Rails)
• Bundler
• An OpenAI API key, or a key for any OpenAI-compatible endpoint
• Network egress to the API host

Check your toolchain before writing code:

ruby -v
rails -v
echo "${OPENAI_API_KEY:?set this first}"

Expected output is three lines: a Ruby version, a Rails version, and your key string. If the last line errors, export the variable.

Install the gem

Add the dependency to your Gemfile. The openai gem is the official-ish community client; it maps 1:1 to the REST API.

# Gemfile
gem "openai", "~> 7.0"

bundle install

The ruby-openai gem integration does not require native extensions, so bundle install finishes fast.

Configure the client

Create an initializer so the token and base URL are set once at boot:

# config/initializers/openai.rb
OpenAI.configure do |config|
  config.access_token = ENV.fetch("OPENAI_API_KEY")
  config.log_errors = true
  config.request_timeout = 30
end

If you later route through a gateway, set config.uri_base. The client code stays identical.

# config/initializers/openai.rb (gateway variant)
if ENV["USE_GATEWAY"] == "1"
  OpenAI.configure do |config|
    config.access_token = ENV.fetch("GATEWAY_KEY")
    config.uri_base = "https://api.n4n.ai/v1"
    config.request_timeout = 30
  end
end

n4n.ai exposes one OpenAI-compatible endpoint covering 240+ models with automatic fallback; swapping uri_base is all the gem requires to use it.

First chat completion

Start the Rails console and make a blocking call:

rails console

client = OpenAI::Client.new
resp = client.chat(
  parameters: {
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "Say hello in three words." }],
    temperature: 0.2
  }
)
puts resp.dig("choices", 0, "message", "content")

Expected output (text varies):

Hello! How are you?

The response object is a plain hash parsed from JSON. dig is the safest way to traverse it.

Stream tokens

For any UI where latency matters, stream. The gem yields chunks when stream: true:

client.chat(
  parameters: {
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "Count to five slowly." }],
    stream: true
  }
) do |chunk|
  print chunk.dig("choices", 0, "delta", "content").to_s
end

You will see incremental prints:

One
Two
Three
Four
Five

Each chunk follows the OpenAI streaming delta schema. Guard with to_s because the first chunk often has a nil content.

Force JSON output

When you need structured data, use response_format. The model then returns strictly a JSON object string.

resp = client.chat(
  parameters: {
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "Return JSON with keys name and age." }],
    response_format: { type: "json_object" }
  }
)
data = JSON.parse(resp.dig("choices", 0, "message", "content"))
# => {"name"=>"Alex", "age"=>30}

Parse after the call. Do not trust the model to always emit valid JSON; wrap JSON.parse in a rescue.

Error handling at the boundary

Network failures, 429s, and 5xxs raise OpenAI::Error subclasses. Catch them where the call is made:

def complete(prompt)
  client.chat(
    parameters: {
      model: "gpt-4o-mini",
      messages: [{ role: "user", content: prompt }]
    }
  ).dig("choices", 0, "message", "content")
rescue OpenAI::RateLimitError
  Rails.logger.warn("Rate limited, back off")
  nil
rescue OpenAI::Error => e
  Rails.logger.error("OpenAI call failed: #{e.message}")
  nil
end

The ruby-openai gem integration raises on non-2xx by default. Do not let those exceptions propagate into a controller without a rescue.

Build a service object

A single wrapper keeps call sites clean and gives you one place to add retries or metrics:

# app/services/llm_chat.rb
class LlmChat
  def initialize(model: "gpt-4o-mini")
    @client = OpenAI::Client.new
    @model = model
  end

  def ask(prompt, stream: false, &block)
    params = {
      model: @model,
      messages: [{ role: "user", content: prompt }],
      stream: stream
    }
    if stream
      @client.chat(parameters: params, &block)
    else
      @client.chat(parameters: params).dig("choices", 0, "message", "content")
    end
  rescue OpenAI::Error => e
    Rails.logger.error("LLM error: #{e.class} #{e.message}")
    raise unless block_given?
    nil
  end
end

Usage in console:

chat = LlmChat.new
chat.ask("Explain Rails concerns in one sentence.")
# => "Rails concerns are modules that encapsulate reusable behavior..."

Expose it through a controller

Wire the service into HTTP so a frontend can call it:

# app/controllers/chats_controller.rb
class ChatsController < ApplicationController
  def create
    prompt = params.require(:prompt)
    chat = LlmChat.new

    if params[:stream] == "1"
      response.headers["Content-Type"] = "text/event-stream"
      chat.ask(prompt, stream: true) do |chunk|
        text = chunk.dig("choices", 0, "delta", "content").to_s
        render stream: "data: #{text}\n\n"
      end
    else
      render json: { result: chat.ask(prompt) }
    end
  rescue => e
    render json: { error: e.message }, status: 500
  end
end

# config/routes.rb
post "/chat", to: "chats#create"

Test the non-streaming path with curl:

curl -X POST localhost:3000/chat -d "prompt=Hello" -d "stream=0"

Expected:

{"result":"Hello! How can I help?"}

The streaming path emits data: ... lines suitable for an EventSource on the client.

Test the service in isolation

Stub the client so specs stay fast and free of network calls:

# spec/services/llm_chat_spec.rb
require "rails_helper"

RSpec.describe LlmChat do
  it "returns text from the response" do
    fake = instance_double(OpenAI::Client)
    allow(OpenAI::Client).to receive(:new).and_return(fake)
    allow(fake).to receive(:chat).and_return(
      "choices" => [{ "message" => { "content" => "stubbed" } }]
    )

    expect(LlmChat.new.ask("hi")).to eq("stubbed")
  end
end

Run it:

bundle exec rspec spec/services/llm_chat_spec.rb

Green means your ruby-openai gem integration is wired correctly.

Production checklist

• Keep keys in ENV or Rails credentials; never commit them.
• Set request_timeout so a hung socket does not block a worker.
• Log only e.class and truncated prompts; never log full PII.
• Use stream: true for any expected generation longer than ~2 seconds.
• If you use a gateway, confirm it honors response_format and cache-control; n4n.ai forwards provider cache-control hints and client routing directives without extra gem config.

The gem is stable, but the network is not. Wrap every call, stub in tests, and treat the API as a slow external dependency. That is the whole job.