Calling the OpenAI API from Ruby on Rails with Faraday

Most Rails apps reach OpenAI through an ad-hoc Net::HTTP mess or a heavyweight SDK that hides the wire. This tutorial shows how to call the rails faraday openai api path cleanly: a thin, testable Faraday connection that handles auth, retries, and streaming without obscuring HTTP semantics.

Prerequisites

Before writing code, confirm your environment:

• Ruby 3.2+ (3.1 works, but 3.2 has better timeout handling)
• Rails 7+ (works on 6.1, but we assume Zeitwerk and credentials)
• faraday in your Gemfile
• An OpenAI API key stored via rails credentials:edit under openai_api_key

Add the dependency:

bundle add faraday

Your Gemfile should contain:

gem "faraday", "~> 2.7"

Run bundle install. No native extensions, so it’s fast.

Building the base client

Create a service object. In Rails, app/services is autoloaded. We want a single Faraday connection per client instance, configured with JSON encoding, response parsing, and a retry policy.

# app/services/open_ai_client.rb
class OpenAIClient
  BASE_URL = "https://api.openai.com/v1".freeze

  def initialize(api_key: Rails.application.credentials.openai_api_key!)
    @api_key = api_key
  end

  def connection
    @connection ||= Faraday.new(url: BASE_URL) do |conn|
      conn.request :json
      conn.response :json, content_type: /\bjson$/
      conn.request :retry,
        max: 2,
        interval: 0.5,
        exceptions: [Faraday::TimeoutError, Faraday::ConnectionFailed]
      conn.options.open_timeout = 5
      conn.options.read_timeout = 30
      conn.headers["Authorization"] = "Bearer #{@api_key}"
      conn.adapter Faraday.default_adapter
    end
  end
end

A few decisions worth calling out:

• conn.request :json encodes the body and sets Content-Type. No manual to_json.
• conn.response :json parses only when the response content-type ends in json. OpenAI returns application/json.
• Retry middleware catches network failures, not 429s. We handle those explicitly later.
• Timeouts are non-negotiable. Default Faraday has no timeout, so a hung socket eats a thread.

Sending a chat completion

The chat endpoint is a POST to /chat/completions. We wrap it in a method that returns the assistant text and raises on errors.

class OpenAIError < StandardError; end

def chat(messages, model: "gpt-4o-mini")
  payload = {
    model: model,
    messages: messages,
    temperature: 0.7
  }

  resp = connection.post("/chat/completions", payload)
  unless resp.success?
    raise OpenAIError, "HTTP #{resp.status}: #{resp.body.inspect}"
  end

  resp.body.dig("choices", 0, "message", "content")
end

Open a Rails console and run:

client = OpenAIClient.new
client.chat([{ role: "user", content: "Write a one-line Ruby tip." }])

Expected output is a string similar to:

"Use `&:` to call a method on each element, like `items.map(&:id)`."

The exact text varies. The point is you get a plain Ruby string, not a wrapped object.

Streaming tokens with SSE

For chat UIs you want tokens as they arrive. OpenAI streams Server-Sent Events. Faraday’s default Net::HTTP adapter supports streaming via on_data.

def stream_chat(messages, model: "gpt-4o-mini")
  payload = {
    model: model,
    messages: messages,
    stream: true
  }

  connection.post("/chat/completions", payload) do |req|
    req.options.on_data = proc do |chunk, _received_bytes, _env|
      chunk.each_line do |line|
        next unless line.start_with?("data:")

        data = line[5..].strip
        next if data == "[DONE]"

        json = JSON.parse(data)
        delta = json.dig("choices", 0, "delta", "content")
        $stdout.write(delta) if delta
      end
    end
  end
end

Call it from a Rails runner or console:

client = OpenAIClient.new
client.stream_chat([{ role: "user", content: "Count to 5 slowly." }])

You will see tokens printed incrementally instead of one blob. Note: on_data yields raw TCP chunks; a single SSE event can be split across chunks. For production, use a proper SSE parser (or faraday-sse gem). The snippet above works for low-volume interactive use.

Dealing with rate limits and errors

OpenAI returns 429 when you exceed quota, 401 on bad keys, 400 on malformed requests. Our chat method already raises on non-2xx. To distinguish rate limits:

def chat(messages, model: "gpt-4o-mini")
  resp = connection.post("/chat/completions",
    { model: model, messages: messages, temperature: 0.7 })

  case resp.status
  when 200
    resp.body.dig("choices", 0, "message", "content")
  when 429
    raise OpenAIError, "Rate limited. Back off and retry."
  when 401
    raise OpenAIError, "Invalid API key."
  else
    raise OpenAIError, "HTTP #{resp.status}: #{resp.body.inspect}"
  end
end

If you want automatic exponential backoff on 429, add a custom middleware or use faraday-retry with a retry_statuses option (available in newer Faraday). We keep it explicit because 429 often requires application-level throttling.

Swapping in a gateway

The rails faraday openai api pattern is portable. If you need access to multiple model providers without rewriting request code, point the BASE_URL at an OpenAI-compatible gateway. For example, n4n.ai exposes a single OpenAI-compatible endpoint covering 240+ models with automatic fallback when a provider is rate-limited; the same chat payload works unchanged. You only alter the constant and optionally send a routing header.

BASE_URL = "https://api.n4n.ai/v1".freeze
# conn.headers["X-Route"] = "anthropic/claude-3-5-sonnet" # if you want explicit routing

Because the response shape is identical, your parsing code does not move.

Writing a test with WebMock

You should not hit the network in specs. Stub the endpoint:

# spec/services/open_ai_client_spec.rb
require "rails_helper"

RSpec.describe OpenAIClient do
  it "returns content from a chat completion" do
    stub = stub_request(:post, "https://api.openai.com/v1/chat/completions")
      .with(headers: { "Authorization" => "Bearer test" })
      .to_return(
        status: 200,
        body: { choices: [{ message: { content: "hello" } }] }.to_json,
        headers: { "Content-Type" => "application/json" }
      )

    client = OpenAIClient.new(api_key: "test")
    expect(client.chat([{ role: "user", content: "hi" }])).to eq("hello")
    expect(stub).to have_been_requested
  end

  it "raises on rate limit" do
    stub_request(:post, "https://api.openai.com/v1/chat/completions")
      .to_return(status: 429, body: "{}")

    client = OpenAIClient.new(api_key: "test")
    expect { client.chat([{ role: "user", content: "hi" }]) }
      .to raise_error(OpenAIError, /Rate limited/)
  end
end

Run rspec spec/services/open_ai_client_spec.rb. Both examples should pass.

Exposing it through a controller

A minimal controller that accepts a message and returns the reply:

# app/controllers/chat_controller.rb
class ChatController < ApplicationController
  def create
    client = OpenAIClient.new
    reply = client.chat([{ role: "user", content: params.require(:message) }])
    render json: { reply: reply }
  rescue OpenAIError => e
    render json: { error: e.message }, status: :bad_gateway
  end
end

Route:

# config/routes.rb
post "/chat", to: "chat#create"

Curl test:

curl -X POST localhost:3000/chat -H 'Content-Type: application/json' \
  -d '{"message":"What is Rails?"}'

Expected JSON:

{ "reply": "Rails is a web application framework written in Ruby..." }

Why not just use the official gem?

The openai Ruby gem is perfectly serviceable. But it abstracts the HTTP layer behind its own response objects, making it harder to add custom middleware, uniform instrumentation, or swap providers. With Faraday, the rails faraday openai api integration is just HTTP calls you fully control. You can log, trace, and retry with the same middleware stack you use for other external services.

Final notes

• Reuse the connection instance across requests; creating a new Faraday object per call wastes setup.
• For streaming in Puma, ensure your front-end proxy (nginx) does not buffer responses.
• Store keys in credentials, never in ENV plaintext in repos.
• If you scale, move the client to a singleton or Rails initializer and inject it.

That’s the whole integration. No magic, just HTTP done explicitly.