A barebones Go CLI that calls an LLM feels broken the moment a request takes more than a second. Adding a go cli progress bar llm users can actually see turns a hanging process into a tool they trust, especially when streaming tokens or waiting on provider fallback.
Step 1: Scaffold a minimal Go CLI that talks to an LLM endpoint
Start with a module and two terminal UI dependencies: briandowns/spinner for indeterminate waits and schollz/progressbar/v3 for bounded progress.
mkdir llmcli && cd llmcli
go mod init llmcli
go get github.com/briandowns/spinner@latest
go get github.com/schollz/progressbar/v3@latestThe CLI should read a prompt and an API base URL from the environment. Keep the HTTP client boring and never log the key. A separated client struct makes testing and later swapping the transport trivial.
package main
import (
"bytes"
"encoding/json"
"flag"
"fmt"
"net/http"
"os"
)
type client struct {
baseURL string
apiKey string
http *http.Client
}
func main() {
prompt := flag.String("p", "", "prompt to send")
flag.Parse()
if *prompt == "" {
fmt.Fprintln(os.Stderr, "provide -p")
os.Exit(1)
}
c := client{
baseURL: os.Getenv("LLM_BASE_URL"),
apiKey: os.Getenv("LLM_API_KEY"),
http: &http.Client{},
}
if c.baseURL == "" || c.apiKey == "" {
fmt.Fprintln(os.Stderr, "set LLM_BASE_URL and LLM_API_KEY")
os.Exit(1)
}
_ = c // used in later steps
}The LLM_BASE_URL should be an OpenAI-compatible /v1/chat/completions endpoint. This keeps the go cli progress bar llm code portable across providers and gateways without rewriting request shapes.
Step 2: Add a spinner for the latency window before first byte
Streaming LLMs have a time-to-first-token (TTFT) that can range from 100 ms to several seconds depending on model load. A static cursor makes users think the process hung. Spin the cursor only until the first token arrives or the full response completes.
import (
"github.com/briandowns/spinner"
"time"
)
func (c *client) complete(prompt string) (string, error) {
s := spinner.New(spinner.CharSets[14], 100*time.Millisecond)
s.Suffix = " contacting provider"
s.Start()
defer s.Stop()
body, _ := json.Marshal(map[string]any{
"model": "gpt-3.5-turbo",
"messages": []map[string]string{{"role": "user", "content": prompt}},
"stream": false,
})
req, _ := http.NewRequest("POST", c.baseURL+"/chat/completions", bytes.NewReader(body))
req.Header.Set("Authorization", "Bearer "+c.apiKey)
req.Header.Set("Content-Type", "application/json")
resp, err := c.http.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
// decode non-streaming response...
return "", nil
}The defer s.Stop() guarantees the spinner clears even on error. Never leave a spinner running after you start printing real output; it corrupts the terminal line and makes logs unreadable.
Why not just print dots
Dots imply progress magnitude that does not exist. A spinner communicates “working, unknown duration” honestly. Use a bar only when you have a real bound, such as a max_tokens ceiling.
Step 3: Stream tokens and drive a progress bar from token estimates
When you enable stream: true, you receive Server-Sent Events. Set max_tokens to a fixed ceiling and treat that as the bar’s total. A rough heuristic: one token ≈ four bytes of English text. Update the bar per chunk.
import (
"bufio"
"strings"
)
func (c *client) stream(prompt string) error {
bar := progressbar.DefaultBytes(200*4, "generating")
defer bar.Finish()
body, _ := json.Marshal(map[string]any{
"model": "gpt-3.5-turbo",
"messages": []map[string]string{{"role": "user", "content": prompt}},
"stream": true,
"max_tokens": 200,
})
req, _ := http.NewRequest("POST", c.baseURL+"/chat/completions", bytes.NewReader(body))
req.Header.Set("Authorization", "Bearer "+c.apiKey)
req.Header.Set("Content-Type", "application/json")
resp, err := c.http.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
scanner := bufio.NewScanner(resp.Body)
scanner.Buffer(make([]byte, 1024*1024), 1024*1024) // SSE lines can be long
for scanner.Scan() {
line := scanner.Text()
if !strings.HasPrefix(line, "data: ") {
continue
}
data := strings.TrimPrefix(line, "data: ")
if data == "[DONE]" {
break
}
var chunk struct {
Choices []struct {
Delta struct{ Content string `json:"content"` }
} `json:"choices"`
}
if json.Unmarshal([]byte(data), &chunk) != nil {
continue
}
if len(chunk.Choices) > 0 {
content := chunk.Choices[0].Delta.Content
if content != "" {
fmt.Print(content)
bar.Add(len(content))
}
}
}
return scanner.Err()
}This go cli progress bar llm technique gives users a visible ceiling. If the model stops early, bar.Finish() snaps to 100% and clears the line. The byte-based bar is approximate, but it is far better than a silent wait.
Step 4: Manage concurrency and signals so the UI never corrupts
A Ctrl-C during a stream must stop the spinner or bar and restore the terminal. Use signal.Notify and a context.Context to cancel the HTTP request.
import (
"context"
"os"
"os/signal"
"syscall"
)
func (c *client) streamSafe(prompt string) error {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
sig := make(chan os.Signal, 1)
signal.Notify(sig, syscall.SIGINT)
go func() {
<-sig
cancel()
}()
body, _ := json.Marshal(map[string]any{
"model": "gpt-3.5-turbo",
"messages": []map[string]string{{"role": "user", "content": prompt}},
"stream": true,
"max_tokens": 200,
})
req, _ := http.NewRequestWithContext(ctx, "POST", c.baseURL+"/chat/completions", bytes.NewReader(body))
req.Header.Set("Authorization", "Bearer "+c.apiKey)
req.Header.Set("Content-Type", "application/json")
resp, err := c.http.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
bar := progressbar.DefaultBytes(200*4, "generating")
defer bar.Finish()
scanner := bufio.NewScanner(resp.Body)
for scanner.Scan() {
if ctx.Err() != nil {
break
}
// ... same SSE parsing as Step 3
}
if ctx.Err() != nil {
fmt.Fprintln(os.Stderr, "\ninterrupted")
return ctx.Err()
}
return nil
}Wrap the spinner and bar creation inside the same context scope. If cancelled, bar.Finish() still writes the final state; call s.Stop() explicitly in the signal handler to avoid a stuck glyph. Restoring the terminal is non-negotiable for a CLI that others will pipe or script.
Step 5: Point the CLI at a gateway with fallback
If you point the CLI at n4n.ai, the OpenAI-compatible endpoint addresses 240+ models and automatic fallback when a provider is degraded means your CLI can skip custom retry loops; the spinner just covers the latency window. Set LLM_BASE_URL=https://api.n4n.ai/v1 and the same code works without branch logic for provider errors.
That single integration point also honors client routing directives and forwards provider cache-control hints, so repeated prompts hit caches and return faster—your spinner spins less. The go cli progress bar llm design stays unchanged; only the environment variable differs.
Step 6: Verify the experience end to end
Build and run against a test prompt:
go build -o llmcli .
LLM_BASE_URL=https://api.openai.com/v1 LLM_API_KEY=sk-... ./llmcli -p "Explain TCP slow start in one paragraph"Success criteria:
- A spinner appears immediately and disappears when the first token prints (or after a non-streaming response returns).
- A progress bar labeled
generatingfills left-to-right as text streams. - Pressing Ctrl-C prints
interruptedand returns to a clean prompt with no leftover animation frames. - With
max_tokensset to 200, the bar reaches full even if the model emits fewer tokens, then clears.
Test the non-streaming path by flipping stream to false in the request body; the spinner should cover the whole wait and stop before any text prints. If you see garbled output, ensure you call bar.Finish() and s.Stop() on every exit path. Terminal width under 80 columns may truncate the bar; progressbar handles this, but verify in a narrow pane.
The go cli progress bar llm pattern is now complete: honest indeterminate state, bounded progress during streaming, and clean shutdown. Ship it.