Cluster Decisions — Optional Sidecar Profiles

Status: proposal (awaiting @diegosouzapw review) Date: 2026-06-20 Refs: #3932, PR #4381

TL;DR

Two opt-in compose profiles (memory, bifrost) for the existing 8-service deployment in docker-compose.yml. Default-up behaviour is unchanged: 3 × omniroute replicas + Caddy + Redis + CliproxyAPI. The two new profiles add Qdrant and Bifrost as optional sidecars, gated by docker compose --profile <name> up. No existing service is removed or replaced.

Why this is conservative

OmniRoute's existing deployment shape is already lean and proven:

• redis:7-alpine handles the rate-limit/cache workload at production scale.
• SQLite + sqlite-vec + FTS5 cover local memory + vector + text-search (see src/lib/memory/vectorStore.ts:108).
• Caddy is already the LB + TLS terminator (docker-compose.yml).
• Bifrost is already integrated as the Tier-1 router in src/app/api/v1/relay/chat/completions/bifrost/route.ts (sidecar proxy with kill switch via BIFROST_ENABLED env var — set =0 to bypass the sidecar and fall through to the TS path).

The two profiles here are scale-out options for deployments that hit the SQLite ceiling — not migrations. Both are default-off.

The two profiles

memory — Qdrant Vector Memory Sidecar

When to flip on:

• 1M embeddings per deployment (sqlite-vec starts to slow at scale).

• Multi-replica deployment that needs shared vector state across omniroute-1/2/3.
• You already have an external Qdrant cluster (Qdrant Cloud, on-prem).

What it adds:

Activation: flip qdrantEnabled = true in the Settings UI or set QDRANT_HOST=qdrant env. See src/lib/memory/qdrant.ts:60 for the precedence rules (settings table → env var → default).

Env vars: QDRANT_HOST, QDRANT_PORT, QDRANT_API_KEY, QDRANT_COLLECTION, QDRANT_VECTOR_SIZE, QDRANT_HNSW_EF_CONSTRUCT (see .env.example lines 1672-1683).

bifrost — Bifrost Tier-1 Router Sidecar

When to flip on:

• You run ≥3 omniroute replicas and want provider rotation centralised in a single Go process.
• You want a single audit/logging surface for upstream-provider requests across all replicas.
• You want horizontal scaling of the Tier-1 routing layer independent of the OmniRoute replicas.

What it adds:

Activation: set BIFROST_BASE_URL=http://bifrost:8080 in .env.example. The existing sidecar proxy route at src/app/api/v1/relay/chat/completions/bifrost/route.ts (added in PR #4381) will pick this up automatically.

Env vars: BIFROST_BASE_URL, BIFROST_API_KEY, BIFROST_STREAMING_ENABLED, BIFROST_TIMEOUT_MS (see .env.example lines 1685-1695).

What this PR explicitly does NOT do

The original issue thread floated a larger cluster rewrite. After auditing the actual workload shape, the following are rejected for the reasons given:

If a future use case proves out one of these, this doc is the place to amend.

4-week rollout (if approved)

1. Wk 1 — Land this PR + verification of opt-in profiles with a 3-replica compose stack.
2. Wk 2 — Bifrost full activation for OpenAI/Claude/Gemini/Ollama (4 of 14+ providers) using the sidecar proxy route at src/app/api/v1/relay/chat/completions/bifrost/route.ts (gated by BIFROST_ENABLED, kill-switchable at runtime).
3. Wk 3 — Qdrant memory profile enabled in a single test deployment; measure latency delta vs sqlite-vec.
4. Wk 4 — Observability healthchecks (docker compose ps exit codes + wget smoke tests); 71-pillar refresh per ADR-041.

Files changed in this PR

Net touched code: 4 production files (docker-compose.yml, qdrant.ts, .env.example, ENVIRONMENT.md), 1 test file (qdrant-wiring.test.ts), 2 doc files (cluster-decisions.md, AGENTS.md).