Intelligent Routing and Traffic Management

5.1 Gateway Routes

Path pattern matching (globs), protocol types (openai-compatible/anthropic/google/cohere/custom), upstream URL, proxy status (active/passthrough), priority, timeout (30s default), custom headers, extensible metadata.

Advanced: Header-based routing (AND/OR), body transformation, path rewriting, query params, content-type filtering, size limits, user agent blocking, IP allowlisting per route, circuit breaker per route, import/export/clone/bulk/reorder.

Routing modes (configurable per route or globally):

• Semantic routing — embedding-similarity match to route by request topic/intent
• Weighted routing — split traffic across model variants by percentage
• Round-robin load balancing — across keys/providers in the same model family
• Least-loaded — picks the key with the lowest in-flight count
• Cost-optimized routing — prefer the cheapest model/provider meeting capability requirements
• Priority / QoS routing — latency-sensitive workloads jump ahead of bulk batch traffic; per-route SLA hints feed circuit-breaker and failover logic
• Failover routing — health-monitored automatic switch to alternate model endpoints when the primary fails or exceeds latency thresholds (see Section 8)

5.2 Tenant Routing Rules

Dynamic processing modes: passthrough (skip DLP/guardrails), proxied (full pipeline), enriched (add context), post_process (enrich responses). Match conditions (AND): model pattern, headers, input patterns, tenant/team/employee IDs, providers, keywords.

5.3 Proxy Mode (Zscaler Integration)

Domain-to-provider mapping, credential modes (replace/passthrough/audit), CIDR trusted IPs, custom user header (X-Zscaler-User).

---