Description

We need a pluggable abstraction layer to support multiple LLM providers (OpenAI, Anthropic, local models, etc.) so Serqet can dynamically route tasks based on cost, latency, and capability.

Goals

Single interface for all model calls
Swap providers without changing agent logic
Enable routing + fallback logic

Tasks

• Create ModelProvider interface
• Implement providers:
• OpenAI
• Anthropic
• Local (Ollama or similar)
• Add config-based provider selection
• Normalize responses (tokens, messages, tool calls)
• Add retry + fallback logic

Acceptance Criteria

Any agent can call model.generate() without knowing provider
Switching providers requires only config change