Models

Otter does not host GPUs for you. Bring your own inference — local Ollama is the default recommendation; any OpenAI-compatible /v1 endpoint works.

Local Ollama (recommended)

ollama pull qwen2.5-coder:7b
# Docker platform base URL: http://host.docker.internal:11434/v1
# Native / CLI base URL:    http://127.0.0.1:11434/v1

Docker UI: open /app/models, choose Local Ollama, save, and test. CLI: run /model inside otter.

Any OpenAI-style /v1 endpoint works. Paste the base URL, pick a model, and set an API key only if the host requires one.

Optional free failover tries alternate local models when the primary completion fails. Toggle it on the Models page in the Docker UI.

Default model env vars live in .env.example. Prefer documenting new providers in this page when you add them.

Local Qwen 7B coding results: Benchmark.