Models

Otter does not host GPUs for you. Configure inference once in the local app under Models.

Local Ollama (recommended)

ollama pull qwen2.5-coder:7b
# Docker platform base URL: http://host.docker.internal:11434/v1
# Native base URL:          http://127.0.0.1:11434/v1

Then open /app/models, choose Local Ollama, save, and test the connection.

Any OpenAI-style /v1 endpoint works. Paste the base URL, pick a model, and set an API key only if the host requires one.

Optional free failover tries alternate local models when the primary completion fails. Toggle it on the Models page.