Local Ollama

Running NIKI with local Ollama inference and no cloud LLM calls.

Local Offline Inference with Ollama

When every agent uses Ollama and optional network features are disabled, LLM prompts stay on the local machine. niki research, configured knowledge URLs, OTLP export, or remote MCP client use can still create outbound traffic.

1. Start Ollama

Ensure Ollama is running locally:

ollama serve
ollama pull qwen2.5-coder:32b

2. Configure niki.toml

Point provider configuration to the Ollama host root. NIKI appends /api/chat; do not add /v1:

[providers.ollama]
base_url      = "http://localhost:11434"
default_model = "qwen2.5-coder:32b"

[agents.planner]
provider = "ollama"
model    = "qwen2.5-coder:32b"

[agents.coder]
provider = "ollama"
model    = "qwen2.5-coder:32b"

[agents.tester]
provider = "ollama"
model    = "qwen2.5-coder:32b"

[agents.reviewer]
provider = "ollama"
model    = "qwen2.5-coder:32b"