Local models on Ollama

[!IMPORTANT] Work in progress — not a final product: Running vibey on local models with Ollama is under active development and not yet a final product. Model profiles and engine features continue to evolve. Folks will be notified on the Discord server once the product is finally done.

Bottom Line Up Front (BLUF) — Install Ollama, vibey, and every krypton interface on your machine:

  1. Install vibey and krypton (one line in Terminal): bash curl -fsSL https://raw.githubusercontent.com/the-vibey-project/vibey/main/scripts/install.sh | sh -s -- --with-krypton vibey connect --yes # plug vibey into Claude Code, Codex, Cursor, VS Code, Claude Desktop & Gemini CLI
  2. Install Ollama and pull the sovereign default model (Heretic v2 of GPT-OSS 20B):
  3. macOS: Download Ollama from ollama.com/download and open it once.
  4. Linux: curl -fsSL https://ollama.com/install.sh | sh
  5. Pull the model (~12.1 GB): bash ollama pull hf.co/mradermacher/gpt-oss-20b-heretic-v2-GGUF:MXFP4_MOE # Or for the stock build: VIBEY_SOVEREIGN_MODEL=stock (ollama pull gpt-oss:20b)
  6. Install any krypton interface (Desktop for macOS & Linux, VS Code extension, Web app, Android & iOS mobile app): See the step-by-step install guide and Downloads.

Sub-doctrine 8.a makes the sovereign path the preferred way to run vibey, not the fallback. This guide is the operator recipe for doing that on one machine with Ollama: the server, the claudeloop profile, the switches, and — just as important — what a local model can and cannot carry today.

There are three local engines. gptossloop — the local runner on GPT-OSS 20B, the sovereign default — is on without any switch. qwenloop is the same runner on a Qwen model (qwen3:14b), and claudeloop-local is the claudeloop binary on a local backend profile; both are opt-in (ADR-0064). With its local engines on, vibey:

  • prefers local engines first for BUILD. Selection runs smooth weighted round-robin within the LOCAL tier (gptossloop, plus qwenloop and claudeloop-local when switched on) and falls back to the paid engines only when no local engine is eligible (ADR-0038);
  • runs DESIGN and DECOMPOSE on the local model when no --provider is given (GptossloopDesignProvider, GptossloopWorkPlanProducer, recorded in the ledger as gptossloop);
  • costs nothing per token: every local engine is priced 0/0, and claudeloop records every local turn at $0.

1. The Ollama server

Start the server with settings sized for agent work. On a 24 GB machine:

export OLLAMA_CONTEXT_LENGTH=32768     # the window every request gets; match it below
export OLLAMA_FLASH_ATTENTION=1        # needed for a quantized KV cache
export OLLAMA_KV_CACHE_TYPE=q8_0       # roughly halves KV-cache memory at 32K
export OLLAMA_MAX_LOADED_MODELS=1      # one model resident; a second one evicts it
ollama serve

With the macOS app, set them with launchctl setenv NAME value and restart the app. Ollama's default context is far smaller than an agent needs: Claude Code's own system prompt and tool definitions are about 14,000 tokens before your plan, so a window left at the default fails on the first request.

Pull the model you mean to run (see the honest ceiling before you choose):

ollama pull your-model:tag

2. The one endpoint setting

VIBEY_OLLAMA_URL is the single setting, in root form. Everything local reads it:

export VIBEY_OLLAMA_URL=http://127.0.0.1:11434
# Default sovereign build: Heretic v2 of GPT-OSS 20B (ADR-0099)
#   hf.co/mradermacher/gpt-oss-20b-heretic-v2-GGUF:MXFP4_MOE
# Select the unmodified OpenAI release instead with:
#   export VIBEY_SOVEREIGN_MODEL=stock        (or: --ollama-model stock)
export VIBEY_OLLAMA_MODEL=your-model:tag     # or --ollama-model on work/worker; beats VIBEY_SOVEREIGN_MODEL

The two sovereign builds (heretic vs stock)

gptossloop and the sovereign DESIGN and DECOMPOSE providers share one two-variant catalogue (src/vibey/domain/sovereign_model.py, ADR-0099):

Variant key Ollama tag Source & license Notes
heretic (default) hf.co/mradermacher/gpt-oss-20b-heretic-v2-GGUF:MXFP4_MOE p-e-w/gpt-oss-20b-heretic-v2 (Apache-2.0 weights; ~12.1 GB download; Heretic AGPL-3.0 tool not used at runtime) Refusal direction removed (49/100 test prompts refused vs 98/100 for stock; KL divergence 0.1248). Native MXFP4_MOE.
stock gpt-oss:20b Ollama library (Apache-2.0; ~14 GB download) OpenAI's release, unmodified. Select with VIBEY_SOVEREIGN_MODEL=stock or --ollama-model stock.

Precedence is --ollama-model > VIBEY_OLLAMA_MODEL > VIBEY_SOVEREIGN_MODEL > heretic. Both --ollama-model and VIBEY_OLLAMA_MODEL also accept the variant keys heretic and stock and resolve them to their full tags.

Verification note (sub-doctrine 10.f): The Hugging Face manifest for hf.co/mradermacher/gpt-oss-20b-heretic-v2-GGUF:MXFP4_MOE has been verified, but the tag has not yet been pulled with ollama pull or run through an end-to-end tool-calling or --sovereign-fit measurement on the reference host. Every measured latency, memory and benchmark figure on this page and in System requirements was recorded against stock gpt-oss:20b.

For the qwenloop storm profile used while developing this repository, see the copyable qwenloop-local.toml and qwenloop-storm.env.example examples. The checked-in storm profile uses qwen3:14b, a 32K context window, a 40-turn local budget, Ollama's OpenAI-compatible endpoint, and the single-model cache settings used for local agent work. Qwen lifecycle desktop notifications stay enabled and use macOS's Ping sound; --desktop-notifications is explicit in the example commands even though it is the default.

  • The DESIGN and DECOMPOSE providers talk to <VIBEY_OLLAMA_URL>/api/chat, in JSON mode with the answer's schema stated in the prompt and checked on the way back; a malformed answer is re-asked once, naming what was wrong. A reply cut short by its output budget (done_reason: length, the reasoning spent every token) is retried once with double the budget, within VIBEY_OLLAMA_CONTEXT, and reasoning level VIBEY_OLLAMA_RETRY_THINK (low by default; none leaves the model's own). A second cut-short reply fails as a capacity shortfall naming VIBEY_OLLAMA_OUTPUT / VIBEY_OLLAMA_CONTEXT. Measured before and after in sovereign-retry-2026-09-29.
  • gptossloop's process gets GPTOSSLOOP_BASE_URL=<VIBEY_OLLAMA_URL>/v1 and GPTOSSLOOP_MODEL=<the model> (--ollama-model, else VIBEY_OLLAMA_MODEL, else VIBEY_SOVEREIGN_MODEL, else the heretic default tag), so it attaches to this server and runs the providers' model.
  • qwenloop's process gets QWENLOOP_BASE_URL=<VIBEY_OLLAMA_URL>/v1 only — never a model: it asks for qwen3:14b unless QWENLOOP_MODEL or its own config names another, so pull that model too.
  • Each variable is set only when you have not set it yourself. Without VIBEY_OLLAMA_URL, each runner keeps its own backend selection. gptossloop reads only GPTOSSLOOP_* and qwenloop only QWENLOOP_*, so naming a model for one never changes the other's.
  • claudeloop-local reads its endpoint from its profile (next step), so keep the profile's base_url equal to VIBEY_OLLAMA_URL.

3. The claudeloop profile

claudeloop-local is the claudeloop binary run with --profile <name>: Claude Code still runs the agent loop, the tools and the edits, but talks to the local server, with the paid key scrubbed. Describe the profile in claudeloop.toml in the project (or ~/.config/claudeloop/config.toml):

[profiles.local]
base_url = "http://127.0.0.1:11434"      # = VIBEY_OLLAMA_URL
model_low = "your-model:tag"             # --preset low     (TRIVIAL, LOW)
model_medium = "your-model:tag"          # --preset medium  (STANDARD)
model_high = "your-model:tag"            # --preset high    (HIGH, MAX)
small_fast_model = "your-model:tag"
context_window = 32768                   # = OLLAMA_CONTEXT_LENGTH
max_output_tokens = 8192                 # Claude Code reserves this out of the window

Every key, and what a local profile changes (a blanked ANTHROPIC_API_KEY, $0 turns, no --effort, web search refused, completion only by a structured verdict), is in claudeloop's own local backend guide. Check the profile with claudeloop directly first:

claudeloop doctor --profile local

Trust its backend-tools check above all: it asks each tier to make one real tool call. A model that fails it cannot do agent work under Claude Code at all.

4. Switch the local engines on

Each local engine has its own switch. The environment variable wins whenever it is set; otherwise [features] in the project's config decides; otherwise the engine's default applies.

Engine Default Environment vibey.toml
gptossloop on VIBEY_FEATURE_GPTOSSLOOP=0 switches it off [features] gptossloop = false
qwenloop off VIBEY_FEATURE_QWENLOOP=1 [features] qwenloop = true
claudeloop-local off VIBEY_FEATURE_CLAUDELOOP_LOCAL=1 [features] claudeloop_local = true

Before ADR-0064 the qwenloop switch turned on the engine that ran gpt-oss:20b; that engine is now gptossloop, on by default. With the qwenloop switch on, vibey worker and vibey doctor print a note: saying so — drop the switch unless you want Qwen as well.

claudeloop-local's own settings:

[engines.claudeloop_local]
profile = "local"             # the [profiles.NAME] above; VIBEY_CLAUDELOOP_LOCAL_PROFILE overrides
context_window = 32768        # keep equal to the profile's and the server's
structured_verdict = false    # claim it only once conformance proves it (below)

The worker reads [features] from the project's stored config, which no vibey new flag writes yet, and vibey doctor reads ./vibey.toml. The environment variables are the switches that reach both — set them for vibey doctor and vibey worker alike.

5. Check, then record

vibey doctor                                   # lists every switched-on local engine
vibey doctor --conformance --record --engine claudeloop-local
vibey doctor --conformance --record --engine gptossloop
vibey doctor --conformance --record --engine qwenloop   # when switched on

Do not leave a model warmed up at a large window. Ollama reloads a model whose context differs from a request's. A warm-up at a very large window, left resident, makes the first real request pay for a reload with a key-value cache sized for that window, and on a laptop the request can time out. vibey doctor prints an ollama line that says so when the loaded model's context is above VIBEY_OLLAMA_CONTEXT, and names the fix: ollama stop gpt-oss:20b.

A prompt that fills the window cuts the answer off. gpt-oss reasons before it answers, and its reasoning counts against the same window as the prompt. A DESIGN prompt grows with every question answered, and response ... stopped: length in vibey llm tail with prompt N tokens + output M tokens adding up to VIBEY_OLLAMA_CONTEXT means the window, not the output budget, ran out. Three settings meet it:

export VIBEY_OLLAMA_CONTEXT=16384
export VIBEY_OLLAMA_THINK=low
export VIBEY_OLLAMA_OUTPUT=6144

A bigger window stops the ledger being trimmed from the middle, VIBEY_OLLAMA_THINK=low keeps the first attempt from spending itself reasoning, and a larger output budget leaves room to write the design spec. The one retry also opens a larger window by itself, up to VIBEY_OLLAMA_RETRY_CONTEXT (32768 by default), but it costs a model reload, so prefer sizing the window right.

A local model is slow to its first turn. Conformance starts a real run and waits 30 seconds for its run directory. gpt-oss:20b on a laptop loads weights and answers a first turn well past that, so run_dir_shape, snapshot_schema, done_marker and structured_verdict fail together for a reason that is the host's speed, not the engine's. Tell conformance how long this host needs:

VIBEY_CONFORMANCE_POLL_SECONDS=420 vibey doctor --conformance --record --engine gptossloop

Measure it rather than guessing: a window long enough to pass once is the number to keep, and one that is far longer than the check ever uses costs nothing, because the wait ends the moment the files appear.

vibey doctor runs claudeloop doctor --profile <name> for claudeloop-local, so the health check probes the backend the run will use. The worker also re-runs each enabled local engine's doctor before every BUILD selection; a local engine whose doctor fails is simply not eligible, and the job goes to the next engine.

STRUCTURED_VERDICT. claudeloop-local does not claim a structured verdict by default. Set structured_verdict = true only for a model you intend to hold to it: conformance then requires a VerdictRendered event from the configured model, and a claim it cannot prove fails conformance and makes the engine ineligible.

Running unattended

A local run parks on gates that are not decisions: an interview question whose default is the answer, an attempt budget you would grant again, a delivery that died with its worker. To let one worker keep going through those, tell it so, for that run:

caffeinate -i vibey -vvv worker --project PROJECT_ID --auto-answer

--auto-answer answers the interview with each question's own default, grants ten more attempts or six more repair rounds when a job runs out, and delivers a dropped job once more. It never answers a spending gate, an approval or a review, and says which gates it left. It stops after --auto-answer-limit answers (30) and says so. Each answer is on the ledger as GateAnswered by auto-answer. Defaults take the narrowest scope, so a question that proposes adding something the intake did not name is answered No: read the spec before vibey design accept.

To go on to BUILD without running vibey design accept yourself, add --auto-accept-design, and --accept-spec-json PATH to import a spec you reviewed first. It accepts once DESIGN has finished, through the same guards as the manual command, never opts into the visual stage, and says what it will do at startup. This is your decision that the spec is right, made in advance, so use it with a spec file you have read, not with whatever the model wrote. The ledger records accepted_by: auto-accept-design.

Watching the model think

A DESIGN or DECOMPOSE call to a local model can run for many minutes, and by default it shows nothing until it returns. Ask for the conversation and follow it from a second terminal:

vibey -vvv worker --project PROJECT_ID
vibey llm tail

-vvv makes the client stream Ollama's reply and write every request and every piece of the reply to the wire log, ~/.local/state/vibey/llm-wire.jsonl. vibey llm tail shows the request (model, context, output budget, both messages), then the reasoning and the answer growing as they are generated, then the model's own counts and tokens per second. A call that stalls shows as a request with nothing after it, which is how a context window set too small or a model still loading looks from the outside.

BUILD and REVIEW too. An engine session is a separate process, so the worker notes each one in the same log and the tail follows it: a banner naming the engine, the work item and the run, then every tool call (turn 3 -> search {...}) and a one-line result (<- search: 3 found, <- shell: exit 1 - ... with the end of the output, where the failure is), the model's streamed answer, its reasoning (thinking: ...), and each turn's time and tokens. When the next item starts, the tail moves to its run on its own, so one terminal shows the whole project:

[22:35:08] === gptossloop session - work item ws - run fb316537 ===
  turn 1 -> search {"glob": "*", "path": "", "query": "package.json"}
    <- search: 10 found
  turn 1 done - 3.1s (model 2.9s) - in 1346 tokens - out 58 tokens
  thinking: The spec wants a /ministry section; check what exists first.
  turn 2 -> shell {"argv": ["npm", "run", "typecheck"]}
    <- shell: exit 1 - src/app/ministry/page.tsx(4,10): error TS2305 ...

Text appears a word at a time, not a token at a time, because the log releases streamed text only at whitespace: a credential the model repeats cannot then be split across two pieces and escape redaction. The log holds your full prompts, so it is created readable by you alone; remove it when you no longer need it. To log without -vvv, set VIBEY_LLM_WIRE_LOG=path.

Measuring a capacity fit

The sovereign DESIGN and DECOMPOSE providers reach Ollama through one client with two ceilings: the context window it asks for (num_ctx: VIBEY_OLLAMA_CONTEXT, default 8192, at least 4096) and the output it allows (num_predict: VIBEY_OLLAMA_OUTPUT, default 2048). vibey doctor --sovereign-fit measures what this host's server actually answers, and records it:

vibey doctor --sovereign-fit                   # writes ~/.local/state/vibey/sovereign-fit.json
vibey doctor --sovereign-fit --fit-output ./sovereign-fit.json
export VIBEY_OLLAMA_FIT=~/.local/state/vibey/sovereign-fit.json   # for work and worker

The probe is scripts/sovereign_probe.py, run with doctor's own interpreter against VIBEY_OLLAMA_URL and VIBEY_OLLAMA_MODEL. It sends one JSON-mode chat request for each combination of context 4096, 8192 and output 512, 1024, 2048, each with a 120-second timeout. A combination is valid when the reply carries non-empty content, and the fastest valid one becomes selected_fit. The record holds measured_at, url, model, revision (git rev-parse HEAD in the directory doctor runs in, else unknown), prompt_shape (the probe's own prompt size), every result, and selected_fit. With no valid combination doctor prints sovereign fit FAIL and exits 1. The file is still written, with selected_fit: null, which the client ignores.

Doctor does not export anything: set VIBEY_OLLAMA_FIT in the environment of vibey work and vibey worker. The client then uses the record only when all of these hold:

  • its url equals VIBEY_OLLAMA_URL (default http://127.0.0.1:11434) exactly, as a string, so a trailing slash is a different server;
  • its model equals the model in effect: --ollama-model, else VIBEY_OLLAMA_MODEL, else VIBEY_SOVEREIGN_MODEL, else hf.co/mradermacher/gpt-oss-20b-heretic-v2-GGUF:MXFP4_MOE (heretic);
  • when VIBEY_REVISION is set, its revision equals it;
  • its selected_fit is valid, with a context of at least 4096 and a positive output, and it carries a prompt_shape.

Otherwise (unset, unreadable, malformed or mismatched) the configured ceilings apply: VIBEY_OLLAMA_CONTEXT and VIBEY_OLLAMA_OUTPUT, 8192 and 2048 by default.

A matching record replaces those two ceilings with its measured context and output, but only for a request no larger than the prompt it was measured with (the record's system plus user characters). A larger request falls back to the built-in ceilings, 8192 and 2048, not to the two variables. The probe's prompt is 42 characters today, so real DESIGN and DECOMPOSE prompts run on the built-in ceilings: the record proves what the server answers at each size, not yet a working size for a full ledger.

The probe script ships in the repository, not in the vibey-engine wheel. Doctor looks for it beside the source tree it runs from, so --sovereign-fit works from a checkout (an editable install) and fails with sovereign fit FAIL from a wheel install.

6. Run

vibey worker                                   # sovereign DESIGN/DECOMPOSE, local-first BUILD
vibey worker --engines gptossloop,claudeloop-local   # never fall back to a paid engine
  • An explicit --provider always wins; claudeloop (paid) is never a default.
  • Verification still rotates: gptossloop's work is reviewed by claudeloop-local (or qwenloop) and the other way round. With one local engine in the pool, the review goes to a paid engine if one is configured, or the engine reviews its own diff and the ledger says so (ADR-0035).
  • A run whose backend cannot serve it — the server down, a model not pulled or out of memory, a window too small — makes claudeloop exit 78. The job parks with an engine_misconfigured gate naming the doctor command that shows the cause. Fix it, then answer the gate to retry: vibey answer <gate-id> --raw '{}'.

Running Claude Code itself on Ollama (a storm)

The same server can carry the interactive Claude Code that drives a storm of lanes. These are the variables a local claudeloop profile sets for Claude Code; export them in the shell that starts claude:

export ANTHROPIC_BASE_URL=http://127.0.0.1:11434
export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_API_KEY=""                         # a paid key must not travel along
export ANTHROPIC_DEFAULT_OPUS_MODEL=your-model:tag
export ANTHROPIC_DEFAULT_SONNET_MODEL=your-model:tag
export ANTHROPIC_DEFAULT_HAIKU_MODEL=your-model:tag
export CLAUDE_CODE_SUBAGENT_MODEL=your-model:tag
export CLAUDE_CODE_MAX_CONTEXT_TOKENS=32768          # = OLLAMA_CONTEXT_LENGTH
export CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1    # no telemetry or update checks
claude

Every model id must be one the server serves: an unset tier falls back to a claude-* id the local server does not have.

The honest ceiling

A local model is not the paid one it displaces, and the defaults say so:

  • One agent at a time on a 24 GB machine. A 14–20B model at a 32K window fills it; a second concurrent agent evicts the first model (OLLAMA_MAX_LOADED_MODELS=1) or does not fit. Run vibey worker -j 1 and one storm lane at a time.
  • Effort tops out at STANDARD. claudeloop-local's HIGH and MAX run the profile's top tier and report achieved=STANDARD, so rotation sees a local model for what it is.
  • Use the endpoint-specific tool smoke before a storm. qwen2.5-coder:14b can write tool calls as text or misclassify the qwenloop-verdict fence when served locally. On this machine, qwen3:14b produced a native read_file call through Ollama's OpenAI-compatible endpoint, so it is the checked-in storm example. This is a compatibility observation, not a guarantee for every serving template; keep the runner's tool smoke and bounded completion guard enabled.
  • Research needs you. A local model has no web access; DESIGN's research stage parks a research_evidence gate asking for <topic>.md in VIBEY_EVIDENCE_DIR rather than inventing a source (ADR-0027).
  • A paid fallback is visible, not yet announced. When no local engine is eligible the job goes to a paid engine; vibey engines and the local engine's health show why, but nothing is written to the ledger at that moment yet. Pin the pool with --engines if a paid run is never acceptable.

Model families

vibey models lists the families vibey knows by name -- Qwen Coder, DeepSeek Coder, Codestral, StarCoder, Gemma, Llama, CodeLlama and CodeGemma -- each with a tag to start from and the ollama pull that fetches it, and then what the Ollama on this machine actually has and says each model can do. Run it before you choose.

What matters is whether a model reports tools. The BUILD runner works through tool calls. A model that reports none is driven through a text tool-call protocol instead (tool_protocol = "auto" in the runner's config; native and text force either way): its tools are described in the prompt, it writes each call as <tool_call>{...}</tool_call>, and the run says when this is in use. It is about twice as slow per turn, and in the measurement behind ADR-0093 it did not make such models build correctly, so reach for it to see what a model does, not to expect a result. Such a model is also good for DESIGN and DECOMPOSE, which ask for JSON and need no tool-calling. A model whose capabilities your Ollama does not report is shown as unknown, and left native.

On a small BUILD task Llama 3.1 8B was correct 0 times in 8 (native or text) and the default gpt-oss:20b 1 time in 3. Give a model that reports a large default context (Llama 3.1 does: 131,072) a num_ctx 32768 derived model before you judge its speed; the first Llama run here took 8 minutes to load without it.

Reporting tools is necessary, not sufficient: Qwen2.5-Coder 14B reports it and still writes its calls out as text. Only a run shows whether a model uses them well. The measurements behind the list, and why a template check is not enough, are in ADR-0093.