Models & providers¶

How to choose a model and a backend wire, and how to swap either without leaving the session.

kin talks to a model over one of two wires. You pick the wire with provider, the model with model, and the endpoint with base_url. Each knob resolves in the same precedence order: CLI flag, then KIN_* env var, then a project .kin/settings.toml, then the global one, then a default. See Configuration for the precedence rules and Environment variables for the full list.

OpenAI-compat vs Anthropic Messages¶

There are two backends, and they speak different protocols.

Backend	`provider`	Protocol	Default model	Notes
OpenAI-compat	`openai`	Chat Completions	`gpt-4o-mini`	The default. Works against OpenAI, OpenRouter, vLLM, LiteLLM, Together, any compat server.
Anthropic Messages	`anthropic`	`/v1/messages`	`claude-opus-4-8`	Native Anthropic, or a vLLM serve that exposes the Anthropic wire.

The OpenAI-compat backend streams text, reasoning, tool-call fragments, and usage, normalizing the quirks of various compat stacks (it trusts assembled tool calls over a stray finish_reason: stop, and reads reasoning from whichever field the provider uses).

The Anthropic Messages backend uses adaptive thinking with summarized display so reasoning streams, sets effort via output_config, preserves thinking-block signatures across turns so they replay on the same model, and supports prefix caching on the frozen system prompt. The thinking.type knob is now knob-shaped (enabled / disabled / adaptive) — Anthropic-compat presets (Z.ai, MiniMax) ride the same knob; real Anthropic defaults to adaptive so behavior is unchanged on a vanilla install.

The KIN_BASE_URL heuristic¶

When you set base_url (via --base-url, KIN_BASE_URL, or a settings file) but leave provider unset, the factory defaults to the Anthropic Messages wire, not the OpenAI-compat one.

The reason is that a base_url almost always points at a self-hosted server (typically vLLM), and on such a server the Anthropic wire is the stronger choice:

Parallel-tool-safe blocks — typed tool_use blocks instead of the streamed-delta reassembly the compat wire has to do.
A real count_tokens — the per-component context breakdown in /tokens works, where the compat wire reports totals only.
Preserved thinking signatures — typed thinking blocks survive across turns for correct replay.

The escape hatch is explicit. Set provider (any of --provider openai, KIN_PROVIDER=openai, or provider = "openai" in a settings file) to force the OpenAI-compat wire (Chat Completions) — for example when base_url points at real OpenAI, OpenRouter, or another GPT-style endpoint.

Note

The heuristic only fires when provider is unset. Anthropic-direct users (no base_url) keep the openai default and opt in with KIN_PROVIDER=anthropic. Setting provider explicitly — by any means — always wins over the heuristic.

base_url handling¶

The two SDKs disagree about what a base URL means, so the factory adjusts for the Anthropic wire.

KIN_BASE_URL is conventionally written the OpenAI way, ending in /v1 (the OpenAI SDK appends /chat/completions to it). The Anthropic SDK appends /v1/messages to whatever you give it, so passing http://host:port/v1 would produce http://host:port/v1/v1/messages and 404.

To let one env var drive both wires, the factory trims a trailing /v1 before handing the URL to the Anthropic backend:

KIN_BASE_URL=http://kin-one:8000/v1
  → OpenAI-compat backend:  http://kin-one:8000/v1   (used as-is)
  → Anthropic backend:      http://kin-one:8000      (trailing /v1 trimmed)

You write the URL once, the OpenAI way; the trim happens only on the Anthropic path. With no base_url set, each backend talks to its provider's public endpoint (api.openai.com / api.anthropic.com).

The vLLM recipe¶

A full environment for a tailnet vLLM serving a Qwen model. With base_url set and provider unset, this runs on the Anthropic Messages wire by way of the heuristic above. KIN_MODEL=default is the wire-side sentinel — vLLM resolves it to the live served-model-name, and the harness surfaces the resolved id in the banner + the top bar (see "The default alias pattern" below). kin-one below is one operator's own tailnet hostname (a worked example, not a fixed name) — substitute your own server's host:port:

KIN_BASE_URL=http://kin-one:8000/v1 \
KIN_MODEL=default \
KIN_MAX_TOKENS=32768 \
uv run kin

To pin the wire to a specific model instead, set KIN_MODEL=<id> (e.g. KIN_MODEL=Qwen/Qwen3.6-35B-A3B).

The context window starts from a conservative default and is refined from the server's max_model_len on the first turn; set KIN_CONTEXT_WINDOW to pin it. Note that sampling is not in this recipe — see below for why.

The `default` alias pattern¶

If you register a stable alias for "whatever model is currently loaded" on your local server (e.g. vLLM's --served-model-name default Ornith-1.0-35B …), hardcode that alias in settings:

# ~/.kin/settings.toml
base_url = "http://kin-one:8000/v1"
model = "default"

The harness sends model="default" verbatim on the wire (so the server keeps routing to the active model across swaps) and surfaces the resolved name — the canonical first served-model-name entry — in the welcome banner and the top bar (left cell, beside ⭘ kin). Sources of the resolved name:

A GET /v1/models probe on session start (silent-degrade; transport errors and empty listings are no-ops).
response.model piggybacked off each completed turn (so a mid-session server swap self-corrects on the next turn without any restart).

The two paths dedup against a per-session cache — a stable server stays quiet. For Anthropic-direct or OpenAI-direct (base_url unset) with model = "default", the banner shows the literal "default" (real Anthropic / OpenAI have no such alias — the alias only makes sense for local serves that route it).

Swapping at runtime¶

Use /model inside the session to change the model without restarting:

/model Qwen/Qwen3.6-35B-A3B   # swap to this model id
/model glm-5.2                # cross-provider swap — Z.ai (Anthropic wire, Bearer)
/model                        # open the cross-provider picker

Keys are entered in `/providers`¶

/providers is the one place to enter an API key. Pick a provider, paste the key (masked), hit enter: it's stored per-provider in the provider_keys[<id>] table and the running session swaps to it live — no restart, no /reload. Each provider keeps its own key, so swapping between Z.ai and MiniMax never overwrites the other's.

`/model` only swaps to providers that have a key¶

Both /model forms surface only providers that are ready to swap:

Bare /model opens a picker with a ✓ key badge on every provider that has a key on file (green for a stored per-provider key; dim when it resolves only via an env var or the shared api_key). Providers with no resolvable key show a needs key hint, and selecting one routes straight into /providers <id> to set it up.
/model <id> swaps silently when the key chain resolves (provider_keys[id] → env → shared api_key). If nothing resolves, it refuses with a hint to run /providers <id> first — there's no inline key prompt.

The picker (`/model` bare)¶

Bare /model opens a keyboard-driven picker spanning every provider — your favorites first (a ★ group), then the built-in presets, your custom [[providers]] endpoints, and the active endpoint's live /v1/models listing. Arrow keys move, enter picks, f toggles a favorite (persisted to model_favorites in your global settings), esc cancels. Each row shows a ✓ key badge (or a needs key hint) so you can see at a glance which providers are ready. Picking a model on a keyed provider does a live cross-wire swap: the harness rebuilds the conversation history into the target wire's native shape and re-points the backend. Picking an unkeyed provider hands off to /providers <id> to enter a key first.

Inline `/model <id>` (no picker)¶

/model <id> swaps in place. The cross-provider path resolves the key for the target preset from provider_keys[id], the env chain (ANTHROPIC_API_KEY / KIN_API_KEY / OPENAI_API_KEY), or the shared api_key; if any of those resolves, the swap is silent. If none does, it refuses with a run /providers <id> first hint — there's no inline key prompt. The [1m] suffix on Z.ai 1M-tier model ids (glm-5.2[1m]) is stripped before the catalog lookup so the plain id matches. History is stored in provider-native shapes; the cross-provider path converts it for you.

Where the keys live¶

# ~/.kin/settings.toml
provider_preset = "zai"            # active preset (built-in id or [[providers]] row)
model = "glm-5.2"                  # default model for the active preset

[provider_keys]                    # one key per provider id — the canonical store
zai = "sk-..."
minimax = "sk-..."

/providers writes here when you paste a key; the table is read-merged, so adding MiniMax never clobbers the Z.ai key. The factory's key-resolution chain (make_backend in src/kin/harness/backends/factory.py) is: caller-supplied api_key > provider_keys[<active_preset_id>] (via _preset_key) > ANTHROPIC_API_KEY env > KIN_API_KEY env > OPENAI_API_KEY env > top-level api_key. The active preset is the most specific selector — a stray ANTHROPIC_API_KEY baked into an image can't silently shadow the Z.ai key you stored in provider_keys.zai. See the settings-toml reference for the full precedence + GLOBAL_ONLY rules.

The mode controls and the rest of the in-session verbs live in Slash commands.

Sampling¶

Sampling knobs apply on the OpenAI-compat wire only. The backend forwards temperature and top_p as standard params and rides top_k and enable_thinking on extra_body (where vLLM and most compat servers read them):

Knob	Env	Effect
`temperature`	`KIN_TEMPERATURE`	0 = greedy, higher = more random
`top_p`	`KIN_TOP_P`	Nucleus sampling
`top_k`	`KIN_TOP_K`	Top-k sampling (via `extra_body`)
`enable_thinking`	`KIN_ENABLE_THINKING`	Qwen3-style soft switch; `false` hard-disables thinking

Warning

The Anthropic Messages backend sends no per-request sampling — current Opus / Fable models reject temperature, top_p, and budget_tokens with a 400. This includes a vLLM serve reached through the heuristic. Set sampling on the server instead, e.g. with vLLM's --override-generation-config. That is also the only place to set server-only knobs like min_p or presence_penalty.

See settings.toml reference for the sampling block and the per-knob scope.

Priority-scheduling passthrough (vLLM)¶

When several kin surfaces share one self-hosted vLLM (interactive sessions, subagent fan-outs, headless cron runs), vLLM can schedule them by priority: serve with --scheduling-policy priority and requests carrying a lower priority integer are scheduled earlier, with genuine KV-swap preemption of lower-priority work when demand spikes (vLLM ≥ 0.23). kin passes this through as request_priority (KIN_REQUEST_PRIORITY env / settings key) → extra_body["priority"] on every request — e.g. KIN_REQUEST_PRIORITY=10 on your cron jobs keeps them behind your live sessions.

Two hard caveats, both server-side facts rather than kin choices:

Serve-gated, and it fails loudly. vLLM errors on any request carrying a non-zero priority when the serve doesn't run --scheduling-policy priority. That is why kin's default is absent from the request — not 0 — and why you should only set this knob against a serve you know runs the priority policy.
OpenAI-compat wire only. The Anthropic /v1/messages protocol has no priority field (verified against vLLM's entrypoints/anthropic/protocol.py), so on the fleet's default wire (the KIN_BASE_URL heuristic picks Anthropic) the knob is silently unsupported — set KIN_PROVIDER=openai if you want priority scheduling. kin doesn't fake it on the Anthropic wire; client-side governance (the per-run token budget) is the primary fleet-safety mechanism either way.

Strict decoding (schema-valid tool calls)¶

Structured decoding makes the server grammar-constrain tool-call arguments to the tool's JSON Schema, so malformed args can't reach the harness. What you actually get depends on the endpoint and the call shape (probed against the fleet's vLLM 0.23.0 — research/probe_strict_decoding.py has the receipts):

Path	Guarantee today
Forced tool calls (the workflow `schema=` structuring call — `force_tool_call`, both wires)	On vLLM, a named/forced `tool_choice` is grammar-constrained unconditionally — on `/v1/chat/completions` and the `/v1/messages` bridge. The OpenAI-compat wire additionally ships a strict-normalized schema + `strict: true` so OpenAI-proper-class endpoints enforce too.
The streaming tool registry (`tool_choice: "auto"`)	Opt-in via `strict_tools` / `KIN_STRICT_TOOLS=1` (OpenAI-compat wire only). Each tool spec is normalized (`additionalProperties: false`, all-required, optionals as `["type","null"]` unions) and stamped `strict: true` at request-build time. vLLM 0.23 ignores `strict` on `auto` (no-op); vLLM ≥ 0.24 and OpenAI-proper grammar-enforce it.

The Anthropic Messages wire never sends strict — real Anthropic rejects unknown tool fields, and the vLLM bridge already constrains forced calls without it. Either way the harness keeps its client-side net: malformed argument JSON still parses to {} (the tool then reports the missing args), and the workflow schema= result is still validated against your original schema — server enforcement makes that net near-never-fire; it doesn't replace it.

A tool schema that can't be normalized losslessly (an open additionalProperties, an unsupported keyword like uniqueItems, an empty enum) is shipped unmodified without strict — better unenforced than a 400. All bundled kin tools normalize cleanly.

Google's DiffusionGemma 26B A4B is a discrete diffusion language model (dLLM): it denoises 256-token canvas blocks in parallel rather than generating one token at a time. vLLM shipped first-class day-0 support in June 2026 (the vLLM blog post walks the architecture — speculative-decoding path, entropy-bound sampler, dynamic per-sequence causal attention). It serves from a normal vllm serve with the model id set.

A few practical notes for kin:

Wire — prefer the heuristic default (Anthropic), with openai as the fallback. vLLM shipped day-0 dLLM support in June 2026 via the speculative-decoding path (originally OpenAI-side), and recent builds (≥ v0.22.1rc1) wire it through /v1/messages too — live-verified on this serve with curl: both wires return HTTP 200 for the dLLM model id, and streaming works on both with the expected chunk cadence. For dLLMs that think, the Anthropic wire is actually the better default: the response is typed thinking + text blocks with a signature, so the harness's existing reasoning handling carries the thinking blocks across turns replay-safe the same way it does on Qwen / MiniMax / Z.ai serves. The OpenAI wire puts reasoning in a bare choices[0].message.reasoning field with no signature — ephemeral. If your serve only exposes the OpenAI-side dLLM integration (an older vLLM build, or a custom router), set KIN_PROVIDER=openai to escape the heuristic; the curl probe at the end of this section is the verification.

The full canonical recipe:
```
KIN_BASE_URL=http://kin-one:8000/v1 \
KIN_MODEL=google/diffusiongemma-26B-A4B-it \
uv run kin
```
KIN_MODEL=default also works — it resolves to RedHatAI/diffusiongemma-26B-A4B-it-FP8-dynamic on this serve (the first alias in /v1/models). The short alias dgemma-26b is a cleaner handle if you've registered it; google/diffusiongemma-26B-A4B-it is the upstream HF id.
Reasoning budget — give it space to think. DiffusionGemma is a reasoning model (the model card describes configurable <|think|> control tokens). A small max_tokens will cut the response off mid-thought (stop_reason: "max_tokens", content: null) — the same shape the harness already salvages on Qwen via loop._salvage_reasoning. Set KIN_MAX_TOKENS generously (8k–32k is fine; the dLLM's per-block throughput is high enough that the wall-clock cost stays modest).
Sampling — server-side, not request-side. DiffusionGemma uses an Entropy-Bound (EB) sampler rather than temperature / top_p / top_k — the model walks canvas positions from most to least confident, accepting tokens until accumulated entropy exceeds a fixed budget, and commits the block once the argmax has been stable for N steps. Configure EB on the vLLM serve (--override-generation-config or the model card's recommended DiffusionSampler settings), not in the request body. The harness's existing "sampling is OpenAI-compat-only" precedent works in your favor here: any KIN_TEMPERATURE / KIN_TOP_P / KIN_TOP_K you set are silently ignored by the dLLM backend, so they don't break anything — they just don't do anything.
Streaming cadence. The vLLM dLLM sampler reports num_sampled = 0 during the denoising loop and only emits num_sampled = 256 on the commit step. From the client's perspective you'll see 256-token chunks landing on the SSE stream rather than token-by-token deltas — the harness consumes them transparently; no KinApp changes needed. Total throughput is roughly 5–6× a comparable autoregressive model (~1,000 tok/s on H100, ~1,290 tok/s on H200 for FP8), so the user-perceived cadence is "small delay, then a 256-token block snaps in, then another, then another" rather than a per-character trickle.
Wire-support probe. A quick curl against your serve confirms whether dLLMs are wired through /v1/messages:
```
curl -sS http://kin-one:8000/v1/messages \
  -H 'content-type: application/json' \
  -d '{"model":"dgemma-26b","max_tokens":256,"messages":[{"role":"user","content":"hi"}]}' \
  | head -c 400
```
Expected on a healthy dLLM serve: HTTP 200 with a typed thinking block + text block in the content array. A 404 NotFoundError means the serve doesn't have a dLLM loaded (check curl /v1/models); a 4xx validation error means the dLLM is loaded but isn't wired through the Anthropic adapter on this vLLM version — escape with KIN_PROVIDER=openai.

Cost display¶

Cost display is opt-in (default off): kin's primary endpoint is local vLLM (no cost at all), and the subscription providers (MiniMax/z.ai) bill flat-rate, so per-token cost is noise for most sessions. Turn it on with /cost on (persists to ~/.kin/settings.toml) or KIN_PRICING=1 / pricing_enabled = true. /cost (bare) reports the current state and, when on, the running total; /cost off turns it back off and resets the running total.

Once on, /tokens shows a running USD cost total alongside the per-component token breakdown, and the StatusBar carries a recessive $0.04 cell on its right edge (next to the mode badge). The figures come from a local, stdlib-only pricing table in harness/pricing.py — keyed by (provider, sanitize_model_id(model)) so the SAME model-id drift tolerance that protects the wire-side display covers the pricing boundary too.

Coverage and rates (2026-07 snapshot, per million tokens, USD):

Provider	Models	Notes
Anthropic	Opus 4.6+ / Sonnet 4.x / Haiku 4.x	Incl. prompt-cache read + write rates. The kin default `claude-opus-4-8` is the current generation ($5/$25), not the legacy 4.1 premium tier.
z.ai	GLM-4.5 / 4.5-Air / 4.6 / 4.7 / 5.x	Fleet default `glm-5.2` at $1.40/$4.40. `glm-4.5-flash` deliberately omitted (not on the current z.ai pricing page).
MiniMax	M3 / M2.x	List price (≤512k input tier for M3).
OpenAI	gpt-4o / 4.1 family	When kin is pointed at real OpenAI.

Unknown / unpriced serves — your local vLLM (Qwen3.6-27B, Ornith-1.0-35B, dgemma-26b, the default alias), a model id the table doesn't carry — surface nothing rather than a misleading $0.00 (free by omission, never a fake zero). Add a row to PRICES in harness/pricing.py when a new fleet model lands; the rate is a snapshot, not a live scrape, so revisit the cited provider page when prices change.

Accuracy boundary (cache-write approximation)¶

The kin Usage shape collapses Anthropic's per-component tokens into three fields: prompt_tokens = input + cache_read + cache_creation, cached_tokens = cache_read, completion_tokens = output. The cache_creation count — which Anthropic bills at ~1.25x the input rate — is folded into prompt_tokens and indistinguishable from a fresh input token at this layer. compute_cost therefore charges cache-creation tokens at the input rate, a documented underestimate. The bound is tight in practice: cache writes happen once per cache lifetime (5 min / 1 h), not per round, so for an agentic loop with many cache hits the missed 25% surcharge applies to a small slice of total tokens.

Edge cases¶

Provider swap (/providers or /model cross-provider): the running total resets because the new provider's per-token rate differs from the old one (Opus $5/M → GLM $1.40/M makes pre-swap cents incomparable to post-swap). The pre-swap total rides forward in a system-note footnote so you can still see what the prior provider cost.
Resume: usage events aren't durably stored in the journal, so a resumed session starts the counter at $0 and accumulates only the new turns. /tokens notes this honestly.
Opt-in / off: /cost off (or KIN_PRICING=0 / pricing_enabled = false) disables the accumulator entirely (no table lookup, no display cell) and clears the running total. This is also the default — nothing to opt out of unless you've already turned it on. See pricing_enabled.

Effort picker¶

/effort <level> flips a per-serve reasoning-depth knob live, and /effort (bare) opens a picker that's curated for the active backend — not a server-cap probe. The vocabulary differs by wire and (on the OpenAI-compat wire) by endpoint, so the right thing to ask the user for is wire-shaped:

Active backend	Vocabulary	Wire knob
`AnthropicBackend` (real Anthropic, vLLM Anthropic serve, Z.ai/MiniMax presets)	`low` / `medium` / `high` / `xhigh` / `max`	`output_config.effort`
`OpenAIBackend` against OpenAI / OpenRouter	`low` / `medium` / `high`	top-level `reasoning_effort`
`OpenAIBackend` against Z.ai OpenAI-compat (GLM-5.2+)	`auto` / `max` / `xhigh` / `high` / `medium` / `low` / `minimal` / `none`	top-level `reasoning_effort`
`OpenAIBackend` against a vLLM OpenAI-compat serve (Qwen-style)	`auto` / `on` / `off`	`extra_body.chat_template_kwargs.enable_thinking` (`auto` = unset)

The picker treats the (default) row as "leave it to the server / wire defaults" — pick it on cold start to wipe the override. Picking always invalidates the prompt cache, so inline /effort <level> pops a confirmation modal first; the picker is the confirmation by itself.

The setter is HUMAN-ONLY — the model has no set_effort tool and reaches the knob only through propose_settings effort=..., which still requires your approval before anything changes. Resuming from the journal restores the active level; the active wire knob stays put so a resume_config() against anthropic.py / openai.py re-points it on reload. See Slash commands for the /effort row + the cache-invalidation caveat.

First-party presets¶

For MiniMax and Z.ai, the harness ships a curated preset that bundles the URL, model id, and auth style. Set provider_preset = "minimax" in ~/.kin/settings.toml (or KIN_PRESET=minimax uv run kin, or uv run kin --preset minimax) and the harness picks the right wire and auth header automatically. The /providers command in the TUI walks the same catalogue interactively and writes the choice back to settings.

See Provider presets for the catalogue, the /providers modal, custom endpoints, and the GLOBAL_ONLY rationale.