MTPLX Release Notes

A fix for Macs with 8, 16 and 32 GB

2.12.1 refused ordinary prompts there with the model the app picks for them, and 2.12.2 serves them.

Prompts priced at what they use

2.12.1 charged every model the 27B's 3 GiB per prompt chunk. Each prompt is now priced at what it was measured to use: 0.19 GiB for a 59-token prompt on the 4B.

Smaller chunks before a refusal

A prompt that does not fit runs in 1,024 or 512-token chunks instead of being refused.

Served again

2,600 to 7,800-token prompts on a 16 GB Mac with Bonsai 2 27B, 6,800 and 7,800-token prompts on a 32 GB Mac with the 27B, and a one-line prompt on an 8 GB Mac with 3 GiB free.

Memory guard, rewritten.

A stability release for long coding sessions: the memory guard prices every request before its prompt is read and releases idle conversations before it refuses anything, long sessions keep their cache through screenshots, and Flash-Next keeps its compiled verifier past 140K tokens.

Every request is priced at its largest moment, idle conversations are released before anything is refused, and a refusal says what holds the memory. A real Pi compaction of a 205K-token Flash-Next session was served with no refusal and swap flat (#567).

Long sessions keep their cache.

In a Pi session on Flash-Next that grew to 204,981 tokens with 12 screenshots, 86 of 87 turns restored from RAM and all 50,769 verify steps ran compiled.

More room on 128 GB Macs.

The default memory limit is 90 GiB instead of 96, and

--memory-limit max

uses everything outside macOS's reserve on a Mac that runs only the server (#548).

Fixes.

The Metal shared event leak in long non-streaming runs (#544), JSON schema output that looped on whitespace (#547), NaN output with 4-bit and 8-bit KV cache (#526), the app badge stuck on Degraded (#528), screenshots after tool calls in streaming clients (#581), and

mtplx start

downloading a model you had already picked (#573).

Contributions.

First-token log-probabilities, faster tokenizing of agent turns, prompt scoring on Gemma 4 and more, by @jvmenen, @shiftedx, @pierre427, @vahako, @freddyhaddad and @bpmforge.

Ternary Bonsai 2 27B

Prism ML's ternary 27B with image input is 8.85 GB with the Qwen3.8-27B draft head and runs on 16 GB Macs. Two new GPU kernels make it faster than the 4-bit 27B: 64.4 against 52.6 tok/s after a 4,061-token prompt and 57.1 against 51.0 after a 16,350-token prompt, with a peak of 11.4 GB against 23.9 (#515).

MiMo V2.6 Qwen 9B

Xiaomi's coding and agent fine-tune of Qwen 3.5 9B, as an 8.70 GB 6-bit pack. On 16 to 31 GB Macs it is the second suggestion, after Bonsai 2.

Flash-Next reads prompts faster

On the same Mac and prompts as 2.11.3, the first token of a 4,061-token prompt arrives after 2.88 s instead of 5.26 s, and of a 65,502-token prompt after 60.1 s instead of 85.5 s, with decode 13 percent faster at that length. New prompt kernels add another 9 to 16 percent.

Flash-Next Optimized Quality

An 8-bit build for Macs with 256 GB or more, a 169.96 GB download. It has not run on a 256 GB Mac yet, so it is listed second there. Forge now converts Flash-Next source models (#508 by @bpmforge).

Sessions stay warm

Stop and send again reuses 10,918 of 10,928 prompt tokens instead of processing 26,294 again, a 149,649-token session restores instead of taking 570 s to the first token on every turn (#499), and a save to SSD pauses for a waiting request in under 200 ms (#505).

Cleaner answers and safer memory

Quoted tool markup no longer moves the thinking into the answer, Gemma 4 streamed replies finish again (#517), the repetition stop that cut a Tetris board is off by default, and the memory guards count memory outside the GPU allocator (#516, #500 by @ArctifoxNL).

Eight exactness defects fixed with tests, speculative decoding measured exact by token id on both flagship models, warm agent turns that restore the exact state, Flash-Next decode up to 27 percent faster at long context, 261k-token prompts on Flash-Next, a session cache that cleans itself, and a set of app and agent-harness repairs.

Exact speculative decoding, measured.

Eight defects in the decode path are fixed, each with a test. At temperature 0 and at the native sampler the fast path matches the plain path by token id on Flash-Next and on the 27B, and warm turns restore the state they were saved with.

Faster where it counts.

Flash-Next decodes a 9k-token code prompt at 79.3 tok/s against 62.5 on 2.11.2 on the same Mac, a 109k-token OpenCode turn went from 48.8 to 61.8 tok/s, and 261k-token prompts decode instead of running out of GPU memory (PR #482, davidtai).

Conversations with tools stay warm.

A 19k-token app chat with web search went from 15 s to the first token on every turn to 0.6 s, Hermes turns keep their cache across the model's own token boundaries, and a mid-conversation system message no longer re-prefills the history (#477).

The session cache looks after itself.

The SSD store reconciles every time it opens and eviction gives way to the request in flight (#493), mtplx gc sweeps it by hand (PR #502, ArctifoxNL), deep sessions persist on 64 GB Macs (PR #496, Dizzler7), and the idle timer is a setting (#481).

App, packs and agents.

The app keeps its port (#503) and no longer kills a busy daemon (#487), ordered model folders (PR #387, Philip John Basile), Turkish and thirteen localized languages (#470), a working draft head on 32 GB Macs (#483), Gemma 4 warm turns (PR #283, Craig Tollifson), OpenCode 2 (#498) and the Responses API (PR #219, Philip John Basile).

A correctness release for every Mac that is not an M5, seven session-bank and memory fixes for long agent sessions, native vision for Flash-Next in Hermes, OpenCode and Pi, a verifier-depth fix that restores Flash-Next's compiled verify route in agent turns, and a set of app and CLI repairs.

27B correct again on M1 to M4.

2.11 turned on a flash-decoding verify route that needs the M5 GPU's tensor units and shipped it without a hardware gate; from 8,192 tokens on, other Macs produced unrelated text, loops or another language. The route now runs only on an M5-class GPU on macOS 26.2 or newer; every other Mac serves the packed kernel from 2.10.2 (#459, #464, #467, #461, #469).

Long agent sessions keep their cache.

A deep conversation keeps its session at its turn boundaries (#446), later conversations are admitted after a restart (#454), AR-only runtimes restore again (#465), and a request that would cross the memory limit is refused with a structured 507 before prefill instead of swapping the Mac to a panic (#450, #447).

Flash-Next sees your agent's screenshots.

Hermes, OpenCode and Pi advertise image input when the pack has a vision tower, images keep sparse attention, and a warm image turn is admitted as warm: a fifteen-screenshot Hermes session completed at a 92 GiB peak.

Verifier depth from measured costs.

The expected-value depth policy measures draft and verify cost per depth on the running machine, so Flash-Next's compiled verify route carries 90 to 96 percent of cycles in agent turns instead of about 11.

App and CLI repairs.

Free space measured on the volume that holds your models (#466), a stall watchdog setting (#448), an Adaptive depth switch, a hero gauge that holds through tool loops, one-shot mtplx run on Flash-Next (#463), and the Hermes profile fixes (#460).

Honest memory refusals, a correct and resilient Anthropic bridge for Claude Code, and sharper stop diagnostics.

Honest memory refusals.

A prompt that cannot fit is answered upfront with a structured HTTP 507 naming the shortfall, instead of dying mid-prefill and being logged as a client cancellation.

Claude Code fixed twice.

Context accounting no longer double-counts cached prefixes on session-cache hits, and long first turns (large MCP toolsets reach 165k tokens) survive the client's 300-second stream timeout during prefill. A measured 165k-token first turn completes, then follow-ups serve 99.8 percent of the prompt from cache at 3.9 s to first token.

Explicit off means off.

An explicit compile kill-switch now wins over profile auto-arming everywhere, including the path that silently re-armed it.

Diagnosable stops.

Request logs record why a response ended: model EOS, stop sequence, length cap, or repetition stop.

Faster long-prompt processing on M4 and M5 Macs, working image input for Flash-Next, and fixes for 96 GB and M2/M3 machines.

Sparse prefill for Flash-Next.

A 98k-token prompt processes 35 percent faster at 8 GB lower peak memory, and a full 262k cold prompt completes in 355 s at 87 GB where 2.10.0 climbed to 119 GB and produced nothing.

Flash-Next reads images.

The packs always shipped their vision weights; the runtime now serves them in app chat, over the API, and in Pi.

96 GB and M2/M3 Macs.

The preload memory check scales with the machine so the packs load on 96 GB, and the kernels that crashed M2 and M3 GPUs now fall back automatically.

No more empty answers.

A first turn with tools declared that ended inside the reasoning channel now continues to a visible answer instead of returning an empty message.

The engine plans memory for your Mac, decode holds its speed deep into long context, Qwen3.8 Flash-Next runs native from day 0, and coding agents got their wall clock cut by two thirds.

Faster everywhere

Against stock 2.9.2 on the same Mac: +15 percent decode at 3k context, +29 at 88k, +54 at 147k, and +41 percent prefill.

Memory planned for your Mac

A 48 GB Mac resolves a context window that actually fits and serves 33 tok/s where 2.9.2 sat in swap at 3 to 4. The pressure banner now names whose pressure it is.

Qwen3.8 Flash-Next, native

The 125B MoE preview measures 61 tok/s plain and 63 to 76 with its own MTP head on an M5 Max. The 32 GB n-gram table streams from SSD, so both packs fit 96 GB Macs.

Agents finish faster and stop dying

The same multi-file task dropped from 150 to 44 seconds, mid-session first token is 0.11 s, and the run killers are fixed: dropped tools, phantom cancels, silent tool errors.

Long work holds up

KV q8 costs about 4 percent for double the context headroom, a 34k-token answer no longer decays from 86 to 25 tok/s, rewrites run 19 percent faster than fresh writes, and 100k-token sessions survive a restart.

Your transcript is yours: agent serving is passthrough by default, greedy decoding gets faster below 12k context, and the model forge gets a correctness fix that rescues collapsed draft acceptance.

Your transcript is yours.

MTPLX no longer compacts tool results, trims file reads, or injects steering text into agent transcripts unless you explicitly turn a rewrite on. The app and the CLI both stopped exporting the legacy compaction settings.

Faster greedy decode.

Chained greedy drafting is on by default for temperature 0 under 12k context: measured +2.5 to +9.8 percent across 0.5k to 8k prompts, fenced off where it lost.

Forge correctness.

The MTP norm convention is decided once per tensor set, rescuing packs whose draft acceptance had collapsed to 0 to 2 percent, and the runtime refuses a double-shifted trunk with a clear error.

Vision fixes.

Images survive message canonicalization on retried turns and near-prefix cache restores. Both were silent vision-drop bugs.

Your config is yours too.

Managed client configs only update files MTPLX wrote itself and never overwrite a config you have customized.

Agent coding sessions run to completion: long-context crash fixes, no hidden output caps, reasoning preserved across turns, and a built-in flight recorder.

Long sessions no longer die.

Fixed a paged-cache bug that could truncate and crash agent sessions near 19,000 tokens, and a shutdown segfault on quit.

No hidden output caps.

OpenCode, Pi, and Hermes injected default ceilings are stripped; explicit caps you set are honored on every lane.

The model thinks once.

Prior reasoning survives across agent turns instead of being re-derived, and warm turns reuse the cache instead of re-reading the whole session.

Turbo profile truth.

The turbo fast path now applies exactly what it advertises and reports it in /health — if you benchmarked turbo on 2.9.0, re-run it.

Flight recorder.

Every request records a per-second telemetry log, and mtplx trace turns any coding session into a full diagnosis report. Local-only, a few MB a day.

Faster responses, smoother streaming, smaller model downloads, and one-click model updates.

Faster generation.

Decode is 15–20% faster on typical workloads and up to 60% faster on code-heavy output.

Smoother streaming.

Visible freezes fell by 95%, and the worst measured stall dropped from 725 ms to 109 ms.

Much smaller updates.

Existing models now update with a 240–450 MB delta instead of downloading 15–21 GB again.

Smaller models, updated in one click.

Qwen 3.8 packs are up to 610 MB smaller, and the app now shows and installs model updates directly.

The streaming-quality release.

2.8.2's field reports kept coming back to one thing: chats that freeze for half a second and land in bursts — but only when a human was actually using the app. Hands-off testing stayed clean, which is exactly how it survived. 2.8.3 closes that whole class, plus every streaming bug found on the way to it.

Streaming stays smooth while you touch the app.

The window-measurement guard ran in a run-loop phase macOS skips while input events keep arriving — precisely when you scroll or move the mouse. Now it runs every turn: 40 s of continuous wheel-scrolling went from 70 UI stalls (18.7 s frozen) to one, and 30 s of cursor movement from 91 stalls to zero.

Scrolling up mid-generation no longer fights you.

User scrolling wins instantly — trackpad, momentum, and classic wheel mice alike — and following re-engages when you return to the bottom.

The transcript can't go blank, thinking is plain text, tables stream correctly.

The mid-generation blank-out, the self-rewriting reasoning ticker, and whitespace-free freezes (tables, URLs, minified code) are all fixed, with a quadratic detokenizer path closed on exactly that content.

Every request records a stream-smoothness census.

Producer gap percentiles land in every request record — including cancelled runs, which used to log nothing and were exactly the runs people complained about.

The 2.8 line.

This release is about trust: people started benchmarking MTPLX seriously and running long agentic sessions against it, and both groups found real problems. 2.8 closes that work, and 2.8.1 is the build that ships it to the desktop, together with a vision cache fix our own release gate caught the same morning.

Long agent sessions no longer stall their cache.

Sessions used to quietly stop reusing their prompt cache around 38k tokens and re-prefill a growing suffix every turn. The committed frontier now advances on every turn, tool-call turns included, so a 45k-token conversation keeps turn-delta prefills only.

Every number the server reports is one you can bench against.

mtplx_stats is always populated, temperature 0 is exact, the logprobs contract is parser-safe, and /health now reports any degradation (compiled-verify fallbacks, overridden profile keys, kernel bails) instead of looking like turbo while running slow.

A different image can never read another image's cache.

The 2.8.1 fix: cache keys for vision turns are derived from the actual image bytes, and the raw session frontier no longer commits image histories. Identical pixels still restore the full prefix; different pixels stop cold before the image.

Sharing the API over your network is a one-liner.

mtplx serve --host 0.0.0.0 --api-key-file ~/.mtplx/api-key
creates the key on first run and prints the LAN base URL other devices should use. Keyless non-localhost binds still refuse, on purpose.

256k context, for real.

The flagship's full 262,144-token window fits and runs on a 128 GB Mac: a 260k-token prompt completes end to end with MTP still engaged.

An emergency fix for

mtplx pull

. On 2.7.1 and older, re-pulling a model that changed upstream can corrupt your local copy. Upgrade before you pull.

Pull no longer corrupts files that changed upstream.

The downloader treated a complete local file whose size no longer matched the server as an interrupted download, and appended the remote tail onto the old content — corrupting

config.json

and the safetensors index. Stale files are now re-fetched whole; genuinely interrupted downloads still resume.

Already hit by it?
Delete the model's

config.json

and

model.safetensors.index.json

, then pull again on 2.7.2. The full notes have the details.

The Qwen 3.8 models can see again.

All six published 3.8 repos were re-published with their vision towers restored, and forge now grafts the tower on every build — failing closed rather than publishing blind. Existing installs pick the repair up as a ~0.9 GB delta.

A bug-fix release. It clears the known-issues list 2.7.0 shipped with.

xhigh stays selected.

Picking it in Inference settings snapped back to medium, and
mtplx config set reasoning_effort xhigh
was refused. Both places carried their own copy of the effort list and neither knew about xhigh, so the save was rejected whole and the picker reverted. Every place that accepts an effort level now reads the same list.

KV cache quantization reaches Qwen 3.8.

The toggle displayed q8 while the launch path recognized only Qwen 3.5 and 3.6.

Honest diagnostics.

mtplx doctor
names the model it actually checked, and turbo's profile note reports the real 32,768-token compiled-verify fence.

A new build can't offer you an older one.

The build number derived for 2.7.1 came out below the shipped 2.7.0, so a fresh install proposed 2.7.0 to itself.