MTPLX Release Notes

Honest memory refusals, a correct and resilient Anthropic bridge for Claude Code, and sharper stop diagnostics.

Honest memory refusals.

A prompt that cannot fit is answered upfront with a structured HTTP 507 naming the shortfall, instead of dying mid-prefill and being logged as a client cancellation.

Claude Code fixed twice.

Context accounting no longer double-counts cached prefixes on session-cache hits, and long first turns (large MCP toolsets reach 165k tokens) survive the client's 300-second stream timeout during prefill. A measured 165k-token first turn completes, then follow-ups serve 99.8 percent of the prompt from cache at 3.9 s to first token.

Explicit off means off.

An explicit compile kill-switch now wins over profile auto-arming everywhere, including the path that silently re-armed it.

Diagnosable stops.

Request logs record why a response ended: model EOS, stop sequence, length cap, or repetition stop.

Faster long-prompt processing on M4 and M5 Macs, working image input for Flash-Next, and fixes for 96 GB and M2/M3 machines.

Sparse prefill for Flash-Next.

A 98k-token prompt processes 35 percent faster at 8 GB lower peak memory, and a full 262k cold prompt completes in 355 s at 87 GB where 2.10.0 climbed to 119 GB and produced nothing.

Flash-Next reads images.

The packs always shipped their vision weights; the runtime now serves them in app chat, over the API, and in Pi.

96 GB and M2/M3 Macs.

The preload memory check scales with the machine so the packs load on 96 GB, and the kernels that crashed M2 and M3 GPUs now fall back automatically.

No more empty answers.

A first turn with tools declared that ended inside the reasoning channel now continues to a visible answer instead of returning an empty message.

The engine plans memory for your Mac, decode holds its speed deep into long context, Qwen3.8 Flash-Next runs native from day 0, and coding agents got their wall clock cut by two thirds.

Faster everywhere

Against stock 2.9.2 on the same Mac: +15 percent decode at 3k context, +29 at 88k, +54 at 147k, and +41 percent prefill.

Memory planned for your Mac

A 48 GB Mac resolves a context window that actually fits and serves 33 tok/s where 2.9.2 sat in swap at 3 to 4. The pressure banner now names whose pressure it is.

Qwen3.8 Flash-Next, native

The 125B MoE preview measures 61 tok/s plain and 63 to 76 with its own MTP head on an M5 Max. The 32 GB n-gram table streams from SSD, so both packs fit 96 GB Macs.

Agents finish faster and stop dying

The same multi-file task dropped from 150 to 44 seconds, mid-session first token is 0.11 s, and the run killers are fixed: dropped tools, phantom cancels, silent tool errors.

Long work holds up

KV q8 costs about 4 percent for double the context headroom, a 34k-token answer no longer decays from 86 to 25 tok/s, rewrites run 19 percent faster than fresh writes, and 100k-token sessions survive a restart.

Your transcript is yours: agent serving is passthrough by default, greedy decoding gets faster below 12k context, and the model forge gets a correctness fix that rescues collapsed draft acceptance.

Your transcript is yours.

MTPLX no longer compacts tool results, trims file reads, or injects steering text into agent transcripts unless you explicitly turn a rewrite on. The app and the CLI both stopped exporting the legacy compaction settings.

Faster greedy decode.

Chained greedy drafting is on by default for temperature 0 under 12k context: measured +2.5 to +9.8 percent across 0.5k to 8k prompts, fenced off where it lost.

Forge correctness.

The MTP norm convention is decided once per tensor set, rescuing packs whose draft acceptance had collapsed to 0 to 2 percent, and the runtime refuses a double-shifted trunk with a clear error.

Vision fixes.

Images survive message canonicalization on retried turns and near-prefix cache restores. Both were silent vision-drop bugs.

Your config is yours too.

Managed client configs only update files MTPLX wrote itself and never overwrite a config you have customized.

Agent coding sessions run to completion: long-context crash fixes, no hidden output caps, reasoning preserved across turns, and a built-in flight recorder.

Long sessions no longer die.

Fixed a paged-cache bug that could truncate and crash agent sessions near 19,000 tokens, and a shutdown segfault on quit.

No hidden output caps.

OpenCode, Pi, and Hermes injected default ceilings are stripped; explicit caps you set are honored on every lane.

The model thinks once.

Prior reasoning survives across agent turns instead of being re-derived, and warm turns reuse the cache instead of re-reading the whole session.

Turbo profile truth.

The turbo fast path now applies exactly what it advertises and reports it in /health — if you benchmarked turbo on 2.9.0, re-run it.

Flight recorder.

Every request records a per-second telemetry log, and mtplx trace turns any coding session into a full diagnosis report. Local-only, a few MB a day.

Faster responses, smoother streaming, smaller model downloads, and one-click model updates.

Faster generation.

Decode is 15–20% faster on typical workloads and up to 60% faster on code-heavy output.

Smoother streaming.

Visible freezes fell by 95%, and the worst measured stall dropped from 725 ms to 109 ms.

Much smaller updates.

Existing models now update with a 240–450 MB delta instead of downloading 15–21 GB again.

Smaller models, updated in one click.

Qwen 3.8 packs are up to 610 MB smaller, and the app now shows and installs model updates directly.

The streaming-quality release.

2.8.2's field reports kept coming back to one thing: chats that freeze for half a second and land in bursts — but only when a human was actually using the app. Hands-off testing stayed clean, which is exactly how it survived. 2.8.3 closes that whole class, plus every streaming bug found on the way to it.

Streaming stays smooth while you touch the app.

The window-measurement guard ran in a run-loop phase macOS skips while input events keep arriving — precisely when you scroll or move the mouse. Now it runs every turn: 40 s of continuous wheel-scrolling went from 70 UI stalls (18.7 s frozen) to one, and 30 s of cursor movement from 91 stalls to zero.

Scrolling up mid-generation no longer fights you.

User scrolling wins instantly — trackpad, momentum, and classic wheel mice alike — and following re-engages when you return to the bottom.

The transcript can't go blank, thinking is plain text, tables stream correctly.

The mid-generation blank-out, the self-rewriting reasoning ticker, and whitespace-free freezes (tables, URLs, minified code) are all fixed, with a quadratic detokenizer path closed on exactly that content.

Every request records a stream-smoothness census.

Producer gap percentiles land in every request record — including cancelled runs, which used to log nothing and were exactly the runs people complained about.

The 2.8 line.

This release is about trust: people started benchmarking MTPLX seriously and running long agentic sessions against it, and both groups found real problems. 2.8 closes that work, and 2.8.1 is the build that ships it to the desktop, together with a vision cache fix our own release gate caught the same morning.

Long agent sessions no longer stall their cache.

Sessions used to quietly stop reusing their prompt cache around 38k tokens and re-prefill a growing suffix every turn. The committed frontier now advances on every turn, tool-call turns included, so a 45k-token conversation keeps turn-delta prefills only.

Every number the server reports is one you can bench against.

mtplx_stats is always populated, temperature 0 is exact, the logprobs contract is parser-safe, and /health now reports any degradation (compiled-verify fallbacks, overridden profile keys, kernel bails) instead of looking like turbo while running slow.

A different image can never read another image's cache.

The 2.8.1 fix: cache keys for vision turns are derived from the actual image bytes, and the raw session frontier no longer commits image histories. Identical pixels still restore the full prefix; different pixels stop cold before the image.

Sharing the API over your network is a one-liner.

mtplx serve --host 0.0.0.0 --api-key-file ~/.mtplx/api-key
creates the key on first run and prints the LAN base URL other devices should use. Keyless non-localhost binds still refuse, on purpose.

256k context, for real.

The flagship's full 262,144-token window fits and runs on a 128 GB Mac: a 260k-token prompt completes end to end with MTP still engaged.

An emergency fix for

mtplx pull

. On 2.7.1 and older, re-pulling a model that changed upstream can corrupt your local copy. Upgrade before you pull.

Pull no longer corrupts files that changed upstream.

The downloader treated a complete local file whose size no longer matched the server as an interrupted download, and appended the remote tail onto the old content — corrupting

config.json

and the safetensors index. Stale files are now re-fetched whole; genuinely interrupted downloads still resume.

Already hit by it?
Delete the model's

config.json

and

model.safetensors.index.json

, then pull again on 2.7.2. The full notes have the details.

The Qwen 3.8 models can see again.

All six published 3.8 repos were re-published with their vision towers restored, and forge now grafts the tower on every build — failing closed rather than publishing blind. Existing installs pick the repair up as a ~0.9 GB delta.

A bug-fix release. It clears the known-issues list 2.7.0 shipped with.

xhigh stays selected.

Picking it in Inference settings snapped back to medium, and
mtplx config set reasoning_effort xhigh
was refused. Both places carried their own copy of the effort list and neither knew about xhigh, so the save was rejected whole and the picker reverted. Every place that accepts an effort level now reads the same list.

KV cache quantization reaches Qwen 3.8.

The toggle displayed q8 while the launch path recognized only Qwen 3.5 and 3.6.

Honest diagnostics.

mtplx doctor
names the model it actually checked, and turbo's profile note reports the real 32,768-token compiled-verify fence.

A new build can't offer you an older one.

The build number derived for 2.7.1 came out below the shipped 2.7.0, so a fresh install proposed 2.7.0 to itself.

Qwen3.8 support 🎉

Qwen3.8-27B came out on 14 August; this release runs it the way the model card says, with three MTPLX builds tuned for it and FP16 versions of all three for M1 and M2 Macs.

Qwen 3.8, served properly.

Official sampler (1.0 / 0.95 / 20), reasoning effort xhigh, medium and low with medium as the coding default, thinking preserved in history. Bare Speed (16.0 GB), Optimized Speed (20.4 GB, recommended) and Optimized Quality (29.4 GB), each with its calibration stamped in its own metadata.

New default.

Macs with 32 GB or more now default to Qwen 3.8 Optimized Speed; M1 and M2 get its FP16 build automatically. Under 32 GB still routes to the 9B.

Compiled verify to 32k.

The compiled verify window moves from 12k to 32k tokens of context: +6.9% at 20k on Qwen 3.8 Bare Speed, peak memory flat at 20k and lower at 30k.

Coding agents uncapped.

OpenCode and Pi no longer send an output cap for MTPLX models; Pi sessions restore their banked prefix from RAM.

Fixes.

The SSD session cache no longer walks its whole store on every write or health poll (idle CPU 35% down to 0.2% on an 816k-file bank); the macOS 27 slider crash is fixed (thanks @joshlacal); mtplx pull names the mirror knob when huggingface.co is unreachable.

Concurrency

Speculative decoding used to be a single-user feature: the moment two requests hit the daemon at once, everyone fell back to plain batching. 2.6.0 removes that trade-off.

Concurrent speculative decoding

--scheduler-mode mtp_batch serves independent requests through fixed-width MTP cohorts (three-wide and eight-wide) with per-row stats and honesty controls, and the session bank composes with it.

Embeddings and reranking

New /v1/embeddings and /v1/rerank endpoints (Cyb3rb1ade).

LiquidAI LFM2

LFM2 and LFM2.5 models run on MTPLX (davidtai).

Temperature-0 exactness

A real correctness fix to greedy decoding under prefill partitioning, plus a serial-lane sampling speedup.

Agent sessions got the attention this cycle, especially Pi. Warm turns stay warm.

Faster warm turns.

Tool turns restore from the stable boundary instead of re-processing ~200 tokens per round; a postcommit about to finish is briefly waited for instead of thrown away (a 1,449-token re-prefill became 436 tokens, first token 2.7 s down to 1.1 s).

No background stalls in your turn.

SSD cache work no longer slips into the gap between a request's internal jobs; the SSD tier skips candidates that cannot win.

The cache tells you what it is doing.

Resolved cache budget printed at startup, per-session and total (#229, #230).

A small release focused on the agent lane and the API surface, from a day of head-to-head benchmarking.

Agent requests no longer stall behind background cache work.

Background commits yield the moment any request is admitted, whichever session it belongs to (worst measured case before: a follow-up turn 44% slower).

Warm follow-ups got faster.

A byte-identical transcript hits an exact-match encode cache and gets 77 to 92 ms back per request.

API honesty.

Fixes for places where the API misled external tools.

Hotfix for the 2.5.1 long-response slowdown. Long answers no longer start fast and decay: the speculative verifier now hands off cleanly when a response outgrows its compiled window.

Long responses hold their speed.

The compiled-to-eager verifier handoff settles all state once at the ownership boundary instead of dragging unfinished GPU work through the rest of the answer.

Identical output.

The fix changes execution order only. Tokens, acceptance, and peak memory are unchanged, and the handoff is visible in verifier stats so it cannot regress silently.

Faster MLX for existing installs.

The minimum MLX version is now 0.32, converging older runtime environments to the stack fresh installs already run.