raullenchai Release Notes

What's new in v0.16.5

Rapid-MLX 0.16.5 adds an experimental mixed-precision DeepSeek V4.1 Flash build for Macs with at least 224 GB of memory, makes streamed text match buffered replies when tool-call markup is split across tokens or rides on a tool-call chunk, and checks authentication before image and video uploads are read. Desktop chats in Chinese get Chinese titles and identity, and interrupted replies no longer vanish.

Models

DeepSeek V4.1 Flash 1017 (experimental): new alias deepseek-v41-flash-1017 serves rapid-mlx/DeepSeek-V4.1-Flash-1017-MLX at a pinned revision. It needs at least 224 GB of memory and is greedy-only and text-only, with no tool calling; inputs are capped at 8,192 tokens and outputs at 4,096 tokens. Decode runs at about 19 tok/s on M3 Ultra. (#4953)

Desktop chat

Interrupted replies stay visible: switching chats while a reply is being written keeps the partial reply, marked as stopped with Retry, instead of letting it vanish. Earlier interrupted replies are restored on launch, and chat titles still arrive after you switch away. (#4940)

Chinese identity and titles: the default assistant instruction now includes a Chinese line, so Chinese greetings describe what the assistant can write, translate, and explain, and tool answers to Chinese questions stay in Chinese more often (#4929). Chat titles follow the language of your first message, and follow-up suggestions are written in your voice rather than as the assistant offering help (#4944).

Calmer date and weather answers: date and time questions no longer show the "may be a guess" caution, and a weather question about a state or region shows a quiet "needs a more specific place" note instead of a red failure card. Real tool failures still show the red card. (#4945)

Server and API

Streamed tool-call markup no longer leaks: streaming /v1/chat/completions no longer shows tool-call markup such as <|tool_call>…<tool_call|> or [Calling tool: …] when it is split across tokens, so streamed text matches the buffered reply. (#4950)

Streamed logprobs stay with their text: with logprobs: true, text held back and released just before a structured tool call carries only its own tokens' logprobs. (#4960)

Streaming text parity on Responses and Messages: streaming /v1/responses and /v1/messages no longer depend on how text is chunked; text that arrives on the same chunk as a tool call is now streamed, and /v1/messages keeps leading whitespace. (#4963)

Buffered replies keep tool-like text: buffered /v1/responses and /v1/messages replies return tool-call-like text unchanged when no tools are offered or tool_choice is none, matching streaming. (#4927)

Large tool lists on native-XML models: with tool_choice: "auto", models whose chat template writes tool calls in native XML (such as Qwen3.5) no longer return HTTP 400 for tool lists over the grammar limits; such requests use normal tool-call parsing, up to 1024 tools, 256 KiB or depth 64. (#4928)

Authentication before uploads are read: with --api-key set, /v1/images/edits answers an unauthenticated request with 401 before size limits or multipart parsing run (#4924), and the JSON depth check no longer reads image-edit uploads before authentication; the upload size cap applies to POST only (#4962). Video upload endpoints parse API keys like every other route (#4949), and POST /v1/videos/extend accepts uploads up to the video route's own limit and answers unauthenticated large uploads with 401 instead of 413 (#4955).

TensorFold timeouts release the server: on the default serial TensorFold path, a non-streaming request that times out cancels its generation, and shutdown cancels in-flight jobs, so later requests and process exit no longer hang. (#4932)

All changes

• chore: bump version to 0.16.5 (#4968) (5897aa8)
• feat(deepseek-v41): serve the mixed-precision DeepSeek-V4.1-Flash-1017 build (#4953) (764bb2c)
• Revert "fix(desktop): clearer cached-model rows in first-run setup (#4964)" (#4967) (0585473)
• fix(api): align streaming and buffered text on Responses and Messages (#4963) (ec83784)
• fix(desktop): clearer cached-model rows in first-run setup (#4964) (03345fa)
• fix: authenticate image edit uploads before the depth gate reads them (#4962) (2c3d25e)
• chore(mirror): retire three owner-approved R2 aliases (#4961) (f0b506d)
• fix(chat): keep tool-call logprobs off held text released before the call (#4960) (58d557b)
• fix: exempt video extend uploads from the generic body cap (#4955) (029ab04)
• fix(chat): hold back split tool-call markup in streaming content (#4950) (2619e87)
• fix(server): share bearer parsing in video upload auth gate (#4923) (#4949) (76f0a12)
• test: stub port preflight in audio-fork serve_command test (#4947) (dfc2307)
• fix(desktop): chat titles follow the user's language; chips stay in the user's voice (#4944) (c0e6d38)
• fix(desktop): no guess caution on date questions, no red card for weather clarifications (#4945) (a56a065)
• fix(tools): skip the schema-size 400 when auto grammar is declined (#4928) (abb927d)
• test(desktop): make chat-depth golden baselines locale-independent (#4943) (0556f82)
• test: pin host memory for MLLM exact-prefix cache tests (#4942) (4547bbb)
• fix(tensorfold): cancel serial non-stream jobs on timeout and close (#4932) (7a64e83)
• fix(desktop): keep interrupted replies visible and title chats after a switch (#4940) (0c2de48)
• fix(desktop): add a Chinese line to the default assistant instruction (#4929) (d1a7e18)
• fix(images): authenticate image edits before body limit and parsing (#4924) (eff0f45)
• fix(responses,messages): keep tool-like text in buffered replies when no tools parse (#4927) (c8e9d51)

Install: brew upgrade rapid-mlx or pip install -U rapid-mlx==0.16.5 (or just rapid-mlx upgrade).

What's new in v0.16.4

Rapid-MLX 0.16.4 makes Desktop chat simpler and more dependable: Personal Intelligence is replaced by the ordinary Chat path, tool-call replies are no longer empty or full of raw markup, weather and web tools ask less and answer more, and you can attach Word documents. The server gains /v1/rerank, a cache-warming API, MiniCPM-V 4.7 and LFM2.5 support, and faster concurrent and multimodal decoding.

Changes to note

• Personal Intelligence removed: Desktop now uses the ordinary Chat path. Local file and command tools are gone from Chat; the built-in tools are Web Search, Browse Web Page, Weather, and Read Attached Document again, plus any connectors you add. Files you created are not touched, and the privacy copy now explains when online tools send data to their services. /v1/models cards no longer include personal_intelligence_profile or personal_intelligence_qualification; /v1/agent/runs remains available. (#4860, #4865)
• Image endpoints require the API key: when the server runs with --api-key, every /v1/images route now requires it. (#4851)
• Bounded retrieval requests: rerank requests now default to at most 1000 documents and embedding requests to 2048 inputs; larger requests return HTTP 400 (raise or disable the limits with --rerank-max-documents / --embedding-max-inputs). Empty rerank documents or instructions return 400, and a full retrieval queue returns 503 with Retry-After: 1. jina-reranker-v3 also scores large requests faster. (#4839)

Desktop chat

• No more empty replies or raw tool-call markup: Qwen models using native XML tool calls no longer produce empty replies, and Desktop gives accurate retry guidance when a reply stops early (#4876). Chats that reach their tool budget show an answer or a clear fallback instead of raw markup (#4879), and an unreadable tool request is retried once before Desktop explains that live information could not be checked (#4885).
• A general assistant by default: Desktop sends a short default assistant instruction, so the starter model describes itself as a general assistant for writing, translation, explanation, and code rather than only its tools. It appears in the Effective System Prompt preview, and your custom instructions still apply after it. (#4903)
• Fewer unrelated tool calls: plain writing and coding chats set off far fewer unrelated web searches and page visits, and the document reader is offered only when a file is attached (#4875). The per-message local time is now a neutral note in its own row before your message, so email drafts are no longer answered as if already sent, and tool replies keep the language of Chinese questions. Chat templates that require strict user/assistant alternation now accept consecutive user turns (#4861, #4894).
• Titles and follow-ups for starter chats: default starter conversations now get generated titles and follow-up suggestions. (#4908)
• Word documents in chat: attach, drop, or paste a .docx file and ask about it. Rapid reads only the document text, with strict size and time limits; files it cannot read safely get a message suggesting PDF or TXT. (#4895)
• Pasting a file attaches it: pasting a file into the composer attaches it, or explains which file types are supported, instead of inserting its local path. (#4867)
• Start & Send: when the chosen chat model is downloaded but stopped, Send starts it and sends your draft once it is ready; the draft stays put if startup is cancelled or fails (#4878). With an empty composer, the disabled button explains in its tooltip and to VoiceOver that you type a message first (#4906).
• Drafts stay with their conversation: unsent text is saved locally across restarts, with a separate draft for New Chat. (#4866, #4886)
• Bold and italic in Chinese, Japanese, and Korean replies: emphasis that starts or ends on full-width punctuation, such as **注意：**请, now renders instead of showing literal **. (#4913)
• One approval per web page fetch: a redirect between a site's bare domain and www., or from http to https on the same host, no longer asks for approval twice. A redirect to a different site still asks, and the prompt says which site redirected. (#4919)
• Cleaner first chat: finishing setup no longer drops a setup message into your new conversation, and the empty chat no longer shows a Star on GitHub button. (#4905)
• Private files are owner-only from creation: chat history, folders, drafts, connector settings, and local caches are written with owner-only permissions from the start instead of being tightened after the write. (#4904)
• Generated videos play in the app: clicking a finished video's preview plays it instead of quitting Desktop. (#4764)

Weather

• Daily forecasts: tomorrow, next-few-days, and this-weekend questions use the weather data source with local dates and weekdays instead of fetching web pages, and Settings describes the tool as current conditions plus a daily forecast of up to 7 days. (#4893, #4920)
• More places understood: US, Canadian, and Australian state and province abbreviations and country aliases such as USA and UK are recognized (#4898). Asking about a state or region tells the model to pick or ask for a city instead of retrying (#4915). Chinese cities such as 德州 (Dezhou) are found, province names such as 山东省 or "Shandong Province" are accepted, and a call naming only a state or province no longer fails as unreadable (#4921).

Server and API

• Document reranking with /v1/rerank: start the server with --rerank-model <hf-id> to score and sort documents against a query (Qwen3-Reranker, mxbai-rerank-v2, jina-reranker-v3, and BGE reranker v2 cross-encoders such as bge-reranker-v2-m3). It runs alongside a chat model, an embedding model, or on its own, and embedding and rerank work no longer blocks other requests or health checks. Contributed by @weizenspreu. (#4832)
• Warm the cache before a request: POST /v1/cache/prefill prefills a fixed system and tools prefix or a revisioned draft without generating text, so the next request starts sooner (#4884). The exact system-and-tools prefix is cached so an agent's first real turn reuses it (#4883).
• Consistent tool-call text handling: buffered replies keep model-emitted tool-call markup as text when no tools were offered, matching streaming (#4887), and /v1/responses does the same under tool_choice: "none" (#4902). Unfinished tool payloads stay out of visible replies (#4774), fenced XML tool-call examples, including ones with CR or CRLF line endings, stay as content (#4775, #4802), and Nemotron prose containing a literal closing tag after a finished call is no longer withheld (#4768).
• /v1/responses honors max_tokens: it is treated as max_output_tokens instead of being ignored; conflicting values return 400. (#4815)
• Oversized uploads get an HTTP 413: the server drains the rest of an oversized body (at most about 64 MiB, within 10 s) and returns a JSON error naming the cap and --max-request-bytes, instead of resetting the connection. (#4831)
• Accurate health after idle unload: /healthz reports model_loaded: false while an idle-unloaded model is in standby, matching /metrics, and unload and reload transitions are logged. (#4834)
• Local pages work from localhost and 127.0.0.1 alike: a page served from http://localhost:<port> can call the server on the same port when it is addressed as 127.0.0.1 or [::1]; other origins stay blocked. (#4816)
• Agent tests are safer: rapid-mlx agents codex --test uses the address saved by --setup (#4807), and agents --test no longer opens browsers or apps during a sign-in flow (#4882).
• TensorFold lane setting is never silently ignored: RAPID_MLX_TENSORFOLD_LANES above 1, or an invalid value, on a model not served through a qualified TensorFold profile now stops rapid-mlx serve at startup with an error. (#4808)
• MTP defaults work with copied model caches: aliases resolve a sole cached snapshot without refs/main, and unavailable or ambiguous configs explain why plain decode is used. (#4801)
• PFlash threshold warning: the server warns at startup when --pflash-threshold-basis uncached cannot apply and falls back to the total prompt length; the CLI reference now documents the sink/tail token budget. (#4767)
• Cache prefill works on Python 3.10: prefill cancellation no longer relies on a Python 3.11-only call. (#4900)

Models

• MiniCPM-V 4.7 (experimental): minicpm-v4.7-1b-8bit and minicpm-v4.7-35b-a3b-mxfp4 serve through the multimodal lane with tool calling and reasoning parsing preconfigured. (#4745)
• LFM2.5-24B-A2B: rapid-mlx serve lfm2.5-24b-a2b-4bit loads LiquidAI's official 4-bit MLX build with tool calling and reasoning parsing preconfigured. (#4852)

Performance

• Long prompts prefill concurrently when memory allows: the scheduler no longer over-estimates long-prompt prefill memory (about 300 GB estimated vs about 45 GB measured for a 32k-token prompt on a 35B MoE hybrid model), so concurrent 32k-token requests are no longer forced to run one at a time. (#4835)
• Faster text on multimodal Qwen MoE models: text-only requests on the default vision-capable lane of Qwen3.6-35B-A3B-class checkpoints keep single-request caches, use fused MoE and GDN projections, and no longer run slower than --no-mllm. Short 2–8-token blocks produce byte-identical output to the unfused model. (#4825, #4837, #4841)
• Vision-model servers keep text prefix caches across restarts: text-only prompt-prefix caches are restored after a restart; image-bearing prefixes are never persisted. (#4829)
• Faster opt-in MTP cohorts under load: continuous self-MTP forms a cohort only with at most 3 live requests and never grows past 3 lanes; with 4 or more concurrent requests the rest use plain batched decode, measured 23–24% faster for short prompts and 6–13% faster at 8k–16k-token prompts than the previous cohort. Cohort rollback overhead is lower too. (#4822, #4777)

Packaging

• Smaller Desktop app: the bundled runtime no longer ships build-only headers, test suites, or Intel code slices, and only the math font the app renders with is included — about 14 MB off the DMG and more than 110 MB and 11,000 files off the installed app. (#4797, #4798)
• Local Search works on stock Macs: the default index location and index folders under /tmp or /var no longer fail with a 503 because of standard macOS deny ACLs or directory aliases; unsafe ancestor permissions still fail with a message naming the path. (#4814)
• Packaged license notices: installed packages include the license and notice files for the vendored speculative-decoding (MTP) code and the MiniCPM-V 4.7 vision model code. (#4812)

All changes

• chore: bump version to 0.16.4 (#4922) (e014e81)
• fix(desktop): resolve Chinese cities and region-only weather calls (#4921) (6ff6f53)
• fix(desktop): tell the model when a weather location is a state or region (#4915) (5e923d3)
• fix(desktop): create private store files owner-only from the start (#4904) (7df53b1)
• fix(desktop): describe weather tool as current conditions plus forecast (#4920) (8c97392)
• fix(desktop): do not re-ask approval for same-site www redirects (#4919) (87d73de)
• fix(mac): explain disabled Start & Send with an empty composer (#4906) (de7b2d4)
• fix(mac): drop seeded setup turn and empty-state star button (#4905) (7bab11e)
• fix(desktop): render CJK emphasis next to full-width punctuation (#4913) (9888d6c)
• fix(desktop): gate background titles on live text concurrency (#4908) (fbfb9fc)
• fix(responses): pass tool-call markup through as text under tool_choice none (#4902) (de4dc90)
• fix(desktop): send a short default assistant instruction (#4873) (#4903) (fe6d9a3)
• fix(engine): make prefill cancellation checks Python 3.10 compatible (#4900) (fcc834c)
• test: restore mac-mini full_unit baseline (#4897) (bc0f969)
• fix(desktop): send the local-time note as its own row (#4894) (c9b4c80)
• feat(desktop): attach Word .docx files with bounded text extraction (#4895) (922af62)
• fix(desktop): accept state abbreviations and country aliases in weather lookup (#4898) (96039e7)
• fix(desktop): add a bounded daily forecast to the weather tool (#4893) (85a10bb)
• Add bounded prefix and draft prefill API (#4884) (65eec0d)
• test(desktop): refresh golden AX snapshots for Start & Send (#4890) (313c44b)
• Cache exact system and tools prefix for first agent turn (#4883) (90b2540)
• fix(agents): agents --test no longer opens browsers or apps (#4882) (3ce532d)
• fix(tools): do not parse tool calls for requests that offer no tools (#4887) (25d474a)
• fix(desktop): recover unreadable tool requests in chat (#4885) (365157e)
• fix(desktop): keep the composer editor alive across conversation switches (#4886) (5c931cf)
• fix(desktop): Start & Send for a downloaded, stopped model (#4878) (758c916)
• fix(tools): keep auto tool grammar on the template's native wire (#4876) (28bbd6b)
• fix(desktop): tool-budget final round answers instead of showing raw tool-call markup (#4879) (a39aa8b)
• fix(desktop): stop priming needless tool calls on plain chat turns (#4875) (3eb8c2c)
• fix(desktop): attach or explain pasted files instead of inserting their paths (#4867) (4a1fea1)
• fix(desktop): keep unsent drafts with their conversation (#4866) (33cbe69)
• refactor(desktop): restore pre-Personal-Intelligence built-in tool set (#4865) (2db268a)
• feat(models): add lfm2.5-24b-a2b-4bit alias (#4852) (4dd48a5)
• fix(desktop): stop framing per-turn clock context as a sent message (#4861) (329fed0)
• refactor: remove Personal Intelligence; local tools off by default in Chat (#4860) (1c82eac)
• fix(server): require the API key on every /v1/images route (#4851) (fc31434)
• ops(ci-runners): document the 60 s launchd stop bound and upgrade procedure (#4843) (099b723)
• ops(ci-runners): harden tart runner scripts (#4840) (5028e16)
• fix(moe): make fused short-block gate/up byte-identical to stock (#4841) (a7f40e1)
• fix(rerank): bound retrieval requests and speed up listwise scoring (#4839) (f7270e8)
• perf(qwen): fuse GDN input projections on the multimodal language module (#4837) (44ca17f)
• feat(models): add /v1/rerank and --rerank-model (carries #4558) (#4832) (3918e7c)
• fix(scheduler): stop over-estimating long-prompt prefill workspace (#4824) (#4835) (39ca237)
• fix(server): keep /healthz model_loaded in sync with idle unload (#4834) (13e17ee)
• fix(mllm): persist text-only vision-lane prefix cache across restarts (#4829) (897ffab)
• fix(server): return 413 instead of resetting oversized uploads (#4828) (#4831) (9709491)
• perf(qwen): close most of the default-lane text-only gap on multimodal Qwen MoE (#4823) (#4825) (893508e)
• perf(mtp): cap continuous self-MTP cohorts at 3 concurrent requests (#4795) (#4822) (d84585a)
• fix(local-search): accept stock macOS ancestor ACLs and symlinked index paths (#4814) (7d8caf1)
• docs(ops): runbook and scripts for ephemeral tart Linux CI runners (#4817) (7116ded)
• fix(packaging): ship MTP and MiniCPM-V 4.7 license notices (#4812) (f1d1332)
• ci: route only test-matrix shards to CI_LINUX_RUNNER; light jobs get their own switch (#4819) (429e19f)
• fix(mtp): resolve alias config from a sole cached snapshot without refs/main (#4801) (6c3951c)
• fix(agents): codex --test uses the endpoint saved by --setup (#4807) (586794f)
• fix(server): treat loopback Origin spellings as same-origin (#4816) (231e950)
• fix(responses): honor max_tokens as an alias of max_output_tokens (#4815) (3de1c6c)
• ci: add opt-in Linux runner variable and single-Python PR matrix (#4813) (bd3e1e5)
• fix(serve): refuse RAPID_MLX_TENSORFOLD_LANES outside a TensorFold lane (#4808) (57171f5)
• build(desktop): ship only the math font the app renders with (#4798) (f1481e0)
• build(desktop): drop build-only payload from the bundled runtime (#4797) (48abe80)
• fix(ci): tighten main guard revert attribution and nightly recovery (#4804) (21a79a2)
• test: cover in-place server global mutations and soak stream diagnostics (#4803) (fea648b)
• fix(tools): treat CR and CRLF as line boundaries in Markdown fence tracking (#4802) (aa2eeea)
• chore: stop bundling integration tests no agent profile can run (#4792) (f27f697)
• chore: remove unreferenced functions and methods (#4791) (aecd9fd)
• refactor(desktop): remove unreferenced Swift views and helpers (#4800) (3ca297c)
• build(desktop): strip the build-machine debug map from the app executable (#4799) (e149412)
• fix(tools): keep Nemotron closing-tag prose after a completed call (#4768) (946e82d)
• fix(tools): keep fenced XML tool-call examples as content (#4775) (023cf4d)
• fix(pflash): warn when the uncached threshold basis cannot apply; document sink/tail budget (#4767) (7c2fe12)
• ci: remove the retired merge queue and its candidate routes (#4790) (142032d)
• perf(mtp): cut continuous cohort rollback overhead (#4589) (#4777) (e38cae0)
• ci: add opt-in merge-on-green merger, main guard, and nightly runs (#4773) (a3bc677)
• fix(tools): hide unfinished tool payloads behind literal markers and forced calls (#4774) (7d0a79d)
• fix(scheduler): release slotted generator and batch state on retirement (#4772) (5e7e3d8)
• test: restore server launch globals between tests (#4751) (581e5e7)
• test(soak): record response details for empty-answer errors (#4639) (0967835)
• test(pr_validate): size default xdist workers by host load (#4754) (ca50937)
• test(pr_validate): stop before tests when codex review blocks (#4750) (965a370)
• ci: skip desktop GUI jobs when the diff cannot affect the app (#4752) (7133d3e)
• ci: remove the automatic spam-suspected issue labeler (#4766) (c2dc90f)
• fix(desktop): play video previews with AVPlayerView to stop the AVKit SwiftUI crash (#4764) (7be1c00)
• feat(models): add MiniCPM-V 4.7 1B and 35B-A3B support (#4745) (1429558)
• docs(release-notes): add opt-in TensorFold concurrent decoding to 0.16.3 (#4763) (f86660d)

Install: brew upgrade rapid-mlx or pip install -U rapid-mlx==0.16.4 (or just rapid-mlx upgrade).

[0.16.5] — 2026-10-19

Rapid-MLX 0.16.5 adds an experimental mixed-precision DeepSeek V4.1 Flash build for Macs with at least 224 GB of memory, makes streamed text match buffered replies when tool-call markup is split across tokens or rides on a tool-call chunk, and checks authentication before image and video uploads are read. Desktop chats in Chinese get Chinese titles and identity, and interrupted replies no longer vanish.

Models

• DeepSeek V4.1 Flash 1017 (experimental): new alias deepseek-v41-flash-1017 serves rapid-mlx/DeepSeek-V4.1-Flash-1017-MLX at a pinned revision. It needs at least 224 GB of memory and is greedy-only and text-only, with no tool calling; inputs are capped at 8,192 tokens and outputs at 4,096 tokens. Decode runs at about 19 tok/s on M3 Ultra. (#4953)

Desktop chat

• Interrupted replies stay visible: switching chats while a reply is being written keeps the partial reply, marked as stopped with Retry, instead of letting it vanish. Earlier interrupted replies are restored on launch, and chat titles still arrive after you switch away. (#4940)
• Chinese identity and titles: the default assistant instruction now includes a Chinese line, so Chinese greetings describe what the assistant can write, translate, and explain, and tool answers to Chinese questions stay in Chinese more often (#4929). Chat titles follow the language of your first message, and follow-up suggestions are written in your voice rather than as the assistant offering help (#4944).
• Calmer date and weather answers: date and time questions no longer show the "may be a guess" caution, and a weather question about a state or region shows a quiet "needs a more specific place" note instead of a red failure card. Real tool failures still show the red card. (#4945)

Server and API

• Streamed tool-call markup no longer leaks: streaming /v1/chat/completions no longer shows tool-call markup such as <|tool_call>…<tool_call|> or [Calling tool: …] when it is split across tokens, so streamed text matches the buffered reply. (#4950)
• Streamed logprobs stay with their text: with logprobs: true, text held back and released just before a structured tool call carries only its own tokens' logprobs. (#4960)
• Streaming text parity on Responses and Messages: streaming /v1/responses and /v1/messages no longer depend on how text is chunked; text that arrives on the same chunk as a tool call is now streamed, and /v1/messages keeps leading whitespace. (#4963)
• Buffered replies keep tool-like text: buffered /v1/responses and /v1/messages replies return tool-call-like text unchanged when no tools are offered or tool_choice is none, matching streaming. (#4927)
• Large tool lists on native-XML models: with tool_choice: "auto", models whose chat template writes tool calls in native XML (such as Qwen3.5) no longer return HTTP 400 for tool lists over the grammar limits; such requests use normal tool-call parsing, up to 1024 tools, 256 KiB or depth 64. (#4928)
• Authentication before uploads are read: with --api-key set, /v1/images/edits answers an unauthenticated request with 401 before size limits or multipart parsing run (#4924), and the JSON depth check no longer reads image-edit uploads before authentication; the upload size cap applies to POST only (#4962). Video upload endpoints parse API keys like every other route (#4949), and POST /v1/videos/extend accepts uploads up to the video route's own limit and answers unauthenticated large uploads with 401 instead of 413 (#4955).
• TensorFold timeouts release the server: on the default serial TensorFold path, a non-streaming request that times out cancels its generation, and shutdown cancels in-flight jobs, so later requests and process exit no longer hang. (#4932)

[0.16.4] — 2026-10-18

Rapid-MLX 0.16.4 makes Desktop chat simpler and more dependable: Personal Intelligence is replaced by the ordinary Chat path, tool-call replies are no longer empty or full of raw markup, weather and web tools ask less and answer more, and you can attach Word documents. The server gains /v1/rerank, a cache-warming API, MiniCPM-V 4.7 and LFM2.5 support, and faster concurrent and multimodal decoding.

Changes to note

• Personal Intelligence removed: Desktop now uses the ordinary Chat path. Local file and command tools are gone from Chat; the built-in tools are Web Search, Browse Web Page, Weather, and Read Attached Document again, plus any connectors you add. Files you created are not touched, and the privacy copy now explains when online tools send data to their services. /v1/models cards no longer include personal_intelligence_profile or personal_intelligence_qualification; /v1/agent/runs remains available. (#4860, #4865)
• Image endpoints require the API key: when the server runs with --api-key, every /v1/images route now requires it. (#4851)
• Bounded retrieval requests: rerank requests now default to at most 1000 documents and embedding requests to 2048 inputs; larger requests return HTTP 400 (raise or disable the limits with --rerank-max-documents / --embedding-max-inputs). Empty rerank documents or instructions return 400, and a full retrieval queue returns 503 with Retry-After: 1. jina-reranker-v3 also scores large requests faster. (#4839)

Desktop chat

• No more empty replies or raw tool-call markup: Qwen models using native XML tool calls no longer produce empty replies, and Desktop gives accurate retry guidance when a reply stops early (#4876). Chats that reach their tool budget show an answer or a clear fallback instead of raw markup (#4879), and an unreadable tool request is retried once before Desktop explains that live information could not be checked (#4885).
• A general assistant by default: Desktop sends a short default assistant instruction, so the starter model describes itself as a general assistant for writing, translation, explanation, and code rather than only its tools. It appears in the Effective System Prompt preview, and your custom instructions still apply after it. (#4903)
• Fewer unrelated tool calls: plain writing and coding chats set off far fewer unrelated web searches and page visits, and the document reader is offered only when a file is attached (#4875). The per-message local time is now a neutral note in its own row before your message, so email drafts are no longer answered as if already sent, and tool replies keep the language of Chinese questions. Chat templates that require strict user/assistant alternation now accept consecutive user turns (#4861, #4894).
• Titles and follow-ups for starter chats: default starter conversations now get generated titles and follow-up suggestions. (#4908)
• Word documents in chat: attach, drop, or paste a .docx file and ask about it. Rapid reads only the document text, with strict size and time limits; files it cannot read safely get a message suggesting PDF or TXT. (#4895)
• Pasting a file attaches it: pasting a file into the composer attaches it, or explains which file types are supported, instead of inserting its local path. (#4867)
• Start & Send: when the chosen chat model is downloaded but stopped, Send starts it and sends your draft once it is ready; the draft stays put if startup is cancelled or fails (#4878). With an empty composer, the disabled button explains in its tooltip and to VoiceOver that you type a message first (#4906).
• Drafts stay with their conversation: unsent text is saved locally across restarts, with a separate draft for New Chat. (#4866, #4886)
• Bold and italic in Chinese, Japanese, and Korean replies: emphasis that starts or ends on full-width punctuation, such as **注意：**请, now renders instead of showing literal **. (#4913)
• One approval per web page fetch: a redirect between a site's bare domain and www., or from http to https on the same host, no longer asks for approval twice. A redirect to a different site still asks, and the prompt says which site redirected. (#4919)
• Cleaner first chat: finishing setup no longer drops a setup message into your new conversation, and the empty chat no longer shows a Star on GitHub button. (#4905)
• Private files are owner-only from creation: chat history, folders, drafts, connector settings, and local caches are written with owner-only permissions from the start instead of being tightened after the write. (#4904)
• Generated videos play in the app: clicking a finished video's preview plays it instead of quitting Desktop. (#4764)

Weather

• Daily forecasts: tomorrow, next-few-days, and this-weekend questions use the weather data source with local dates and weekdays instead of fetching web pages, and Settings describes the tool as current conditions plus a daily forecast of up to 7 days. (#4893, #4920)
• More places understood: US, Canadian, and Australian state and province abbreviations and country aliases such as USA and UK are recognized (#4898). Asking about a state or region tells the model to pick or ask for a city instead of retrying (#4915). Chinese cities such as 德州 (Dezhou) are found, province names such as 山东省 or "Shandong Province" are accepted, and a call naming only a state or province no longer fails as unreadable (#4921).

Server and API

• Document reranking with /v1/rerank: start the server with --rerank-model <hf-id> to score and sort documents against a query (Qwen3-Reranker, mxbai-rerank-v2, jina-reranker-v3, and BGE reranker v2 cross-encoders such as bge-reranker-v2-m3). It runs alongside a chat model, an embedding model, or on its own, and embedding and rerank work no longer blocks other requests or health checks. Contributed by @weizenspreu. (#4832)
• Warm the cache before a request: POST /v1/cache/prefill prefills a fixed system and tools prefix or a revisioned draft without generating text, so the next request starts sooner (#4884). The exact system-and-tools prefix is cached so an agent's first real turn reuses it (#4883).
• Consistent tool-call text handling: buffered replies keep model-emitted tool-call markup as text when no tools were offered, matching streaming (#4887), and /v1/responses does the same under tool_choice: "none" (#4902). Unfinished tool payloads stay out of visible replies (#4774), fenced XML tool-call examples, including ones with CR or CRLF line endings, stay as content (#4775, #4802), and Nemotron prose containing a literal closing tag after a finished call is no longer withheld (#4768).
• /v1/responses honors max_tokens: it is treated as max_output_tokens instead of being ignored; conflicting values return 400. (#4815)
• Oversized uploads get an HTTP 413: the server drains the rest of an oversized body (at most about 64 MiB, within 10 s) and returns a JSON error naming the cap and --max-request-bytes, instead of resetting the connection. (#4831)
• Accurate health after idle unload: /healthz reports model_loaded: false while an idle-unloaded model is in standby, matching /metrics, and unload and reload transitions are logged. (#4834)
• Local pages work from localhost and 127.0.0.1 alike: a page served from http://localhost:<port> can call the server on the same port when it is addressed as 127.0.0.1 or [::1]; other origins stay blocked. (#4816)
• Agent tests are safer: rapid-mlx agents codex --test uses the address saved by --setup (#4807), and agents --test no longer opens browsers or apps during a sign-in flow (#4882).
• TensorFold lane setting is never silently ignored: RAPID_MLX_TENSORFOLD_LANES above 1, or an invalid value, on a model not served through a qualified TensorFold profile now stops rapid-mlx serve at startup with an error. (#4808)
• MTP defaults work with copied model caches: aliases resolve a sole cached snapshot without refs/main, and unavailable or ambiguous configs explain why plain decode is used. (#4801)
• PFlash threshold warning: the server warns at startup when --pflash-threshold-basis uncached cannot apply and falls back to the total prompt length; the CLI reference now documents the sink/tail token budget. (#4767)
• Cache prefill works on Python 3.10: prefill cancellation no longer relies on a Python 3.11-only call. (#4900)

Models

• MiniCPM-V 4.7 (experimental): minicpm-v4.7-1b-8bit and minicpm-v4.7-35b-a3b-mxfp4 serve through the multimodal lane with tool calling and reasoning parsing preconfigured. (#4745)
• LFM2.5-24B-A2B: rapid-mlx serve lfm2.5-24b-a2b-4bit loads LiquidAI's official 4-bit MLX build with tool calling and reasoning parsing preconfigured. (#4852)

Performance

• Long prompts prefill concurrently when memory allows: the scheduler no longer over-estimates long-prompt prefill memory (about 300 GB estimated vs about 45 GB measured for a 32k-token prompt on a 35B MoE hybrid model), so concurrent 32k-token requests are no longer forced to run one at a time. (#4835)
• Faster text on multimodal Qwen MoE models: text-only requests on the default vision-capable lane of Qwen3.6-35B-A3B-class checkpoints keep single-request caches, use fused MoE and GDN projections, and no longer run slower than --no-mllm. Short 2–8-token blocks produce byte-identical output to the unfused model. (#4825, #4837, #4841)
• Vision-model servers keep text prefix caches across restarts: text-only prompt-prefix caches are restored after a restart; image-bearing prefixes are never persisted. (#4829)
• Faster opt-in MTP cohorts under load: continuous self-MTP forms a cohort only with at most 3 live requests and never grows past 3 lanes; with 4 or more concurrent requests the rest use plain batched decode, measured 23–24% faster for short prompts and 6–13% faster at 8k–16k-token prompts than the previous cohort. Cohort rollback overhead is lower too. (#4822, #4777)

Packaging

• Smaller Desktop app: the bundled runtime no longer ships build-only headers, test suites, or Intel code slices, and only the math font the app renders with is included — about 14 MB off the DMG and more than 110 MB and 11,000 files off the installed app. (#4797, #4798)
• Local Search works on stock Macs: the default index location and index folders under /tmp or /var no longer fail with a 503 because of standard macOS deny ACLs or directory aliases; unsafe ancestor permissions still fail with a message naming the path. (#4814)
• Packaged license notices: installed packages include the license and notice files for the vendored speculative-decoding (MTP) code and the MiniCPM-V 4.7 vision model code. (#4812)

What's new in v0.16.3

Rapid-MLX 0.16.3 is a reliability update with cleaner tool-call replies, better resilience under memory pressure during concurrent serving, safer model loading, Desktop crash fixes, and more helpful Local Search.

Highlights

• Concurrent long prompts queue when memory is tight: long prompts run in groups that fit available memory, so bursts of concurrent requests no longer abort with out-of-memory errors. (#4738)
• The server recovers from out-of-memory errors without a restart: after a batch fails with insufficient memory, the memory it held is released so later requests can succeed again. (#4742)
• Truncated tool calls no longer leak markup: replies cut off by the token limit while starting a tool call no longer show partial tool-call markup as text in non-streamed Chat Completions, Messages, and Responses output. (#4748)
• Clearer replies after unavailable tool calls: unavailable-tool notices now apply to every tool-call format, including Nemotron and Gemma 4. Calls to undeclared tools no longer appear as raw markup in reply text or as returned tool calls. (#4732)
• Safer model startup after incomplete downloads: checkpoints with missing shards or tensors are refused at startup with a cause and a next step instead of reporting ready and producing invalid replies. Truncated checkpoints and checkpoints without weights show the same concise guidance without a traceback. (#4731)
• Logprobs requests work with MTP speculative decoding: requests for token log probabilities no longer fail with HTTP 500 or crash the server when MTP speculative decoding is active. (#4733)
• Correct concurrent Nemotron-H text responses: when Nemotron 3.5 text models (Nemotron-H architecture) serve several prompts of different lengths at once, shorter prompts no longer produce different or prematurely stopped responses than when sent alone. Prefix-cache restores stay consistent too. (#4711)
• Opt-in concurrent decoding for qualified TensorFold profiles: setting RAPID_MLX_TENSORFOLD_LANES to 2, 4, or 8 lets up to that many requests decode together in shared rounds on the qwen3.8-27b-tensorfold and nemotron-3.5-lightning-tensorfold profiles, with output matching one-at-a-time decoding. Other profiles refuse the setting before loading weights, and the default (unset or 1) keeps serial decoding. (#4739)
• Fresh installs no longer break image generation: Rapid-MLX keeps Pillow below 13 to avoid an incompatible release that prevented the image runtime from loading on supported Python versions. (#4741)
• Experimental Local Search keeps indexed folders fresh: newly saved or edited files in added folders appear automatically without manual Reindex, with Desktop refreshing stale folders when Search opens and the engine rescanning folders in the background about every 30 minutes. The Whole Mac location refreshes only on that background interval. (#4735)
• Local Search shows the closest files instead of an empty page: when a meaning-based search finds no match, experimental Local Search shows up to three of the most similar files under a "No exact matches · Closest files" header. Searches that already return results are unchanged, and keyword-only search still shows no results. (#4736)
• Clear guidance when macOS blocks a Local Search folder: adding a folder that macOS privacy settings block now keeps the Search page open and explains how to allow access, with Open Privacy Settings and Try Again buttons, instead of a generic error and a page that cannot recover. (#4661)
• Quick Look in Local Search no longer crashes Desktop: choosing Quick Look on a search result or pressing Space opens the preview reliably instead of occasionally quitting the app. (#4737)
• Desktop no longer quits when you open What's New after an update: clicking "See what's new" on the post-update banner opens the release notes and dismisses the banner without crashing. The "Star on GitHub" prompt is fixed too. (#4729)

All changes

• feat: opt-in concurrent decoding for qualified TensorFold profiles (#4739) (76b8296)
• chore: bump version to 0.16.3 (#4743) (81e846e)
• feat(local-search): show the closest files instead of an empty result page (#4736) (db704e6)
• fix(local-search): readable recovery when macOS denies folder access (#4661) (f93ddeb)
• fix(tools): hide truncated tool-call markup in non-streamed replies (#4748) (73a4367)
• fix(scheduler): release retired generator memory after a generation error (#4742) (4ee4e8a)
• fix(tools): handle undeclared tool calls the same way for every parser (#4732) (db07d16)
• fix(serve): refuse incomplete checkpoints instead of loading with missing weights (#4731) (8e14147)
• fix(scheduler): budget concurrent prefill memory before admitting a cohort (#4738) (96c7fac)
• fix(mtp): logprobs requests no longer 500 with MTP speculative decoding (#4733) (7ada140)
• fix(desktop): stop Local Search Quick Look from crashing the app (#4737) (6d7e132)
• test(pr_validate): run full_unit in parallel with pytest-xdist (#4740) (2d92e61)
• feat(local-search): keep indexed folders fresh without a manual reindex (#4735) (f20f5a1)
• test: update content locks for the Pillow <13 pin (#4744) (8ec8b50)
• fix(deps): keep Pillow below 13 (#4741) (e121bb5)
• Fix Nemotron-H recurrent state after padded batched prefill (#4711) (adc88c3)
• fix(desktop): stop the What's New banner from crashing the app (#4729) (a1b3b1d)
• fix(release): retry stapling after Apple accepts notarization (#4727) (52ca6a9)

Install: brew upgrade rapid-mlx or pip install -U rapid-mlx==0.16.3 (or just rapid-mlx upgrade).

[0.16.3] — 2026-10-17

Rapid-MLX 0.16.3 is a reliability update with cleaner tool-call replies, better resilience under memory pressure during concurrent serving, safer model loading, Desktop crash fixes, and more helpful Local Search.

Highlights

• Concurrent long prompts queue when memory is tight: long prompts run in groups that fit available memory, so bursts of concurrent requests no longer abort with out-of-memory errors. (#4738)
• The server recovers from out-of-memory errors without a restart: after a batch fails with insufficient memory, the memory it held is released so later requests can succeed again. (#4742)
• Truncated tool calls no longer leak markup: replies cut off by the token limit while starting a tool call no longer show partial tool-call markup as text in non-streamed Chat Completions, Messages, and Responses output. (#4748)
• Clearer replies after unavailable tool calls: unavailable-tool notices now apply to every tool-call format, including Nemotron and Gemma 4. Calls to undeclared tools no longer appear as raw markup in reply text or as returned tool calls. (#4732)
• Safer model startup after incomplete downloads: checkpoints with missing shards or tensors are refused at startup with a cause and a next step instead of reporting ready and producing invalid replies. Truncated checkpoints and checkpoints without weights show the same concise guidance without a traceback. (#4731)
• Logprobs requests work with MTP speculative decoding: requests for token log probabilities no longer fail with HTTP 500 or crash the server when MTP speculative decoding is active. (#4733)
• Correct concurrent Nemotron-H text responses: when Nemotron 3.5 text models (Nemotron-H architecture) serve several prompts of different lengths at once, shorter prompts no longer produce different or prematurely stopped responses than when sent alone. Prefix-cache restores stay consistent too. (#4711)
• Fresh installs no longer break image generation: Rapid-MLX keeps Pillow below 13 to avoid an incompatible release that prevented the image runtime from loading on supported Python versions. (#4741)
• Experimental Local Search keeps indexed folders fresh: newly saved or edited files in added folders appear automatically without manual Reindex, with Desktop refreshing stale folders when Search opens and the engine rescanning folders in the background about every 30 minutes. The Whole Mac location refreshes only on that background interval. (#4735)
• Local Search shows the closest files instead of an empty page: when a meaning-based search finds no match, experimental Local Search shows up to three of the most similar files under a "No exact matches · Closest files" header. Searches that already return results are unchanged, and keyword-only search still shows no results. (#4736)
• Clear guidance when macOS blocks a Local Search folder: adding a folder that macOS privacy settings block now keeps the Search page open and explains how to allow access, with Open Privacy Settings and Try Again buttons, instead of a generic error and a page that cannot recover. (#4661)
• Quick Look in Local Search no longer crashes Desktop: choosing Quick Look on a search result or pressing Space opens the preview reliably instead of occasionally quitting the app. (#4737)
• Desktop no longer quits when you open What's New after an update: clicking "See what's new" on the post-update banner opens the release notes and dismisses the banner without crashing. The "Star on GitHub" prompt is fixed too. (#4729)

What's new in v0.16.2

Rapid-MLX 0.16.2 adds new text-model support and experimental Local Search, and improves concurrent serving, cache reuse, agent setup, and Desktop Share Compute.

Highlights

• NVIDIA Nemotron 3.5 Bolt 74B-A9B text support: run local text chat, reasoning, and tool calling with automatic reasoning and tool-call configuration. Choose rapid-mlx/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-MLX-3bit (31.7 GB download, 48 GB+ Macs) or rapid-mlx/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-MLX-4bit (43.3 GB download, 64 GB+ Macs). Start either with rapid-mlx serve <repo>. Supports text only; image/video input and speculative drafter support are unavailable. Download size differs from runtime memory; memory needs vary with context length and concurrent requests. (#4679)
• NVIDIA Nemotron 3.5 Super VL 120B-A12B text support: run the 120B model for local text chat, reasoning, and tool calling. Choose rapid-mlx/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-MLX-4bit (69.5 GB download, 128 GB+ Macs) or rapid-mlx/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-MLX-3bit (mixed precision, 42.8 GB download, targets 64 GB Macs at short context). Start either with rapid-mlx serve <repo>. Supports text only; image input and speculative decoding are unavailable. (#4695)
• Experimental on-device Local Search: opt in through Settings → Experimental, select folders, and find documents, images, and sampled video frames by filename, exact text, or meaning, with open, Finder reveal, and Quick Look actions. Meaning-based search requires the search model; keyword results remain available when it is unavailable, with clearer status guidance. The feature is off by default. (#4610, #4637, #4648)
• Ternary Bonsai 2 small models: new aliases bonsai2-2b-2bit, bonsai2-4b-2bit, and bonsai2-9b-2bit (0.43 / 0.95 / 2.0 GB) bring the 2-bit Hadamard format to small-memory Macs with image input, tool calling, and reasoning. (#4703)
• Safer Local Search startup and shutdown: enabling Search on non-loopback or inherited listeners requires an API key before model preflight or indexing begins. Disabling Search stops background indexing without deleting the index. (#4671)
• Better Local Search result ordering: meaning-based results receive a bounded ranking boost when the file contains query terms that are rare across indexed files, helping distinguish similar documents and find matches deep inside long files. Filename and exact-text search priorities stay unchanged. (#4638)
• More responsive concurrent text serving: qualified Qwen3.5 4B and 9B vision-enabled servers can batch text requests while preserving image routing, and eligible long text prefills yield between chunks so concurrent requests can advance. Benefits depend on the model, hardware, and workload. (#4575, #4577, #4584)
• Better warm-session cache reuse: the qualified experimental Qwen27 text runtime retains prompt prefixes at message boundaries. (#4626)
• Steady concurrent Qwen3.6 27B text serving: after a burst of concurrent text requests, a larger later burst no longer fails with HTTP 503 and leaves the server failing every larger batch. (#4722)
• PFlash threshold basis: opt-in --pflash-threshold-basis uncached lets --pflash auto compare only uncached tokens against its threshold with the memory-aware prefix cache, preserving full context and cache reuse when the remainder is below the threshold. total remains the default; cold prompts above the threshold still compress, and block-aware/paged caches retain total-token semantics. (#4684)
• Safer speculative decoding: automatic MTP falls back to ordinary decoding for unqualified targets, and grammar-constrained requests retain ordinary constrained decoding. (#4569, #4615)
• Configurable continuous MTP depth: continuous MTP honors num_speculative_tokens values of 1 or 2. Values of 3 or higher clamp to 2 with an explicit warning once per server process after admission; the default remains 2. Cohorts reduced to depth 1 under memory pressure can continue at that depth. (#4683)
• Clean forced tool-call responses: a truncated terminal tool-call header no longer leaks into visible content or reasoning in streamed or buffered forced-call responses, and a forced streamed call that runs out of tokens before any call is produced now ends with an explicit error, matching the buffered response, instead of an empty stream. Narration and valid structured tool calls are preserved. (#4678, #4710)
• Truthful finish reason for truncated tool calls: when max_tokens ends generation in the middle of a tool call, Chat Completions reports finish_reason: "length", Responses reports incomplete, and Messages reports stop_reason: "max_tokens", instead of signalling a completed call whose arguments may be incomplete. (#4715)
• Clearer replies after unavailable tool calls: when a model calls a tool the request did not declare and removing that call leaves no visible reply or valid tool call, chat completions explain that the tool is unavailable instead of returning an empty reply. Applies to streamed and buffered responses. (#4680, #4710)
• Less unused Gemma 4 prefill output work: eligible ordinary text/image prefills compute only the final output position while retaining the full KV cache. On Apple M5 Max with 36 GiB, Gemma 4 E2B 4-bit warm HTTP time to first content token fell 47.0% for 220-token text (50.65 → 26.84 ms), 61.3% for 796-token text (121.57 → 47.07 ms), and 24.6% for the tested 280-token image/text request (112.73 → 85.04 ms), with caches and speculation disabled. Already-chunked requests were neutral; results depend on model, hardware, and workload. (#4658)
• Steadier idle resource use and safer runtime qualification: repeated idle unload/reload cycles no longer accumulate the same retained host-memory graphs and text-stream threads, and optional Qwen3.5 fused decoding qualifies against the installed runtime with a safe fallback when checks fail. (#4542, #4596, #4493)
• Actionable image validation: invalid image parameters return HTTP 400 with the model’s validation message. Asset-independent checks run before download or load; HiDream’s token limit is checked after its assets load and also returns HTTP 400. (#4573)
• Easier model startup and recovery: interrupted model pulls retry transient failures and give access guidance; local path, architecture, and incomplete-checkpoint failures explain the next step. Missing serving runtimes offer repair through the active installer where supported. (#4549, #4545, #4618, #4621)
• Simpler local agent setup: additional coding-agent integrations preserve existing settings and use isolated integration checks, while interactive chat-server startup provides working request and setup commands without exposing the server key. (#4538, #4550, #4617)
• Working speech speed control: the Text-to-Speech Speed setting (speed on /v1/audio/speech) now changes playback duration for Qwen3-TTS, Chatterbox, and IndexTTS while preserving pitch; models with native speed control keep using it. (#4686)
• Readable Desktop text with a Text size setting: Desktop interface text now renders at 12 pt or larger, and Settings → Appearance → Text size enlarges interface text up to 1.5×, including chat messages, code blocks, the message composer, and the model list. Changes apply immediately and persist across relaunches. (#4707)
• Safer plain sharing: plain shares prove their generated share key during tunnel registration, keys stay out of connection URLs and protocol frames, and connection diagnostics redact keys. Clients older than 0.16.2 cannot register plain shares with the updated relay. (#4698)
• More predictable Share Compute: Desktop saves the provider account in Keychain and reuses its registered node on reconnect. Qwen27 sharing selects the qualified experimental text runtime with in-memory prefix reuse and a 48 GiB Desktop memory floor; Stop leaves sharing and personal models unloaded so the Mac’s resources are released. (#4641, #4650)

All changes

• chore: bump version to 0.16.2 (82e0571)
• ci(desktop): raise the bundle size cap to 1340 MiB for 0.16.2 (#4725) (8656279)
• release: 0.16.2 final engine train (#4715 #4722) (#4723) (1762fe1)
• fix(desktop): unblock the Swift 6.1 release build for Local Search thumbnails (#4718) (c5e91a6)
• fix(mac): enforce 12pt minimum UI text and add text size setting (#4707) (5ad45e3)
• fix(chat): suppress truncated XML tool headers and keep undeclared-tool rejections (#4710) (27bb383)
• feat(models): support Bonsai 2 small Hadamard models (#4703) (db41f4c)
• security(share): prove the share key on plain-share tunnel registration (#4698) (11ae839)
• fix(audio): honor speed for TTS models that ignore it (#4686) (ed77e0b)
• release: 0.16.2 release train (#4684 #4683 #4678 #4695 #4680 #4660 #4638) (#4700) (620b02f)
• fix(local-search): close release exposure blockers (#4671) (6721603)
• perf: reduce unused Gemma 4 ordinary prefill output work (#4658) (1ad4cf9)
• test(desktop): isolate golden-flow personas from host defaults (#4682) (a1cbf62)
• docs(nemotron): remove internal hosts and scratch paths from Bolt records (#4690) (ab94211)
• feat(engine): load Nemotron 3.5 Bolt text model (heterogeneous Puzzle MoE) (#4679) (bc3c4f2)
• fix(ci): distinguish tool marker prose from wire leaks (#4672) (2cdbe29)
• fix(share): remember provider account and unload models on stop (#4650) (6fb35fb)
• fix(desktop): explain why meaning-based Local Search is off (#4648) (5c70e23)
• fix(compute): default Qwen27 serving to cached TensorFold (#4641) (3b57495)
• feat(desktop): experimental Local Search surface (opt-in, off by default) (#4637) (6d617f5)
• fix(engine): yield shared executor between text prefill chunks (#4584) (04db89e)
• feat(local-search): experimental on-device search over user-selected folders (engine) (#4610) (7ac7ad5)
• test(mac): settle chat-restore before its answered AX baseline (#4630) (063237c)
• fix(tensorfold): retain Qwen27 prefixes at message boundaries (#4626) (33b9e6e)
• Show a working next step after serve is ready (#4617) (b638c2a)
• Repair missing serving runtimes using active installer (#4621) (6a884cb)
• fix(serve): explain local checkpoint startup failures (#4618) (1ee17ca)
• fix(mtp): fail closed for grammar-constrained requests (#4615) (3f6528a)
• ci: add controlled macOS runner fallback (#4561) (0474c19)
• test(mac): settle hosted attachment drops (#4612) (afc9bac)
• test(mac): await selected model before start (#4607) (3965cbf)
• test(mac): stabilize dirty-rect timing gate (#4603) (6b87053)
• test(mac): stabilize shared Desktop merge gates (#4600) (5372881)
• Fix MLX stream thread growth across text idle unloads (#4596) (23410e3)
• fix(ci): retain admission publication rejection evidence (#4560) (9582ddd)
• test(perf): add first-token prefill contention benchmark (#4579) (9ea3459)
• fix(image): return 400 for invalid image requests before loading weights (#4573) (8f076c1)
• docs: lead README with M5 Max benchmark results (#4585) (1f2b85e)
• fix(gdn): qualify Qwen3.5 convolution SiLU per runtime (#4493) (bec47ab)
• fix(engine): batch Qwen3.5 4B text on vision lane (#4577) (d127eaf)
• Add Goose and Crush local agent integrations (#4550) (a660493)
• fix(engine): batch dense Qwen3.5 9B text on vision lane (#4575) (8a0cdae)
• fix: fall back when automatic MTP target is unqualified (#4569) (0bd71ff)
• Add Cline agent setup and refresh harness integration guides (#4538) (cf3a898)
• Fix host-memory retention across idle model unloads (#4542) (d327146)
• Improve BYOM serve path and architecture guidance (#4545) (e70cc71)
• fix(cli): recover interrupted model pulls with actionable Hub guidance (#4549) (1799e23)

Install: brew upgrade rapid-mlx or pip install -U rapid-mlx==0.16.2 (or just rapid-mlx upgrade).

[0.16.2] — 2026-10-16

Rapid-MLX 0.16.2 adds new text-model support and experimental Local Search, and improves concurrent serving, cache reuse, agent setup, and Desktop Share Compute.

Highlights

• NVIDIA Nemotron 3.5 Bolt 74B-A9B text support: run local text chat, reasoning, and tool calling with automatic reasoning and tool-call configuration. Choose rapid-mlx/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-MLX-3bit (31.7 GB download, 48 GB+ Macs) or rapid-mlx/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-MLX-4bit (43.3 GB download, 64 GB+ Macs). Start either with rapid-mlx serve <repo>. Supports text only; image/video input and speculative drafter support are unavailable. Download size differs from runtime memory; memory needs vary with context length and concurrent requests. (#4679)
• NVIDIA Nemotron 3.5 Super VL 120B-A12B text support: run the 120B model for local text chat, reasoning, and tool calling. Choose rapid-mlx/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-MLX-4bit (69.5 GB download, 128 GB+ Macs) or rapid-mlx/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-MLX-3bit (mixed precision, 42.8 GB download, targets 64 GB Macs at short context). Start either with rapid-mlx serve <repo>. Supports text only; image input and speculative decoding are unavailable. (#4695)
• Experimental Local Search is off by default. Opt in through Settings → Experimental, choose folders, and search documents, images, and sampled video frames by filename, exact text, or meaning, then open, reveal, or Quick Look results. Meaning-based search requires the search model; keyword results and clearer availability guidance remain when it cannot load.
• Ternary Bonsai 2 small models: new aliases bonsai2-2b-2bit, bonsai2-4b-2bit, and bonsai2-9b-2bit (0.43 / 0.95 / 2.0 GB) bring the 2-bit Hadamard format to small-memory Macs with image input, tool calling, and reasoning. (#4703)
• Safer Local Search startup and shutdown: enabling Search on non-loopback or inherited listeners requires an API key before model preflight or indexing begins. Disabling Search stops background indexing without deleting the index. (#4671)
• Better Local Search result ordering: meaning-based results receive a bounded ranking boost when the file contains query terms that are rare across indexed files, helping distinguish similar documents and find matches deep inside long files. Filename and exact-text search priorities stay unchanged. (#4638)
• Qualified Qwen3.5 4B and 9B vision-enabled servers batch concurrent text requests while preserving image routing; eligible long text prefills yield between chunks so other requests can advance. Benefits depend on the model, hardware, and workload.
• Better warm-session cache reuse: the qualified experimental Qwen27 text runtime retains prompt prefixes at message boundaries. (#4626)
• Steady concurrent Qwen3.6 27B text serving: after a burst of concurrent text requests, a larger later burst no longer fails with HTTP 503 and leaves the server failing every larger batch. (#4722)
• PFlash threshold basis: opt-in --pflash-threshold-basis uncached lets --pflash auto compare only uncached tokens against its threshold with the memory-aware prefix cache, preserving full context and cache reuse when the remainder is below the threshold. total remains the default; cold prompts above the threshold still compress, and block-aware/paged caches retain total-token semantics. (#4684)
• Safer speculative decoding: automatic MTP falls back to ordinary decoding for unqualified targets, and grammar-constrained requests retain ordinary constrained decoding. (#4569, #4615)
• Configurable continuous MTP depth: continuous MTP honors num_speculative_tokens values of 1 or 2. Values of 3 or higher clamp to 2 with an explicit warning once per server process after admission; the default remains 2. Cohorts reduced to depth 1 under memory pressure can continue at that depth. (#4683)
• Clean forced tool-call responses: a truncated terminal tool-call header no longer leaks into visible content or reasoning in streamed or buffered forced-call responses, and a forced streamed call that runs out of tokens before any call is produced now ends with an explicit error, matching the buffered response, instead of an empty stream. Narration and valid structured tool calls are preserved. (#4678, #4710)
• Truthful finish reason for truncated tool calls: when max_tokens ends generation in the middle of a tool call, Chat Completions reports finish_reason: "length", Responses reports incomplete, and Messages reports stop_reason: "max_tokens", instead of signalling a completed call whose arguments may be incomplete. (#4715)
• Clearer replies after unavailable tool calls: when a model calls a tool the request did not declare and removing that call leaves no visible reply or valid tool call, chat completions explain that the tool is unavailable instead of returning an empty reply. Applies to streamed and buffered responses. (#4680, #4710)
• Less unused Gemma 4 prefill output work: eligible ordinary text/image prefills compute only the final output position while retaining the full KV cache. On Apple M5 Max with 36 GiB, Gemma 4 E2B 4-bit warm HTTP time to first content token fell 47.0% for 220-token text (50.65 → 26.84 ms), 61.3% for 796-token text (121.57 → 47.07 ms), and 24.6% for the tested 280-token image/text request (112.73 → 85.04 ms), with caches and speculation disabled. Already-chunked requests were neutral; results depend on model, hardware, and workload. (#4658)
• Idle unload/reload cycles release retained host-memory graphs and reuse text-stream workers. Optional Qwen3.5 fused decoding checks the installed runtime and falls back safely when qualification fails.
• Invalid image settings report the model’s validation message as HTTP 400. Asset-independent parameter checks (steps, dimensions, guidance, negative prompt, character limits, and input-image rules) reject requests before any download or load. HiDream’s token limit is only evaluated once the model’s assets are available and also returns HTTP 400; a cold request downloads and loads the model first.
• Model pulls retry transient failures, local path and checkpoint errors explain recovery, and missing serving runtimes offer repair through the active installer where supported.
• Additional local coding-agent integrations preserve existing settings and isolate integration checks. Interactive chat-server startup shows working request and setup commands without exposing the server key.
• Working speech speed control: the Text-to-Speech Speed setting (speed on /v1/audio/speech) now changes playback duration for Qwen3-TTS, Chatterbox, and IndexTTS while preserving pitch; models with native speed control keep using it. (#4686)
• Readable Desktop text with a Text size setting: Desktop interface text now renders at 12 pt or larger, and Settings → Appearance → Text size enlarges interface text up to 1.5×, including chat messages, code blocks, the message composer, and the model list. Changes apply immediately and persist across relaunches. (#4707)
• Safer plain sharing: plain shares prove their generated share key during tunnel registration, keys stay out of connection URLs and protocol frames, and connection diagnostics redact keys. Clients older than 0.16.2 cannot register plain shares with the updated relay. (#4698)
• Share Compute remembers the provider account in Keychain and reuses its registered node. Qwen27 sharing defaults to the qualified experimental text runtime with in-memory prefix reuse and a 48 GiB Desktop memory floor; Stop leaves sharing and personal models unloaded.

What's new in v0.16.1

Rapid-MLX 0.16.1 is a reliability and safety update for local serving, multimodal requests, installation, and coding-agent setup.

Highlights

Safer local serving: browser-origin and Host validation now reject drive-by access and DNS rebinding by default, while service qualification uses the trusted destination correctly. (#4512, #4521)

More reliable vision requests: Anthropic image input is preserved, Gemma 4 12B image reading is restored, and qualified dense hybrid vision/text requests can batch without crossing capability boundaries. (#4490, #4489, #4454)

Actionable model lifecycle failures: impossible model loads are rejected before healthy residents are evicted, low-memory switches are identified as retryable, and startup or residency failures now explain the recovery action. (#4473, #4427, #4496, #4510)

Installer and agent setup fixes: streamed curl | bash installation no longer loses input to child package managers, and generated profiles better match current client configuration and server-root URL behavior. (#4537, #4446, #4540, #4541)

Serving and cache correctness: dedicated embedding-only serving is available, recurrent hybrid KV quantization is selective, shared prompt state is charged once, and MCP cold startup and teardown are more robust. (#4457, #4408, #4383, #4464, #4475)

All changes

• fix(desktop): preserve vision capability during restore (#4566) (df1f4db)
• chore: bump version to 0.16.1 (ff6ee14)
• fix(validation): block SDK startup failures before baseline waiver (#4518) (61eb406)
• Fix OpenHands CLI profile for current environment override (#4541) (568ad21)
• docs: align Qwen Code profile guidance with current release (#4544) (af3cc7d)
• Audit OpenCode, Hermes, and Pi harness setup (#4540) (a142d6f)
• Clarify capability errors with actionable serve commands (#4547) (bf9fa6d)
• fix(install): protect streamed script stdin from child processes (#4537) (2928a26)
• fix: explain local model startup failures (#4539) (c91b00c)
• feat(cli): suggest a model to serve after discovery commands (#4543) (e58a655)
• fix(share): resolve profile repository for legacy prompt cleanup (#4529) (87a14c3)
• test: bound synthetic sidecar checkpoint fixtures (#4533) (6364ee8)
• feat(share): pool status line and first-run download progress (#4516) (ac8725d)
• test: qualify M5 lane formats across compiler updates (#4495) (cd5933e)
• fix(service): restore qualification under default Host protection (#4521) (cb6608c)
• fix: report vision cache persistence capability and actual cache stats (#4485) (55c01e5)
• fix(engine): batch qualified dense hybrid vision text requests (#4454) (25ea648)
• fix(benchmark): honor ignore_eos on native text generation (#4484) (7e7c58d)
• fix: retain live macOS diagnostics across observer clock baselines (#4501) (f28462e)
• fix(residency): make eviction recovery actionable (#4510) (1c39d4c)
• fix(anthropic): preserve image input in vision requests (#4490) (3669ca2)
• fix(server): reject drive-by browser access and DNS rebinding by default (#4512) (f0c8e99)
• fix(residency): return actionable model load errors (#4496) (46d638f)
• fix(agents): accept server-root URLs in Codex setup (#4446) (dd8ea07)
• test: keep XML tokenizer validation offline and independent of test order (#4504) (deac709)
• fix(mcp): keep SDK teardown in the connection lifecycle task (#4475) (6c6fae1)
• docs(gdn): qualify exact long-history state on M4 Pro (#4499) (91a37a4)
• fix(residency): reject impossible loads before evicting residents (#4473) (b3f06e4)
• fix: restore Gemma 4 12B image reading on the MLLM path (#4489) (529a27e)
• fix: release GLM startup resources on failure (#4487) (9409d6e)
• test(integrations): keep PydanticAI clients on one event loop (#4486) (cb10831)
• test(gdn): qualify exact native state across model histories (#4465) (ff7a544)
• test: qualify Nemotron long-chat prefix reuse (#4472) (f71d60b)
• fix(gdn): qualify Qwen4 convolution SiLU before fused decode (#4447) (30a7c0f)
• test: stabilize Apple numerical and golden fixture validation (#4469) (1c9597a)
• fix: reject Anthropic custom tools missing input_schema (#4463) (34e2f31)
• fix: restore MCP cold startup in serve mode (#4464) (b5926fc)
• test(mac): record per-journey startup and execution cost (#4453) (e6e7c0d)
• fix(share): prevent pool prompt-cache persistence and fail closed on TLS (#4458) (5f76643)
• feat(cli): support dedicated embedding-only serving and services (#4457) (9fd9d80)
• test: make Apple offline unit fixtures self-contained (#4449) (9bb5110)
• feat(ci): queue validated PRs without a second operator visit (#4450) (f715f46)
• fix(glm): bound shard and repack materialization on shared Macs (#4445) (dfaa9cc)
• fix(gdn): match stock convolution SiLU rounding for Qwen3.5 (#4443) (ef9c139)
• test(perf): measure hybrid checkpoint correctness with paired orders (#4433) (226f6bc)
• fix(mtp): restore cold drafter loading and qualify native decode on M5 (#4420) (d66f01e)
• fix(tools): honor required tool choice with native XML templates (#4432) (9cdca6a)
• fix(server): report retryable low-memory model switching (#4427) (52d1ad4)
• ci: extend source preflight to isolated engine CPU contracts (#4424) (d2dc1a9)
• fix(vision): enforce native context window before prefill (#4426) (c0cd9d6)
• fix(desktop): allow a custom Hugging Face download endpoint (#4411) (e44e959)
• perf(ci): avoid duplicate full source checks for release preparation (#4421) (646b804)
• fix(cache): support selective live KV quantization for recurrent hybrids (#4408) (19c0062)
• feat(release): advance preparation without manual polling (#4402) (3a8fb7d)
• fix(audio): preserve STT upload container suffixes (#4410) (a9eab3b)
• fix(desktop): keep update offers clear of model controls (#4412) (c36183c)
• feat(release): automate resumable metadata and preflight preparation (#4400) (61abe4a)
• fix(benchmark): keep healthy long serving rounds alive (#4409) (f41a500)
• fix(release): accept approved environment before and after frozen-policy cleanup (#4403) (a826d1a)
• fix(desktop): isolate GUI journeys from live update releases (#4405) (5c84378)
• fix(release): upload large Desktop archives through S3 multipart (#4399) (9cff18b)
• fix(release): fetch frozen source ancestry before detection (#4397) (c172a32)
• test: read SIGTERM readiness from raw pipe (#4394) (90f3312)
• fix(release): preserve audited 0.16.0 source on exact frozen branch (#4391) (3d67178)
• fix(cache): charge prompt state shared by cached sessions once (#4383) (f152732)
• feat(system-one): serve OpenJev typed decisions on native MLX (#4374) (837d6d6)
• feat(cli): support cache-only model loading (#4387) (4584269)
• Lock automatic prefix cache persistence across servers (#4381) (bf3f199)
• feat(scheduler): bound a streaming request's stall while another prompt prefills (#4338) (7df2996)
• fix(agents): support OpenCode 2.x provider and service flow (#4380) (541bb78)
• Add daemon-mode agent soak and 30-minute validation (#4379) (ff0b81a)
• test: pin serve_command flag resolution in a characterization snapshot (#4076) (22c4c96)
• ci: stop superseded engine workloads on cancellation (#4236) (cd09584)
• test: pin a cross-protocol streaming contract for chat, Responses and Anthropic (#4073) (a503ea9)
• feat(scheduler): store a snapshot where a prompt stops sharing a stored prefix (#4366) (7413d1f)
• feat(system-one): serve Cloudflare Clef on native MLX (#4365) (9adb97b)
• fix(ci): use paired confidence bounds for stress benchmarks (#4370) (c04f564)
• feat(scheduler): wait for a running prefill of the same prompt prefix (#4345) (167bd27)
• fix(packaging): sign every bundled Mach-O before notarization (#4346) (e65e28a)
• feat(system-one): serve Decider typed decisions on native MLX (#4341) (f2e0c58)
• Keep streamed tool arguments byte-identical to non-streaming JSON (#4344) (63db9ba)
• Measure and guard multi-turn agent prefix reuse (#4340) (22a7160)
• Add long-running agent soak harness and three-hour baseline (#4348) (d7ca3c4)
• fix(desktop): make first-run model downloads resumable and recoverable (#4342) (3460fd2)
• Surface protocol-correct context overflow errors to agent clients (#4343) (b5b92ff)
• feat(mllm): store a snapshot where a prompt stops sharing a stored prefix (#4351) (67e2513)
• fix(ci): parse Git database candidate messages (#4349) (721847c)
• ci: bind candidate qualification to queue lineage (#4335) (427a47f)
• fix(mirror): load a cached pinned snapshot in offline mode (#4332) (8c0435c)
• ci: include CLI snapshot in source preflight (#4333) (e77bbeb)
• fix(tensorfold): answer unservable requests as client errors (#4314) (e28562c)
• fix(tensorfold): keep prompt state on the GLM-5.3 Flash profile so follow-up turns reuse it (#4323) (531f466)
• feat(packaging): ship vision, image and video runtimes in the base install (#4309) (230b58b)
• fix(desktop): localize experimental pages in Simplified Chinese (#4313) (48d8180)
• fix(speculative): keep the non-stream answer in content when thinking is off (#4319) (37c1c1c)
• fix(tensorfold): keep prompt state on the family profiles so follow-up turns reuse it (#4317) (aa84928)
• fix(tensorfold): make request sampling take effect on TensorFold profiles (#4310) (082d722)
• test(desktop): preserve destination acknowledgement before retrying a drop (#4321) (4b9af99)
• feat(telemetry): closed command prop on app_opened (#4121) (e556222)
• fix(tensorfold): pass conversation boundaries so follow-up turns reuse the prompt (#4308) (b6553f9)
• fix(cache): keep stride checkpoints on a cold hybrid boundary snapshot (#4299) (19651e9)
• fix(release): exclude measured non-agentic alias from random sampling (#4295) (8c88dff)
• fix(agents): finish setup when the server is not running yet (#4304) (0ecacae)
• feat(tensorfold): add DeepSeek V4 Flash accelerated profile (#4301) (60d0983)
• fix(devtools): fail closed before Qwen4 conversion (#4294) (ba5cd69)
• fix(cli): name failure class in support request consent (#4293) (b6bef14)
• fix(recommendations): omit stale speed projections (#4302) (4ad083d)
• fix(desktop): reap catalog probes on app shutdown (#4298) (2b2e962)
• chore(mirror): list qwen3.8-flash-next-tensorfold as intentionally unmirrored (#4296) (8ba6cb2)
• feat(tensorfold): add Bonsai 2 27B and Gemma 4 26B accelerated profiles (#4289) (8a5cde2)

Community contributors

@guo — test: pin a cross-protocol streaming contract for chat, Responses and Anthropic

@guo — test: pin serve_command flag resolution in a characterization snapshot

Install: brew upgrade rapid-mlx or pip install -U rapid-mlx==0.16.1 (or just rapid-mlx upgrade).

[0.16.1] — 2026-10-12

Rapid-MLX 0.16.1 is a reliability and safety update for local serving, multimodal requests, installation, and coding-agent setup.

Highlights

Local server Host and browser-origin protections reject drive-by access and DNS rebinding while preserving trusted service health probes.

Anthropic and Gemma image requests retain their media inputs, and qualified dense hybrid vision/text requests batch safely.

Model startup and residency failures provide actionable recovery guidance without evicting healthy models for impossible loads.

Streamed installation protects its input from child package managers, and coding-agent profiles follow current client configuration behavior.

Embedding-only serving, recurrent hybrid cache quantization, shared prompt accounting, and MCP lifecycle handling are more reliable.

What's new in v0.16.0

Rapid-MLX 0.16.0 expands packaged multimodal and local decision workflows, adds guarded accelerated model profiles, and improves long-running server, agent, cache, and Desktop reliability. Performance varies by model, hardware, and workload; this release makes no general throughput or coding-quality claim.

Highlights

Vision, image, and video runtimes in the base package. The packaged runtime now includes the dependencies used by supported vision understanding, image generation, and video generation flows, with a documented video workflow. Model-specific weights and capability checks still apply; inclusion of a runtime does not make every model or modality supported. (#4309)

Local typed decisions. System One can serve Decider and native MLX Clef models for structured answers and candidate ranking. Native Clef supports its validated image and video inputs. OpenJev adds text-only answer and rank decisions from the pinned published conversion or an explicit local conversion; its published weights are CC BY-NC 4.0 and are restricted to non-commercial use. (#4341, #4365, #4374)

More guarded accelerated profiles. Bonsai 2 27B, Gemma 4 26B, and DeepSeek V4 Flash gain experimental TensorFold profiles. Related fixes preserve conversation boundaries and follow-up state, apply requested sampling, keep thinking-disabled answers in content, and return actionable client errors for requests the profile cannot serve. Each profile keeps its shipped model, runtime, hardware, and capability guards and fails closed outside them. (#4289, #4301, #4308, #4310, #4314, #4317, #4319, #4323)

More reliable shared prefixes and streaming. Multi-turn and multimodal requests retain reusable prompt boundaries, concurrent matching prompts can wait for an in-flight prefill, divergence snapshots are stored at the correct boundary, and automatic prefix-cache persistence is coordinated across server processes. Streaming receives a bounded scheduling opportunity during another request's long prefill, and streamed tool arguments remain byte-consistent with buffered JSON. These are correctness and responsiveness controls, not a universal speedup claim. (#4299, #4340, #4344, #4345, #4351, #4366, #4338, #4381)

Current agent setup and protocol behavior. Setup can finish before the local server starts and supports the current OpenCode provider and service configuration. Agent clients receive protocol-correct context-limit errors, streaming behavior is pinned across supported API protocols, and reproducible long-running daemon checks cover session stability. These checks validate integration behavior; they do not rate model quality. (#4304, #4073, #4343, #4348, #4379, #4380)

Operator control over model artifact downloads. A CLI flag, environment setting, and configuration option can prevent automatic missing model-artifact downloads while preserving complete cached models and explicit rapid-mlx pull. An uncached model receives a typed refusal and recovery guidance. This control is not general offline mode: BYOM preflight may still query repository metadata; use the existing Hugging Face offline settings when zero Hub traffic is required. This work incorporates Kenny Niehage's contribution from #4290, adopted in #4387.

Desktop and operational reliability

• First-run model downloads can resume after interruption and surface recoverable readiness states. (#4342)

• Drag-and-drop retry preserves destination acknowledgement, experimental pages add Simplified Chinese localization, and app shutdown or re-onboarding terminates and reaps catalog probes. (#4321, #4313, #4298)

• Recommendation cards keep model tier, footprint, capability, and caveat guidance while removing unsupported legacy speed projections. (#4302)

• Pinned snapshots already present in the shared cache can load offline, and the signed Mac package now signs and verifies every bundled Mach-O executable before notarization. (#4332, #4346)

• Support-request consent failures identify their failure class. (#4293)

Developer and release safeguards

Qwen4 conversion validates the pinned source and converter before staging, and release sampling excludes an explicitly non-agentic alias without changing deterministic engine, parser, or replay coverage. (#4294, #4295)

Candidate lineage, source preflight, cancellation, release signing inventory, characterization coverage, telemetry schema coverage, and mirror inventory received fail-closed repairs. These changes improve delivery safety and do not add product behavior. (#4236, #4333, #4335, #4349, #4076, #4370, #4121, #4296)

All changes

• test(release): bind multipart upload ordering and frozen repair paths (09e378b)
• fix(release): upload large Desktop archives through S3 multipart (5ac3995)
• test(release): isolate running render deadline from setup latency (27a3097)
• fix(release): fetch frozen source ancestry before detection (c5674b8)
• chore: bump version to 0.16.0 (2980430)
• test: read SIGTERM readiness from raw pipe (642d74d)
• test: bind frozen branch to full CI lanes (c7dd5ec)
• test: isolate frozen release history fixture (24980de)
• ci: run full gates on frozen release branch (de6b450)
• ci: cover frozen release policy checks (49283f5)
• fix: allow frozen release contract test (cbbbb34)
• test: bind frozen bump metadata contract (8b74a61)
• ci: allow exact frozen 0.16 release ref (f6d653e)
• feat(system-one): serve OpenJev typed decisions on native MLX (#4374) (837d6d6)
• feat(cli): support cache-only model loading (#4387) (4584269)
• Lock automatic prefix cache persistence across servers (#4381) (bf3f199)
• feat(scheduler): bound a streaming request's stall while another prompt prefills (#4338) (7df2996)
• fix(agents): support OpenCode 2.x provider and service flow (#4380) (541bb78)
• Add daemon-mode agent soak and 30-minute validation (#4379) (ff0b81a)
• test: pin serve_command flag resolution in a characterization snapshot (#4076) (22c4c96)
• ci: stop superseded engine workloads on cancellation (#4236) (cd09584)
• test: pin a cross-protocol streaming contract for chat, Responses and Anthropic (#4073) (a503ea9)
• feat(scheduler): store a snapshot where a prompt stops sharing a stored prefix (#4366) (7413d1f)
• feat(system-one): serve Cloudflare Clef on native MLX (#4365) (9adb97b)
• fix(ci): use paired confidence bounds for stress benchmarks (#4370) (c04f564)
• feat(scheduler): wait for a running prefill of the same prompt prefix (#4345) (167bd27)
• fix(packaging): sign every bundled Mach-O before notarization (#4346) (e65e28a)
• feat(system-one): serve Decider typed decisions on native MLX (#4341) (f2e0c58)
• Keep streamed tool arguments byte-identical to non-streaming JSON (#4344) (63db9ba)
• Measure and guard multi-turn agent prefix reuse (#4340) (22a7160)
• Add long-running agent soak harness and three-hour baseline (#4348) (d7ca3c4)
• fix(desktop): make first-run model downloads resumable and recoverable (#4342) (3460fd2)
• Surface protocol-correct context overflow errors to agent clients (#4343) (b5b92ff)
• feat(mllm): store a snapshot where a prompt stops sharing a stored prefix (#4351) (67e2513)
• fix(ci): parse Git database candidate messages (#4349) (721847c)
• ci: bind candidate qualification to queue lineage (#4335) (427a47f)
• fix(mirror): load a cached pinned snapshot in offline mode (#4332) (8c0435c)
• ci: include CLI snapshot in source preflight (#4333) (e77bbeb)
• fix(tensorfold): answer unservable requests as client errors (#4314) (e28562c)
• fix(tensorfold): keep prompt state on the GLM-5.3 Flash profile so follow-up turns reuse it (#4323) (531f466)
• feat(packaging): ship vision, image and video runtimes in the base install (#4309) (230b58b)
• fix(desktop): localize experimental pages in Simplified Chinese (#4313) (48d8180)
• fix(speculative): keep the non-stream answer in content when thinking is off (#4319) (37c1c1c)
• fix(tensorfold): keep prompt state on the family profiles so follow-up turns reuse it (#4317) (aa84928)
• fix(tensorfold): make request sampling take effect on TensorFold profiles (#4310) (082d722)
• test(desktop): preserve destination acknowledgement before retrying a drop (#4321) (4b9af99)
• feat(telemetry): closed command prop on app_opened (#4121) (e556222)
• fix(tensorfold): pass conversation boundaries so follow-up turns reuse the prompt (#4308) (b6553f9)
• fix(cache): keep stride checkpoints on a cold hybrid boundary snapshot (#4299) (19651e9)
• fix(release): exclude measured non-agentic alias from random sampling (#4295) (8c88dff)
• fix(agents): finish setup when the server is not running yet (#4304) (0ecacae)
• feat(tensorfold): add DeepSeek V4 Flash accelerated profile (#4301) (60d0983)
• fix(devtools): fail closed before Qwen4 conversion (#4294) (ba5cd69)
• fix(cli): name failure class in support request consent (#4293) (b6bef14)
• fix(recommendations): omit stale speed projections (#4302) (4ad083d)
• fix(desktop): reap catalog probes on app shutdown (#4298) (2b2e962)
• chore(mirror): list qwen3.8-flash-next-tensorfold as intentionally unmirrored (#4296) (8ba6cb2)
• feat(tensorfold): add Bonsai 2 27B and Gemma 4 26B accelerated profiles (#4289) (8a5cde2)

Community contributors

@guo — test: pin a cross-protocol streaming contract for chat, Responses and Anthropic

@guo — test: pin serve_command flag resolution in a characterization snapshot

Install: brew upgrade rapid-mlx or pip install -U rapid-mlx==0.16.0 (or just rapid-mlx upgrade).

[0.16.0] — 2026-10-09

Rapid-MLX 0.16.0 expands packaged multimodal and typed-decision workflows and improves model management, agent sessions, shared-prefix handling, and release reliability.

Added

The base package includes supported vision, image-generation, and video-generation runtime dependencies and the documented video workflow. Model weights and per-model capability guards still apply.

System One adds native MLX Decider and Clef answer/rank backends. Clef supports its validated media inputs. OpenJev adds text-only typed answers and ranking; its published weights are CC BY-NC 4.0 and restricted to non-commercial use.

Experimental accelerated profiles cover Bonsai 2 27B, Gemma 4 26B, and DeepSeek V4 Flash under their pinned runtime, model, hardware, and capability guards.

Operators can prevent automatic missing model-artifact downloads through the CLI, environment, or configuration while preserving cached models and explicit pull. BYOM metadata checks may still access the Hub; general offline operation remains a separate setting. This includes Kenny Niehage's contribution from #4290, adopted in #4387.

Agent setup supports the current OpenCode provider and service flow and can finish while the local server is stopped.

Changed

Accelerated profile requests preserve conversation boundaries and follow-up state, apply supported sampling controls, keep thinking-disabled answers in content, and reject unservable requests with actionable client errors.

Shared-prefix scheduling coordinates matching in-flight prefills, records exact divergence boundaries, persists prefix state safely across processes, and gives streaming requests a bounded opportunity to advance during another long prefill. No universal performance gain is claimed.

Streamed tool-call arguments retain the same JSON bytes as buffered responses, and agent-facing context-limit errors follow each supported protocol.

Recommendation cards retain tiers, footprints, capabilities, and caveats without unsupported legacy speed projections.

Pinned cached snapshots can load offline, and support-request consent failures identify their failure class.

Fixed

First-run model downloads resume after interruption and show recoverable readiness states.

Desktop attachment retry keeps destination acknowledgement; experimental pages add Simplified Chinese localization; catalog probes are terminated and reaped during quit and re-onboarding.

The signed package inventories, signs, and verifies every bundled Mach-O executable before notarization.

Long-running agent soak, cross-protocol streaming, conversion preflight, release sampling, candidate lineage, source preflight, cancellation, and stress-decision contracts received fail-closed regression coverage.

What's new in v0.15.7

Rapid-MLX 0.15.7 adds native text and code embeddings, improves local agent setup, and strengthens Desktop and inference reliability.

Highlights

EmbeddingGemma 2 text/code embeddings: serve normalized 768-dimensional vectors through /v1/embeddings, with standard 4-bit and BF16 aliases, optional smaller dimensions, and an explicit 8192-token limit. This release supports text/code inputs; images, audio and video are unsupported. (#4279)

More reliable agent setup: Continue and Cline configuration is written where their clients read it; keyed Continue connections preserve the configured server credential. Qwen Code previews redact retained secrets while keeping the saved configuration intact. (#4257, #4274, #4275)

Warmer coding-agent turns: prompt reuse avoids repeated full-prompt tokenization; MTP scheduling and reproducibility receive further fixes. Improvements depend on the model, hardware and workload. (#4214, #4221, #4238)

Desktop polish: broader Simplified Chinese coverage, clearer timeout handling and more reliable saved Codex model identity. (#4147, #4146, #4249)

Server controls and request timing: --disable-disk-caches disables optional disk-cache writes while preserving in-memory reuse; --log-file redirects server logs. Chat and Responses can expose request-scoped time to first token and mean token interval. (#4272, #4151)

LTX-2.5 video extension jobs and model mirror/prefix-cache reliability fixes. Computer use remains experimental.

Model catalog

qwopus-27b-8bit is retired because its upstream repository is unavailable; use qwopus-27b-4bit.

qwen3.8-27b-abliterated-4bit follows the publisher’s relocated oQ4e build.

All changes

• fix(release): advance Desktop build number for 0.15.7 (53d943f)
• feat(tensorfold): add Nemotron 3.5 and Qwen3.8 Flash Next accelerated profiles (#4264) (3d00b30)
• chore: bump version to 0.15.7 (61fb9e8)
• feat: support native EmbeddingGemma 2 text and code embeddings (#4279) (3766d27)
• feat(api): expose request-scoped TTFT and mean token interval (#4151) (dc14ee7)
• feat(cli): one switch to stop optional disk-cache writes and redirect server logs (#4272) (1a69734)
• fix(agents): forward keyed credential to Continue setup (#4274) (dfd7cff)
• fix(agents): redact preserved credentials in Qwen setup preview (#4275) (62f0d5d)
• fix(pr_validate): treat only a root setup.py as a packaging change (#4271) (e86e180)
• fix(agents): preserve Qwen Code providers during setup (#4247) (deb60a0)
• ci: enable controlled mapped candidate validation and trusted admission (#4242) (668740f)
• test: drain video jobs before extension teardown (#4267) (1956275)
• ci: wait for published attestation workflow completion (#4259) (5170693)
• fix(launch): write the config files Continue and Cline actually read (#4257) (859b80e)
• perf(mtp): charge round overhead to drafting rounds only on steep hosts (#4256) (d3886c8)
• feat(tensorfold): qualify both accelerated profiles on one TensorFold 0.6.6 runtime (#4251) (470f516)
• ci: scope main Desktop allocation to the complete push diff (#4255) (4c4f1c5)
• test: reuse immutable outputs in distribution model (#4253) (fb2604c)
• fix(desktop): preserve subprocess timeout cause (#4249) (6b25bec)
• perf(mtp): pipeline parked MTP rounds like plain decode (#4238) (fc767c3)
• ci: keep accompanying docs neutral in source CPU preflight (#4241) (c82c0c7)
• test(telemetry): separate atomic claims from bounded fail-open waits (#4240) (0e91c4c)
• fix(mirror): prove configs, tokenizers and templates by git blob id on default-branch pulls (#4234) (96fca67)
• feat(video): add LTX-2.5 video extension jobs (#4233) (eda8160)
• ci: preflight ordinary source PRs on CPU before full candidate validation (#4232) (facce5c)
• test(ci): bound oversized artifact case names (#4230) (e8c04f3)
• ci: reuse admission API connections with fresh proof reads (#4227) (4c7137c)
• fix(mtp): make greedy MTP output reproducible run to run (#4221) (abd33fa)
• test(cua): no-model fixtures that hold the hands to an outside oracle (#4211) (243a6a6)
• fix(cua): harden the hands contract found by no-model fixtures (#4210) (3979070)
• ci: start full admission alongside candidate qualification (#4222) (c83053b)
• fix(cache): keep a free-disk reserve when persisting the prefix cache (#4170) (31e1d9b)
• perf(engine): stop re-tokenizing the whole prompt on warm agent turns (#4214) (1d49467)
• ci: deploy trusted full candidate admission consumer (#4216) (5407081)
• test: make the full unit run green again on Apple Silicon hosts (#4198) (ec3543f)
• ci: deploy candidate execution shadow and authenticated evidence transport (#4209) (a925878)
• docs(readme): new hero banner and demo GIF (#4212) (73a2890)
• refactor(cua): the hands execute; approvals belong to the brain (#4205) (9fa70c9)
• ci: add trusted candidate qualification producer with mapped routing disabled (#4203) (9dc9a94)
• fix(cua): perception and hands polish from mock regression (#4178) (96a179c)
• ci: preserve latest candidate barrier for full main backstop (#4192) (cc05786)
• ci: revalidate exact main full qualification prerequisite (#4191) (d7ab0cd)
• ci: observe combined candidate routing from trusted full evidence (#4187) (08816d5)
• fix(mirror): download pinned-revision image, video and speculative checkpoints through the mirror (#4193) (8a4c76b)
• ci: verify complete mapped test execution against base (#4186) (6584038)
• docs(readme): lead with the measured mlx-lm comparison; brand cleanup (#4184) (84b0a29)
• chore(mirror): mirror gpt-oss-120b-mxfp4-q4 (#4189) (70a70a1)
• test(cua): keep the focus_only refusal tests off the real window server (#4181) (afa2ac7)
• ci: map isolated telemetry test maintenance to source canary (#4180) (16491b2)
• fix(catalog): resolve mirror drift for the 26 aliases in #4141 (#4172) (3165558)
• feat(cua): mock web-task evals for the perception session (#4175) (6d4e5cd)
• test: preserve banner purity across mixed renders (#4176) (6a0b317)
• feat(cua): perception session — stable refs, receipts, whole-page walk and user-only steps (#4168) (056cbb2)
• ci: roll out an opt-in mapped source canary with full candidate qualification (#4139) (d558f2c)
• test: honor explicit integration server opt-in (#4166) (c55b6ed)
• test: verify actual repository shard enrollment (#4161) (3c45249)
• feat(cua): background menus, menu paths, modifier clicks, Electron, hidden renderers and AX scroll (#4165) (4f4c473)
• feat(cua): hands for off-Space windows — exact key-window switching, background fill, drag (#4156) (986caeb)
• ci: validate exact full-evidence matrix identities (#4159) (96bef0e)
• test: give oversized TextEdit fixture a short pytest id (#4117) (1f0bbf8)
• fix(cli): guide external cache permission recovery (#4145) (c46df01)
• fix(agents): verify saved Codex model identity (#4146) (21c55a5)
• fix(desktop): complete zh-Hans on sidebar, chat home, Settings and model lists (#4147) (0e6898c)
• ci: keep Codecov upload advisory (#4150) (dbbd815)
• docs: document prefix-cache persistence opt-out (#4149) (d22e0bf)

Install: brew upgrade rapid-mlx or pip install -U rapid-mlx==0.15.7 (or just rapid-mlx upgrade).

[0.15.7] — 2026-10-07

Rapid-MLX 0.15.7 adds native text and code embeddings, improves local agent setup, and strengthens Desktop and inference reliability.

Highlights

• EmbeddingGemma 2 text/code embeddings: serve normalized 768-dimensional vectors through /v1/embeddings, with standard 4-bit and BF16 aliases, optional smaller dimensions, and an explicit 8192-token limit. This release supports text/code inputs; images, audio and video are unsupported. (#4279)
• More reliable agent setup: Continue and Cline configuration is written where their clients read it; keyed Continue connections preserve the configured server credential. Qwen Code previews redact retained secrets while keeping the saved configuration intact. (#4257, #4274, #4275)
• Warmer coding-agent turns: prompt reuse avoids repeated full-prompt tokenization; MTP scheduling and reproducibility receive further fixes. Improvements depend on the model, hardware and workload. (#4214, #4221, #4238)
• Desktop polish: broader Simplified Chinese coverage, clearer timeout handling and more reliable saved Codex model identity. (#4147, #4146, #4249)
• Server controls and request timing: --disable-disk-caches disables optional disk-cache writes while preserving in-memory reuse; --log-file redirects server logs. Chat and Responses can expose request-scoped time to first token and mean token interval. (#4272, #4151)
• LTX-2.5 video extension jobs and model mirror/prefix-cache reliability fixes. Computer use remains experimental.

Model catalog

• qwopus-27b-8bit is retired because its upstream repository is unavailable; use qwopus-27b-4bit.
• qwen3.8-27b-abliterated-4bit follows the publisher’s relocated oQ4e build.

Fixed

• Simplified Chinese now covers the sidebar, the chat home screen and message box, the Settings categories and panels, Model Management, and the model picker, which previously stayed in English when macOS was set to Chinese. Experimental tabs (Benchmark, Share Compute, Computer Use, Video) are not yet fully translated.