MLflow Release Notes

We're excited to announce MLflow 3.13.0rc0, which deepens agent observability, tightens permissions, and broadens deployment options:

Major New Features

RBAC + Admin UI: Major overhaul of MLflow's Role-Based Access Control — legacy per-resource permission tables collapsed into role_permissions, unified per-user permission APIs under /mlflow/users/permissions/*, workspace USE permission lets users create experiments and registered models, default roles are seeded on workspace creation, prompt is promoted to a first-class RBAC resource_type, and a new 4-page Admin UI (account widget, /account page, Platform Admin pages, backend auth endpoints) opens to workspace managers scoped per their workspace. (#22855, #22857, #22859, #22928, #22929, #22941, #22973, #23086, #23247, #23248, #23337, #23379, @PattaraS)

Coding-Agent Tracing as Plugins: Claude Code, OpenClaw, Ollama, and OpenAI Codex are now wired into the AI Gateway as first-class assistant providers, plus a Claude Code TypeScript plugin with a setup wizard and settings.local.json support. The legacy Python autolog hook for mlflow autolog claude is replaced by the new official plugin, and a coding-agent endpoint creation flow is now available directly in the AI Gateway UI. (#20414, #22098, #22566, #22717, #23218, #23285, #23339, #23430, #23517, @B-Step62, @joelrobin18, @Gkrumbach07, @SuperSonnix71, @TomeHirata)

Trace Archival: End-to-end trace archival across the tracking stack. Includes archival configuration models, OTLP and artifact helpers, SQLAlchemy archival passes, archive-aware retrieval fallback, plus workspace/experiment/server-level archival settings in the UI. Read archived traces back seamlessly. (#23359, @mprahl)

Helm Charts for Kubernetes Deployment: First-class Helm chart for deploying MLflow to Kubernetes clusters — production-ready configuration, ingress, persistence, and appVersion wired to the released MLflow image. Get from helm install to a running tracking server without writing your own manifests. (#21973, @WeichenXu123)

mlflow.genai.test_agent for Automated Agent Stress-Testing: New API for stress-testing GenAI agents — generate adversarial inputs, replay them through your agent, and review the resulting traces in MLflow. Wires into the existing evaluation flow and assessment APIs. (#22990, @serena-ruan)

OpenTelemetry Span Links: Tracing now supports the OpenTelemetry Link entity via LiveSpan.add_link(), letting you connect causally related spans across traces. (#22797, @khaledsulayman)

Database Replica Routing: The SQL tracking store now supports reader/writer instance routing for database replicas, so read-heavy MLflow deployments can scale horizontally without overloading the primary. (#22910, @ravidarbha)

Stay tuned for the full release, which will include even more features and bug fixes.

To try out this release candidate, please run:

pip install mlflow==3.13.0rc0

MLflow 3.12.0 includes several major features and improvements

Major New Features

🖼️ Multimodal Tracing: Users can now store multimodal content in tracing spans as artifact attachments instead of inline binary data. We've also patched the UI to support the new mlflow-attachment:// style URI, with rich rendering available for PDFs, audio, and images.

🤖 Codex, Gemini, Qwen coding agent tracing support: Similar to our Claude Code tracing integration, we've now added support for the Codex, Gemini, and Qwen coding agent platforms as well!

🛡️ Gateway guardrails: You can now set guardrails on your gateway endpoints to prevent unsafe or non-compliant model inputs and outputs. Try it out in the MLflow UI!

⚡ Trace table pagination: The traces tab is now paginated, rather than fetching all traces up to a limit of 1000. This improves initial load time, and makes the page feel more responsive overall.

Breaking Changes

[Scoring] Deprecate enable_mlserver in pyfunc serving backend (#22994, @B-Step62)

Other Assorted Features & Improvements

[UI] Add coding agents section to AI Gateway quick start (#23006, @TomeHirata)
[Tracing] feat: record caller in gateway traces from request headers (#22926, @TomeHirata)
[] Run guardrails on passthrough endpoints; skip response_format for non-chat payloads (#22856, @TomeHirata)
[] Gateway: preserve client auth header for subscription-based CLI tools (claude-cli, Codex-Desktop, GeminiCLI) (#22915, @TomeHirata)
[Tracing] Expose Codex and Qwen Code hooks as installable CLI binaries (#22853, @kriscon-db)
[Tracking / UI] Add Portkey as a supported AI Gateway provider (#22830, @sairavuri-sudo)
[Tracking] Cache successful basic-auth credential checks to eliminate per-request PBKDF2 (#22817, @PattaraS)
[] Add last_updated_at field to model catalog entries (#22838, @copilot-swe-agent)
[Evaluation] Enable third-party scorer registration in OSS MLflow (#22634, @smoorjani)
[] Add platform-side telemetry fields to Gateway invocation and budget events (#22557, @PattaraS)
[Tracing] Auto-start SQL warehouse before V4/V5 MLflow tracing calls (#22798, @artjen)
[Tracing] Migrate Claude Code tracing to TypeScript-based plugin for simpler installation and runtime handling. (#22338, @B-Step62)
[Tracing] Add TypeScript Qwen Code tracing via Stop hook (#22411, @kriscon-db)
[] Support model_kwargs in DeepEval scorers for LLM parameter control (#22494, @debu-sinha)
[] Add Application Default Credentials auth mode for Vertex AI gateway (#22754, @harupy)
[UI] Reorganize Settings into section-based routing with sub-sidebar navigation (#22743, @serena-ruan)
[Tracing] Add TypeScript Codex CLI tracing via notify hook (#22410, @kriscon-db)
[Tracing / UI] Add Share button to trace detail view (#22608, @alkispoly-db)
[Docs / Models] Add mlflow.diffusers flavor for diffusion model LoRA adapters (#22253, @Rasaboun)
[Tracing] Add size limit for trace attachments (#22575, @kriscon-db)
[Tracing] Add GeminiCliTranslator for Gemini CLI OTLP span type mapping (#22409, @kriscon-db)
[Tracing] Add JSON OTLP encoding support for trace ingestion (#22408, @kriscon-db)
[Tracing] Extract service.name from OTLP resource attributes for usage telemetry (#22407, @kriscon-db)
[Tracing] Add MLflow tracing spans to guardrail execution in gateway (#22581, @TomeHirata)
[Tracing / UI] Add rendering size guards for large media content (#22574, @kriscon-db)
[Tracking] #21037 Add presigned upload URL endpoint for S3 artifact uploads (#21039, @henishborad)
[UI] Add Guardrails tab to endpoint editor (#22360, @TomeHirata)
[UI] Add click-to-expand modal for trace attachment images (#22461, @kriscon-db)
[UI] Enhance Gateway quick start cards with logos, multi-model options, and compact variant (#22513, @xq-yin)
[Tracing] Add @mlflow/vercel for better Vercel AI SDK tracing in Databricks UC (#22105, @dbrx-euirim)
[UI] [UI] Refactor API keys page: bulk delete, inline drawer editing, consistent list pattern (#22485, @xq-yin)
[] Add Guardrail base class and JudgeGuardrail implementation (#21964, @TomeHirata)
[Tracing] Add multimodal demo traces with synthetic image and audio data (#22465, @kriscon-db)
[] [FR] #22217 Add configurable provider allow/deny list to AI Gateway (#22291, @etirelli)
[UI] [UI] Add bulk duplicate and delete actions to AI Gateway endpoints table (#22404, @xq-yin)
[UI] Redesign endpoint details model section with unified Model card (#22376, @xq-yin)
[] Add X-MLflow-Gateway-Duration-Ms and X-MLflow-Gateway-Overhead-Duration-Ms response headers (#22229, @PattaraS)
[UI] Redesign AI Gateway endpoint details: Overview tab, starter code, and Try in Browser (#22284, @xq-yin)
[] Add guardrail proto definitions, server handlers, and REST store (#21962, @TomeHirata)
[Build] Add aiohttp as a core dependency of mlflow (#22189, @TomeHirata)
[] Add gateway guardrail DB layer: schema, store methods, and tests (#21960, @TomeHirata)
[UI] Add quick-start template cards to AI Gateway empty state (#22249, @xq-yin)
[Tracing / UI] Add freeform Notes section to trace assessments pane (#21676, @smoorjani)
[Tracing / UI] [UI] Improve traces table visual design and UX (#20424, @ridgupta26)
[UI] Improve Image Grid chart UX for multi-run image comparison (#22118, @Rasaboun)
[Models] Add base_model_path parameter to save PEFT adapter-only with local base model reference (#22052, @rpathade)
[Tracking] Add mlflow db move-resources command to move resources between workspaces (#21263, @mprahl)
[Tracking] feat(tracking): Add composite index on metrics (run_uuid, key, step) (#21727, @ravidarbha)

Bug fixes

[Tracing / Tracking] Redact presigned URL credentials in urllib3 retry logs (#22995, @B-Step62)
[Tracing] Fix assessment logging silently dropped in distributed tracing (#22963, @serena-ruan)
[] Filter _list_budget_windows results by active workspace when request is workspace-scoped (#22885, @copilot-swe-agent)
[Models / Scoring] Fix uv custom index URLs omitted from model requirements.txt (#22921, @serena-ruan)
[Tracing / UI] Set mlflow.message.format on gateway passthrough spans to enable Chat tab (#22916, @TomeHirata)
[Server-infra] Add RBAC regression tests + gateway validator fix (#22862, @PattaraS)
[Server-infra] Include role-based grants in list_accessible_workspace_names (#22864, @PattaraS)
[Tracing] Preserve cache_read tokens in claude_code tracing for cache observability (#22683, @dgokeeffe)
[Tracing] Fix InferenceTableSpanProcessor init for opentelemetry-sdk 1.41.0 (#22867, @harupy)
[UI] Fix audio artifact viewer failing in non-default workspaces (#22811, @harupy)
[Tracking] Fix O(n^2) body buffering in WSGIMiddleware for large uploads (#22729, @harupy)
[Tracking] Propagate workspace_id as X-Databricks-Org-Id header for SPOG support (#22554, @B-Step62)
[UI] Fix sessions pagination not auto-loading after virtualizer settles (#22808, @daniellok-db)
[Tracing] Fix _get_token_usage dropping zero-valued token counts (#22748, @copilot-swe-agent)
[Tracing] Fix _get_token_usage dropping cache token fields (#22818, @harupy)
[UI] Remove rendering size guards from artifact image and video views (#22816, @kriscon-db)
[UI] Disable Create Guardrail in AddGuardrailModal until a Guardrail Model endpoint is selected (#22766, @copilot-swe-agent)
[UI] Disable Guardrails tab in EditEndpointFormRenderer when endpoint.experiment_id is null, refresh i18n messages, and fix JS type-check follow-ups (#22757, @copilot-swe-agent)
[] Fix OPENAI_API_KEY requirement when guardrails use gateway:/ judge model (#22769, @TomeHirata)
[Tracking] Fix ABFSS URI double-encoding in _parse_abfss_uri (#22759, @artjen)
[UI] Persist immediate gateway usage-tracking updates and preserve dirty edits in useEditEndpointForm (#22734, @copilot-swe-agent)
[] Fix Vertex AI gateway global location handling for Gemini 3 models (#22696, @harupy)
[Tracing] Fix ValueError: Circular reference detected crash in dump_span_attribute_value (pydantic_ai autolog) (#22693, @barry3406)
[Tracing] Add Bedrock Converse models to model catalog (#22515, @ajgassner)
[Tracing] Fix incorrect cost calculation for Anthropic cached requests (#22516, @ajgassner)
[UI] Disable Create button when experiment name is empty (#22649, @ssam18)
[Tracing] Fix AsyncHttpxClientWrapper AttributeError by avoiding deepcopy in TraceJSONEncoder (#22742, @harupy)
[Evaluation] Fix JSON parsing failure for judge model responses containing newlines (#22736, @harupy)
[Evaluation] Fix non-ASCII characters being escaped in judge template variables (#22728, @harupy)
[Tracking] Fix pandas 3.x datetime64 resolution compatibility in cast_df_types_according_to_schema (#22705, @copilot-swe-agent)
[Tracing] Fix conversation simulator adding wrapper span that breaks custom scorers (#21321, @B-Step62)
[] Fix Bedrock tool-use chat response parsing (#22677, @lavaFreak)
[UI] Fix AI Gateway provider dropdown by flattening OpenAI / Azure OpenAI (#22701, @harupy)
[Evaluation / Tracing] Preserve predict_fn signature in simulation turn tracing (#22610, @rogalski)
[Tracing / Tracking] fix: Extract cached_tokens in OpenAI streaming responses to correct cost tracking (#22620, @Rishabh-git10)
[Tracing / UI] Auto-select first endpoint and API key in issue detection modal (#22633, @serena-ruan)
[Model Registry] Use SQL MAX() in create_model_version instead of loading all rows (#22635, @neolunar7)
[Tracing] Add modality/tool pricing extraction to dev/update_model_catalog.py, backfill anthropic.json / gemini.json, and skip reasoning modality pricing (#22699, @copilot-swe-agent)
[Tracing] Fix pydantic-ai run_stream_sync autologging broken by AgentSpec forward reference in _returns_sync_streamed_result (#22666, @copilot-swe-agent)
[UI] Unify execution duration cell rendering in session header rows (#22631, @daniellok-db)
[] Implement guardrail execution in gateway API handlers (#22306, @TomeHirata)
[Tracing / Tracking] Fast-path trace-id search in frontend to avoid spans.content scan (#22433, @harupy)
[UI] Fix tag cutoff in prompts table (#22507, @nananosirova)
[Scoring] Fix pandas 3.0 StringDtype mismatch in cast_df_types_according_to_schema (#22537, @copilot-swe-agent)
[Tracing] Fix exception propagation in @mlflow.trace within @trace_disabled context (#22501, @harupy)
[UI] [UI] Show header and breadcrumb on gateway usage page when no endpoints exist (#22445, @xq-yin)
[Model Registry] Fix performance degradation for single-tenant installation (#22405, @abaland)
[UI] Render attachments inline in Details & Timeline span view (#22451, @kriscon-db)
[UI] Skip markdown rendering for oversized chat content to prevent browser freezes (#22464, @kriscon-db)
[UI] Render attachment images embedded in complex JSON structures (#22462, @kriscon-db)
[UI] Handle input_file content type in Responses API chat rendering (#22466, @kriscon-db)
[UI] Fall back to OpenAI parser for LangChain traces with OpenAI-format data (#22456, @kriscon-db)
[UI] Combine multi-part Responses API input into a single chat message (#22459, @kriscon-db)
[UI] Render text content before audio parts in chat messages (#22460, @kriscon-db)
[Tracing] Handle Gemini SDK bytes repr format in inline_data extraction (#22453, @kriscon-db)
[UI] Show all output fields alongside attachment renderings in summary view (#22449, @kriscon-db)
[UI] Handle single dict contents in Gemini chat input normalization (#22455, @kriscon-db)
[UI] Fix audio player not loading on first render in attachment view (#22450, @kriscon-db)
[UI] Allow Gemini chat rendering when role field is omitted (#22454, @kriscon-db)
[Evaluation] Add server side check for registering custom scorer (#22381, @WeichenXu123)
[UI] [UI] Cache gateway provider/model queries to eliminate redundant fetches (#22440, @xq-yin)
[Tracing] Eager-load trace relationships in search_traces (#22431, @harupy)
[Model Registry] Fix cross-workspace copy_model_version failure caused by get_logged_model call (#22262, @TomeHirata)
[UI] [UI] Fix endpoint details to display all API keys for multi-model endpoints (#22406, @xq-yin)
[UI] Fix tag cutoff in experiments table (#21871, @nananosirova)
[Tracking] Fix rename_experiment in SqlAlchemyStore to enforce 500-char name limit (#22418, @copilot-swe-agent)
[Tracking] Fix download_artifacts and list_artifacts returning wrong status for missing artifacts (#22310, @Karim-siala)
[Docs / Tracing] Enable async trace logging by default for OSS MLflow (#22304, @PattaraS)
[UI] Show Save/Cancel buttons only when endpoint has unsaved changes (#22367, @xq-yin)
[UI] Remove mp4 from AUDIO_EXTENSIONS to fix video artifact preview (#22350, @copilot-swe-agent)
[Evaluation] Fix MetaPromptOptimizer failing on prompts with no template variables (#22301, @alkispoly-db)
[Prompts / UI] Fix misleading placeholder text in PromptsListFilters (#22321, @copilot-swe-agent)
[Evaluation] Fix make_judge with bool/numeric types not reporting aggregated metrics in evaluate() (#22302, @alkispoly-db)
[UI] Fix dark mode CSS not consistently applied in compare runs page (#22254, @DaoDaoNoCode)
[Tracking] Fix 500 error in start_trace when merging traces with existing metrics (#22257, @alkispoly-db)
[Model Registry / Tracing / Tracking] Fix _all_tables_exist to derive expected_tables from Base.metadata dynamically (#22128, @copilot-swe-agent)
[UI] Hide 'Show more columns' button when runs are selected (#21867, @nananosirova)
[Evaluation] Support T | None as top-level feedback_value_type in make_judge (#22201, @copilot-swe-agent)
[UI] Fix dataset details drawer: local source display, profile parsing, broken link (#22143, @Rasaboun)
[Tracking] Fix --static-prefix not applied to /api/ REST routes (#22159, @TomeHirata)
[Tracing / Tracking] Map user.id span attribute to mlflow.trace.user during OTel ingestion (#22150, @ajgassner)

Documentation updates

[Docs] Docs: Add AI Gateway CLI Tools pages (Claude Code, Codex, Gemini CLI) (#22918, @TomeHirata)
[Docs / Tracing / UI] Add documentation for Codex CLI, Gemini CLI, and Qwen Code tracing (#22412, @kriscon-db)
[Docs] Add Live Demo documentation page for demo.mlflow.org (#22840, @serena-ruan)
[Docs] Docs: Adds Middleware in the tracing section for integrated observability platforms (#22791, @luharshekhar2000-dot)
[Docs / UI] Rename guardrail stage terminology to Pre-LLM / Post-LLM across guardrail_utils.py, Gateway API, UI, docs, and guardrailValidation.ts (#22767, @copilot-swe-agent)
[Docs] Fix EvaluationDataset type references in GenAI datasets docs (#22761, @harupy)
[Docs] docs: add guardrails documentation for AI Gateway (#22577, @TomeHirata)
[] Allow different PGPORT in docker compose (#22646, @riklopfer)
[Docs] Remove docs/docs/genai/governance/ai-gateway/legacy/* and redirect legacy AI Gateway routes to ai-gateway/index (#22579, @copilot-swe-agent)
[Docs] Remove unused parameter from invocation() function example in manual tracing docs (#22524, @prithvipal)
[Docs] Document uv run --frozen flag for offline/no-network usage in CLAUDE.md (#22505, @copilot-swe-agent)
[Docs] Add documentation for multimodal trace attachments (#21783, @kriscon-db)
[Docs] Add AI Gateway Performance & Benchmarks docs page (#22390, @copilot-swe-agent)
[Docs] Update changelog for 3.11.0 release (#22394, @WeichenXu123)

Small bug fixes and documentation updates

#22993, #22978, #21482, #21321, #22559, #21319, #22483, #22346, #22091, @B-Step62; #22919, #22925, #22854, #22821, #22786, #22820, #22615, #22819, #22781, #22703, #22565, #22200, #22193, @serena-ruan; #22869, #22863, #22843, #22839, #22837, #22815, #22765, #22762, #22749, #22708, #22486, #22481, #22471, #22401, #22366, #22278, #22281, @harupy; #22848, #22775, #22764, #22763, #22772, #22527, #22435, #22358, #22357, #22356, #22355, #22261, #22259, #22202, #22234, #22243, @TomeHirata; #22824, @vinh412; #22805, #22804, #22803, #22802, #22801, #22800, #22447, #22632, #22735, #22697, #22691, #22508, #22628, #22567, #22482, #22441, #22273, #22272, #22190, #22434, #22154, #22147, #22144, #22268, #22241, #22187, #22074, @daniellok-db; #22557, #22722, #22807, #22721, #22558, #22589, #22319, #22145, #21789, @PattaraS; #22825, #22783, #22789, #22787, #22784, #22782, #22757, #22771, #22770, #22758, #22756, #22755, #22744, #22737, #22715, #22718, #22714, #22713, #22711, #22710, #22707, #22704, #22700, #22692, #22686, #22690, #22689, #22688, #22684, #22685, #22682, #22670, #22669, #22668, #22667, #22664, #22663, #22662, #22655, #22657, #22659, #22601, #22580, #22578, #22576, #22563, #22550, #22549, #22548, #22547, #22545, #22544, #22541, #22540, #22539, #22538, #22511, #22532, #22529, #22528, #22526, #22523, #22525, #22519, #22518, #22517, #22514, #22512, #22509, #22510, #22506, #22504, #22490, #22488, #22480, #22478, #22477, #22470, #22468, #22443, #22438, #22436, #21203, #22417, #22403, #22413, #22399, #22395, #22391, #22392, #22388, #22379, #22348, #22347, #22343, #22341, #22342, #22340, #22339, #22337, #22330, #22329, #22328, #22316, #22309, #22305, #22250, #22207, #22204, #22198, #22177, #22174, @copilot-swe-agent; #22795, @rollyjoel; #22629, #22730, #22614, #22573, #22334, #22502, #22448, #22196, #22263, #22186, #22185, #22184, #21956, #22183, #22246, #21955, @kriscon-db; #22627, #22238, @xsh310; #22295, #22212, @smoorjani; #22463, #22370, #22365, #22240, #22226, @xq-yin; #22473, #22439, #22437, @WeichenXu123; #22387, #21419, @mprahl; #22382, #22373, @SeldonAi; #21848, @alkispoly-db; #22160, @amotl

MLflow 3.12.0 is a release focused on improving our LLM observability workflows, making tracing more accessible, feature-rich, and performant.

🖼️ Multimodal Tracing

Users can now store multimodal content in tracing spans as artifact attachments instead of inline binary data. We've also patched the UI to support the new mlflow-attachment:// style URI, with rich rendering available for PDFs, audio, and images.

This feature works out of the box with autologging, but manual attachment management is also possible. Visit the documentation page to learn more.

🤖 Codex, Gemini, Qwen coding agent tracing support

Similar to our Claude Code tracing integration, we've now added support for the Codex, Gemini, and Qwen coding agent platforms as well! For intructions on how to get started, check out the doc pages at:

• Codex
• Gemini CLI
• Qwen Code

🛡️ Gateway guardrails

You can now set guardrails on your gateway endpoints to prevent unsafe or non-compliant model inputs and outputs. Try it out in the MLflow UI, and visit the documentation page to learn more!

⚡ Trace table pagination

The traces tab is now paginated, rather than fetching all traces up to a limit of 1000. This improves initial load time, and makes the page feel more responsive overall.

Full Changelog

For a comprehensive list of changes, see the release change log.

What's Next

Get Started

Install MLflow 3.12.0 to try these new features:

pip install mlflow==3.12.0

Share Your Feedback

We'd love to hear about your experience with these new features:

• GitHub Issues - Report bugs or request features
• MLflow Roadmap - See what's coming next and share your ideas
• ⭐ Star us on GitHub - Show your support for the project

Learn More

• Join our upcoming webinar to see these features in action
• Check out the MLflow documentation for detailed guides

We're excited to announce MLflow 3.12.0rc0, which brings powerful new capabilities for agent developers:

Major New Features

• Automatic Tracing for more AI Coding Assistants: First-class tracing for Claude Code, Codex, Qwen Code, and Gemini CLI, now distributed as standalone TypeScript-based plugins installable as CLI binaries. Drop one into your CLI of choice and capture every prompt, tool call, and turn out-of-the-box. (#22338, #22410, #22411, #22409, #22853, @B-Step62, @kriscon-db)
• OpenClaw Tracing Plugin: New TypeScript-based tracing plugin for OpenClaw. Install via npm and get full automatic tracing of OpenClaw sessions, no SDK changes required. Captures every prompt, tool call, and response across the session, allowing you to inspect agent behavior directly in the MLflow UI. (#22717)
• AI Gateway Guardrails: Add safety checks to any agent endpoint with the new Guardrail base class and JudgeGuardrail implementation. Configure pre-LLM and post-LLM guardrails directly from the Gateway endpoint editor, with full DB persistence, REST API, and gateway-level execution. Protect production agents from unsafe inputs and outputs without writing wrapper code. Tracing spans for guardrail execution are included. (#21964, #21960, #21962, #22306, #22360, #22577, #22581, #22767, @TomeHirata)
• Multimodal Trace Attachments: Trace agents that handle images, audio, and files, and view them directly in the MLflow UI. Click-to-expand image modals, inline rendering in span Details and Timeline views, audio playback, multi-part chat normalization for Gemini and OpenAI Responses API, and size guards for large media so the UI stays responsive. (#22465, #22461, #22451, #22462, #22466, #22460, #22449, #22450, #22574, #22575, #21783, @kriscon-db)
• mlflow.diffusers Flavor: New first-class flavor for diffusion models. Save and serve LoRA adapters for image-generation pipelines with the same mlflow.<flavor>.log_model ergonomics as PyTorch or Transformers. (#22253, @Rasaboun)

Stay tuned for the full release, which will include even more features and bug fixes.

To try out this release candidate, please run:

pip install mlflow==3.12.0rc0

MLflow 3.11.1 includes several major features and improvements.

Major New Features

• 🔍 Automatic Issue Identification: Automatically identify quality issues in your agent with AI! Use the new "Detect Issues" button in the traces table to analyze selected traces and surface potential problems across categories like correctness, safety, and performance. Issues are linked directly to traces for easy investigation and debugging. Docs (#21431, #21204, #21165, #21163, #21161, @smoorjani, @serena-ruan)

• 💰 Gateway Budget Alerts & Limits: Control your AI Gateway spending with configurable budget policies! Set spending limits by time window (daily, weekly, or monthly), receive alerts before hitting limits, and prevent runaway costs with automatic request blocking. The new budget management UI lets you track spending, configure webhooks for notifications, and monitor violations across all your gateway endpoints. Docs (#21116, #21534, #21569, #21473, #21108, @TomeHirata, @copilot-swe-agent)

• 📊 Trace Graph View: Visualize complex trace hierarchies with an interactive graph view! Navigate multi-level trace structures, understand parent-child relationships at a glance, and debug complex systems more effectively with a visual representation of your trace topology. Docs (#20607, @joelrobin18)

• 🌐 Native OpenTelemetry GenAI Convention Support: MLflow now natively supports the OpenTelemetry GenAI Semantic Conventions for trace export! When exporting traces via OTLP with MLFLOW_ENABLE_OTEL_GENAI_SEMCONV enabled, MLflow automatically translates them to follow the OTel GenAI semantic conventions, enabling seamless integration with OTel-compatible observability platforms while preserving GenAI-specific metadata. Docs (#21494, #21495, @B-Step62)

• 🔧 OpenCode Tracing Integration: Debug smarter with OpenCode CLI integration! Track and analyze code execution flows directly from your development workflow, making it easier to identify performance bottlenecks and trace issues back to specific code paths. Docs (#20133, @joelrobin18)

• ⚡ Native UV Support for Model Dependencies: Automatic dependency inference now supports UV! MLflow automatically detects UV projects and captures exact, locked dependencies from your lockfile when logging models, ensuring reproducible environments. Docs (#20344, #20935, @debu-sinha)

• 🔒 Pickle-Free Model Serialization: Enhance security with pickle-free model formats! MLflow now supports safer model serialization using torch.export and skops formats, with improved controls when MLFLOW_ALLOW_PICKLE_DESERIALIZATION=False. Comprehensive documentation guides you through migrating existing models to pickle-free formats for production deployments. Docs (#21404, #21188, #20774, @WeichenXu123)

Breaking Changes

• ⚠️ TypeScript SDK Package Renaming: The MLflow TypeScript SDK packages have been renamed to use npm organization scoping. If you're using the TypeScript SDK, update your package.json dependencies and import statements: mlflow-tracing → @mlflow/core, mlflow-openai → @mlflow/openai, mlflow-anthropic → @mlflow/anthropic, mlflow-gemini → @mlflow/gemini. All packages are now at version 0.2.0. (#20792, @B-Step62)

• Remove MLFLOW_ENABLE_INCREMENTAL_SPAN_EXPORT environment variable (#22182, @PattaraS)

• Remove litellm and gepa from genai extras (#22059, @TomeHirata)

• Block / and : in Registered Model names (#21458, @Bhuvan-08)

Features

• [Evaluation] Allow MetaPromptOptimizer to work without litellm (#22233, @TomeHirata)
• [Tracking] Update Databricks API calls to use new gRPC APIs instead of py4j APIs (#22205, @WeichenXu123)
• [Build] Add aiohttp as a core dependency of mlflow (#22189, @TomeHirata)
• [Evaluation] Extend _get_provider_instance with groq, deepseek, xai, openrouter, ollama, databricks, vertex_ai (#22148, @kriscon-db)
• [UI] Move native providers to non-LiteLLM in gateway UI (#22203, @TomeHirata)
• [Tracing / Tracking] Add trace_location parameter to create_experiment (#22075, @dbrx-euirim)
• [Gateway] Complete Bedrock provider with Converse API support (#21999, @TomeHirata)
• [Gateway] Add native Vertex AI gateway provider (#21998, @TomeHirata)
• [Gateway] Add native Databricks gateway provider (#21997, @TomeHirata)
• [Gateway] Add native Ollama gateway provider (#21995, @TomeHirata)
• [Gateway] Add native xAI (Grok) gateway provider (#21993, @TomeHirata)
• [Tracing] Use bulk upsert in log_spans() to eliminate per-span ORM overhead (#21954, @harupy)
• [Tracing] Add builtin cost_per_token to remove litellm dependency for cost tracking (#22046, @TomeHirata)
• [Evaluation] Remove LiteLLM hard dependency from the discovery pipeline and judge adapters (#21739, @harupy)
• [Evaluation] Add pipelined predict-score execution for mlflow.genai.evaluate (#20940, @alkispoly-db)
• [Tracing / Tracking] Default trace location table_prefix to experiment ID in set_experiment (#21815, @danielseong1)
• [Tracking] Add default uvicorn log config with timestamps (#21838, @harupy)
• [Tracing / UI] Add Session ID filter to GenAI traces table filter dropdown (#21794, @daniellok-db)
• [Evaluation / UI] Add Default Credential Chain auth mode for Bedrock/SageMaker in AI Gateway (#21061, @timsolovev)
• [UI] Add multi metric bar chart support (#21258, @RenzoMXD)
• [Tracking] Add TCP keepalive to HTTP sessions to detect stale connections and reduce timeout hangs (#21514, @mobaniha)
• [Evaluation] Add proxy URL support for make_judge (#21185, @yukimori)
• [UI] Improve run group filter to use grouping criteria instead of run IDs (#21072, @daniellok-db)
• [UI] Add tool selector to Tool Calls charts and fix dark mode/sizing (#20865, @B-Step62)
• [UI] Graph View Traces + OpenAI (#20607, @joelrobin18)
• [UI] Show run description in chart tooltip (#21580, @KaushalVachhani)
• [Evaluation / Tracing / UI] Add bulk judge execution from traces table toolbar with status feedback (#21270, @PattaraS)
• [Gateway] Add Redis-backed BudgetTracker for distributed gateway deployments (#21504, @TomeHirata)
• [Tracing / Tracking] Add trace location param to set_experiment (#21385, @danielseong1)
• [Build / Tracking] Add azure extra for Azure Blob Storage support in full Docker image (#21582, @harupy)
• [UI] Add budget violation indicator to gateway budget list page (#21569, @copilot-swe-agent)
• [Evaluation] [5/5] Add discover_issues() pipeline and public API (#21431, @smoorjani)
• [UI] Add Structured Output (JSON Schema) Support to the MLflow Prompts UI (#21394, @kennyvoo)
• [Tracing] Auto-inject tracing context headers in autologging (#21490, @TomeHirata)
• [UI] Add budget alert webhooks UI and fix budgets table borders (#21534, @TomeHirata)
• [Model Registry / Prompts / UI] Add webhooks management UI to settings page (#21483, @TomeHirata)
• [Tracing] Opencode CLI (#20133, @joelrobin18)
• [Models] Add uv_groups and uv_extras params for uv dependency group control (#20935, @debu-sinha)
• [Tracing] Add GenAI Semantic Convention translator for OTLP trace export (#21494, @B-Step62)
• [Tracking] Add polars dataset support to autologging (#21507, @harupy)
• [Tracing] Add mlflow.tracing.context() API for injecting metadata/tags without wrapper spans (#21318, @B-Step62)
• [UI] Add budget dates and current spending for gateway budgets (#21473, @TomeHirata)
• [Tracing / UI] Improve DSPy trace chat view readability (#21296, @B-Step62)
• [UI] Add Kubernetes request auth provider plugin (#21176, @HumairAK)
• [Tracking] Add IS NULL/IS NOT NULL support for tags and params in search_runs (#21283, @TomeHirata)
• [Tracing / UI] Display clickable gateway trace link in trace explorer (#21316, @TomeHirata)
• [UI] Add session selection support with checkbox, actions, and row alignment (#21324, @B-Step62)
• [Models] Add UV package manager support for automatic dependency inference (#20344, @debu-sinha)
• [Evaluation / UI] Add feature flag to control evaluation runs issues panel visibility (#21406, @serena-ruan)
• [Tracing / UI] Add cached tokens display to Token Usage chart (#21295, @TomeHirata)
• [UI] Add budget policies management UI for AI Gateway (#21116, @TomeHirata)
• [UI] Allow multiple judge selection in Run judge on trace modal (#21322, @B-Step62)
• [Docs / Tracking] Add admin-only authorization to webhook CRUD operations (#21271, @TomeHirata)
• [Evaluation / Tracking] Add SqlIssue database table for storing experiment issues (#21165, @serena-ruan)
• [Model Registry / Prompts] Support search_prompt_versions in OSS SQLAlchemy store (#21315, @TomeHirata)
• [Evaluation / Tracing / UI] Add issue detection button to traces table toolbar with feature flag (#21204, @serena-ruan)
• [Docs / Tracing / UI] Add inline audio player for input_audio content parts in trace UI (#21302, @TomeHirata)
• [Evaluation / Tracing] Add IssueReference assessment type to store issue links with traces (#21163, @serena-ruan)
• [Evaluation / Tracing] Add issue management protos with create, update, get, and search APIs (#21161, @serena-ruan)
• [UI] Add IS NULL/IS NOT NULL operators for trace tags in search UI (#21280, @TomeHirata)
• [Docs / Tracing] Add IS NULL/IS NOT NULL support for trace tags in search_traces (#21277, @TomeHirata)
• [Tracing] Add steer message tracing support for Claude Code (#21265, @harupy)
• [Models / Tracking] Add support for transformers 5.x (#20728, @KUrushi)
• [Gateway] Add WEEKS to BudgetDurationUnit enum (#21196, @copilot-swe-agent)
• [UI] Add try-it page on Gateway usage example modal (#21077, @PattaraS)
• [Docs / Tracing / Tracking] Add mlflow.otel.autolog() for OTEL-based tracing integrations (Langfuse, Arize/Phoenix) (#20954, @alkispoly-db)
• [Gateway] Add SQL schema and SQLAlchemy CRUD for gateway budget policies (#21108, @TomeHirata)
• [UI] Add global gateway logs tab to usage page (#21126, @TomeHirata)
• [Tracking] [MLflow Demo] Add server availability handling checks (#20349, @BenWilson2)
• [Tracking] [MLflow Demo] Add scorers demo (#20287, @BenWilson2)
• [Docs / Tracking] Add Backblaze B2 artifact repository (b2://) (#20731, @jeronimodeleon)
• [Docs / Tracking] Add support for multipart download with presigned URLs for S3 compatible object storages (#20352, @etirelli)
• [Tracing] MCP server expansion (#19830, @joelrobin18)
• [Tracing / UI] Include response body in HTTP error messages with 1000 character limit (#20794, @copilot-swe-agent)

Bug fixes

• [Gateway] Fix DatabricksProvider to use OpenAI-compatible endpoint URLs (#22393, @TomeHirata)
• [Evaluation] Fix: use EvalResult.scorer_stats for multi-turn scorer stat aggregation (#22364, @copilot-swe-agent)
• [Scoring / Tracing] Revert "Register InferenceTableSpanProcessor alongside DatabricksUCTableSpanProcessor in model serving (#22332)" (#22362, @smurching)
• [Scoring / Tracing] Warn when UCSchemaLocation destination is set in Databricks model serving (trace: null) (#22332, @smurching)
• [Tracing / UI] Support tool_reference content blocks in Anthropic Chat UI parser (#22331, @B-Step62)
• [Tracing] Fix online scoring failure when trace spans are stored in artifact repo (#20784, @Mr-Neutr0n)
• [UI] Fix adding a tag with empty value silently failing without user feedback in the Experiments table (#22320, @WeichenXu123)
• [Docs / Models] Bump minimum uv version requirement from 0.5.0 to 0.6.10 (#22313, @copilot-swe-agent)
• [Scoring] Fix: exclude Serverless from use_dbconnect_artifact path in spark_udf (#22300, @franciffu723)
• [UI] Fix assistant crash on unknown CLI message types (#21928, @SuperSonnix71)
• [Tracing / Tracking] Fix mlflow-skinny: guard numpy-dependent imports in mlflow.types (#22211, @Suraj-kumar00)
• [Tracing / UI] Fix dropdown showing wrong selection state before endpoints load in issue detection modal (#22236, @serena-ruan)
• [Tracing] Normalize get_provider_name() to align with model_prices_and_context_window.json (#22223, @TomeHirata)
• [Tracking / UI] Fix log_image with slash-containing keys: replace # with ~ as path separator (#22172, @copilot-swe-agent)
• [Evaluation] Fix discovery pipeline _call_llm_via_gateway to handle gateway:/ URIs (#22153, @TomeHirata)
• [UI] Auto-dismiss and fade-out judge run notifications in trace UI (#22137, @copilot-swe-agent)
• [Evaluation / Tracking] Add polars version guard in polars_dataset.py to fix import failure with polars<1 (#22085, @TomeHirata)
• [Tracking] Fix huey_consumer.py path resolution when venv bin dir is not on PATH (#22126, @copilot-swe-agent)
• [UI] Fix sidebar navigation highlighting for run detail pages (#20860, @daniellok-db)
• [Tracing] Lowercase model_provider in calculate_cost_by_model_and_token_usage (#22134, @TomeHirata)
• [Gateway] Fix misleading "Discarded unknown message" log in Anthropic gateway provider (#21942, @copilot-swe-agent)
• [UI] Fix selected run URL param not updating in eval runs table (#22135, @daniellok-db)
• [Tracing / Tracking] Fix trace export DB contention by disabling incremental span export for gateway (#21721, @PattaraS)
• [Tracing / Tracking] Expand session-level assessment filters to return all session traces (#21792, @daniellok-db)
• [Evaluation] Support inference_params for built-in scorers (#21943, @debu-sinha)
• [UI] Fix assistant stream killed by unhandled rate_limit_event from Claude Code CLI (#22067, @forrestmurray-db)
• [UI] Fix gateway UI not showing custom model name during endpoint edit (#22068, @TomeHirata)
• [Evaluation] Fix Anthropic structured outputs compatibility in gateway adapter (#21922, @harupy)
• [UI] Remove assessment type dropdown and align terminology (#21379, @B-Step62)
• [Tracking] Fix NextMethod() S3 dispatch error in R mlflow_get_run_context (#21957, @daniellok-db)
• [Models / Tracking] Enforce auth on logged model artifact download AJAX endpoint (#21708, @B-Step62)
• [Scoring] Fix tar path traversal vulnerability in extract_archive_to_dir (#21824, @TomeHirata)
• [Scoring] Fix Starlette 1.0 compatibility in mlflow/pyfunc/scoring_server/init.py (#21908, @copilot-swe-agent)
• [Tracing] [TS SDK] Port smart preview truncation from Python SDK (#21826, @B-Step62)
• [UI] Fix trace drawer width using context instead of prop drilling (#21830, @B-Step62)
• [UI] fix: use both registrations and tags for consistent registered model display (#20671) (#21555, @s-zx)
• [Tracking] Fix autologging overwriting user's warnings.showwarning handler (#21707, @mango766)
• [Tracing] Remove trace limit in issue discovery to annotate all affected traces (#21736, @serena-ruan)
• [Scoring] fix: accept Sequence instead of list in to_chat_completions_input (#21724, @mr-brobot)
• [Tracking] Set UV_PROJECT_ENVIRONMENT in run_uv_sync to install into the correct Python environment (#21750, @copilot-swe-agent)
• [Tracing / UI] Fix chat UI rendering for OTel GenAI traces with non-standard attributes (#21215, @B-Step62)
• [Build] Fix build-system in examples/uv-dependency-management/pyproject.toml (#21752, @copilot-swe-agent)
• [Tracing] fix: avoid deepcopy in dataclass JSON serialization in TraceJSONEncoder (#21668, @raulblazquezbullon)
• [Tracing] Support artifact-repo traces in batch_get_traces (#21650, @harupy)
• [Evaluation / Tracing] Fall back to agentic judge mode when trace inputs/outputs are missing (#21306, @TomeHirata)
• [UI] Fix chat/session view for LangGraph: deduplicate accumulated messages (#21279, @B-Step62)
• [UI] Show server error detail in Try It panel for budget limit errors (#21568, @TomeHirata)
• [Evaluation] Fix conversation simulator default model encoding on Databricks (#21644, @smoorjani)
• [UI] Delete model definitions when endpoint is deleted from UI (#21649, @TomeHirata)
• [UI] Hide _issue_discovery_judge feedback from traces UI (#21648, @harupy)
• [Prompts] Clarify OSS register_prompt tag behavior (#21600, @yangbaechu)
• [Prompts / Tracing / UI] Make Prompt column clickable in trace view (#21304, @copilot-swe-agent)
• [UI] Fix dataset link not clickable for external source type (#21342, @smoorjani)
• [Tracing / Tracking] Add audio content normalization for LangChain messages (#21533, @elliotllliu)
• [UI] Add tooltips to display full budget and spend amounts in gateway budgets table (#21573, @copilot-swe-agent)
• [Tracking / UI] downsample rows in SQL, update db index (#20928, @sscheele)
• [Models] Skip _maybe_save_model for Databricks ACL-protected artifact URIs (#21602, @mohammadsubhani)
• [UI] Make Try-It UI footer always visible in gateway endpoint modal (#21583, @copilot-swe-agent)
• [Tracing / Tracking] Fix trace assessment filtering and MSSQL pagination syntax errors (#21273, @copilot-swe-agent)
• [Tracing] Fix trace sampling to ensure parent-child consistency (#21524, @harupy)
• [Tracking] Add Azure Government Cloud (usgovcloudapi.net) support to WASBS URI parsing (#21519, @ahringer)
• [Gateway] Change default MLFLOW_GATEWAY_BUDGET_REFRESH_INTERVAL from 60 to 600 seconds (#21565, @copilot-swe-agent)
• [Evaluation / Tracking] Fix scorer re-registration raising RESOURCE_ALREADY_EXISTS in auth layer (#21560, @harupy)
• [Tracking] Harden check when MLFLOW_ALLOW_PICKLE_DESERIALIZATION is disabled (#21404, @WeichenXu123)
• [Tracing] Fix trace ID collisions when random seed is set to fixed value (#21418, @WeichenXu123)
• [UI] Remove "Rate Limiting [Coming Soon]" placeholder from gateway UI (#21559, @copilot-swe-agent)
• [Gateway] Remove policy ID from budget limit exceeded error, show budget reset time instead (#21557, @copilot-swe-agent)
• [Evaluation / Tracking] Fix Strands autolog tool input format for SpanType.TOOL (#21552, @LeviLong01)
• [Tracing] Fix AttributeError in OpenAI autolog by excluding run_config from span attributes (#21454, @MarkVasile)
• [Gateway] Fix singular/plural unit in budget limit exceeded error message (#21538, @copilot-swe-agent)
• [UI] Invalidate budget windows cache on budget policy create/edit/delete (#21535, @copilot-swe-agent)
• [Evaluation] Fix field-based make_judge prompt missing feedback_value_type (#21058, @yangbaechu)
• [Tracing] Set MODEL_PROVIDER across autologging integrations for cost breakdown (#21288, @B-Step62)
• [Evaluation] Fix gateway provider support in third-party judge integrations (ragas, deepeval, phoenix, trulens) (#21414, @copilot-swe-agent)
• [Gateway] Update Anthropic gateway to use GA structured outputs API (#21436, @TomeHirata)
• [Tracking] Adds builtin skops trusted types for LightGBM models (#21412, @WeichenXu123)
• [Tracing / UI] Fix UI flickering in trace review modal during background refetches (#21290, @daniellok-db)
• [Tracking] Add wildcard subdomain support to CORS origins validation (#21468, @arnewouters)
• [UI] Fix refresh button on evaluation runs page to also refresh traces and assessments (#21332, @B-Step62)
• [Models] Fix skops serialization format detection in _load_pyfunc (#21480, @copilot-swe-agent)
• [UI] Fix Shift+Enter not creating newlines in assistant chat input (#21341, @smoorjani)
• [UI] Make retrieved document source URLs clickable in span details view (#21340, @smoorjani)
• [Evaluation / Tracing] Fix AttributeError when trace is None in genai evaluation (#19616, @omarfarhoud)
• [Tracking] Fix CrewAI autologging compatibility with crewai >= 1.10 (#21376, @WeichenXu123)
• [Tracing] Remove span name deduplication suffix from TypeScript SDK (#21382, @B-Step62)
• [Evaluation] Fix LLM judge authentication failure when basic-auth is enabled (#21323, @PattaraS)
• [UI] Fix stored XSS via unsafe YAML parsing of MLmodel artifacts (#21435, @harupy)
• [Tracing / UI] Fix Pydantic AI Chat UI rendering for InstrumentedModel LLM spans (#21410, @B-Step62)
• [Models] Fix transformers 5.3.0 compatibility for removed pipeline classes (#21426, @harupy)
• [Tracing / UI] Fix Chat UI not rendering for Google ADK traces (#21274, @B-Step62)
• [Tracking] Fix image artifact filename mangling caused by URL encoding of % separator (#21269, @harupy)
• [Tracking] Fix: MLFLOW_ALLOW_PICKLE_DESERIALIZATION=False safety control is ineffective for pyfunc flavor (#21188, @WeichenXu123)
• [Tracing / UI] Fix Pydantic AI autologging: auto-enable instrumentation and fix Chat UI (#21278, @B-Step62)
• [Tracing] Fix span type not translated for OTel spans when MLflow SDK is active (#21307, @B-Step62)
• [UI] Remove redundant "Hide assessments" toggle button (#21378, @B-Step62)
• [Tracking] Fix Mistral autologging compatibility with mistralai >= 2.0 (#21374, @WeichenXu123)
• [Tracking] Fix pydantic-ai autologging compatibility with pydantic-ai >= 1.63.0 (#21373, @WeichenXu123)
• [Tracing / Tracking] Fix Claude Code autologging import collision with local mlflow folders (#21343, @smoorjani)
• [Prompts] Fix stale prompt cache after prompt deletion (#21381, @yangbaechu)
• [Tracing / Tracking] Fix flush_trace_async_logging AttributeError with non-default tracer provider (#21105, @cgrierson-smartsheet)
• [UI] Fix session assessments panel terminology (#21336, @smoorjani)
• [UI] Improve quality chart readability and styling in overview tab (#21325, @B-Step62)
• [Tracing] Support uv run in Claude Code tracing hooks (#21327, @copilot-swe-agent)
• [Tracing / UI] Fix Chat tab not rendering for non-OpenAI model names in OpenAI autolog spans (#21356, @TomeHirata)
• [UI] Fix false 'endpoint deleted' warning after endpoint rename (#21333, @TomeHirata)
• [UI] Fix broken image rendering in trace chat collapsed preview (#21291, @harupy)
• [UI] Fix tag key validation UI contradiction (#21140, @KaushalVachhani)
• [Tracing] Use correct env key for Claude Code settings environment variables (#21344, @smoorjani)
• [UI] Fix truncated model names in Cost Breakdown donut chart (#21310, @TomeHirata)
• [Evaluation / Tracing] Fix ConversationSimulator validation for predict_fn signatures and context fields (#21171, @yangbaechu)
• [UI] [ML-63097] Fix broken LLM judge documentation links (#21347, @smoorjani)
• [Tracing / Tracking] Add authentication support to OTLP exporter headers (#21230, @giulio-leone)
• [Evaluation / Tracking] Fix deletion of assessments associated with a run (#20624, @retrowhiz)
• [Models] Fix _deduplicate_requirements merging marker-differentiated requirements (#21098, @harupy)
• [UI] Fix Tags functionality in Recent Experiments table on Home page (#20907, @joelrobin18)
• [Tracing] Fix MCP fn_wrapper handling of Click UNSET defaults (#20953) (#20962, @yangbaechu)
• [Evaluation] Enable Databricks LLM fallback for available tools extraction (#21017, @xsh310)
• [UI] Fix sorting for timestamp columns in ExperimentListTable (#20908, @joelrobin18)
• [UI] Fix tag value input being cleared when entered before key (#20910, @joelrobin18)
• [Docs] Fix LiteLLM model URI format in eval quickstart docs (#20941, @copilot-swe-agent)
• [Tracing] Fix SpanEvent timestamp resolution to use nanoseconds (#20828, @copilot-swe-agent)
• [Tracking] Escape regex special chars in search_experiments LIKE filter (#16667, @joelrobin18)

Documentation updates

• [Docs] docs: clarify uv dependency management vs MLFLOW_LOCK_MODEL_DEPENDENCIES, add uv workspace limitation (#22312, @copilot-swe-agent)
• [Docs] Document supported provider environment variables for judge models (#22195, @kriscon-db)
• [Docs] Add relative duration examples for uv --exclude-newer (#22133, @copilot-swe-agent)
• [Docs] Add secure installs documentation page (#22036, @harupy)
• [Evaluation] Add documentation for issue detection (#22057, @serena-ruan)
• [Tracing] Add OpenHands integration doc (#21933, @B-Step62)
• [Docs / Tracing] Fix MLFLOW_ENABLE_ASYNC_TRACE_LOGGING docs to reflect OSS default behavior (#21731, @copilot-swe-agent)
• [Docs] Add note for pickle-free model doc (#21732, @WeichenXu123)
• [Docs] Add experiment note to the pickle-free model format doc page (#21709, @WeichenXu123)
• [Docs] Add Guide: Deploy MLflow to Google Cloud (#21599, @WeichenXu123)
• [Docs] Add Guide: Deploy MLflow to Azure cloud (#21128, @WeichenXu123)
• [Docs / Tracing] Add Goose tracing integration documentation (#21190, @B-Step62)
• [Docs] Expand Koog integration doc (#21218, @B-Step62)
• [Docs / Tracing] Add 'Combine with MLflow SDK' section to OTel integration guides (#21298, @TomeHirata)
• [Docs] docs: add Budget Tracker Strategies guideline to AI Gateway budget page (#21633, @copilot-swe-agent)
• [Docs] Add tracking URI note to mlflow-skinny README (#21638, @harupy)
• [Docs] Add Guide: Deploy MLflow to AWS cloud (#20729, @WeichenXu123)
• [Docs / Models] Deprecate generate_signature_output in favor of input_example (#21556, @shivamshinde123)
• [Docs] Claude MCP setup instructions to use .mcp.json or CLI (#21609, @copilot-swe-agent)
• [Docs] [1/3] Document OTel attribute mapping (#21478, @B-Step62)
• [Docs] docs: Add OpenAI Responses API examples to gateway passthrough documentation (#21545, @copilot-swe-agent)
• [Docs] Add standalone multimodal content in traces documentation (#21357, @kriscon-db)
• [Docs] Add documentation page for Budget Alerts & Limits (#21121, @TomeHirata)
• [Docs / Models] Add documentation for pickle-free model formats (#20774, @WeichenXu123)
• [Docs / Prompts] Update prompt registry docs to use MLflow 3.x API examples (#21267, @copilot-swe-agent)
• [Docs] docs: Add single quotes to install commands with extras to prevent zsh errors (#21227, @mshavliuk)
• [Docs] Add Amazon Nova bedrock model examples for mlflow.metrics.genai (#21063, @ManasVardhan)
• [Docs] Update SSO oidc plugin doc: add google identity platform / AWS cognito / Azure Entra ID configuration guide (#20591, @WeichenXu123)

Small bug fixes and documentation updates

#22377, #22258, #22260, #22259, #21988, #22000, #21994, #21992, #21991, #21990, #21989, #21986, #21918, #21919, #20739, #21753, #21784, #21785, #21786, #21653, #21647, #21558, #21572, #21567, #21571, #21540, #21544, #21542, #21120, #21114, #21112, #21198, #21111, #21289, #21110, #20743, #21109, #20960, #21125, #21124, #22369, #22261, #22234, #22220 @TomeHirata; #22370, #22277, @xq-yin; #22346, #22311, #22091, #21930, #22235, #22232, #22086, #22008, #21975, #21866, #21940, #21920, #21931, #21820, #21830, #21825, #21810, #21788, #21712, #21620, #21702, #21479, #21495, #21506, #21377, #21010, @B-Step62; #22166, @SomtochiUmeh; #22283, #22083, #22200, #22248, #21317, #22242, #22173, #22239, #22193, #22197, #22224, #22225, #22222, #22138, #22175, #22176, #21932, #22053, #22039, #21860, #21268, #21863, #21833, #21917, #21864, #21804, #21854, #21803, #21840, #21837, #21802, #21831, #21835, #21797, #21821, #21758, #21793, #21747, #21796, #21746, #21718, #21756, #21787, #21745, #21741, #21734, #21719, #21715, #21713, #21716, #21661, #21701, #21407, #21589, #21655, #21703, #21664, #21700, #21663, #21662, #21623, #21622, #21619, #21618, #21546, #21654, #21597, #21625, #21596, #21595, #21594, #21613, #21593, #21592, #21591, #21590, #21588, #21579, #21578, #21577, #21575, #21536, #21531, #21510, #21509, #21492, #21500, #21363, #21499, #21498, #21532, #21497, #21502, #21491, #21361, #21409, #21360, #21408, #21437, #21189, #21187, #21167, #21398, #21209, #21208, #21166, #21207, #21286, #21284, #21367, #21366, #21365, #21206, #21164, #21162, #21297, #21303, #21261, #21090, @serena-ruan; #22179, #22191, #22178, #22007, #22004, #21852, #21799, #21832, #21759, #21717, #21657, #21624, #21541, @kriscon-db; #22374, #21503, #20795, #21372, @WeichenXu123; #22192, #22139, #22136, #22108, #22107, #22104, #22103, #22099, #22094, #22093, #22090, #22087, #22079, #22065, #22064, #22058, #22062, #22051, #22048, #22044, #22035, #22026, #22025, #22021, #21965, #21961, #21959, #21952, #21937, #21936, #21926, #21924, #21916, #21914, #21912, #21911, #21901, #21900, #21897, #21896, #21894, #21891, #21888, #21887, #21885, #21880, #21879, #21878, #21876, #21877, #21875, #21868, #21862, #21861, #21859, #21834, #21808, #21822, #21807, #21766, #21782, #21761, #21757, #21742, #21740, #21737, #21733, #21730, #21729, #21728, #21710, #21694, #21683, #21684, #21677, #21675, #21672, #21671, #21670, #21652, #21651, #21634, #21629, #21627, #21621, #21610, #21543, #21529, #21527, #21523, #21513, #21511, #21508, #21501, #21496, #21486, #21485, #21481, #21477, #21474, #21472, #21471, #21464, #21462, #21457, #21459, #21456, #21455, #21452, #21451, #21449, #21448, #21442, #21441, #21415, #21411, #21402, #21397, #21375, #21369, #21330, #21353, #21335, #21331, #21328, #21285, #21251, #21239, #21235, #21229, #21228, #21224, #21210, #21153, #21147, #21139, #21122, #21113, #21117, #21095, #21091, #21092, #21089, #21088, #21056, #21047, #21044, #21043, #21042, #21040, #21031, #21032, #21024, #21021, #21018, #21011, #21016, #20997, #20994, #20985, #20982, #20981, #20978, #20967, #20966, #20942, #20921, #20916, #20911, #20905, #20909, #20906, #20823, #20820, #20811, #20810, #20788, #20798, #20786, #20754, #20745, #20735, #20724, #20722, #20721, @copilot-swe-agent; #22140, #22180, #22171, #22014, #22009, #22005, #22001, #21921, #21858, #21780, #21640, #21612, #21643, #21563, #21537, #21525, #21275, #21145, #21358, #21272, #21068, #21066, #20789, #21026, #20790, @daniellok-db; #22131, @amotl; #22080, #22056, #22019, #22017, #22018, #22015, #22016, #22013, #22011, #22002, #21996, #21985, #21984, #21981, #21971, #21939, #21838, #21806, #21798, #21748, #21666, #21665, #21642, #21637, #21566, #21539, #21405, #21484, #21439, #21440, #21395, #21195, #21194, #21150, #21029, #21014, #20987, #20738, @harupy; #22115, @sebneira; #21641, #21632, #21614, @dbrx-euirim; #21849, #20939, #20938, #20937, @alkispoly-db; #21846, #21764, #21678, #21644, #21636, #21342, #21430, #21429, #21428, #21446, #21427, #21336, #21339, #21344, #21345, #21033, @smoorjani; #21853, @bbqiu; #21827, @kevin-lyn; #21350, #21777, #21696, #21691, #21515, #21488, #21386, @dbczumar; #21528, #21673, #21549, @debu-sinha; #21658, #21600, #21493, #21393, @yangbaechu; #21660, #21639, #21443, #21444, @PattaraS; #21581, @kennyvoo; #21219, @mprahl; #21424, #21232, #21184, #21178, @danielseong1; #21173, @nananosirova; #20388, @mdalvz0000; #21080, @ManasVardhan

MLflow 3.11.1 is a major release that significantly advances MLflow's AI Observability, security, and governance capabilities. This release brings automated quality issue detection for agents, fine-grained spending controls for AI Gateway, interactive trace graph visualization, native OpenTelemetry GenAI semantic convention support, and safer pickle-free model serialization — alongside broad improvements to tracing integrations, evaluation pipelines, and the MLflow UI.

Automatic Issue Identification

Automatically surface quality problems in your agent without manual inspection! Use the new Detect Issues button in the traces table to analyze selected traces with AI and identify potential problems across categories like correctness, safety, and performance. Detected issues are linked directly to the relevant traces, making it easy to investigate root causes and debug your agent at scale.

Gateway Budget Alerts & Limits

Take control of your AI Gateway spending with configurable budget policies. Set spending limits by time window (daily, weekly, or monthly), receive alerts before hitting limits, and block runaway costs automatically when thresholds are exceeded. The new budget management UI lets you track current spending, configure webhook notifications, and monitor violations across all gateway endpoints — all without writing any code.

Trace Graph View

Navigate complex agent interactions with a new interactive graph view for traces. Visualize multi-level trace hierarchies, understand parent-child span relationships at a glance, and debug intricate multi-agent systems more effectively with a visual representation of your trace topology.

Native OpenTelemetry GenAI Convention Support

MLflow now natively supports the OpenTelemetry GenAI Semantic Conventions for trace export. When exporting traces via OTLP with MLFLOW_ENABLE_OTEL_GENAI_SEMCONV enabled, MLflow automatically translates spans to follow the OTel GenAI semantic conventions — enabling seamless integration with OTel-compatible observability platforms while preserving all GenAI-specific metadata.

OpenCode Tracing Integration

Debug smarter with the new OpenCode CLI tracing integration. OpenCode is an open-source, terminal-based AI coding assistant. Track and analyze code execution flows directly from your development workflow, making it easier to identify performance bottlenecks and trace issues back to specific code paths without leaving your terminal.

Native UV Support for Model Dependencies

Automatic dependency inference now supports UV. MLflow detects UV projects and captures exact, locked dependencies — including SHA-256 hashes for every package — from your lockfile when logging models, ensuring fully reproducible environments when serving or sharing models that were built with UV. This provides a safer approach against supply chain attacks: if an attacker publishes a modified package under an existing version number, the hash check fails and installation is blocked.

Pickle-Free Model Serialization

Enhance the security of your ML pipelines with pickle-free model formats. MLflow now supports safer model serialization using torch.export and skops formats, with improved controls when MLFLOW_ALLOW_PICKLE_DESERIALIZATION=False. Comprehensive documentation guides you through migrating existing models to pickle-free formats for production deployments.

Breaking Changes

• TypeScript SDK Package Renaming: The MLflow TypeScript SDK packages have been renamed to use npm organization scoping. Update your package.json dependencies: mlflow-tracing → @mlflow/core, mlflow-openai → @mlflow/openai, mlflow-anthropic → @mlflow/anthropic, mlflow-gemini → @mlflow/gemini. All packages are now at version 0.2.0.
• The MLFLOW_ENABLE_INCREMENTAL_SPAN_EXPORT environment variable has been removed.
• litellm and gepa have been removed from genai extras.
• / and : are now blocked in Registered Model names.

Full Changelog

For a comprehensive list of changes, see the release change log, and check out the latest documentation on mlflow.org.

What's Next

Get Started

Install MLflow 3.11.1 to try these new features:

pip install mlflow==3.11.1

Share Your Feedback

We'd love to hear about your experience with these new features:

• GitHub Issues - Report bugs or request features
• MLflow Roadmap - See what's coming next and share your ideas
• ⭐ Star us on GitHub - Show your support for the project

Learn More

Check out the MLflow documentation for detailed guides

MLflow 3.11.1 includes several major features and improvements.
Major New Features:
• Automatic Issue Identification: Automatically identify quality issues in your agent with AI! Use the new "Detect Issues" button in the traces table to analyze selected traces and surface potential problems across categories like correctness, safety, and performance. Issues are linked directly to traces for easy investigation and debugging.
• Gateway Budget Alerts & Limits: Control your AI Gateway spending with configurable budget policies! Set spending limits by time window (daily, weekly, or monthly), receive alerts before hitting limits, and prevent runaway costs with automatic request blocking. The new budget management UI lets you track spending, configure webhooks for notifications, and monitor violations across all your gateway endpoints.
• Trace Graph View: Visualize complex trace hierarchies with an interactive graph view! Navigate multi-level trace structures, understand parent-child relationships at a glance, and debug complex systems more effectively with a visual representation of your trace topology.
• Native OpenTelemetry GenAI Convention Support: MLflow now natively supports the OpenTelemetry GenAI Semantic Conventions for trace export! When exporting traces via OTLP with MLFLOW_ENABLE_OTEL_GENAI_SEMCONV enabled, MLflow automatically translates them to follow the OTel GenAI semantic conventions, enabling seamless integration with OTel-compatible observability platforms while preserving GenAI-specific metadata.
• OpenCode Tracing Integration: Debug smarter with OpenCode CLI integration! Track and analyze code execution flows directly from your development workflow, making it easier to identify performance bottlenecks and trace issues back to specific code paths.
• Native UV Support for Model Dependencies: Automatic dependency inference now supports UV! MLflow automatically detects UV projects and captures exact, locked dependencies from your lockfile when logging models, ensuring reproducible environments.
• Pickle-Free Model Serialization: Enhance security with pickle-free model formats! MLflow now supports safer model serialization using torch.export and skops formats, with improved controls when MLFLOW_ALLOW_PICKLE_DESERIALIZATION=False. Comprehensive documentation guides you through migrating existing models to pickle-free formats for production deployments.
Breaking Changes:
• TypeScript SDK Package Renaming: The MLflow TypeScript SDK packages have been renamed to use npm organization scoping. If you're using the TypeScript SDK, update your package.json dependencies and import statements: mlflow-tracing → @mlflow/core, mlflow-openai → @mlflow/openai, mlflow-anthropic → @mlflow/anthropic, mlflow-gemini → @mlflow/gemini. All packages are now at version 0.2.0. (#20792, @B-Step62)
• Remove MLFLOW_ENABLE_INCREMENTAL_SPAN_EXPORT environment variable (#22182, @PattaraS)
• Remove litellm and gepa from genai extras (#22059, @TomeHirata)
• Block / and : in Registered Model names (#21458, @Bhuvan-08)
Features:
• [Evaluation] Allow MetaPromptOptimizer to work without litellm (#22233, @TomeHirata)
• [Tracking] Update Databricks API calls to use new gRPC APIs instead of py4j APIs (#22205, @WeichenXu123)
• [Build] Add aiohttp as a core dependency of mlflow (#22189, @TomeHirata)
• [Evaluation] Extend _get_provider_instance with groq, deepseek, xai, openrouter, ollama, databricks, vertex_ai (#22148, @kriscon-db)
• [UI] Move native providers to non-LiteLLM in gateway UI (#22203, @TomeHirata)
• [Tracing / Tracking] Add trace_location parameter to create_experiment (#22075, @dbrx-euirim)
• [Gateway] Complete Bedrock provider with Converse API support (#21999, @TomeHirata)
• [Gateway] Add native Vertex AI gateway provider (#21998, @TomeHirata)
• [Gateway] Add native Databricks gateway provider (#21997, @TomeHirata)
• [Gateway] Add native Ollama gateway provider (#21995, @TomeHirata)
• [Gateway] Add native xAI (Grok) gateway provider (#21993, @TomeHirata)
• [Tracing] Use bulk upsert in log_spans() to eliminate per-span ORM overhead (#21954, @harupy)
• [Tracing] Add builtin cost_per_token to remove litellm dependency for cost tracking (#22046, @TomeHirata)
• [Evaluation] Remove LiteLLM hard dependency from the discovery pipeline and judge adapters (#21739, @harupy)
• [Evaluation] Add pipelined predict-score execution for mlflow.genai.evaluate (#20940, @alkispoly-db)
• [Tracing / Tracking] Default trace location table_prefix to experiment ID in set_experiment (#21815, @danielseong1)
• [Tracking] Add default uvicorn log config with timestamps (#21838, @harupy)
• [Tracing / UI] Add Session ID filter to GenAI traces table filter dropdown (#21794, @daniellok-db)
• [Evaluation / UI] Add Default Credential Chain auth mode for Bedrock/SageMaker in AI Gateway (#21061, @timsolovev)
• [UI] Add multi metric bar chart support (#21258, @RenzoMXD)
• [Tracking] Add TCP keepalive to HTTP sessions to detect stale connections and reduce timeout hangs (#21514, @mobaniha)
• [Evaluation] Add proxy URL support for make_judge (#21185, @yukimori)
• [UI] Improve run group filter to use grouping criteria instead of run IDs (#21072, @daniellok-db)
• [UI] Add tool selector to Tool Calls charts and fix dark mode/sizing (#20865, @B-Step62)
• [UI] Graph View Traces + OpenAI (#20607, @joelrobin18)
• [UI] Show run description in chart tooltip (#21580, @KaushalVachhani)
• [Evaluation / Tracing / UI] Add bulk judge execution from traces table toolbar with status feedback (#21270, @PattaraS)
• [Gateway] Add Redis-backed BudgetTracker for distributed gateway deployments (#21504, @TomeHirata)
• [Tracing / Tracking] Add trace location param to set_experiment (#21385, @danielseong1)
• [Build / Tracking] Add azure extra for Azure Blob Storage support in full Docker image (#21582, @harupy)
• [UI] Add budget violation indicator to gateway budget list page (#21569, @copilot-swe-agent)
• [Evaluation] [5/5] Add discover_issues() pipeline and public API (#21431, @smoorjani)
• [UI] Add Structured Output (JSON Schema) Support to the MLflow Prompts UI (#21394, @kennyvoo)
• [Tracing] Auto-inject tracing context headers in autologging (#21490, @TomeHirata)
• [UI] Add budget alert webhooks UI and fix budgets table borders (#21534, @TomeHirata)
• [Model Registry / Prompts / UI] Add webhooks management UI to settings page (#21483, @TomeHirata)
• [Tracing] Opencode CLI (#20133, @joelrobin18)
• [Models] Add uv_groups and uv_extras params for uv dependency group control (#20935, @debu-sinha)
• [Tracing] Add GenAI Semantic Convention translator for OTLP trace export (#21494, @B-Step62)
• [Tracking] Add polars dataset support to autologging (#21507, @harupy)
• [Tracing] Add mlflow.tracing.context() API for injecting metadata/tags without wrapper spans (#21318, @B-Step62)
• [UI] Add budget dates and current spending for gateway budgets (#21473, @TomeHirata)
• [Tracing / UI] Improve DSPy trace chat view readability (#21296, @B-Step62)
• [UI] Add Kubernetes request auth provider plugin (#21176, @HumairAK)
• [Tracking] Add IS NULL/IS NOT NULL support for tags and params in search_runs (#21283, @TomeHirata)
• [Tracing / UI] Display clickable gateway trace link in trace explorer (#21316, @TomeHirata)
• [UI] Add session selection support with checkbox, actions, and row alignment (#21324, @B-Step62)
• [Models] Add UV package manager support for automatic dependency inference (#20344, @debu-sinha)
• [Evaluation / UI] Add feature flag to control evaluation runs issues panel visibility (#21406, @serena-ruan)
• [Tracing / UI] Add cached tokens display to Token Usage chart (#21295, @TomeHirata)
• [UI] Add budget policies management UI for AI Gateway (#21116, @TomeHirata)
• [UI] Allow multiple judge selection in Run judge on trace modal (#21322, @B-Step62)
• [Docs / Tracking] Add admin-only authorization to webhook CRUD operations (#21271, @TomeHirata)
• [Evaluation / Tracking] Add SqlIssue database table for storing experiment issues (#21165, @serena-ruan)
• [Model Registry / Prompts] Support search_prompt_versions in OSS SQLAlchemy store (#21315, @TomeHirata)
• [Evaluation / Tracing / UI] Add issue detection button to traces table toolbar with feature flag (#21204, @serena-ruan)
• [Docs / Tracing / UI] Add inline audio player for input_audio content parts in trace UI (#21302, @TomeHirata)
• [Evaluation / Tracing] Add IssueReference assessment type to store issue links with traces (#21163, @serena-ruan)
• [Evaluation / Tracing] Add issue management protos with create, update, get, and search APIs (#21161, @serena-ruan)
• [UI] Add IS NULL/IS NOT NULL operators for trace tags in search UI (#21280, @TomeHirata)
• [Docs / Tracing] Add IS NULL/IS NOT NULL support for trace tags in search_traces (#21277, @TomeHirata)
• [Tracing] Add steer message tracing support for Claude Code (#21265, @harupy)
• [Models / Tracking] Add support for transformers 5.x (#20728, @KUrushi)
• [Gateway] Add WEEKS to BudgetDurationUnit enum (#21196, @copilot-swe-agent)
• [UI] Add try-it page on Gateway usage example modal (#21077, @PattaraS)
• [Docs / Tracing / Tracking] Add mlflow.otel.autolog() for OTEL-based tracing integrations (Langfuse, Arize/Phoenix) (#20954, @alkispoly-db)
• [Gateway] Add SQL schema and SQLAlchemy CRUD for gateway budget policies (#21108, @TomeHirata)
• [UI] Add global gateway logs tab to usage page (#21126, @TomeHirata)
• [Tracking] [MLflow Demo] Add server availability handling checks (#20349, @BenWilson2)
• [Tracking] [MLflow Demo] Add scorers demo (#20287, @BenWilson2)
• [Docs / Tracking] Add Backblaze B2 artifact repository (b2://) (#20731, @jeronimodeleon)
• [Docs / Tracking] Add support for multipart download with presigned URLs for S3 compatible object storages (#20352, @etirelli)
• [Tracing] MCP server expansion (#19830, @joelrobin18)
• [Tracing / UI] Include response body in HTTP error messages with 1000 character limit (#20794, @copilot-swe-agent)
Bug fixes:
• [Gateway] Fix DatabricksProvider to use OpenAI-compatible endpoint URLs (#22393, @TomeHirata)
• [Evaluation] Fix: use EvalResult.scorer_stats for multi-turn scorer stat aggregation (#22364, @copilot-swe-agent)
• [Scoring / Tracing] Revert "Register InferenceTableSpanProcessor alongside DatabricksUCTableSpanProcessor in model serving (#22332)" (#22362, @smurching)
• [Scoring / Tracing] Warn when UCSchemaLocation destination is set in Databricks model serving (trace: null) (#22332, @smurching)
• [Tracing / UI] Support tool_reference content blocks in Anthropic Chat UI parser (#22331, @B-Step62)
• [Tracing] Fix online scoring failure when trace spans are stored in artifact repo (#20784, @Mr-Neutr0n)
• [UI] Fix adding a tag with empty value silently failing without user feedback in the Experiments table (#22320, @WeichenXu123)
• [Docs / Models] Bump minimum uv version requirement from 0.5.0 to 0.6.10 (#22313, @copilot-swe-agent)
• [Scoring] Fix: exclude Serverless from use_dbconnect_artifact path in spark_udf (#22300, @franciffu723)
• [UI] Fix assistant crash on unknown CLI message types (#21928, @SuperSonnix71)
• [Tracing / Tracking] Fix mlflow-skinny: guard numpy-dependent imports in mlflow.types (#22211, @Suraj-kumar00)
• [Tracing / UI] Fix dropdown showing wrong selection state before endpoints load in issue detection modal (#22236, @serena-ruan)
• [Tracing] Normalize get_provider_name() to align with model_prices_and_context_window.json (#22223, @TomeHirata)
• [Tracking / UI] Fix log_image with slash-containing keys: replace # with ~ as path separator (#22172, @copilot-swe-agent)
• [Evaluation] Fix discovery pipeline _call_llm_via_gateway to handle gateway:/ URIs (#22153, @TomeHirata)
• [UI] Auto-dismiss and fade-out judge run notifications in trace UI (#22137, @copilot-swe-agent)
• [Evaluation / Tracking] Add polars version guard in polars_dataset.py to fix import failure with polars<1 (#22085, @TomeHirata)
• [Tracking] Fix huey_consumer.py path resolution when venv bin dir is not on PATH (#22126, @copilot-swe-agent)
• [UI] Fix sidebar navigation highlighting for run detail pages (#20860, @daniellok-db)
• [Tracing] Lowercase model_provider in calculate_cost_by_model_and_token_usage (#22134, @TomeHirata)
• [Gateway] Fix misleading "Discarded unknown message" log in Anthropic gateway provider (#21942, @copilot-swe-agent)
• [UI] Fix selected run URL param not updating in eval runs table (#22135, @daniellok-db)
• [Tracing / Tracking] Fix trace export DB contention by disabling incremental span export for gateway (#21721, @PattaraS)
• [Tracing / Tracking] Expand session-level assessment filters to return all session traces (#21792, @daniellok-db)
• [Evaluation] Support inference_params for built-in scorers (#21943, @debu-sinha)
• [UI] Fix assistant stream killed by unhandled rate_limit_event from Claude Code CLI (#22067, @forrestmurray-db)
• [UI] Fix gateway UI not showing custom model name during endpoint edit (#22068, @TomeHirata)
• [Evaluation] Fix Anthropic structured outputs compatibility in gateway adapter (#21922, @harupy)
• [UI] Remove assessment type dropdown and align terminology (#21379, @B-Step62)
• [Tracking] Fix NextMethod() S3 dispatch error in R mlflow_get_run_context (#21957, @daniellok-db)
• [Models / Tracking] Enforce auth on logged model artifact download AJAX endpoint (#21708, @B-Step62)
• [Scoring] Fix tar path traversal vulnerability in extract_archive_to_dir (#21824, @TomeHirata)
• [Scoring] Fix Starlette 1.0 compatibility in mlflow/pyfunc/scoring_server/__init__.py (#21908, @copilot-swe-agent)
• [Tracing] [TS SDK] Port smart preview truncation from Python SDK (#21826, @B-Step62)
• [UI] Fix trace drawer width using context instead of prop drilling (#21830, @B-Step62)
• [UI] fix: use both registrations and tags for consistent registered model display (#20671) (#21555, @s-zx)
• [Tracking] Fix autologging overwriting user's warnings.showwarning handler (#21707, @mango766)
• [Tracing] Remove trace limit in issue discovery to annotate all affected traces (#21736, @serena-ruan)
• [Scoring] fix: accept Sequence instead of list in to_chat_completions_input (#21724, @mr-brobot)
• [Tracking] Set UV_PROJECT_ENVIRONMENT in run_uv_sync to install into the correct Python environment (#21750, @copilot-swe-agent)
• [Tracing / UI] Fix chat UI rendering for OTel GenAI traces with non-standard attributes (#21215, @B-Step62)
• [Build] Fix build-system in examples/uv-dependency-management/pyproject.toml (#21752, @copilot-swe-agent)
• [Tracing] fix: avoid deepcopy in dataclass JSON serialization in TraceJSONEncoder (#21668, @raulblazquezbullon)
• [Tracing] Support artifact-repo traces in batch_get_traces (#21650, @harupy)
• [Evaluation / Tracing] Fall back to agentic judge mode when trace inputs/outputs are missing (#21306, @TomeHirata)
• [UI] Fix chat/session view for LangGraph: deduplicate accumulated messages (#21279, @B-Step62)
• [UI] Show server error detail in Try It panel for budget limit errors (#21568, @TomeHirata)
• [Evaluation] Fix conversation simulator default model encoding on Databricks (#21644, @smoorjani)
• [UI] Delete model definitions when endpoint is deleted from UI (#21649, @TomeHirata)
• [UI] Hide _issue_discovery_judge feedback from traces UI (#21648, @harupy)
• [Prompts] Clarify OSS register_prompt tag behavior (#21600, @yangbaechu)
• [Prompts / Tracing / UI] Make Prompt column clickable in trace view (#21304, @copilot-swe-agent)
• [UI] Fix dataset link not clickable for external source type (#21342, @smoorjani)
• [Tracing / Tracking] Add audio content normalization for LangChain messages (#21533, @elliotllliu)
• [UI] Add tooltips to display full budget and spend amounts in gateway budgets table (#21573, @copilot-swe-agent)
• [Tracking / UI] downsample rows in SQL, update db index (#20928, @sscheele)
• [Models] Skip _maybe_save_model for Databricks ACL-protected artifact URIs (#21602, @mohammadsubhani)
• [UI] Make Try-It UI footer always visible in gateway endpoint modal (#21583, @copilot-swe-agent)
• [Tracing / Tracking] Fix trace assessment filtering and MSSQL pagination syntax errors (#21273, @copilot-swe-agent)
• [Tracing] Fix trace sampling to ensure parent-child consistency (#21524, @harupy)
• [Tracking] Add Azure Government Cloud (usgovcloudapi.net) support to WASBS URI parsing (#21519, @ahringer)
• [Gateway] Change default MLFLOW_GATEWAY_BUDGET_REFRESH_INTERVAL from 60 to 600 seconds (#21565, @copilot-swe-agent)
• [Evaluation / Tracking] Fix scorer re-registration raising RESOURCE_ALREADY_EXISTS in auth layer (#21560, @harupy)
• [Tracking] Harden check when MLFLOW_ALLOW_PICKLE_DESERIALIZATION is disabled (#21404, @WeichenXu123)
• [Tracing] Fix trace ID collisions when random seed is set to fixed value (#21418, @WeichenXu123)
• [UI] Remove "Rate Limiting [Coming Soon]" placeholder from gateway UI (#21559, @copilot-swe-agent)
• [Gateway] Remove policy ID from budget limit exceeded error, show budget reset time instead (#21557, @copilot-swe-agent)
• [Evaluation / Tracking] Fix Strands autolog tool input format for SpanType.TOOL (#21552, @LeviLong01)
• [Tracing] Fix AttributeError in OpenAI autolog by excluding run_config from span attributes (#21454, @MarkVasile)
• [Gateway] Fix singular/plural unit in budget limit exceeded error message (#21538, @copilot-swe-agent)
• [UI] Invalidate budget windows cache on budget policy create/edit/delete (#21535, @copilot-swe-agent)
• [Evaluation] Fix field-based make_judge prompt missing feedback_value_type (#21058, @yangbaechu)
• [Tracing] Set MODEL_PROVIDER across autologging integrations for cost breakdown (#21288, @B-Step62)
• [Evaluation] Fix gateway provider support in third-party judge integrations (ragas, deepeval, phoenix, trulens) (#21414, @copilot-swe-agent)
• [Gateway] Update Anthropic gateway to use GA structured outputs API (#21436, @TomeHirata)
• [Tracking] Adds builtin skops trusted types for LightGBM models (#21412, @WeichenXu123)
• [Tracing / UI] Fix UI flickering in trace review modal during background refetches (#21290, @daniellok-db)
• [Tracking] Add wildcard subdomain support to CORS origins validation (#21468, @arnewouters)
• [UI] Fix refresh button on evaluation runs page to also refresh traces and assessments (#21332, @B-Step62)
• [Models] Fix skops serialization format detection in _load_pyfunc (#21480, @copilot-swe-agent)
• [UI] Fix Shift+Enter not creating newlines in assistant chat input (#21341, @smoorjani)
• [UI] Make retrieved document source URLs clickable in span details view (#21340, @smoorjani)
• [Evaluation / Tracing] Fix AttributeError when trace is None in genai evaluation (#19616, @omarfarhoud)
• [Tracking] Fix CrewAI autologging compatibility with crewai >= 1.10 (#21376, @WeichenXu123)
• [Tracing] Remove span name deduplication suffix from TypeScript SDK (#21382, @B-Step62)
• [Evaluation] Fix LLM judge authentication failure when basic-auth is enabled (#21323, @PattaraS)
• [UI] Fix stored XSS via unsafe YAML parsing of MLmodel artifacts (#21435, @harupy)
• [Tracing / UI] Fix Pydantic AI Chat UI rendering for InstrumentedModel LLM spans (#21410, @B-Step62)
• [Models] Fix transformers 5.3.0 compatibility for removed pipeline classes (#21426, @harupy)
• [Tracing / UI] Fix Chat UI not rendering for Google ADK traces (#21274, @B-Step62)
• [Tracking] Fix image artifact filename mangling caused by URL encoding of % separator (#21269, @harupy)
• [Tracking] Fix: MLFLOW_ALLOW_PICKLE_DESERIALIZATION=False safety control is ineffective for pyfunc flavor (#21188, @WeichenXu123)
• [Tracing / UI] Fix Pydantic AI autologging: auto-enable instrumentation and fix Chat UI (#21278, @B-Step62)
• [Tracing] Fix span type not translated for OTel spans when MLflow SDK is active (#21307, @B-Step62)
• [UI] Remove redundant "Hide assessments" toggle button (#21378, @B-Step62)
• [Tracking] Fix Mistral autologging compatibility with mistralai >= 2.0 (#21374, @WeichenXu123)
• [Tracking] Fix pydantic-ai autologging compatibility with pydantic-ai >= 1.63.0 (#21373, @WeichenXu123)
• [Tracing / Tracking] Fix Claude Code autologging import collision with local mlflow folders (#21343, @smoorjani)
• [Prompts] Fix stale prompt cache after prompt deletion (#21381, @yangbaechu)
• [Tracing / Tracking] Fix flush_trace_async_logging AttributeError with non-default tracer provider (#21105, @cgrierson-smartsheet)
• [UI] Fix session assessments panel terminology (#21336, @smoorjani)
• [UI] Improve quality chart readability and styling in overview tab (#21325, @B-Step62)
• [Tracing] Support uv run in Claude Code tracing hooks (#21327, @copilot-swe-agent)
• [Tracing / UI] Fix Chat tab not rendering for non-OpenAI model names in OpenAI autolog spans (#21356, @TomeHirata)
• [UI] Fix false 'endpoint deleted' warning after endpoint rename (#21333, @TomeHirata)
• [UI] Fix broken image rendering in trace chat collapsed preview (#21291, @harupy)
• [UI] Fix tag key validation UI contradiction (#21140, @KaushalVachhani)
• [Tracing] Use correct env key for Claude Code settings environment variables (#21344, @smoorjani)
• [UI] Fix truncated model names in Cost Breakdown donut chart (#21310, @TomeHirata)
• [Evaluation / Tracing] Fix ConversationSimulator validation for predict_fn signatures and context fields (#21171, @yangbaechu)
• [UI] [ML-63097] Fix broken LLM judge documentation links (#21347, @smoorjani)
• [Tracing / Tracking] Add authentication support to OTLP exporter headers (#21230, @giulio-leone)
• [Evaluation / Tracking] Fix deletion of assessments associated with a run (#20624, @retrowhiz)
• [Models] Fix _deduplicate_requirements merging marker-differentiated requirements (#21098, @harupy)
• [UI] Fix Tags functionality in Recent Experiments table on Home page (#20907, @joelrobin18)
• [Tracing] Fix MCP fn_wrapper handling of Click UNSET defaults (#20953) (#20962, @yangbaechu)
• [Evaluation] Enable Databricks LLM fallback for available tools extraction (#21017, @xsh310)
• [UI] Fix sorting for timestamp columns in ExperimentListTable (#20908, @joelrobin18)
• [UI] Fix tag value input being cleared when entered before key (#20910, @joelrobin18)
• [Docs] Fix LiteLLM model URI format in eval quickstart docs (#20941, @copilot-swe-agent)
• [Tracing] Fix SpanEvent timestamp resolution to use nanoseconds (#20828, @copilot-swe-agent)
• [Tracking] Escape regex special chars in search_experiments LIKE filter (#16667, @joelrobin18)
Documentation updates:
• [Docs] docs: clarify uv dependency management vs MLFLOW_LOCK_MODEL_DEPENDENCIES, add uv workspace limitation (#22312, @copilot-swe-agent)
• [Docs] Document supported provider environment variables for judge models (#22195, @kriscon-db)
• [Docs] Add relative duration examples for uv --exclude-newer (#22133, @copilot-swe-agent)
• [Docs] Add secure installs documentation page (#22036, @harupy)
• [Evaluation] Add documentation for issue detection (#22057, @serena-ruan)
• [Tracing] Add OpenHands integration doc (#21933, @B-Step62)
• [Docs / Tracing] Fix MLFLOW_ENABLE_ASYNC_TRACE_LOGGING docs to reflect OSS default behavior (#21731, @copilot-swe-agent)
• [Docs] Add note for pickle-free model doc (#21732, @WeichenXu123)
• [Docs] Add experiment note to the pickle-free model format doc page (#21709, @WeichenXu123)
• [Docs] Add Guide: Deploy MLflow to Google Cloud (#21599, @WeichenXu123)
• [Docs] Add Guide: Deploy MLflow to Azure cloud (#21128, @WeichenXu123)
• [Docs / Tracing] Add Goose tracing integration documentation (#21190, @B-Step62)
• [Docs] Expand Koog integration doc (#21218, @B-Step62)
• [Docs / Tracing] Add 'Combine with MLflow SDK' section to OTel integration guides (#21298, @TomeHirata)
• [Docs] docs: add Budget Tracker Strategies guideline to AI Gateway budget page (#21633, @copilot-swe-agent)
• [Docs] Add tracking URI note to mlflow-skinny README (#21638, @harupy)
• [Docs] Add Guide: Deploy MLflow to AWS cloud (#20729, @WeichenXu123)
• [Docs / Models] Deprecate generate_signature_output in favor of input_example (#21556, @shivamshinde123)
• [Docs] Claude MCP setup instructions to use .mcp.json or CLI (#21609, @copilot-swe-agent)
• [Docs] [1/3] Document OTel attribute mapping (#21478, @B-Step62)
• [Docs] docs: Add OpenAI Responses API examples to gateway passthrough documentation (#21545, @copilot-swe-agent)
• [Docs] Add standalone multimodal content in traces documentation (#21357, @kriscon-db)
• [Docs] Add documentation page for Budget Alerts & Limits (#21121, @TomeHirata)
• [Docs / Models] Add documentation for pickle-free model formats (#20774, @WeichenXu123)
• [Docs / Prompts] Update prompt registry docs to use MLflow 3.x API examples (#21267, @copilot-swe-agent)
• [Docs] docs: add single quotes to install commands with extras to prevent zsh errors (#21227, @mshavliuk)
• [Docs] Add Amazon Nova bedrock model examples for mlflow.metrics.genai (#21063, @ManasVardhan)
• [Docs] Update SSO oidc plugin doc: add google identity platform / AWS cognito / Azure Entra ID configuration guide (#20591, @WeichenXu123)
For a comprehensive list of changes, see the release change log, and check out the latest documentation on mlflow.org.

Major New Features

• 🔍 Automatic Issue Identification: Automatically identify quality issues in your agent with AI! Use the new "Detect Issues" button in the traces table to analyze selected traces and surface potential problems across categories like correctness, safety, and performance. Issues are linked directly to traces for easy investigation and debugging. (#21431, #21204, #21165, #21163, #21161, @smoorjani, @serena-ruan)

• 💰 Gateway Budget Alerts & Limits: Control your AI Gateway spending with configurable budget policies! Set spending limits by time window (daily, weekly, or monthly), receive alerts before hitting limits, and prevent runaway costs with automatic request blocking. The new budget management UI lets you track spending, configure webhooks for notifications, and monitor violations across all your gateway endpoints. (#21116, #21534, #21569, #21473, #21108, @TomeHirata, @copilot-swe-agent)

• 📊 Trace Graph View: Visualize complex trace hierarchies with an interactive graph view! Navigate multi-level trace structures, understand parent-child relationships at a glance, and debug complex systems more effectively with a visual representation of your trace topology. (#20607, @joelrobin18)

• 🌐 Native OpenTelemetry GenAI Convention Support: MLflow now natively supports the OpenTelemetry GenAI Semantic Conventions for trace export! When exporting traces via OTLP with MLFLOW_ENABLE_OTEL_GENAI_SEMCONV enabled, MLflow automatically translates them to follow the OTel GenAI semantic conventions, enabling seamless integration with OTel-compatible observability platforms while preserving GenAI-specific metadata. (#21494, #21495, @B-Step62)

• 🔧 Opencode Tracing Integration: Debug smarter with Opencode CLI integration! Track and analyze code execution flows directly from your development workflow, making it easier to identify performance bottlenecks and trace issues back to specific code paths. (#20133, @joelrobin18)

• ⚡ UV Package Manager Support: Automatic dependency inference now supports UV! MLflow automatically detects UV projects and captures exact, locked dependencies from your lockfile when logging models, ensuring reproducible environments. (#20344, #20935, @debu-sinha)

• 🔒 Pickle-Free Model Serialization: Enhance security with pickle-free model formats! MLflow now supports safer model serialization using torch.export and skops formats, with improved controls when MLFLOW_ALLOW_PICKLE_DESERIALIZATION=False. Comprehensive documentation guides you through migrating existing models to pickle-free formats for production deployments. (#21404, #21188, #20774, @WeichenXu123)

Breaking Changes

• ⚠️ TypeScript SDK Package Renaming: The MLflow TypeScript SDK packages have been renamed to use npm organization scoping. If you're using the TypeScript SDK, update your package.json dependencies and import statements: mlflow-tracing → @mlflow/core, mlflow-openai → @mlflow/openai, mlflow-anthropic → @mlflow/anthropic, mlflow-gemini → @mlflow/gemini. All packages are now at version 0.2.0. (#20792, @B-Step62)

Stay tuned for the full release, which will be packed with even more features and bugfixes.

To try out this release candidate, please run:

pip install mlflow==3.11.0rc0

MLflow 3.10.1 is a patch release that contains some minor feature enhancements, bug fixes, and documentation updates.

Features

• [UI] Add try-it page on Gateway usage example modal (#21077, @PattaraS)
• [UI] Filter gateway experiments from the experiment list page (#21130, @copilot-swe-agent)

Bug fixes

• [UI] Fix "View full dashboard" link in gateway usage tab when workspace is enabled (#21191, @copilot-swe-agent)
• [UI] Persist AI Gateway default passphrase security banner dismissal to localStorage (#21292, @copilot-swe-agent)
• [Evaluation] Demote unused parameters log message from WARNING to DEBUG in instructions judge (#21294, @copilot-swe-agent)
• [UI] Clear "All" time selector when switching to overview tab (#21371, @daniellok-db)
• [Prompts / UI] Fix Traces view in Prompts tab not being scrollable (#21282, @TomeHirata)
• [UI] Fix judge builder instruction textarea (#21299, @daniellok-db)
• [UI] Fix group mode to aggregate "Additional runs" as "Unassigned" group in charts (#21155, @copilot-swe-agent)
• [UI] Fix artifact download when workspaces are enabled (#21074, @timsolovev)
• [Tracing] Fix NOT NULL constraint on assessments.trace_id during trace export (#21348, @dbczumar)
• [Tracking] Fix 403 Forbidden for artifact list via query param when default_permission=NO_PERMISSIONS (#21220, @copilot-swe-agent)
• [UI] [ML-63097] Fix broken LLM judge documentation links (#21347, @smoorjani)
• [Tracing] Fix Run Judge failed with litellm.InternalServerError: Invalid response object. (#21262, @PattaraS)
• [Tracing / UI] Update Action menu: indentation to avoid confusion (#21266, @PattaraS)
• [Model Registry] Fix MlflowClient.copy_model_version for the case that copy UC model across workspaces (#21212, @WeichenXu123)
• [UI] Fix empty description box rendering for sanitized-empty experiment descriptions (#21223, @copilot-swe-agent)
• [Artifacts] Fix single artifact downloading through HttpArtifactRepository (#12955, @Koenkk)
• [Tracing] Fix find_last_user_message_index skipping skill content injections (#21119, @alkispoly-db)
• [Tracing] Fix retrieval context extraction when span outputs are stored as strings (#21213, @smoorjani)
• [UI] Fix visibility toggle button in chart tooltip not working (#21071, @daniellok-db)
• [UI] Move gateway experiment filtering to server-side query to fix inconsistent page sizes (#21138, @copilot-swe-agent)
• [Gateway] Downgrade spurious warning to debug log for gateway endpoints with fallback_config but no FALLBACK models (#21123, @copilot-swe-agent)
• [Tracing] Fix MCP fn_wrapper to pass None for optional params with UNSET defaults (#21051, @yangbaechu)
• [Tracking] Add CASCADE to logged_model tables experiment_id foreign keys (#20185, @harupy)
• [Tracing] Fix MCP fn_wrapper handling of Click UNSET defaults (#20953) (#20962, @yangbaechu)

Documentation updates

• [Docs] Update SSO oidc plugin doc: add google identity platform / AWS cognito / Azure Entra ID configuration guide (#20591, @WeichenXu123)
• [Docs / Tracing] Fix distributed tracing rendering and improve doc (#21070, @B-Step62)
• [Docs] docs: Add single quotes to install commands with extras to prevent zsh errors (#21227, @mshavliuk)
• [Docs / Model Registry] Fix outdated docstring claiming models:/ URIs are unsupported in register_model (#21197, @copilot-swe-agent)
• [Docs] Replace MinIO with RustFS in docker-compose setup (#21099, @jmaggesi)

Small bug fixes and documentation updates

#20740, #21148, #21149, #21096, @TomeHirata; #21368, #21118, @B-Step62; #21384, #21345, #21236, #21106, #21033, #21115, #21034, @smoorjani; #21326, #21133, #21036, @copilot-swe-agent; #21293, @daniellok-db; #21175, @caponetto; #21305, #21264, @serena-ruan; #21216, @justinwei-db; #21038, #21082, @bbqiu; #21143, #20733, @mprahl; #20488, @mdalvz0000; #21142, @EPgg92; #21094, @PattaraS

Features

• [UI] Add try-it page on Gateway usage example modal (#21077, @PattaraS)
• [UI] Filter gateway experiments from the experiment list page (#21130, @copilot-swe-agent)

Bug fixes

• [UI] Fix "View full dashboard" link in gateway usage tab when workspace is enabled (#21191, @copilot-swe-agent)
• [UI] Persist AI Gateway default passphrase security banner dismissal to localStorage (#21292, @copilot-swe-agent)
• [Evaluation] Demote unused parameters log message from WARNING to DEBUG in instructions judge (#21294, @copilot-swe-agent)
• [UI] Clear "All" time selector when switching to overview tab (#21371, @daniellok-db)
• [Prompts / UI] Fix Traces view in Prompts tab not being scrollable (#21282, @TomeHirata)
• [UI] Fix judge builder instruction textarea (#21299, @daniellok-db)
• [UI] Fix group mode to aggregate "Additional runs" as "Unassigned" group in charts (#21155, @copilot-swe-agent)
• [UI] Fix artifact download when workspaces are enabled (#21074, @timsolovev)
• [Tracing] Fix NOT NULL constraint on assessments.trace_id during trace export (#21348, @dbczumar)
• [Tracking] Fix 403 Forbidden for artifact list via query param when default_permission=NO_PERMISSIONS (#21220, @copilot-swe-agent)
• [UI] [ML-63097] Fix broken LLM judge documentation links (#21347, @smoorjani)
• [Tracing] Fix Run Judge failed with litellm.InternalServerError: Invalid response object. (#21262, @PattaraS)
• [Tracing / UI] Update Action menu: indentation to avoid confusion (#21266, @PattaraS)
• [Model Registry] Fix MlflowClient.copy_model_version for the case that copy UC model across workspaces (#21212, @WeichenXu123)
• [UI] Fix empty description box rendering for sanitized-empty experiment descriptions (#21223, @copilot-swe-agent)
• [Artifacts] Fix single artifact downloading through HttpArtifactRepository (#12955, @Koenkk)
• [Tracing] Fix find_last_user_message_index skipping skill content injections (#21119, @alkispoly-db)
• [Tracing] Fix retrieval context extraction when span outputs are stored as strings (#21213, @smoorjani)
• [UI] Fix visibility toggle button in chart tooltip not working (#21071, @daniellok-db)
• [UI] Move gateway experiment filtering to server-side query to fix inconsistent page sizes (#21138, @copilot-swe-agent)
• [Gateway] Downgrade spurious warning to debug log for gateway endpoints with fallback_config but no FALLBACK models (#21123, @copilot-swe-agent)
• [Tracing] Fix MCP fn_wrapper to pass None for optional params with UNSET defaults (#21051, @yangbaechu)
• [Tracking] Add CASCADE to logged_model tables experiment_id foreign keys (#20185, @harupy)
• [Tracing] Fix MCP fn_wrapper handling of Click UNSET defaults (#20953) (#20962, @yangbaechu)

Documentation updates

• [Docs] Update SSO oidc plugin doc: add google identity platform / AWS cognito / Azure Entra ID configuration guide (#20591, @WeichenXu123)
• [Docs / Tracing] Fix distributed tracing rendering and improve doc (#21070, @B-Step62)
• [Docs] docs: Add single quotes to install commands with extras to prevent zsh errors (#21227, @mshavliuk)
• [Docs / Model Registry] Fix outdated docstring claiming models:/ URIs are unsupported in register_model (#21197, @copilot-swe-agent)
• [Docs] Replace MinIO with RustFS in docker-compose setup (#21099, @jmaggesi)

For a comprehensive list of changes, see the release change log, and check out the latest documentation on mlflow.org.