MLflow Release Notes

Major New Features

• 🔍 Automatic Issue Identification: Automatically identify quality issues in your agent with AI! Use the new "Detect Issues" button in the traces table to analyze selected traces and surface potential problems across categories like correctness, safety, and performance. Issues are linked directly to traces for easy investigation and debugging. (#21431, #21204, #21165, #21163, #21161, @smoorjani, @serena-ruan)

• 💰 Gateway Budget Alerts & Limits: Control your AI Gateway spending with configurable budget policies! Set spending limits by time window (daily, weekly, or monthly), receive alerts before hitting limits, and prevent runaway costs with automatic request blocking. The new budget management UI lets you track spending, configure webhooks for notifications, and monitor violations across all your gateway endpoints. (#21116, #21534, #21569, #21473, #21108, @TomeHirata, @copilot-swe-agent)

• 📊 Trace Graph View: Visualize complex trace hierarchies with an interactive graph view! Navigate multi-level trace structures, understand parent-child relationships at a glance, and debug complex systems more effectively with a visual representation of your trace topology. (#20607, @joelrobin18)

• 🌐 Native OpenTelemetry GenAI Convention Support: MLflow now natively supports the OpenTelemetry GenAI Semantic Conventions for trace export! When exporting traces via OTLP with MLFLOW_ENABLE_OTEL_GENAI_SEMCONV enabled, MLflow automatically translates them to follow the OTel GenAI semantic conventions, enabling seamless integration with OTel-compatible observability platforms while preserving GenAI-specific metadata. (#21494, #21495, @B-Step62)

• 🔧 Opencode Tracing Integration: Debug smarter with Opencode CLI integration! Track and analyze code execution flows directly from your development workflow, making it easier to identify performance bottlenecks and trace issues back to specific code paths. (#20133, @joelrobin18)

• ⚡ UV Package Manager Support: Automatic dependency inference now supports UV! MLflow automatically detects UV projects and captures exact, locked dependencies from your lockfile when logging models, ensuring reproducible environments. (#20344, #20935, @debu-sinha)

• 🔒 Pickle-Free Model Serialization: Enhance security with pickle-free model formats! MLflow now supports safer model serialization using torch.export and skops formats, with improved controls when MLFLOW_ALLOW_PICKLE_DESERIALIZATION=False. Comprehensive documentation guides you through migrating existing models to pickle-free formats for production deployments. (#21404, #21188, #20774, @WeichenXu123)

Breaking Changes

• ⚠️ TypeScript SDK Package Renaming: The MLflow TypeScript SDK packages have been renamed to use npm organization scoping. If you're using the TypeScript SDK, update your package.json dependencies and import statements: mlflow-tracing → @mlflow/core, mlflow-openai → @mlflow/openai, mlflow-anthropic → @mlflow/anthropic, mlflow-gemini → @mlflow/gemini. All packages are now at version 0.2.0. (#20792, @B-Step62)

Stay tuned for the full release, which will be packed with even more features and bugfixes.

To try out this release candidate, please run:

pip install mlflow==3.11.0rc0

MLflow 3.10.1 is a patch release that contains some minor feature enhancements, bug fixes, and documentation updates.

Features

• [UI] Add try-it page on Gateway usage example modal (#21077, @PattaraS)
• [UI] Filter gateway experiments from the experiment list page (#21130, @copilot-swe-agent)

Bug fixes

• [UI] Fix "View full dashboard" link in gateway usage tab when workspace is enabled (#21191, @copilot-swe-agent)
• [UI] Persist AI Gateway default passphrase security banner dismissal to localStorage (#21292, @copilot-swe-agent)
• [Evaluation] Demote unused parameters log message from WARNING to DEBUG in instructions judge (#21294, @copilot-swe-agent)
• [UI] Clear "All" time selector when switching to overview tab (#21371, @daniellok-db)
• [Prompts / UI] Fix Traces view in Prompts tab not being scrollable (#21282, @TomeHirata)
• [UI] Fix judge builder instruction textarea (#21299, @daniellok-db)
• [UI] Fix group mode to aggregate "Additional runs" as "Unassigned" group in charts (#21155, @copilot-swe-agent)
• [UI] Fix artifact download when workspaces are enabled (#21074, @timsolovev)
• [Tracing] Fix NOT NULL constraint on assessments.trace_id during trace export (#21348, @dbczumar)
• [Tracking] Fix 403 Forbidden for artifact list via query param when default_permission=NO_PERMISSIONS (#21220, @copilot-swe-agent)
• [UI] [ML-63097] Fix broken LLM judge documentation links (#21347, @smoorjani)
• [Tracing] Fix Run Judge failed with litellm.InternalServerError: Invalid response object. (#21262, @PattaraS)
• [Tracing / UI] Update Action menu: indentation to avoid confusion (#21266, @PattaraS)
• [Model Registry] Fix MlflowClient.copy_model_version for the case that copy UC model across workspaces (#21212, @WeichenXu123)
• [UI] Fix empty description box rendering for sanitized-empty experiment descriptions (#21223, @copilot-swe-agent)
• [Artifacts] Fix single artifact downloading through HttpArtifactRepository (#12955, @Koenkk)
• [Tracing] Fix find_last_user_message_index skipping skill content injections (#21119, @alkispoly-db)
• [Tracing] Fix retrieval context extraction when span outputs are stored as strings (#21213, @smoorjani)
• [UI] Fix visibility toggle button in chart tooltip not working (#21071, @daniellok-db)
• [UI] Move gateway experiment filtering to server-side query to fix inconsistent page sizes (#21138, @copilot-swe-agent)
• [Gateway] Downgrade spurious warning to debug log for gateway endpoints with fallback_config but no FALLBACK models (#21123, @copilot-swe-agent)
• [Tracing] Fix MCP fn_wrapper to pass None for optional params with UNSET defaults (#21051, @yangbaechu)
• [Tracking] Add CASCADE to logged_model tables experiment_id foreign keys (#20185, @harupy)
• [Tracing] Fix MCP fn_wrapper handling of Click UNSET defaults (#20953) (#20962, @yangbaechu)

Documentation updates

• [Docs] Update SSO oidc plugin doc: add google identity platform / AWS cognito / Azure Entra ID configuration guide (#20591, @WeichenXu123)
• [Docs / Tracing] Fix distributed tracing rendering and improve doc (#21070, @B-Step62)
• [Docs] docs: Add single quotes to install commands with extras to prevent zsh errors (#21227, @mshavliuk)
• [Docs / Model Registry] Fix outdated docstring claiming models:/ URIs are unsupported in register_model (#21197, @copilot-swe-agent)
• [Docs] Replace MinIO with RustFS in docker-compose setup (#21099, @jmaggesi)

Small bug fixes and documentation updates

#20740, #21148, #21149, #21096, @TomeHirata; #21368, #21118, @B-Step62; #21384, #21345, #21236, #21106, #21033, #21115, #21034, @smoorjani; #21326, #21133, #21036, @copilot-swe-agent; #21293, @daniellok-db; #21175, @caponetto; #21305, #21264, @serena-ruan; #21216, @justinwei-db; #21038, #21082, @bbqiu; #21143, #20733, @mprahl; #20488, @mdalvz0000; #21142, @EPgg92; #21094, @PattaraS

Features

• [UI] Add try-it page on Gateway usage example modal (#21077, @PattaraS)
• [UI] Filter gateway experiments from the experiment list page (#21130, @copilot-swe-agent)

Bug fixes

• [UI] Fix "View full dashboard" link in gateway usage tab when workspace is enabled (#21191, @copilot-swe-agent)
• [UI] Persist AI Gateway default passphrase security banner dismissal to localStorage (#21292, @copilot-swe-agent)
• [Evaluation] Demote unused parameters log message from WARNING to DEBUG in instructions judge (#21294, @copilot-swe-agent)
• [UI] Clear "All" time selector when switching to overview tab (#21371, @daniellok-db)
• [Prompts / UI] Fix Traces view in Prompts tab not being scrollable (#21282, @TomeHirata)
• [UI] Fix judge builder instruction textarea (#21299, @daniellok-db)
• [UI] Fix group mode to aggregate "Additional runs" as "Unassigned" group in charts (#21155, @copilot-swe-agent)
• [UI] Fix artifact download when workspaces are enabled (#21074, @timsolovev)
• [Tracing] Fix NOT NULL constraint on assessments.trace_id during trace export (#21348, @dbczumar)
• [Tracking] Fix 403 Forbidden for artifact list via query param when default_permission=NO_PERMISSIONS (#21220, @copilot-swe-agent)
• [UI] [ML-63097] Fix broken LLM judge documentation links (#21347, @smoorjani)
• [Tracing] Fix Run Judge failed with litellm.InternalServerError: Invalid response object. (#21262, @PattaraS)
• [Tracing / UI] Update Action menu: indentation to avoid confusion (#21266, @PattaraS)
• [Model Registry] Fix MlflowClient.copy_model_version for the case that copy UC model across workspaces (#21212, @WeichenXu123)
• [UI] Fix empty description box rendering for sanitized-empty experiment descriptions (#21223, @copilot-swe-agent)
• [Artifacts] Fix single artifact downloading through HttpArtifactRepository (#12955, @Koenkk)
• [Tracing] Fix find_last_user_message_index skipping skill content injections (#21119, @alkispoly-db)
• [Tracing] Fix retrieval context extraction when span outputs are stored as strings (#21213, @smoorjani)
• [UI] Fix visibility toggle button in chart tooltip not working (#21071, @daniellok-db)
• [UI] Move gateway experiment filtering to server-side query to fix inconsistent page sizes (#21138, @copilot-swe-agent)
• [Gateway] Downgrade spurious warning to debug log for gateway endpoints with fallback_config but no FALLBACK models (#21123, @copilot-swe-agent)
• [Tracing] Fix MCP fn_wrapper to pass None for optional params with UNSET defaults (#21051, @yangbaechu)
• [Tracking] Add CASCADE to logged_model tables experiment_id foreign keys (#20185, @harupy)
• [Tracing] Fix MCP fn_wrapper handling of Click UNSET defaults (#20953) (#20962, @yangbaechu)

Documentation updates

• [Docs] Update SSO oidc plugin doc: add google identity platform / AWS cognito / Azure Entra ID configuration guide (#20591, @WeichenXu123)
• [Docs / Tracing] Fix distributed tracing rendering and improve doc (#21070, @B-Step62)
• [Docs] docs: Add single quotes to install commands with extras to prevent zsh errors (#21227, @mshavliuk)
• [Docs / Model Registry] Fix outdated docstring claiming models:/ URIs are unsupported in register_model (#21197, @copilot-swe-agent)
• [Docs] Replace MinIO with RustFS in docker-compose setup (#21099, @jmaggesi)

For a comprehensive list of changes, see the release change log, and check out the latest documentation on mlflow.org.

MLflow 3.10.0 Release Notes

MLflow 3.10.0 is a major release that enhances MLflow's AI Observability and evaluation capabilities, while also making these features easier to use, both for new users and organizations operating at scale. This release brings multi-workspace support, evaluation and simulation for chatbot conversations, cost tracking for your traces, usage tracking for your AI Gateway endpoints, and a number of UI enhancements to make your apps and agent development much more intuitive.

Workspace Support in MLflow Tracking Server

MLflow now supports multi-workspace environments. Users can organize experiments, models, prompts, with a coarser level of granularity and logically isolate them in a single tracking server. To enable this feature, pass the --enable-workspaces flag to the mlflow server command, or set the MLFLOW_ENABLE_WORKSPACES environment variable to true.

Multi-turn Evaluation & Conversation Simulation

MLflow now supports multi-turn evaluation, including evaluating existing conversations with session-level scorers and simulating conversations to test new versions of your agent, without the toil of regenerating conversations. Use the session-level scorers introduced in MLflow 3.8.0 and the brand new session UIs to evaluate the quality of your conversational agents and enable automatic scoring to monitor quality as traces are ingested.

Trace Cost Tracking

Gain visibility into your LLM spending! MLflow now automatically extracts model information from LLM spans and calculates costs, with a new UI that renders model and cost data directly in your trace views. Additionally, costs are aggregated and broken down in the "Overview" tab, giving you granular insights into your LLM spend patterns.

Navigation bar redesign

As we continue to add more features to the MLflow UI, we found that navigation was getting cluttered and overwhelming, with poor separation of features for different workflow types. We've redesigned the navigation bar to be more intuitive and easier to use, with a new sidebar that provides a more relevant set of tabs for both GenAI apps and agent developers, as well as classic model training workflows. The new experience also gives more space to the main content area, making it easier to focus on the task at hand.

MLflow Demo Experiment

New to MLflow GenAI? With one click, launch a pre-populated demo and explore LLM tracing, evaluation, and prompt management in action. No configuration, no code required. This feature is available in the MLflow UI's homepage, and provides a comprehensive overview of the different functionality that MLflow has to offer.

Gateway Usage Tracking

Monitor your AI Gateway endpoints with detailed usage analytics. A new "Usage" tab shows request patterns and metrics, with trace ingestion that links gateway calls back to your experiments for end-to-end AI observability. To turn this feature on for your AI Gateway endpoints, make sure to check the "Enable usage tracking" toggle in your endpoint settings.

In-UI Trace Evaluation

Run custom or pre-built LLM judges directly from the traces and sessions UI, no code required! This enables quick evaluation of individual traces and individual without context switching to the Python SDK. In order to use this feature, make sure to set up an AI gateway endpoint, as you'll need to select an endpoint to use when running LLM judges.

For a comprehensive list of changes, see the release change log.

Major New Features:

• 🏢 Organization Support in MLflow Tracking Server: MLflow now supports multi-workspace environments. Users can organize experiments, models, prompts, with a coarser level of unit and logically isolate them in a single tracking server. (#20702, #20657, @mprahl, @Gkrumbach07, @B-Step62)
• 💬 Multi-turn Evaluation & Conversation Simulation: MLflow now supports multi-turn evaluation, including evaluating existing conversations with session-level scorers and simulating conversations to test new versions of your agent, without the toil of regenerating conversations. Use the session-level scorers introduced in MLflow 3.8.0 and the brand new session UIs to evaluate the quality of your conversational agents and enable automatic scoring to monitor quality as traces are ingested. (#20243, #20377, #20289, @smoorjani)
• 💰 Trace Cost Tracking: Gain visibility into your LLM spending! MLflow now automatically extracts model information from LLM spans and calculates costs, with a new UI that renders model and cost data directly in your trace views. (#20327, #20330, @serena-ruan)
• 🎯 Navigation bar redesign: We've redesigned the navigation to provide a frictionless experience. A new workflow type selector in the top-level navbar lets you quickly switch between GenAI and Classical ML contexts, with streamlined sidebars that reduce visual clutter. (#20158, #20160, #20161, #20699, @ispoljari, @daniellok-db)
• 🎮 MLflow Demo Experiment: New to MLflow GenAI? With one click, launch a pre-populated demo and explore tracing, evaluation, and prompt management in action. No configuration, no code required. (#19994, #19995, #20046, #20047, #20048, #20162, @BenWilson2)
• 📊 Gateway Usage Tracking: Monitor your AI Gateway endpoints with detailed usage analytics. A new usage page shows request patterns and metrics, with trace ingestion that links gateway calls back to your experiments for end-to-end observability. (#20357, #20358, #20642, @TomeHirata)
• ⚡ In-UI Trace Evaluation: Users can now run custom or pre-built LLM judges directly from the traces and sessions UI. This enables quick evaluation of individual traces and individual without context switching to the python SDK. (#20360, @hubertzub-db, @danielseong1)

Features:

• [UI] Add sliding animation to workflow switch component (#20831, @daniellok-db)
• [Tracing] Display cached tokens in trace UI (#20957, @TomeHirata)
• [Evaluation] Move select traces button to be next to Run judge (#20992, @PattaraS)
• [Gateway] Distributed tracing for gateway endpoints (#20864, @TomeHirata)
• [Gateway] Add user selector in the gateway usage page (#20944, @TomeHirata)
• [Docs] [MLflow Demo] Docs for GenAI Demo (#20240, @BenWilson2)
• [UI] Move Getting Started above experiments list and make collapsible (#20691, @B-Step62)
• [Model Registry / Tracking] Add mlflow migrate-filestore command (#20615, @harupy)
• [UI] Add visual indicator for demo experiment in experiment list (#20787, @B-Step62)
• [Scoring] Enable parquet content_type in the scoring server input for pyfunc (#20630, @TFK1410)
• [UI] feat(ui): Add workspace landing page, multi-workspace support, and qu… (#20702, @Gkrumbach07)
• [Tracking] Merge workspace feature branch into master (#20657, @B-Step62)
• [Gateway] Add Gateway Usage Page (#20642, @TomeHirata)
• [Gateway] Add usage section in endpoint page (#20357, @TomeHirata)
• [UI] [ MLflow Demo ] UI updates for MLflow Demo interfaces (#20162, @BenWilson2)
• [Build] Support comma-separated rules in # clint: disable= comments (#20651, @copilot-swe-agent)
• [Build / Docs / Models / Projects / Scoring] Replace virtualenv with python -m venv in virtualenv env_manager path (#20640, @copilot-swe-agent)
• [Tracing] Add per-decorator sampling_ratio_override parameter to @mlflow.trace (#19784, @harupy)
• [Evaluation / Tracking] Add mlflow datasets list CLI command (#20167, @alkispoly-db)
• [Gateway] Add trace ingestion for Gateway endpoints (#20358, @TomeHirata)
• [Tracing] feat(typescript-anthropic): add streaming support (#20384, @rollyjoel)
• [Evaluation] Add delete dataset records API (#19690, @joelrobin18)
• [UI] Add tooltip link to navigate to traces tab with time range filter (#20466, @serena-ruan)
• [Tracking] [MLflow Demo] Add mlflow demo cli command (#20048, @BenWilson2)
• [Evaluation] Add an SDK for distillation from conversation to goal/persona (#20289, @smoorjani)
• [Tracing] Livekit Agents Integration in MLflow (#20439, @joelrobin18)
• [Tracing / UI] Enable running scorers/judges from trace details drawer in UI (#20518, @danielseong1)
• [Gateway] link gateway and experiment (#20356, @TomeHirata)
• [Prompts] Add optimization backend APIs to auth control (#20392, @chenmoneygithub)
• [Tracing] Add an SDK for search sessions to get complete sessions (#20288, @smoorjani)
• [Tracing] Reasoning in Chat UI Mistral + Chat UI (#19636, @joelrobin18)
• [Evaluation] Add TruLens third-party scorer integration (#19492, @debu-sinha)
• [Evaluation / Tracing] Add Guardrails AI scorer integration (#20038, @debu-sinha)
• [Tracking] [MLflow Demo] Add Prompt demo data (#20047, @BenWilson2)
• [Tracking] [MLflow Demo] Add Eval simulation data (#20046, @BenWilson2)
• [Tracking] [MLflow Demo] Add trace data for demo (#19995, @BenWilson2)
• [Tracking] Support get_dataset(name=...) in OSS environments (#20423, @alkispoly-db)
• [UI] Add session comparison UI with goal/persona matching (#20377, @smoorjani)
• [UI] Model and cost rendering for spans (#20330, @serena-ruan)
• [UI] [1/x] Support span model extraction and cost calculation (#20327, @serena-ruan)
• [Evaluation] Make conversation simulator public and easily subclassable (#20243, @smoorjani)
• [Prompts] Add progress tracking for prompt optimization job (#20374, @chenmoneygithub)
• [Prompts] Prompt Optimization backend PR 3: Add Get, Search, and Delete prompt optimization job APIs (#20197, @chenmoneygithub)
• [Prompts] Track intermediate candidates and evaluation scores in gepa optimizer (#20198, @chenmoneygithub)
• [Tracking] [MLflow Demo] Base implementation for demo framework (#19994, @BenWilson2)
• [Prompts] Prompt Optimization backend PR 2: Add CreatePromptOptimizationJob and CancelPromptOptimizationJob (#20115, @chenmoneygithub)
• [Tracing] Support shift+select for Traces (#20125, @B-Step62)
• [UI] Ml61127/remove experiment type selector inside experiment page (#20161, @ispoljari)
• [UI] Ml61126/remove nested sidebars within gateway and experiments tab (#20160, @ispoljari)
• [UI] [ML-61124]: add selector for workflow type in top level navbar (#20158, @ispoljari)
• [Prompts / UI] Feat/render md in prompt registry (#19615, @iyashk)
• [Prompts] [Prompt Optimization Backend PR #1] Wrap prompt optimize in mlflow job (#20001, @chenmoneygithub)
• [Tracking] Add --experiment-name option to mlflow experiments get command (#19929, @alkispoly-db)
• Bug fixes:
• [Tracing / UI] Fix infinite fetch loop in trace detail view when num_spans metadata mismatches (#20596, @coldzero94)
• [UI] fix:implement dark mode in experiment correctly (#20974, @intelliking)
• [Evaluation] Fix 'Select traces' do not show new traces in Judge UI (#20991, @PattaraS)
• [Tracing / Tracking] Fix RecursionError in strands, semantic_kernel, and haystack autologgers with shared tracer provider (#20809, @cgrierson-smartsheet)
• [Tracking] fix(tracking): Fix IntegrityError in log_batch when duplicate metrics span multiple key batches (#20807, @aws-khatria)
• [Tracing] Support native tool calls in CrewAI 1.9.0+ autolog tests (#20742, @TomeHirata)
• [Evaluation] Fix retrieval_relevance assessments logged to wrong span with missing chunk index (#20998, @smoorjani)
• [Evaluation] Fix missing session metadata on failed session-level scorer assessments (#20988, @smoorjani)
• [Tracking] Enhance path validation in check_tarfile_security for windows (#20924, @TomeHirata)
• [Docs] Fix admonition link underlines not rendering (#20990, @copilot-swe-agent)
• [Tracking] Rebuild SearchTraces V2 request body on ENDPOINT_NOT_FOUND fallback (#20963, @brendanmaguire)
• [Build] Add model version search filtering based on user permissions (#20964, @TomeHirata)
• [Tracing] Display notebook trace viewer when workspace is on (#20947, @TomeHirata)
• [Tracing] Add MLFLOW_GATEWAY_RESOLVE_API_KEY_FROM_FILE flag to prevent local file inclusion in API gateway (#20965, @TomeHirata)
• [Tracking] Fix Claude Agent SDK tracing by capturing messages from receive_messages (#20778, @smoorjani)
• [Build / Tracking] Add missing authentication for fastapi routes (#20920, @TomeHirata)
• [Evaluation] Fix guardrails scorer compatibility with guardrails-ai 0.9.0 (#20934, @smoorjani)
• [UI] Fix duplicated title and add icons to Experiments/Prompts page headers (#20813, @B-Step62)
• [Tracing] Trace UI papercut: highlight searched text and change search box hint's wording. (#20841, @PattaraS)
• [Prompts] Fix arbitrary file read via prompt tag validation bypass in Model Registry (#20833, @TomeHirata)
• [Tracking] Fix RestException crash on null error_code and incorrect except clause (#20903, @copilot-swe-agent)
• [UI] Fix Disable action button in Traces Tab (#20883, @joelrobin18)
• [UI] Fix experiment rename modal not refreshing experiment details (#20882, @joelrobin18)
• [Build] Skip workspace header when workspace is disabled (#20904, @TomeHirata)
• [UI] Block CORS for ajax paths (#20832, @TomeHirata)
• [UI] [UI] Improve empty states across Experiments, Models, Prompts, and Gateway pages (#20044, @ridgupta26)
• [UI] UI: Improve empty states for Traces and Sessions tabs (#20034, @ridgupta26)
• [Build] Validate webhook url to fix SSRF vulnerability (#20747, @TomeHirata)
• [Scoring / Tracing] Fix TypeError in online scoring config endpoint when basic-auth is enabled (#20783, @copilot-swe-agent)
• [Tracing] Fix experiment_id type error in gateway config resolver (#20764, @copilot-swe-agent)
• [UI] Fix docs link to respect workflow type (GenAI vs ML) (#20752, @copilot-swe-agent)
• [Tracking] Fix: Do not emit pickle warning when user calls mlflow.pyfunc.log_model with loader_module param (#20727, @WeichenXu123)
• [Tracing] Change cache config to prevent search bounce (#20688, @PattaraS)
• [Evaluation] Fix multiple align() calls on MemoryAugmentedJudge (#20708, @smoorjani)
• [Evaluation] Batch embedding calls for Databricks endpoints to avoid size limit errors (#20685, @smoorjani)
• [Evaluation] Fix the UI for MemAlign-ed scorers (#20632, @smoorjani)
• [Tracing] Fix type hints lost with @mlflow.trace decorator (#20648, @veeceey)
• [Evaluation] Use JSONAdapter for best-effort structured outputs in MemAlign predictions (#20679, @smoorjani)
• [Tracking] Fix mlflow demo URL to use experiment ID instead of name (#20678, @copilot-swe-agent)
• [Tracking] Fix circular import in FileStore caused by PromptVersion import (#20677, @copilot-swe-agent)
• [Scoring / Tracing] Fix error handling for streaming request (#20610, @TomeHirata)
• [Models] Fix warning message: add space and documentation link for pickle security (#20656, @copilot-swe-agent)
• [Evaluation] Fix SHAP compatibility for shap >= 0.47 (#20623, @copilot-swe-agent)
• [Prompts] Fix the deadlock between run linking and trace linking (#20620, @TomeHirata)
• [Tracking] Fix FTP artifact path handling on Windows with Python 3.11+ (#20622, @copilot-swe-agent)
• [Evaluation] Fix failed judge call error propagation (#20601, @AveshCSingh)
• [Tracking] Fix off-by-one error in _validate_max_retries and _validate_backoff_factor (#20597, @vb-dbrks)
• [Prompts] Fix bug: linking prompt to experiments does not work for default experiments (#20588, @PattaraS)
• [Build] Fix Docker full image tags not being published for versioned releases (#20589, @copilot-swe-agent)
• [Prompts] Implement locking mechanism to prevent race conditions during prompt linking (#20586, @TomeHirata)
• [Prompts] Revert "Fix bug: linking prompt to experiments does not work for defa… (#20585, @PattaraS)
• [Prompts] Fix bug: linking prompt to experiments does not work for default experiments (#20562, @PattaraS)
• [Model Registry] Fix N+1 query issue in search_registered_models (#20493, @Karim-siala)
• [Tracking] Fix optimistic pagination in SQLAlchemy store _search_runs and handle max_results=None (#20547, @copilot-swe-agent)
• [UI] Add cancel button for LLM judge evaluations in trace details drawer (#20519, @danielseong1)
• [UI] Fix incorrect 'Trace level' label in session judges modal (#20520, @danielseong1)
• [Tracing] fix: allow overriding notebook trace iframe base URL (#20485, @TatsuyaHayashino)
• [Prompts] Include the prompt model config in the optimized prompt (#20431, @chenmoneygithub)
• [Tracing / UI] Fix Anthropic trace UI rendering for tool_result with image content (#20190, @joncarter1)
• [Tracking] Enforce authorization on AJAX proxy artifact APIs (#20035, @mprahl)
• [Tracking] Ensure server-provided artifact root is reused on MLflowClient calls (#19336, @mprahl)
• [UI] Fix trace selection not registering in SelectTracesModal (#20099, @joelrobin18)

Documentation updates:

• [Docs] Add documentation for mlflow migrate-filestore command (#20616, @harupy)
• [Docs] Document X-MLFLOW-WORKSPACE header for AI Gateway endpoints with workspace fallback behavior (#20984, @copilot-swe-agent)
• [Docs] Fix outdated server-features references to server-info (#20948, @copilot-swe-agent)
• [Docs / Tracing] Remove span attributes filtering from search traces documentation (#20858, @copilot-swe-agent)
• [Docs] Add Modal as a supported deployment target with full documentation (#20032, @debu-sinha)
• [Docs] Add gateway usage tracking doc page (#20748, @TomeHirata)
• [Docs / Evaluation] Fix MemAlign bug bash issues (#20712, @veronicalyu320)
• [Docs] Fix docs: trace spans are stored in database, not artifact storage (#20668, @B-Step62)
• [Prompts] Change header level for "Automatic Prompt Linking" section in use-prompts-in-apps.mdx (#20661, @PattaraS)
• [Docs] Multi-turn evaluation launch documentation (#20443, @smoorjani)
• [Prompts] Update use-prompts-in-apps.mdx with a section for prompt linking under traced method (#20593, @PattaraS)
• [Docs] docs: Add missing targets arg in huggingface dataset docs (#20637, @KarelZe)
• [Build] Display rule names instead of IDs in clint error output (#20592, @copilot-swe-agent)
• [Docs] Detailed guide for setting up SSO with mlflow-oidc-auth plugin (#20556, @WeichenXu123)
• [Prompts] Mark prompt registry APIs as stable. (#20507, @PattaraS)
• [Docs] code-based scorer examples (#20407, @SomtochiUmeh)
• [Docs] Custom judges section (#20393, @SomtochiUmeh)
• [Docs] (mostly) copy over eval datasets article from managed docs (#19787, @achen530)
• [Docs] Add the RAG built-in judges section (#20369, @SomtochiUmeh)
• [Docs] Fix ToolAgent name formatting in ag2 documentation and examples (#20470, @Umakanth555)
• [Docs] Add collection_name parameter to CrewAI knowledge configuration in docs and example (#20469, @Umakanth555)
• [Docs] Update index and predefined judges pages (#20368, @SomtochiUmeh)
• [Docs] docs: Clarify -full Docker image availability from v3.9.0 onwards (#20223, @copilot-swe-agent)
• [Docs] Generalize Knowledge Cutoff Note in CLAUDE.md beyond model names (#20165, @copilot-swe-agent)

Small bug fixes and documentation updates:

#20959, #20915, #20986, #20956, #20912, #20955, #20943, #20919, #20776, #20826, #20781, #20767, #20761, #20760, #20763, #20762, #20687, #20746, #20682, #20667, #20658, #20578, #20559, #20495, #20497, @TomeHirata; #21006, #20980, #20707, #20777, @bbqiu; #20950, #21008, #20877, #20822, #20817, #20813, #20816, #20796, #20815, #20765, #20716, #20689, #20744, #20690, #20451, #20502, #20252, #20314, #20210, @B-Step62; #21000, #20975, #20806, #20449, #20686, #20603, #20573, #20572, #20584, #20551, #20526, #20550, #20523, #20525, #20453, #20478, #20452, #20438, #20474, #20460, #20457, #20459, #20456, #20444, #20418, #20285, #20284, #20283, #20282, #20281, #20280, #20051, @smoorjani; #21005, #21007, #20880, #20857, #20802, #20779, #20717, #20713, #20714, #20692, #20693, #20683, #20675, #20665, #20674, #20673, #20663, #20662, #20659, #20652, #20649, #20650, #20647, #20646, #20641, #20638, #20635, #20634, #20633, #20626, #20625, #20621, #20619, #20618, #20617, #20606, #20564, #20581, #20570, #20568, #20566, #20558, #20560, #20543, #20554, #20537, #20536, #20532, #20530, #20528, #20512, #20505, #20501, #20498, #20496, #20491, #20490, #20489, #20487, #20486, #20484, #20483, #20482, #20441, #20436, #20427, #20417, #20400, #20399, #20397, #20395, #20396, #20391, #20342, #20341, #20332, #20326, #20316, #20315, #20305, #20300, #20299, #20297, #20293, #20268, #20262, #20260, #20251, #20250, #20244, #20235, #20228, #20227, #20226, #20220, #20202, #20186, #20172, #20152, #20150, #19984, #20102, #20098, #20095, #20093, #20094, #20091, #20090, #20089, #20088, #20087, #20086, #20085, #20084, #20083, #20082, #20081, #20080, #20077, #20076, #20075, #20070, #20067, #20069, #20020, #20026, @copilot-swe-agent; #20793, #20791, #20768, @WeichenXu123; #20979, #20701, #20609, #20608, #20569, #20535, #20481, #20318, #20224, #20149, #20119, #20068, #20014, #20016, #20019, @harupy; #20973, @Gkrumbach07; #21003, #20936, #20730, #20041, #20381, @xsh310; #20989, #20830, #20766, #20759, #20758, #20757, #20756, #20699, #20697, #20696, #20695, #20694, #20255, #20254, #20253, #20248, #20247, #20010, #20009, #19999, #19998, #19976, #19975, #19974, #19973, #19971, @daniellok-db; #20976, @aravind-segu; #20725, #20339, #20565, #20660, #20455, #20440, #20404, #20403, #20402, #20567, #20542, #20541, #20540, #20557, #20503, #20506, #20500, #20499, #20467, #20338, #20337, #20331, #20462, #20329, #20328, #20323, @serena-ruan; #20737, @jamesbxwu; #20862, #20861, @PattaraS; #20805, #20705, #20373, @mprahl; #20773, @etirelli; #20753, @etscript; #20629, #19758, @justinwei-db; #20711, @kevin-lyn; #20576, @nisha2003; #20553, #20521, @danielseong1; #20548, @bartosz-grabowski; #20504, @smivv; #20527, @BenWilson2; #20363, #20364, @rollyjoel; #20494, @dbczumar; #20360, #20340, #20313, #20312, #20276, #20275, #20261, #20233, #19484, @hubertzub-db; #20359, @LiberiFatali; #20386, @chenmoneygithub; #20159, @ispoljari

Major New Features

• 🏢 Organization Support in MLflow Tracking Server: MLflow now supports multi-workspace environments! You can organize your experiments and resources across different workspaces with a new landing page that lets you navigate between them seamlessly. (#20702, #20657, @mprahl, @Gkrumbach07, @B-Step62)
• 💬 Multi-turn Conversation Simulation: Building on the conversation simulator introduced in 3.9, we've made it fully public and easily subclassable. You can now create custom simulation scenarios, compare sessions with goal/persona matching, and distill conversations into reusable goal/persona pairs for comprehensive agent testing. (#20243, #20377, #20289, @smoorjani)
• 💰 Trace Cost Tracking: Gain visibility into your LLM spending! MLflow now automatically extracts model information from LLM spans and calculates costs, with a new UI that renders model and cost data directly in your trace views. (#20327, #20330, @serena-ruan)
• 🎯 Top-level GenAI/Classical ML Split: We've redesigned the navigation to provide a frictionless experience. A new workflow type selector in the top-level navbar lets you quickly switch between GenAI and Classical ML contexts, with streamlined sidebars that reduce visual clutter. (#20158, #20160, #20161, #20699, @ispoljari, @daniellok-db)
• 🎮 MLflow Demo Experiment: Get started with MLflow faster than ever! The new mlflow demo CLI command generates a fully-populated demo environment with sample traces, prompts, and evaluation data so you can explore MLflow's features hands-on without any setup. (#19994, #19995, #20046, #20047, #20048, #20162, @BenWilson2)
• 📊 Gateway Usage Tracking: Monitor your AI Gateway endpoints with detailed usage analytics. A new usage page shows request patterns and metrics, with trace ingestion that links gateway calls back to your experiments for end-to-end observability. (#20357, #20358, #20642, @TomeHirata)

Stay tuned for the full release, which will be packed with even more features and bugfixes.

To try out this release candidate, please run:

pip install mlflow==3.10.0rc0

MLflow 3.9.0 Release

MLflow 3.9.0 is a major release focused on AI Observability and Evaluation capabilities, bringing powerful new features for building, monitoring, and optimizing AI agents. This release introduces an AI-powered assistant, comprehensive dashboards for agent performance, a new judge optimization algorithm, judge builder UI, continuous monitoring with LLM judges, and distributed tracing.

1. MLflow Assistant Powered by Claude Code

MLflow Assistant transforms coding agents like Claude Code into experienced AI engineers by your side. Unlike typical chatbots, the assistant is aware of your codebase and context —it's not just a Q&A tool, but a full-fledged AI engineer that can find root causes for issues, set up quality tests, and apply LLMOps best practices to your project.

Key capabilities include:

• No additional costs: Use your existing Claude Code subscription. MLflow provides the knowledge and integration at no cost.
• Context-rich assistance: Understands your local codebase, project structure, and provides tailored recommendations—not generic advice.
• Complete dev-loop: Goes beyond Q&A to fetch MLflow data, read your code, and add tracing, evaluation, and versioning to your project.
• Fully customizable: Add custom skills, sub-agents, and permissions. Everything runs on your machine with full transparency.

Open the MLflow UI, navigate to the Assistant panel in any experiment page, and follow the setup wizard to get started.

2. Dashboards for Agent Performance Metrics

A new "Overview" tab in GenAI experiments provides pre-built charts and visualizations for monitoring agent performance at a glance. Monitor key metrics like latency, request counts, and quality scores without manual configuration. Identify performance trends and anomalies across your agent deployments, and get tool call summaries to understand how your agents are utilizing available tools.

Navigate to any GenAI experiment and click the "Overview" tab to access the dashboard. Charts are automatically populated based on your trace data. Have a specific visualization need? Request additional charts via GitHub Issues.

3. MemAlign: A New Judge Optimizer Algorithm

MemAlign is a new optimization algorithm that learns evaluation guidelines from past feedback and dynamically retrieves relevant examples at runtime. Improve judge accuracy by learning from human feedback patterns, reduce prompt engineering effort with automatic guideline extraction, and adapt judge behavior dynamically based on the input being evaluated.

Use the MemAlignOptimizer to optimize your judges with historical feedback.

4. Configuring and Building a Judge with Judge Builder UI

A new visual interface lets you create and test custom LLM judge prompts without writing code. Iterate quickly on judge criteria and scoring rubrics with immediate feedback, test judges on sample traces before deploying to production, and export validated judges to the Python SDK for programmatic integration.

Navigate to the "Judges" section in the MLflow UI and click "Create Judge." Define your evaluation criteria, scoring rubric, and test your judge against sample traces. Once satisfied, export the configuration to use with the MLflow SDK.

5. Continuous Online Monitoring with MLflow LLM Judges

Automatically run LLM judges on incoming traces without writing any code, enabling continuous quality monitoring of your agents in production. Detect quality issues in real-time as traces flow through your system, leverage pre-defined judges for common evaluations like safety, relevance, groundedness, and correctness, and get actionable assessments attached directly to your traces.

Go to the "Judges" tab in your experiment, select from pre-defined judges or use your custom judges, and configure which traces to evaluate. Assessments are automatically attached to matching traces as they arrive.

6. Distributed Tracing for Tracking End-to-end Requests

Track requests across multiple services with context propagation, enabling end-to-end visibility into distributed AI systems. LLM tracing maintains trace continuity across microservices and external API calls, debug issues that span multiple services with a unified trace view, and understand latency and errors at each step of your distributed pipeline.

Use the get_tracing_context_headers_for_http_request and set_tracing_context_from_http_request_headers functions to inject and extract trace context.

For a comprehensive list of changes, see the release change log.

We're excited to announce MLflow 3.9.0, which includes several notable updates:
Major New Features:
🔮 MLflow Assistant: Figuring out the next steps to debug your apps and agents can be challenging. We're excited to introduce the MLflow Assistant, an in-product chatbot that can help you identify, diagnose, and fix issues. The assistant is backed by Claude Code, and directly passes context from the MLflow UI to Claude. Click on the floating "Assistant" button in the bottom right of the MLflow UI to get started!
📈 Trace Overview Dashboard: You can now get insights into your agent's performance at a glance with the new "Overview" tab in GenAI experiments. Many pre-built statistics are available out of the box, including performance metrics (e.g. latency, request count), quality metrics (based on assessments), and tool call summaries. If there are any additional charts you'd like to see, please feel free to raise an issue in the MLflow repository!
✨ AI Gateway: We're revamping our AI Gateway feature! AI Gateway provides a unified interface for your API requests, allowing you to route queries to your LLM provider(s) of choice. In MLflow 3.9.0, the Gateway server is now located directly in the tracking server, so you don't need to spin up a new process. Additional features such as passthrough endpoints, traffic splits, and fallback models are also available, with more to come soon! For more detailed information, please take a look at the docs.
🔎 Online Monitoring with LLM Judges: Configure LLM judges to automatically run on your traces, without having to write a line of code! You can either use one of our pre-defined judges, or provide your own prompt and instructions to create custom metrics. Head to the new "Judges" tab within the GenAI Experiment UI to get started.
🤖 Judge Builder UI: Define and iterate on custom LLM judge prompts directly from the UI! Within the new "Judges" tab, you can create your own prompt for an LLM judge, and test-run it on your traces to see what the output would be. Once you're happy with it, you can either use it for online monitoring (as mentioned above), or use it via the Python SDK for your evals.
🔗 Distributed Tracing: Trace context can now be propagated across different services and processes, allowing you to truly track request lifecycles from end to end. The related APIs are defined in the mlflow.tracing.distributed module (with more documentation to come soon).
📚 MemAlign - a new judge optimizer algorithm: We're excited to introduce MemAlignOptimizer, a new algorithm that makes your judges smarter over time. It learns general guidelines from past feedback while dynamically retrieving relevant examples at runtime, giving you more accurate evaluations.
Features:
[Gateway] Add LiteLLM provider to support many other providers (#19394, @TomeHirata)
[Gateway] Add passthrough support for Anthropic Messages API (#19423, @TomeHirata)
[Gateway] Add passthrough support for Gemini generateContent and streamGenerateContent APIs (#19425, @TomeHirata)
[Gateway] Add routing strategy and fallback configuration support for gateway endpoints (#19483, @TomeHirata)
[Gateway] Deprecate Unity Catalog function integration in AI Gateway (#19457, @harupy)
[Gateway / UI] Create List API Keys landing page (#19441, @BenWilson2)
[Gateway / UI] Add Create API Keys functionality (#19442, @BenWilson2)
[Gateway / UI] Add delete and update capabilities for API Keys (#19446, @BenWilson2)
[Gateway / UI] Add endpoint listing page and tab layout (#19474, @BenWilson2)
[Gateway / UI] Add Create endpoint page and enhance provider select (#19475, @BenWilson2)
[Gateway / UI] Add Model select functionality for endpoint creation (#19477, @BenWilson2)
[Gateway / UI] Add Auth config to endpoint creation (#19494, @BenWilson2)
[Gateway / UI] Add the Endpoint Edit Page (#19502, @BenWilson2)
[Gateway / UI] Refactor the provider display for better UX (#19503, @BenWilson2)
[Gateway / UI] Create Endpoint details page (#19537, @BenWilson2)
[Gateway / UI] Add security notice banner (#19538, @BenWilson2)
[Gateway / UI] Create common editable combo box with extra modal select (#19546, @BenWilson2)
[Evaluation] Introduce MemAlign as a new optimizer for judge alignment (#19598, @smoorjani)
[Evaluation] Parallelize LLM calls in MemAlign guideline distillation (#20291, @veronicalyu320)
[Evaluation] Add GePaAlignmentOptimizer for judge instruction optimization (#19882, @alkispoly-db)
[Evaluation] Add Fluency scorer for evaluating text quality (#19414, @alkispoly-db)
[Evaluation] Add KnowledgeRetention built-in scorer (#19436, @alkispoly-db)
[Evaluation] Implement automatic discovery for builtin scorers (#19443, @alkispoly-db)
[Evaluation] Add Phoenix (Arize) third-party scorer integration (#19473, @debu-sinha)
[Evaluation] Add gateway provider support for scorers (#19470, @danielseong1)
[Evaluation] Introduce a conversation simulator into mlflow.genai (#19614, @smoorjani)
[Evaluation] Integrate conversation simulation into mlflow.genai.evaluate (#19760, @smoorjani)
[Evaluation] Make conversation simulator work with datasets (#19845, @SomtochiUmeh)
[Evaluation] Support for conversational datasets with persona, goal, and context (#19686, @SomtochiUmeh)
[Evaluation] Introduce conversational guidelines scorer (#19729, @smoorjani)
[Evaluation] Update tool call correctness judge to accept expected tool calls (#19613, @smoorjani)
[Evaluation] Support trace parsing fallback using Databricks model (#19654, @AveshCSingh)
[Evaluation] Documentation for online evaluation / scoring (#20103, @dbczumar)
[Evaluation] Job backend: Update job backend to use static names rather than function full names (#19430, @WeichenXu123)
[Evaluation] Job backend: support job cancellation (#19565, @WeichenXu123)
[Tracing] Support distributed tracing (#19920, @WeichenXu123)
[Tracing] Trace Metrics backend (#19271, @serena-ruan)
[Tracing] Add IS NULL / IS NOT NULL comparator support for trace metadata filtering (#19720, @dbczumar)
[Tracing] Auto-navigate to Events tab when clicking error spans (#20188, @anshuman-sahu)
[Tracing] Support shift+select for Traces (#20125, @B-Step62)
[Tracing] SpringAI Integration (#19949, @joelrobin18)
[Tracing] Reasoning in Chat UI for OpenAI, Anthropic, Gemini, Langchain, and PydanticAI (#19535, #19541, #19627, #19651, #19657, @joelrobin18)
[UI] Current Page context to assistant (#20139, @joelrobin18)
[UI] Assistant regenerate button (#20066, @joelrobin18)
[UI] Copy button Assistant (#20063, @joelrobin18)
[UI] Overview tab for GenAI experiments (#19521, @serena-ruan)
[UI] Enable Scorers UI feature flags (#19842, @danielseong1)
[UI] Improve LLM judge creation modal UX and variable ordering (#19963, @danielseong1)
[UI] Hide instructions section for built-in LLM judges (#19883, @danielseong1)
[UI] Change model provider and name to dropdown list (#19653, @chenmoneygithub)
[Prompts] Support Jinja2 template in prompt registry (#19772, @B-Step62)
[Prompts] Support metaprompting in mlflow.genai.optimize_prompts() (#19762, @chenmoneygithub)
[Prompts] Add option to delegate saving dspy model to dspy.module.save API (#19704, @WeichenXu123)
[Prompts / UI] Add traces mode to prompts details page and implement filtered traces (#19599, @TomeHirata)
[Tracking] Support mlflow.genai.to_predict_fn for app invocation endpoints (#19779, @jennsun)
[Tracking] Add log_stream API for logging binary streams as artifacts (#19104, @harupy)
[Tracking] Add import_checkpoints API for databricks SGC Checkpointing with MLflow (#19839, @WeichenXu123)
[Tracking] Support GC clean up for Historical Jobs (#19626, @joelrobin18)
[Tracking] Add JupyterNotebookRunContext for Tracking local Jupyter notebook as the source (#19162, @iyashk)
[Tracking] Full docker image support with db (#19979, @serena-ruan)
[Tracking] Add react route handling to communicate with the tracking server (#19010, @BenWilson2)
[Tracking] [TypeScript SDK] Simplify Databricks auth by delegating to Databricks SDK (#19434, @simonfaltum)
[Models] Safe model serialization: Support saving pytorch model via torch.export.save, add skops serialization format, and deprecate unsafe pickle/cloudpickle formats (#18759, #18832, #19692, #20151, @WeichenXu123)
Bug fixes:
[Gateway] Fix Anthropic and Gemini streaming for LiteLLM providers (#20398, @TomeHirata)
[Build] Include git submodule contents in Python package build (#20394, @copilot-swe-agent)
[Tracing] Fix duplicate traces in semantic kernel autolog (#20206, @harupy)
[Tracing] Fix Claude autolog to prioritize settings.json over OS environment variables (#20376, @alkispoly-db)
[Evaluation] Fix temperature/json issues with ConversationSimulator on managed (#20236, @xsh310)
[Tracing / UI] Add support for OpenAI function calling inputs in chat UI parsing (#20058, @daniellok-db)
[Tracking] Update checking code for pickle deserialization (#20267, @WeichenXu123)
[Gateway] Fix Vertex AI model configuration (#20242, @TomeHirata)
[UI] Store gateway<>scorer binding correctly (#20176, @TomeHirata)
[Evaluation] Support SparkDF trace handling in eval (#20207, @BenWilson2)
[Evaluation] Fix tool name extraction for tool call correctness (#20201, @smoorjani)
[Prompts] Fix scorers issue in metaprompting (#20173, @chenmoneygithub)
[UI] Propagate Run id context to Assistant (#20138, @joelrobin18)
[Model Registry] Allow for model registration to use KMS auth from different workspace (#20156, @BenWilson2)
[UI] Improve scorer trace picker UX and validation (#20178, @danielseong1)
[Evaluation] Improve MemAlign optimizer for incremental judge alignment (#20049, @veronicalyu320)
[Evaluation] Fix bug with max tokens using max output tokens (#20174, @smoorjani)
[Evaluation] Fix a race condition bug when using DF inputs for genai eval (#20079, @BenWilson2)
[Tracking] Fix DATABRICKS_CONFIG_PROFILE env var detection when fetching databricks credentials (#20112, @daniellok-db)
[Gateway] Move gateway invocation validation to fastapi middleware (#20111, @TomeHirata)
[Prompts] Fix the length check in mlflow.genai.optimize_prompts() (#19993, @chenmoneygithub)
[UI] Fix trace selection not registering in SelectTracesModal (#20099, @joelrobin18)
[UI] Fix LimitOverrunError in Assistant streaming (#20078, @joelrobin18)
[Tracing] CC Token usage (#20022, @joelrobin18)
[Gateway] Remove MLflow-specific auth_mode from LiteLLMConfig (#20059, @TomeHirata)
[UI] Assistant UI fix for dark theme (#20056, @joelrobin18)
[Tracing] Isolate runtime context between opentelemetry and mlflow (#19797, @B-Step62)
[UI] Prevent spurious 404 requests for relative image URLs in markdown (#20003, @harupy)
[Tracing] Support MLflow tracing with OpenTelemetry auto-instrumentation (#19501, @serena-ruan)
[UI] [UI] Fix session selector table column resizing and link behavior (#19927, @danielseong1)
[Gateway] Add Azure provider support in gateway configuration (#19933, @TomeHirata)
[Gateway] Propagate extra auth config to LiteLLM provider (#19931, @TomeHirata)
[Evaluation / UI] Add missing retrieval context error for retrieval scorers (#19895, @danielseong1)
[Evaluation / UI] Improve trace selection UX in scorer/judge UI (#19913, @danielseong1)
[Model Registry / Models] Fix infer_code_paths to capture transitive imports of functions/classes (#19814, @copilot-swe-agent)
[Tracking] fix for addressing rest api call latency in databricks job run (#19886, @WeichenXu123)
[UI] Enable {{trace}} variable support in sample judge evaluation (#19851, @danielseong1)
[Scoring] Check security before extracting tar file (#19557, @WeichenXu123)
[Gateway] Fix authorization header duplication (#19853, @TomeHirata)
[Gateway] Fix Gateway error handling to translate MlflowException to HTTPException (#19728, @danielseong1)
[Gateway] Remove gateway_deprecated decorator - AI Gateway is not deprecated (#19821, @copilot-swe-agent)
[Tracking] Make local artifact location creation lazy to support read-only proxy environments (#19678, @BenWilson2)
[Evaluation] fixed databricks hosted llm failure due to response_schema injection (#19741, @sinanshamsudheen)
[Evaluation] Add @overload annotations to @scorer decorator for proper type inference (#19570, @mr-brobot)
[Tracking] Add debug logging for 500 errors in catch_mlflow_exception (#19781, @harupy)
[Tracing] [Bug fix] Support search traces by string feedback / expectation values (#19719, @dbczumar)
[Tracing / UI] Fix scorer creation UX issues (#19756, @danielseong1)
[Evaluation] Fix KnowledgeRetention model parameter not propagating to inner scorer (#19753, @danielseong1)
[Tracking] [BUG] serve-artifacts is not enabled in docker-compose #19700 (#19701, @zjffdu)
[Tracing] Fix type signature loss in @trace_disabled decorator (#19569, @mr-brobot)
[Tracking] Fix: Return 400 instead of 500 for invalid experiment_id (#19655, @copilot-swe-agent)
[Models] Fix schema enforcement for pandas StringDtype (#19518, @harupy)
[Tracing] Fix Python 3.12 DeprecationWarning from generator.throw() in tracing (#19629, @mr-brobot)
[Evaluation] Fix structured outputs for databricks serving endpoints (#19572, @smoorjani)
[Models / Scoring] Add dict to PyFuncOutput type alias for ResponsesAgent/ChatAgent/ChatModel (#19560, @copilot-swe-agent)
[Tracking] Fix enable_git_model_versioning to work from subdirectories (#19529, @copilot-swe-agent)
Documentation updates:
[Docs] fix: Remove multi_class argument from scikit-learn's LogisticRegression in docs (#20266, @SOORAJTS2001)
[Docs] Add doc for distributed tracing (#20027, @WeichenXu123)
[Docs] Add Judge Builder UI documentation (#20163, @danielseong1)
[Docs] Add framework integration examples for AI Gateway query-endpoint page (#20137, @TomeHirata)
[Docs] Add "Evaluation Examples" article (#19722, @achen530)
[Docs] [1/3] Add gateway tracing guide for LiteLLM, OpenRouter, and Vercel AI Gateway (#20031, @B-Step62)
[Docs] Update prompt optimization doc to include metaprompting (#19966, @chenmoneygithub)
[Docs] Reorganize gateway page structure (#19968, @TomeHirata)
[Build / Docs] Fix broken auth REST API documentation links (#19872, @copilot-swe-agent)
[Docs] Add setup and query documentation for new AI Gateway (#19804, @TomeHirata)
[Docs] Add additional eval dataset serialization examples (#19697, @BenWilson2)
[Docs] ML-60766: Add dataset schema from managed content to SDK reference page (#19676, @achen530)
[Docs / Prompts] Fix duplicate tags argument in register_prompt documentation example (#19591, @copilot-swe-agent)
[Docs] Fix ML-59546 eval quickstart links to wrong place, add notebook version of eval quickstart (#19511, @achen530)
[Docs] Add documentation for KnowledgeRetention scorer (#19478, @alkispoly-db)
Small bug fixes and documentation updates:
#20406, #20122, #20317, #20333, #20361, #20274, #20362, #20249, #20169, #20345, #20252, #20314, #20214, #20215, #20210, #20212, #20142, #20183, #20121, #20141, #20140, #20124, #20073, #20062, #20065, #19893, #19912, #19464, #19857, #19401, #19600, #19555, #19400, #19392, #19393, @B-Step62; #20323, #20263, #19982, #20218, #20143, #20146, #20145, #20064, #20117, #20144, #20110, #20050, #20017, #20116, #20118, #19989, #19953, #19836, #19915, #19955, #19952, #19940, #19939, #19938, #19937, #19877, #19874, #19869, #19867, #19865, #19837, #19835, #19834, #19864, #19873, #19833, #19825, #19876, #19799, #19798, #19793, #19771, #19770, #19635, #19634, #19633, #19632, #19624, #19622, #19621, #19620, #19631, #19619, #19747, #19609, #19608, #19607, #19606, #19604, #19603, #19602, #19601, #19588, #19587, #19581, #19585, #19610, #19590, #19580, #19579, #19578, #19577, #19576, #19234, @serena-ruan; #20378, #20385, #20205, #20237, #20193, #20171, #20155, #20170, #20132, #20097, #20100, #20101, #19736, #19717, #19716, #19759, #19718, #19714, #19713, #19712, #19711, #19840, #19710, #19709, #19708, #19777, #19707, @dbczumar; #20387, #19981, #19964, @bbqiu; #20390, #20334, #20208, #19978, #19980, #19875, #19854, #19816, #19815, #19796, #19806, #19785, #19789, #19769, #19748, #19773, #19782, #19706, #19523, #19505, #19450, #19482, #19458, #19433, #19431, #19455, #19417, #19426, #19424, @harupy; #20355, #20245, #20120, #20229, #20114, #20053, #20012, #19972, #20002, #19991, #19990, #19977, #19986, #19985, #19967, #19957, #19960, #19954, #19945, #19941, #19934, #19917, #19916, #19905, #19904, #19903, #19900, #19899, #19897, #19894, #19892, #19890, #19888, #19887, #19861, #19828, #19818, #19803, #19802, #19791, #19788, #19795, #19790, #19786, #19783, #19767, #19768, #19746, #19735, #19733, #19732, #19726, #19561, #19549, #19544, #19543, #19510, #19486, #19487, #19463, #19871, @copilot-swe-agent; #20308, #20264, #20109, #20181, #20180, #20177, #20134, #20107, #20015, #20007, #20008, #19930, #20006, #20005, #19965, #19942, #19944, #19950, #19936, #19947, #19948, #19946, #19870, #19824, #19823, #19856, #19863, #19858, #19860, #19849, #19822, #19765, #19792, #19764, #19763, #19618, #19453, #19452, #19404, #19390, #19290, @TomeHirata; #20350, #20203, #19675, #19677, #19674, #19476, #19447, @BenWilson2; #20286, #20157, #20051, #20216, #20200, #20213, #20194, #20072, #20195, #20175, #20039, #19844, #19935, #19696, #19451, #19409, @smoorjani; #20209, #20131, #19742, #19969, #19734, #19480, #19351, @daniellok-db; #20204, #20164, #20192, #19997, #19925, #19850, #19914, #19774, #19721, #19673, #19623, #19668, #19496, #19554, #19471, @danielseong1; #20037, #19884, #19846, #19843, #19813, #19454, #19391, #19322, #19388, #19307, #19382, @xsh310; #20130, @iyashk; #20147, #20030, #19962, #19826, @kevin-lyn; #20108, #20071, #19743, #20045, #20042, #19959, #19880, @SomtochiUmeh; #20025, #19662, #19749, #19738, #19419, @WeichenXu123; #19847, @jaceklaskowski; #19820, @Abhiii47; #19800, @shreenidhi2205; #19703, #19693, #19689, #19688, #19664, #19663, #19660, #19534, #19533, #19532, #19531, @hubertzub-db; #19652, @AMRUTH-ASHOK; #19493, #19495, @alkispoly-db; #16372, @mohammadsubhani; #19522, @pmeier