ZenML Release Notes

🎯 Pipeline Execution Control

Pause and Resume Pipeline Runs: Introduced zenml.wait(...) to pause dynamic pipelines while waiting for external inputs, automatically freeing resources until the input is provided. Runs can be resumed automatically (when using remote orchestrators with snapshot support) or manually via zenml pipeline runs resume <ID>. PR #4588

Override Step Inputs on Replay: You can now override step inputs when replaying a pipeline run, giving you more flexibility to rerun pipelines with different data. PR #4590

🔧 Materializers and Data Handling

Dataclass Materializer: Added a built-in materializer for JSON-serializable dataclasses, making it easier to pass structured data between steps. PR #4600

LakeFS Data Versioning Example: New example demonstrating the "pass references, not data" pattern for terabyte-scale datasets. ZenML steps exchange lightweight LakeFS pointers while actual data stays in LakeFS, accessed via its S3-compatible gateway. PR #4559

☁️ Infrastructure and Deployment

Helm Environment Variable Overrides: Environment variables specified in zenml.environment, zenml.secretEnvironment, and worker deployment configurations can now override computed settings from the Helm chart. PR #4595

Secret Environment Variables in Helm: Added support for injecting secret environment variables into the ZenML server deployment via Helm without committing secrets to values.yaml, enabling better GitOps workflows. PR #4606

Docker Build Arguments: Build arguments defined in DockerSettings are now properly declared with ARG instructions in auto-generated Dockerfiles. PR #4612

🔐 Authentication and Credentials

Improved GCP Credentials Refresh: Implemented native GCP credentials refresh using service connector logic, replacing periodic expiration checks for more reliable OAuth2 credential handling. PR #4527

📊 Pipeline Configuration

Step Parameter Schema Storage: Step parameter specifications are now stored for better schema validation when triggering pipeline snapshots. PR #4591

Fixed

Exception Handling for Isolated Steps: Isolated steps in dynamic pipelines now raise the correct exception types instead of always wrapping them in RuntimeError, making error handling more intuitive and consistent. PR #4589

Kubernetes API Request Timeout: Fixed typing issue in the Kubernetes orchestrator and step operator to ensure api_request_timeout is properly applied as an integer value. PR #4605

Client-Server Compatibility: Resolved compatibility issue with clients <=0.92.0 by not passing both pipeline run ID and step run ID to logs requests. PR #4614

Dynamic Pipeline Monitoring: Fixed dictionary size change error that occurred when the main thread modified the _steps_to_monitor dictionary while the monitoring thread was iterating over it. PR #4619

Step Config Template Parameters: Step parameters defined in config templates now correctly take precedence over function default values in dynamic pipelines. PR #4624

📦 Dependencies

Updated Pydantic to version 2.12.5. PR #4552

What's Changed

• Add version 0.93.3 to legacy docs by @github-actions[bot] in #4556
• Add 0.94.0 to the migration tests by @github-actions[bot] in #4564
• Fix kubernetes service selector by @schustmi in #4567
• Update contribution guidelines for PRs and commits by @strickvl in #4587
• Only show named snapshots in CLI by @schustmi in #4583
• Allow helm env vars to override computed settings by @stefannica in #4595
• Enhance changelog workflow to dispatch new release to zenml-projects-backend by @safoinme in #4592
• Adjusted helm chart to have the right permissions for PVC by @AlexejPenner in #4225
• Replaced pngs with avif by @AlexejPenner in #4413
• Handle activation exceptions for individual integrations by @strickvl in #4490
• Update CLAUDE.md and AGENTS.md for recent codebase changes by @strickvl in #4585
• Fix S3 double-close AssertionError on cleanup by @strickvl in #4593
• Bugfix/runner logs for triggered runs by @Json-Andriopoulos in #4601
• Add dataclass materializer by @schustmi in #4600
• Store step parameter schemas by @schustmi in #4591
• Raise correct exceptions for isolated steps and pipeline retries by @schustmi in #4589
• Better gcsfs credentials refresh by @schustmi in #4527
• Step types by @schustmi in #4608
• Allow overwriting step inputs for pipeline replays by @schustmi in #4590
• Update pydantic version to 2.12.5 by @htahir1 in #4552
• Fix api_request_timeout typing in K8S orchestrator by @avishniakov in #4605
• Pricing details on trigger doc page by @Json-Andriopoulos in #4611
• Improve error when not having local extra installed by @schustmi in #4613
• Heartbeat logs and neptune update by @bcdurak in #4581
• Add ARG instructions to Dockerfile for build arguments defined in DockerSettings by @jlamperez in #4612
• Don't pass pipeline run and step run ids to logs request by @schustmi in #4614
• Fix broken image link in self-hosted K8s deploy docs by @strickvl in #4617
• Fix dict size change during iteration error by @schustmi in #4619
• Fix log keys by @schustmi in #4620
• Pause and resume pipeline runs by @schustmi in #4588
• Fix running property of dynamic runner futures by @schustmi in #4622
• Use step parameters from config template over function defaults by @schustmi in #4624
• Add a way to inject secret environment variables into the server deployment (helm) by @spwoodcock in #4606
• Add LakeFS data versioning example by @htahir1 in #4559
• Feature/kitaru analytics support by @safoinme in #4618
• Configurable repository directory by @schustmi in #4623
• Prepare release 0.94.1 by @github-actions[bot] in #4625

New Contributors

@jlamperez made their first contribution in #4612

@spwoodcock made their first contribution in #4606

Full Changelog

0.94.0...0.94.1

Original source Report a problem

Breaking Changes

Old endpoints and client methods for legacy triggers, actions and event sources have been removed. This shouldn't affect you unless you explicitly used those endpoints or methods in your code.

Custom step operator flavors must implement new submit_step and get_step_status methods to work with dynamic pipelines. The legacy launch method will only work in static pipelines as a fallback. The Spark step operator is not yet compatible with dynamic pipelines. PR #4515

🚀 New Integrations

Run:AI Step Operator: ZenML now supports running individual pipeline steps on Run:AI clusters with fractional GPU allocation, enabling more efficient resource utilization for ML workloads. PR #4439

✨ New Features

Step and Pipeline Replays: You can now replay existing step or pipeline runs with the same inputs and configuration. When replaying a pipeline run, you can specify which steps to skip and reuse from the original run. A debug mode is also available to run replays on your active stack with a local orchestrator. PR #4456

Triggers and Native Schedules (PRO): Introduced the Trigger concept for automated pipeline execution. The first supported trigger type is Schedules, which offers lifecycle management, automatic synchronization with orchestrators, and centralized management across stacks. PR #4482

Step Run Filtering by Version: Added the ability to filter step runs by version, making it easier to track and manage specific versions of your pipeline steps. PR #4518

🔧 Improvements

Enhanced Dynamic Pipeline Monitoring: Improved the execution and monitoring of isolated steps in dynamic pipelines. Step submission is now separated from monitoring, preventing thread blocking during step execution. PR #4369

SkyPilot Integration Update: Updated the SkyPilot integration to support version 0.11.x, including migration to the new async API and support for new resource settings. PR #4462

Kubernetes Retry Configuration: Added configurable timeout options for Kubernetes orchestrator and step operator API calls, ensuring proper retry behavior and preventing unnecessary hanging. PR #4525

Git Submodule Support: Code archives now include files from git submodules when uploading to the artifact store, ensuring complete code tracking for repositories with submodules. PR #4496

Fixed

Fixed handling of variadic keyword arguments in pipeline functions, ensuring they are properly flattened before being passed to the pipeline. PR #4528

Fixed Azure integration dependencies by limiting the azure-mgmt-resource version to avoid compatibility issues with the 25.0.0 package split. PR #4516

Fixed handling of external artifacts with None values and reduced chunk size when fetching many step runs to avoid URI too large errors. PR #4551

What's Changed

Add version 0.93.2 to legacy docs by @github-actions[bot] in #4504

Add 0.93.3 to the migration tests by @github-actions[bot] in #4505

Clarify Snapshots vs Deployments in documentation by @strickvl in #4497

Improve dynamic pipeline step monitoring by @schustmi in #4369

add ENV_ZENML_CONTAINER_PYTHON_EXECUTABLE support by @avishniakov in #4501

Run:AI Integration - Step Operator by @safoinme in #4439

Remove Discord notification from release prep and reduce CI runner waste by @strickvl in #4500

Add RLM document analysis example with dynamic pipelines by @strickvl in #4491

Update skypilot Integrations by @safoinme in #4462

Update .gitignore to include Codex settings by @strickvl in #4513

Fix azure integration dependencies by @schustmi in #4516

Run release test pipeline using kubernetes orchestrator by @schustmi in #4517

Allow filtering step runs by version by @schustmi in #4518

Attach log context to run in dynamic pipelines by @schustmi in #4519

Fix broken docs links found via lychee link checker by @strickvl in #4526

Remove wrong error message when running dynamic snapshots by @schustmi in #4533

Split step operator submission and monitoring by @schustmi in #4515

Suggest installing all stack requirements in import error by @AlexejPenner in #4530

Fix handling of variadic keyword args in pipeline function by @schustmi in #4528

Update Run:AI Step-Operator for new submit logic by @safoinme in #4535

Improve k8s client retries by @avishniakov in #4525

Preselect organization and workspace upon zenml login by @bcdurak in #4531

Add PathMaterializer documentation to materializers page by @strickvl in #4534

Fix broken Evidently and W&B documentation links by @strickvl in #4538

Fix logging of existing job status in Kubernetes orchestrator by @kshitijrajsharma in #4549

Feature/triggers v2 and native schedules by @Json-Andriopoulos in #4482

Misc fixes by @schustmi in #4551

Step/Pipeline replays by @schustmi in #4456

!fixup: disable flaky tests by @Json-Andriopoulos in #4554

Include git submodule files in code archive by @apoorvdarshan in #4496

Add docs references for zenml-scoping and zenml-pipeline-authoring skills by @strickvl in #4539

Prepare release 0.94.0rc0 by @github-actions[bot] in #4555

Fix for SDK docs release by @bcdurak in #4560

Small schedule fixes by @schustmi in #4561

Prepare release 0.94.0 by @github-actions[bot] in #4563

New Contributors

@kshitijrajsharma made their first contribution in #4549

@apoorvdarshan made their first contribution in #4496

Full Changelog

0.93.3...0.94.0

Original source Report a problem

You can now replay existing step or pipeline runs with the same inputs and configuration. When replaying a pipeline run, specify which steps to skip and reuse outputs from the original run. A debug mode is also available to run replays on your active stack with a local orchestrator. Note: this release removes old endpoints for legacy triggers, actions, and event sources, and custom step operator flavors must now implement submit_step and get_step_status methods to work with dynamic pipelines.

Original source Report a problem