Watchtower Updates & Release Notes

Watchtower v1.20.0 introduces a significantly updated HTTP API and support for label-based container filtering. It also includes several fixes and improvements, such as improved GCP Artifact Registry support, a correction to a bug impacting the revive-stopped container configuration, the addition of caching for bearer tokens, and parallelized staleness checks.

Note

Legacy notification and HTTP API configurations are deprecated. You are encouraged to update your configurations.

For more info:

• Notification Configuration Deprecation
• HTTP API Configuration Deprecation

Features

• refactor/http-api by @nicholas-fedor in #1939
• feat(filters): add label-based container enable and disable filtering by @nicholas-fedor in #1969

Fixes

• fix(registry): derive bearer service from realm by @llc1123 in #1895
• refactor: restructure registry auth and expand test coverage by @nicholas-fedor in #1951
• refactor(actions): parallelize staleness checks with bounded concurrency by @nicholas-fedor in #1958
• fix: handle local-only images and clarify self-update port messages by @nicholas-fedor in #1960
• fix: ensure revive-stopped configuration reaches scheduled update runs by @nicholas-fedor in #1961
• refactor: consolidate restart-policy updates into SetNoRestartPolicy helper by @nicholas-fedor in #1965
• fix(flags): validate notification-url secret file entries by @nicholas-fedor in #1971

Dependency Updates

• chore(deps): update module github.com/pelletier/go-toml/v2 to v2.4.3 by @renovate[bot] in #1890
• chore(deps): update module golang.org/x/text to v0.39.0 by @renovate[bot] in #1898
• chore(deps): update go module directive to v1.26.5 by @renovate[bot] in #1904
• chore(deps): update module golang.org/x/mod to v0.38.0 by @renovate[bot] in #1914
• chore(deps): update module golang.org/x/net to v0.57.0 by @renovate[bot] in #1917
• chore(deps): update module golang.org/x/tools to v0.48.0 by @renovate[bot] in #1920
• chore(deps): update github.com/google/pprof digest to b9395ee by @renovate[bot] in #1926
• chore(deps): update module github.com/prometheus/common to v0.70.0 by @renovate[bot] in #1928
• chore(deps): update module github.com/gofiber/utils/v2 to v2.2.0 by @renovate[bot] in #1944
• chore(deps): update module github.com/mattn/go-isatty to v0.0.23 by @renovate[bot] in #1947
• chore(deps): update module github.com/docker/cli to v29.6.2+incompatible by @renovate[bot] in #1953
• chore(deps): update module github.com/gofiber/schema to v1.8.3 by @renovate[bot] in #1973
• chore(deps): update go-openapi packages to v0.27.1 by @renovate[bot] in #1975
• chore(deps): update go-openapi packages by @renovate[bot] in #1980
• chore(deps): update module github.com/go-logr/logr to v1.4.4 by @renovate[bot] in #1982
• chore(deps): update module github.com/klauspost/compress to v1.19.1 by @renovate[bot] in #1983
• chore(deps): update module github.com/prometheus/client_golang to v1.24.0 by @renovate[bot] in #1986
• chore(deps): update module github.com/nicholas-fedor/shoutrrr to v0.16.2 by @renovate[bot] in #1991

New Contributors

• @llc1123 made their first contribution in #1895

Full Changelog: v1.19.0...v1.20.0

Features

feat: filter containers by image name by @Greite in #1762

Fixes

fix(notifications): correct deprecation notice url by @nicholas-fedor in #1795

Dependency Updates

chore(deps): update module github.com/felixge/httpsnoop to v1.1.0 by @renovate[bot] in #1788

chore(deps): update module github.com/prometheus/common to v0.69.0 by @renovate[bot] in #1818

chore(deps): update module github.com/moby/moby/api to v1.55.0 by @renovate[bot] in #1822

chore(deps): update module github.com/moby/moby/client to v0.5.0 by @renovate[bot] in #1825

chore(deps): update module github.com/onsi/ginkgo/v2 to v2.32.0 by @renovate[bot] in #1838

chore(deps): update module github.com/onsi/gomega to v1.42.1 by @renovate[bot] in #1842

chore(deps): update module github.com/pelletier/go-toml/v2 to v2.4.2 by @renovate[bot] in #1844

chore(deps): update module golang.org/x/tools to v0.47.0 by @renovate[bot] in #1853

chore(deps): update module github.com/docker/cli to v29.6.1+incompatible by @renovate[bot] in #1856

chore(deps): update module github.com/prometheus/procfs to v0.21.1 by @renovate[bot] in #1870

New Contributors

@Greite made their first contribution in #1762

Full Changelog: v1.18.1...v1.19.0

v1.16.1 fixes Watchtower's handling of locally-built images and also integrates an upstream fix for Matrix notifications.

Fixes

fix(digest): skip registry requests for locally built images by @nicholas-fedor in #1514

Dependency Updates

chore(deps): update google.golang.org/genproto/googleapis/api digest to f93e5f3 by @renovate[bot] in #1510

chore(deps): update module github.com/nicholas-fedor/shoutrrr to v0.14.3 by @renovate[bot] in #1513

chore(deps): update google.golang.org/genproto/googleapis/rpc digest to 3a24fdc by @renovate[bot] in #1515

Full Changelog: v1.16.0...v1.16.1

Watchtower v1.14.4 integrates a minor bug fix and several dependency updates, including Shoutrrr v0.14.0, which adds both MQTT and Twilio notification support.

Bug Fix

Improve graceful handling of notifications by @nicholas-fedor in #1408

Dependency Updates

chore(deps): update module github.com/docker/cli to v29.3.0+incompatible by @renovate[bot] in #1390

chore(deps): update dependency go to v1.26.1 by @renovate[bot] in #1393

chore(deps): update module github.com/nicholas-fedor/shoutrrr to v0.14.0 by @renovate[bot] in #1407

chore(deps): update module golang.org/x/text to v0.35.0 by @renovate[bot] in #1410

chore(deps): update OpenTelemetry and golang.org/x dependencies by @nicholas-fedor in #1411

Documentation Update

Document regex pattern matching support by @nicholas-fedor in #1404

Full Changelog: v1.14.3...v1.14.4

Feature Updates

feat: add Docker registry mirror support for digest comparison by @nicholas-fedor in #1693

feat: add eligible_at timestamp to cooldown deferral logs and notifications by @nicholas-fedor in #1698

feat(api): add read-only /v1/containers endpoint by @barnabasbusa in #1700

Bug Fixes

fix: prevent image pull when within cooldown window by @nicholas-fedor in #1669

fix(container): default empty device CgroupPermissions to 'rwm' for Podman compatibility by @nicholas-fedor in #1673

fix(flags): interpret bare numeric timeout as seconds by @nicholas-fedor in #1675

fix: use shared identifier matching for implicit restart logic by @nicholas-fedor in #1677

fix(container): set Detach true in ExecStart to prevent blocking on command execution by @nicholas-fedor in #1683

fix(scheduling): strip quotes from cron schedule spec before parsing by @nicholas-fedor in #1686

fix: resolve stale predecessor misidentification during self-update by @nicholas-fedor in #1713

fix(notifications): update Shoutrrr MS Teams integration by @nicholas-fedor in #1718

fix: prevent orphaned old Watchtower containers from persisting by @nicholas-fedor in #1743

Dependency Updates

chore(deps): update module go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp to v0.69.0 by @renovate[bot] in #1688

chore(deps): update module github.com/mattn/go-colorable to v0.1.15 by @renovate[bot] in #1701

chore(deps): update module github.com/prometheus/common to v0.68.0 by @renovate[bot] in #1703

chore(deps): update go module directive to v1.26.4 by @renovate[bot] in #1715

chore(deps): update module github.com/prometheus/common to v0.68.1 by @renovate[bot] in #1726

chore(deps): update module github.com/docker/cli to v29.5.3+incompatible by @renovate[bot] in #1731

chore(deps): update github.com/google/pprof digest to 7023385 by @renovate[bot] in #1733

chore(deps): update module golang.org/x/mod to v0.37.0 by @renovate[bot] in #1749

chore(deps): update module golang.org/x/sync to v0.21.0 by @renovate[bot] in #1750

chore(deps): update module github.com/docker/docker-credential-helpers to v0.9.8 by @renovate[bot] in #1752

chore(deps): update module golang.org/x/sys to v0.46.0 by @renovate[bot] in #1753

chore(deps): update module golang.org/x/term to v0.44.0 by @renovate[bot] in #1756

chore(deps): update module golang.org/x/text to v0.38.0 by @renovate[bot] in #1758

chore(deps): update module github.com/nicholas-fedor/shoutrrr to v0.16.1 by @renovate[bot] in #1765

Deprecations

chore(notifications): deprecate legacy notification types for v2 removal by @nicholas-fedor in #1737

New Contributors

@barnabasbusa made their first contribution in #1700

Full Changelog: v1.17.2...v1.18.0

Release v1.17.1 of Watchtower implements a change from the deprecated github.com/docker/docker dependency to the dedicated github.com/moby/moby/client and github.com/moby/moby/api modules. Review #1613 for more info. This release also fixes a longstanding bug when using the no-restart configuration option to skip new container startup.

Important

#1620 corrects a missing label that will cause Watchtower self-updates to fail, such as when it attempts to self-update from v1.17.0 to v1.17.1 . The old container will remain in a stopped state and no new Watchtower container is created.

One-Time Remediation:

Remove the old, stopped container:

docker rm -f $(docker ps -a -q --filter ancestor=nickfedor/watchtower)

Pull the new Watchtower container image:

docker pull nickfedor/watchtower

Start the new Watchtower container image

Fixes

fix(goreleaser): add missing watchtower label to Docker images by @nicholas-fedor in #1620
fix(actions): restore no-restart flag to only skip container start by @nicholas-fedor in #1626

Refactoring

refactor(deps): migrate from docker/docker to moby/moby split packages for v29 compatibility by @nicholas-fedor in #1613

Dependency Updates

chore(deps): update module github.com/onsi/ginkgo/v2 to v2.29.0 by @renovate[bot] in #1630
chore(deps): update module github.com/onsi/gomega to v1.41.0 by @renovate[bot] in #1631
chore(deps): update module github.com/docker/cli to v29.5.1+incompatible by @renovate[bot] in #1635

Full Changelog: v1.17.0...v1.17.1

Release v1.17.0 of Watchtower introduces a minor addition to the HTTP API that enables early returns from POST requests (#1575). More importantly, this release implements a refactored and updated release pipeline that takes advantage of GoReleaser's latest features.

Important

The latest-dev image tag has been replaced by the nightly image tag.

If you are using the nickfedor/watchtower:latest-dev or ghcr.io/nicholas-fedor/watchtower:latest-dev images, you will need to switch to either the nickfedor/watchtower:nightly or ghcr.io/nicholas-fedor/watchtower:nightly images.

Deployments using the latest image tag are unaffected.

New Features

feat(api): add async query parameter support by @nicholas-fedor in #1609

Dependency Updates

chore(deps): update google.golang.org/genproto/googleapis/api digest to 9d38bb4 by @renovate[bot] in #1516

chore(deps): update google.golang.org/genproto/googleapis/rpc digest to 9d38bb4 by @renovate[bot] in #1517

chore(deps): update module google.golang.org/grpc to v1.80.0 by @renovate[bot] in #1518

chore(deps): update module go.opentelemetry.io/otel/metric to v1.43.0 by @renovate[bot] in #1529

chore(deps): update module go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp to v1.43.0 by @renovate[bot] in #1530

chore(deps): update module go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp to v0.68.0 by @renovate[bot] in #1537

chore(deps): update module golang.org/x/term to v0.42.0 by @renovate[bot] in #1549

chore(deps): update module golang.org/x/text to v0.36.0 by @renovate[bot] in #1550

chore(deps): update module github.com/docker/go-connections to v0.7.0 by @renovate[bot] in #1561

chore(deps): update module github.com/mattn/go-isatty to v0.0.22 by @renovate[bot] in #1578

chore(deps): update module github.com/onsi/ginkgo/v2 to v2.28.3 by @renovate[bot] in #1581

chore(deps): update module github.com/masterminds/semver/v3 to v3.5.0 by @renovate[bot] in #1584

chore(deps): update module github.com/pelletier/go-toml/v2 to v2.3.1 by @renovate[bot] in #1587

chore(deps): update module github.com/fsnotify/fsnotify to v1.10.1 by @renovate[bot] in #1590

chore(deps): update module github.com/docker/cli to v29.4.3+incompatible by @renovate[bot] in #1591

chore(deps): update module github.com/nicholas-fedor/shoutrrr to v0.15.0 by @renovate[bot] in #1592

chore(deps): update github.com/google/pprof digest to 92041b7 by @renovate[bot] in #1593

chore(deps): update go module directive to v1.26.3 by @renovate[bot] in #1597

chore(deps): update module github.com/docker/docker-credential-helpers to v0.9.7 by @renovate[bot] in #1599

chore(deps): update module golang.org/x/sys to v0.44.0 by @renovate[bot] in #1600

chore(deps): update module golang.org/x/mod to v0.36.0 by @renovate[bot] in #1602

chore(deps): update module golang.org/x/net to v0.54.0 by @renovate[bot] in #1603

chore(deps): update module golang.org/x/tools to v0.45.0 by @renovate[bot] in #1605

Full Changelog: v1.16.1...v1.17.0

Overview

Release 1.16.0 introduces new advanced features, a number of bug and security-related fixes, and several dependency updates.

You are highly encouraged to review Watchtower's documentation if you are utilizing advanced features, such as the HTTP API, and/or are interested in using new, experimental features, such as the ephemeral self-updates or image cooldowns.

Important

Watchtower's HTTP API now explicitly requires using POST requests for the /v1/update endpoint and GET requests for the /v1/metrics endpoint.

Rate limiting has also been introduced and can be configured to better fit your use case.

Bug fixes include, but aren't limited to, improvements to error handling, race conditions, retry logic for Docker daemon connection issues, port conflicts, and self-update cleanup. In the event of authentication failures during image pulls, warnings will be propagated to users via notifications. This may potentially result in increased noise in the event of misconfigured registries.

If you're using Watchtower without any advanced features, then you should see a slight performance improvement when Watchtower is checking images and a deduplication of "Found new image" and "Removing image" notification entries.

Note

Given the nature of the new features and number of changes being released, please reference the Discussion's Support section for v1.16.0 support before creating new bug reports.

Features

feat: add ephemeral self-update capability by @nicholas-fedor in #1491

feat: add image cooldown supply-chain defense mechanism by @nicholas-fedor in #1495

feat: add HTTP API rate limiting and security improvements by @nicholas-fedor in #1505

Bug Fixes

fix: handle container removed between check and stop by @nicholas-fedor in #1475

fix: replace fragile string-based error checking with typed errors by @nicholas-fedor in #1476

fix: differentiate auth, not-found, and transient errors during image pull by @nicholas-fedor in #1477

fix(container): validate port bindings before ContainerCreate by @nicholas-fedor in #1478

fix: replace global image inspect mutex with per-image keyed mutex by @nicholas-fedor in #1479

fix: prefer Watchtower-labeled container when multiple share hostname by @nicholas-fedor in #1480

fix: skip self-update when exposed ports would cause conflict by @nicholas-fedor in #1481

fix: make excess Watchtower cleanup non-fatal with increased retry window by @nicholas-fedor in #1482

fix: deduplicate image removal and grouped notification entries by @nicholas-fedor in #1483

fix: retry transient Docker daemon connection failures during container listing by @nicholas-fedor in #1492

fix(api): correct self-update and add skipped container tracking in HTTP API by @nicholas-fedor in #1506

Dependency Updates

chore(deps): update module github.com/docker/cli to v29.3.1+incompatible by @renovate[bot] in #1472

chore(deps): update google.golang.org/genproto/googleapis/api digest to d5a96ad by @renovate[bot] in #1497

chore(deps): update google.golang.org/genproto/googleapis/rpc digest to d5a96ad by @renovate[bot] in #1498

chore(deps): update module github.com/nicholas-fedor/shoutrrr to v0.14.2 by @renovate[bot] in #1504

Full Changelog: v1.15.0...v1.16.0

Overview

Version 1.15.0 of Watchtower features the addition of a global Use Docker Compose Depends-On configuration option that provides users the option to disable Watchtower's use of the Docker Compose depends_on configuration option. This minor release also integrates Shoutrrr v0.14.1, which includes an update to Matrix v3 notification support and a minor fix for Mattermost notification titles.

Feature Addition

Add option to disable Docker Compose depends_on dependency detection by @nicholas-fedor in #1457

Bug Fixes

fix: prevent cleanup of images actively used by active containers by @LJspice in #1427

fix: resolve type mismatch after a minor Shoutrrr API change by @nicholas-fedor in #1463

Dependency Updates

chore(deps): update module golang.org/x/net to v0.52.0 by @renovate[bot] in #1432

chore(deps): update module golang.org/x/tools to v0.43.0 by @renovate[bot] in #1433

chore(deps): update module go.yaml.in/yaml/v2 to v3 by @renovate[bot] in #1439

chore(deps): update google.golang.org/genproto/googleapis/api digest to d00831a by @renovate[bot] in #1445

chore(deps): update google.golang.org/genproto/googleapis/rpc digest to d00831a by @renovate[bot] in #1444

chore(deps): update module go.yaml.in/yaml/v2 to v2.4.4 by @renovate[bot] in #1440

chore(deps): update module github.com/pelletier/go-toml/v2 to v2.3.0 by @renovate[bot] in #1461

chore(deps): update module github.com/nicholas-fedor/shoutrrr to v0.14.1 by @renovate[bot] in #1462

New Contributors

@LJspice made their first contribution in #1427

Full Changelog: v1.14.4...v1.14.5

Overview

This patch release includes several improvements and fixes to improve Watchtower's reliability. Container operations now support proper cancellation and timeouts. Startup failures are now visible as warnings without enabling debug mode. The HTTP API now prevents concurrent update conflicts by returning HTTP 429 when an update is already running.

Note

Containers without proper stop signaling are now expected to take the full 30-second timeout period to stop, unless configured to use a different stop signal or timeout.

Bug Fixes

• Update container startup failures to log at warning level by @nicholas-fedor in #1332
• Fix context propagation to enable proper cancellation and timeout handling by @nicholas-fedor in #1335
• fix(api): return HTTP 429 for full updates when another update is running by @veeceey in #1304

Dependency Updates

• Update indirect Go dependencies by @nicholas-fedor in #1343
• chore(deps): update transitive dependencies by @nicholas-fedor in #1373

Documentation Updates

• Add status code and concurrency behavior information to HTTP API update endpoint documentation by @nicholas-fedor in #1342

New Contributors

• @veeceey made their first contribution in #1304

Full Changelog: v1.14.2...v1.14.3

Overview

This is a patch release that integrates a fix for Watchtower's handling of container dependencies, adds two minor performance improvements, and an update to Go version 1.26.0.

Bug Fixes

• Add service-only matching for cross-project container dependencies by @nicholas-fedor in #1307

Performance Improvements

• Optimize HTTP client creation for registry authentication by @nicholas-fedor in #1287
• Improve digest matching efficiency and reliability by @nicholas-fedor in #1289

Dependency Updates

• chore(deps): update module golang.org/x/text to v0.34.0 by @renovate[bot] in #1303
• chore(deps): update dependency go to v1.26.0 by @renovate[bot] in #1309

Full Changelog

Full Changelog: v1.14.1...v1.14.2

Overview

This is a patch release that introduces several fixes and dependency updates, including improvements to container dependency management and the Compose-based dependency handling that was introduced with version 1.13. The update to Shoutrrr v0.13.2 enables the use of Discord threads, as documented here. Watchtower's multiple notification services documentation was also updated to provide additional clarity.

Bug Fixes

• Resolve prefix matching and circular dependency issues by @nicholas-fedor in #1265
• Prevent spurious safeguard delay when SkipSelfUpdate is enabled by @nicholas-fedor in #1276
• Extract getRawLabelValue to Eliminate Code Duplication and Improve Performance by @nicholas-fedor in #1277
• Fix missing space between fields in notifications by @nicholas-fedor in #1279

Dependency Updates

• chore(deps): update module github.com/onsi/gomega to v1.39.1 by @renovate[bot] in #1261
• chore(deps): update module github.com/onsi/ginkgo/v2 to v2.28.1 by @renovate[bot] in #1262
• chore(deps): update module github.com/docker/cli to v29.2.1+incompatible by @renovate[bot] in #1274
• chore(deps): update module github.com/nicholas-fedor/shoutrrr to v0.13.2 by @renovate[bot] in #1275

Documentation Updates

• Improve multiple notification services documentation by @nicholas-fedor in #1251

Full Changelog: v1.14.0...v1.14.1