Rudder Server Updates & Release Notes

Features

• jobsdb: pending events gauge for the gw jobsdb (#7447) (fc539c4)
• processor: report event_filter drops per reason and stop its succeeded rows (#7454) (5caff53)

Bug Fixes

• destination: authenticate marketo bulk uploads with bearer token (#7510) (3eaffad)
• processor: exclude dropped jobs from tracked users and activation records (#7485) (526602c)
• router: enable live events payload masking by default (#7500) (8b5c22d)
• router: mask multi-request delivery payloads (#7476) (f65572e)
• router: mask reporting sample event secrets (#7497) (833d694)
• router: report destination status for v1 proxy deliveries in transformer_outgoing_request_count (#7482) (bba1c21)

Miscellaneous

• bump sqlconnect-go to v1.34.0 (#7479) (71e8cfa)
• stop setting in_reported_by on report rows (#7458) (814d149)
• sync release v1.90.0 to main branch (#7475) (d05fcbb)
• tooling: move rudder-server test MinIO to Silo (#7469) (935bf59)

Features

• processor: add source_out reporting metric (#7435) (909fba9)
• processor: allow disabling event schemas v2 per source (#7429) (a8c12b2)
• processor: report dest_transformer diff rows for multiplexing destination transformations (#7446) (cd95c32)
• processor: report dest_transformer succeeded for destinations without a destination transformation (#7445) (21f7e41)
• processor: report user_transformer succeeded for destinations without a user transformation (#7440) (f424257)
• router: mask live events destination credentials (#7460) (5fa5fe2)
• warehouse: add Microsoft Fabric destination (#7441) (74940c2)

Bug Fixes

• batchrouter: do not abort warehouse jobs on a zero failure timestamp (#7426) (506f6d5)
• batchrouter: recover snowpipe streaming channels for tables dropped externally (#7434) (93cd81d)
• destination: encode Lytics bulk upload URL values (#7433) (08e6937)
• destination: retry bqstream transport failures (#7464) (cd3dc3e)
• warehouse: cap deltalake validation schema describes (#7444) (d119779)
• warehouse: keep snowflake copy credentials out of logs (#7442) (f91747b)
• warehouse: set explicit lineSep for deltalake csv copy (#7455) (cc22d18)

Miscellaneous

• migrate self-hosted arm64 docker build runners to gha-runner-scale-set (#7425) (5cdffde)
• processor: retire the destination-filter reorder, dedup and gw_ingested reporting flags (#7450) (1040511)
• pull openapi-generator through the ECR pull-through cache (#7449) (6e9f920)

Features

• warehouse: add shared SQL identifier and string literal quoting helpers (#7406) (814f6db)

Bug Fixes

• warehouse: quote azure synapse identifiers (#7363) (d399fd1)
• warehouse: quote bigquery identifiers (#7366) (89de526)
• warehouse: quote clickhouse identifiers (#7420) (490c097)
• warehouse: quote deltalake identifiers (#7368) (08f2f1e)
• warehouse: quote mssql identifiers (#7364) (695b881)
• warehouse: quote postgres load column list and bind schema existence check (#7405) (2ff81c1)
• warehouse: quote redshift identifiers (#7365) (a3e2a3c)
• warehouse: quote snowflake identifiers (#7367) (53d4891)

Miscellaneous

• sync release v1.88.0 to main branch (#7415) (130bdd3)
• warehouse: drop the per dialect identifier quoting wrappers (#7421) (ec0cf52)
• warehouse: drop the v2 suffix now v1 is gone (#7418) (b759c33)
• warehouse: move back to upstream clickhouse-go (#7419) (1a7e776)
• warehouse: remove the ClickHouse v1 driver (#7417) (bd14179)
• warehouse: replace dead clickhouse blockSize with commitEvery (#7427) (0806478)

Features

• warehouse: bigquery workload identity federation authentication (#7361) (d16ab5f)

Bug Fixes

• bound clickHouse v2 s3 copy memory (#7407) (a17a78f)

Miscellaneous

• cleanup python mirroring (#7395) (9f9bcde)
• default failed-keys error capture on (#7404) (db84987)
• sync release v1.87.0 to main branch (#7394) (80353d1)
• tag namespace config http client stats with version and mode (#7401) (d152ae5)

Features

• backend config shadow mode (#7340) (4898165)
• bump rudder-go-kit and go modules to Go 1.27.1 (#7345) (8d95c68)
• router: read transformerProxy from transformer features before env (#7337) (e8f0ea6)

Bug Fixes

• batchrouter: retry failed async manager initialization (#7374) (974d383)
• router: redact credentials from destination URLs in failure responses (#7372) (14659bf)
• router: request fails with unexpected use of stub RoundTripper error (#7386) (2ff491f)
• singer-protocol sources filtered destination config for source type cloud instead of cloudSource (#7389) (f48c997)
• warehouse: instrument the clickhouse v2 load path (#7355) (4d743db)
• warehouse: pass temporary credentials to the clickhouse v2 s3 copy engine (#7353) (4d743db)
• warehouse: retire idle clickhouse v2 connections (#7352) (4d743db)
• warehouse: retry a failed clickhouse v2 block (#7354) (4d743db)

Miscellaneous

• log the candidate error when a backend config shadow sample fails to fetch (#7383) (eaa9b61)
• processor: assemble side-PU report metrics at end of preprocess stage (#7346) (4540876)
• sync release v1.86.0 to main branch (#7356) (6868c6c)

Bug Fixes

• warehouse: instrument the clickhouse v2 load path (#7355) (6456c64)
• warehouse: pass temporary credentials to the clickhouse v2 s3 copy engine (#7353) (c835fc0)
• warehouse: retire idle clickhouse v2 connections (#7352) (3a1407c)
• warehouse: retry a failed clickhouse v2 block (#7354) (72390f6)

Features

• backend config v2-to-v1 mapper (#7315) (887f6d2)
• core: capture the final recorded error for failed rETL records (#7295) (f4dc4d9)
• per-destination visibility at the destination-filter boundary (#7283) (3d83f02)
• user_suppression counter via suppressed dummy jobs (#7325) (77d3d37)
• warehouse: support jsonPaths for gcs datalake (#7333) (e714140)

Bug Fixes

• rsources failed-records pagination drops records across failed-key boundaries (#7329) (0a50439)
• warehouse: stop logging full schemas on table-level schema mismatch (#7339) (f244c14)
• warehouse: upgrade clickhouse v2 driver (#7332) (e2c5cb9)

Miscellaneous

• ci: remove unused Snyk policy file (SEC-409) (#7317) (d60b3c6)
• make reporting 413 handling unconditional (#7313) (d2474e9)
• moving pyt contract tests (#7327) (8316af2)
• processor: pipeline attribution in processor stats (#7326) (3e97f7c)
• skip consent and connection-mode filtering for replays of non-existent connections (#7323) (d2474e9)

Features

• backend config v2 resolver with stub mapper (#7275) (267b973)
• warehouse: clickhouse v2 connection and schema (#7301) (2eb1fa6)
• warehouse: clickhouse v2 ddl operations (#7302) (f2887c5)
• warehouse: clickhouse v2 loading (#7303) (3e48618)
• warehouse: enable the clickhouse v2 driver behind a flag (#7311) (3ce1da5)

Bug Fixes

• race conditions while counting initial pending events can cause wrong counting (#7292) (3efa718)
• replay destinations not using the config corresponding to their connected source type (#7307) (ee7aa93)
• router: abort router-transform on proactive invalid_grant instead of retrying (INT-6779) (#7175) (4d7348d)
• tracking plan merged config is cached without its event type (#7314) (624dd5c)

Miscellaneous

• setattr/delattr hardening (#7291) (2855af4)
• sync release v1.84.0 to main branch (#7306) (fa102b7)
• webhook reasons (#7294) (c298d59)

Features

• add AGENTS.md for agent management and reference (#7290) (2d28b32)
• processor: report duplicate events as dedup/filtered metrics (#7280) (f2655fb)
• warehouse: add clickhouse driver interface (#7296) (239d9a4)

Bug Fixes

• processor: proc destination hydration omitted destination's original id (#7297) (f389df4)
• warehouse: close load file descriptor in identity uploadFile (#7186) (bfd8bc9)

Miscellaneous

• handle oversized reporting payloads (#7293) (7a83281)
• pyt security hardening (#7272) (e13b731)

Features

• add OriginalID and OriginalDestinationID to support replay event transformations (#7274) (28d38e9)
• bingads integration tests (#7253) (fdb8c0a)
• processor: fork event processing for rsources pipelines (#7235) (707ea06)

Bug Fixes

• pyt contract tests data race (#7247) (9e62d0c)

Miscellaneous

• bump sqlconnect-go to v1.30.2 (#7251) (f5d6175)
• jobsdb: multi-consumer compaction improvements (#7248) (b4af86c)
• pyt contract tests for candidate releases (#7252) (149e91a)
• pyt contract tests for transformations auth (#7239) (ac4a859)
• router: support per-partition noOfWorkers override (#7270) (9d6f7cf)
• sync release v1.82.0 to main branch (#7243) (a6e29f2)

Features

• processor: fork event processing for configured isolated destinations (#7207) (6eb91b4)
• processor: isolated destination processing pipeline (#7202) (a684641)
• router: detect and classify transformer-proxy contract breaches (INT-6839) (#7184) (c7a15a8)
• router: gate 296 delivered-with-warning behind per-workspace rollout (#7217) (1d772a1)

Bug Fixes

• apply configured client QPS and Burst settings in restConfig (#7220) (762bc06)
• gcs manager allows non-service account credentials and warehouse ssl keys can escape their containing path (#7227) (3e126b2)
• mid-run pod deaths under concurrent test runs (#7212) (5279b64)
• normalize null BingAds offline conversion fields (#7236) (678f130)
• use correct JSON tag for adjustedConversionTime in bing-ads (#7204) (50df0f7)

Miscellaneous

• add new readme logo (#7200) (ab86afd)
• deps: bump the go-deps group across 1 directory with 32 updates (#7199) (0eaa833)
• jobsdb: graceful shutdown of addNewDSLoop and refreshDSListLoop (#7210) (78e104c)
• remove archiveInPreProcess processor toggle (#7205) (f471b1d)