⚡ Highlights

🧬 Replicated bytea values are no longer corrupted on the way to a Postgres target
🧩 Partitioned tables restore with valid indexes and primary keys, so they keep replicating after a snapshot
🔐 Binaries are built with Go 1.26.6, patching stdlib and module CVEs
Dependency updates

What's Changed

Bug fixes

• Decode bytea hex from wal2json before writing to postgres by @siriusfrk in #1090
• Restore ALTER INDEX ... ATTACH PARTITION with indices and constraints by @siriusfrk in #1094

Dependency updates

• Bump Go to 1.26.6 and patch stdlib / module CVEs by @walidelbiir in #1107
• Bump go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc from 1.44.0 to 1.45.0 by @dependabot[bot] in #1097
• Bump go.opentelemetry.io/otel/exporters/prometheus from 0.66.0 to 0.67.0 by @dependabot[bot] in #1099
• Bump testcontainers-go postgres, opensearch and elasticsearch modules from 0.43.0 to 0.44.0 by @dependabot[bot] in #1098, #1100, #1096
• Bump clickhouse/clickhouse-server from 26.7.2.59-alpine to 26.7.3.19-alpine by @dependabot[bot] in #1095

New Contributors

• @siriusfreak made their first contribution in #1090

Full Changelog: v1.4.0...v1.4.1

v1.4.1 Latest

Latest

Compare

Choose a tag to compare

⚡ Highlights

📊 Native Prometheus scrape endpoint
🎯 Granular object type filtering for pg-to-pg schema snapshots and DDL replication
🛡️ Transformation rules are now validated against unique indexes before a run starts, instead of failing mid-load
🔌 Target connection pool limits are configurable
🪝 Webhook delivery retries transient failures and no longer silently drops still-failing events; adds idempotency headers
🐛 Fixed a DDL parsing bug that silently dropped RENAME COLUMN / DROP COLUMN schema changes
🛂 New preflight check for CREATEROLE when a snapshot restores roles
Dependency updates

What's Changed

New features

• Add prometheus endpoint by @pauliyobo in #1008
• Granular object type filtering for pg to pg snapshot and streaming by @tsg in #735
• Add target CREATEROLE preflight check for roles snapshotting by @lghuy05 in #1055
• Make target connection limits configurable by @subotac in #1048
• Add a uniqueness classification to transformers by @kvch in #1082
• Classify the greenmask and neosync transformers for uniqueness by @kvch in #1084, #1085
• Validate transformation rules against unique indexes by @kvch in #1086

Bug fixes

• Reject masking configurations that mask no characters by @kvch in #1083
• Fix retry idempotency for webhook target by @kvch in #1080
• Stop the statement terminator being captured as part of a column name by @kvch in #1078

Documentation

• Document transformer uniqueness and the unique index check by @kvch in #1088
• Document the destructive --reset snapshot behaviour by @kvch in #1059

Dependency updates

• Bump actions/attest from 4.2.0 to 4.2.2 by @dependabot[bot] in #1065
• Bump go.opentelemetry.io/otel/sdk/metric from 1.44.0 to 1.45.0 by @dependabot[bot] in #1064
• Bump github.com/pgvector/pgvector-go/pgx from 0.4.0 to 0.4.1 by @dependabot[bot] in #1066
• Bump docker/login-action from 4.5.2 to 4.6.0 by @dependabot[bot] in #1061
• Bump clickhouse/clickhouse-server, opensearch and opensearch-dashboards build images by @dependabot[bot] in #1060, #1062, #1063

Details

Prometheus scrape endpoint

The setting instrumentation.metrics.prometheus.enabled exposes a /metrics endpoint on the existing health server. Metrics can now be scraped directly, with no collector in between:

instrumentation:
  metrics:
    prometheus:
      enabled: true # Defaults to false
      endpoint: "/metrics" # Defaults to /metrics

The endpoint returns 404 when disabled.

Object type filtering

Both the schema snapshot and DDL replication for pg-to-pg pipelines can now be restricted to a subset of object categories (tables, sequences, types, indexes, functions, views, triggers, etc.) via include_object_types (allowlist) or exclude_object_types (denylist), independently for snapshot and replication. A DDL statement that touches several object types (e.g. CREATE TABLE with a primary key) is only skipped when every object it touches is excluded.

Transformation rules are validated against unique indexes.

Anonymization is lossy by design: masking with type: id keeps a 6-character prefix, so every value sharing that prefix collapses to the same masked value. Applied to a column covered by a unique index, this previously surfaced as duplicate key value violates unique constraint hours into a data load. Every transformer is now classified as preserved (distinct inputs stay distinct), not_guaranteed (random or hashed output, collisions possible) or lossy (collides by construction).

validate rules and pipeline startup now read pg_index for the tables in scope and check the configured rules against unique indexes, primary keys and unique constraints. With a Postgres target, a lossy transformer on a covered column fails at parse time with transformation rules break a unique index; with other targets (Kafka, Elasticsearch/OpenSearch, webhooks) the same finding is a warning, since there is no unique index to violate. A not_guaranteed transformer always warns. Set allow_uniqueness_loss: true on a column rule to keep a lossy transformer anyway. A custom mask configured to mask zero characters (mask_begin equal to mask_end, or unmask_begin: 0) is now rejected at construction rather than silently passing values through unmasked. Use noop to pass a column through on purpose.

Configure target side connection count

target.postgres.max_connections (PGSTREAM_POSTGRES_WRITER_MAX_CONNECTIONS) now controls the writer pool size, defaulting to 50 and honoring pool_max_conns from the target URL when set explicitly. Previously this setting also sized the schema observer's pool, so a target capped at 90 connections to stay under a 100-connection server limit could still open up to 180. The schema observer is now sized separately, at min(writer pool, 16). So the process now opens at most the configured writer limit plus 16.

Webhook delivery

A failed webhook POST is now retried with configurable backoff (network errors, 429, 5xx); other non-2xx responses are treated as permanent and not retried. A delivery that is still failing after retries is logged and dropped so one broken subscriber can't block delivery to everyone else, and at-least-once semantics apply on restart. New X-Pgstream-LSN / Idempotency-Key headers let subscribers dedupe redelivered events (omitted for snapshot events, which share a zero LSN). DISABLE_RETRIES now actually disables retries.

New Contributors

• @pauliyobo made their first contribution in #1008
• @subotac made their first contribution in #1048

Full Changelog: v1.3.1...v1.4.0

v1.4.0

Compare

Choose a tag to compare

⚡ Highlights

📌 Data snapshots read the columns the schema snapshot captured, so a column added to the source mid-snapshot no longer breaks the load
⚠️ Schema drift during a snapshot is now reported: a warning when it happens during the dump, an error when a table's captured columns are gone
🛂 New preflight check for CREATEDB on the target when the snapshot creates the target database
🔐 Documented how DDL replication can carry privileges from a less trusted source to the target
Dependency updates

What's Changed

New features

• Add target CREATEDB preflight check for snapshot DB creation by @lghuy05 in #996

Bug fixes

• Pin the data snapshot to the columns the schema snapshot saw by @kvch in #1058
• Add catalog helpers for reading a table's column list by @kvch in #1057

Documentation

• Document potential privileges escalation between source and target via DDL replication by @tsg in #1052

Dependency updates

• Bump docker/login-action from 4 to 4.5.2 by @dependabot[bot] in #1054
• Bump actions/attest from 4 to 4.2.0 by @dependabot[bot] in #1053

Full Changelog: v1.3.0...v1.3.1

v1.3.1

Compare

Choose a tag to compare

Changelog

• 474ff1c Added tests and docs for snapshot+stream from a read replica (#1049)
• c06f9bd Fix encoding of empty strings as null bytes (#1051)

What's Changed

• Added tests and docs for snapshot+stream from a read replica by @tsg in #1049
• Fix encoding of empty strings as null bytes by @tsg in #1051

Full Changelog: v1.2.5...v1.3.0

v1.3.0

Compare

Choose a tag to compare

⚡ Highlights

🎲 Template and JSON transformers no longer corrupt their random generator under multi-worker snapshots
📉 New metrics and totals for what ignore_send_errors silently discards, plus a startup warning when suppression is on
🧾 Snapshot failures are logged as errors instead of rendering as {}
🔁 Snapshot request status updates are retried, and report data loss when they still cannot be persisted
🔍 A failed restore now carries the output that explains it, with credentials redacted

What's Changed

Bug fixes

• Pool the templates that share greenmask's funcmap state by @kvch in #1045
• Add a counter for batches dropped by ignore_send_errors by @kvch in #1042
• Log snapshot failures as errors instead of rendering them as {} by @kvch in #1039
• Retry snapshot recorder requests, emit errors by @kvch in #1041
• Add more details about failed statements by @kvch in #1040

CI / build

• Catch both outcomes of the enum type creation race in integration tests by @kvch in #1044
• Update golangci-lint in Makefile to match CI version by @kvch in #1046

Upgrade notes

No configuration changes are required.

Contributors. make lint now installs golangci-lint v2.10.1 to match CI, which reports findings the previously pinned v2.5.0 did not.

Full Changelog: v1.2.4...v1.2.5

v1.2.5

Compare

Choose a tag to compare

⚡ Highlights

🔑 Replicated DELETEs on tables with a user-defined enum primary key no longer fail
🧬 Enum arrays and domains over enums are now handled in delete identities, not just plain enums
🛟 A failed query can no longer take its whole batch down with it, including writes to unrelated tables

What's Changed

Bug fixes

• Fix replicated DELETEs on user-defined enum PKs by @kvch in #1038

Upgrade notes

No configuration changes are required, and the SQL emitted for tables without enum identity columns is unchanged.

Full Changelog: v1.2.3...v1.2.4

v1.2.4

Compare

Choose a tag to compare

⚡ Highlights

🎨 Bulk ingest no longer fails on tables with user-defined enum columns
🔑 Kafka primary_key partitioning no longer collides distinct rows onto one message key
🩺 Two preflight checks that reported false failures on correctly-configured sources (wal2json, replica identity) are fixed
🔍 Preflight can now run source-only checks (--source) and verifies major PostgreSQL version compatibility
Dependency updates

What's Changed

New features

• [Preflight] Option to run only source side checks by @kvch in #1025
• [Preflight] Check major PostgreSQL version by @kvch in #1026

Bug fixes

• Copy user defined enum columns by @kvch in #1035
• Escape keys when using primary key partitioning by @kvch in #1036
• Fix two false-result preflight checks (wal2json, replica_identity) by @kvch in #1034
• Update PostgreSQL version check for source size support by @kvch in #1027

Dependency updates

• Bump actions/github-script from 7 to 9 by @dependabot[bot] in #1030
• Bump actions/download-artifact from 7 to 8 by @dependabot[bot] in #1029
• Bump clickhouse/clickhouse-server from 26.6.1.1193-alpine to 26.7.1.1315-alpine in /build/docker by @dependabot[bot] in #1028

Full Changelog: v1.2.2...v1.2.3

v1.2.3

Compare

Choose a tag to compare

⚡ Highlights

🗜 Configurable session settings for the snapshot index/constraint restore phase (via PGOPTIONS)
🛡 Index constraint settings validated at startup
🩺 Preflight now warns when a snapshot source looks load-balanced (Aurora/RDS reader, instance-spanning pooler)
💥 Fixed a stack-depth overflow on bulk deletes of tables with composite primary keys
🛑 pg_restore now honors context cancellation, so shutdowns and timeouts no longer hang

What's Changed

New features

• Add index restore session settings by @jonasHanhan in #990

Bug fixes

• Fix stack-depth overflow on bulk composite PK deletes by @kvch in #1024
• Honor context cancellation in pg_restore by @kvch in #1023
• [Preflight] Detect load-balanced snapshot sources by @kvch in #1015

New Contributors

• @jonasHanhan made their first contribution in #990

Full Changelog: v1.2.1...v1.2.2

v1.2.2

Compare

Choose a tag to compare

⚡ Highlights

📊 Pipeline phase visibility via /status and a new OTel gauge
🧬 Preflight checks no longer reports custom enums as unsupported types
🗜 Snapshot request unique index hashes table_names: fixes btree overflow with many tables
🐛 Bug fixes (deferred parsing of validated schema objects, comment restore downgraded to warning)
Dependency updates

What's Changed

New features

• Add pipeline phase visibility via /status and OTel gauge by @Shaurya2k06 in #1000

Bug fixes

• Hash table_names in snapshot request unique index to avoid btree overflow by @kvch in #1013
• Do not report custom enums as unsupported types by @kvch in #1005
• Defer parsing of validated schema objects by @tsg in #1003
• Downgrade error on restoring comment to warning by @kvch in #1009

Dependency updates

• Bump actions/setup-go from 6 to 7 by @dependabot[bot] in #1007

New Contributors

• @Shaurya2k06 made their first contribution in #1000

Full Changelog: v1.2.0...v1.2.1

v1.2.1

Compare

Choose a tag to compare