github.com/turbot/tailpipe

Grant contents: write to release workflow (#603)

The Tag Release step runs 'git push origin $VERSION' via the default
github-actions[bot] token. Without an explicit permissions block the
token is read-only (org default), causing:

remote: Permission to turbot/tailpipe.git denied to github-actions[bot].
fatal: ... error: 403

Mirrors the permissions block steampipe's release workflow already has.

Dependencies

• Upgrade containerd and crypto packages to remediate high and moderate vulnerabilities.

Bug Fixes

• Build: Restored CentOS/RHEL 9 compatibility by pinning the build image to an older libstdc++/GCC baseline. Previous build linked against newer GLIBCXX symbols, causing Tailpipe to fail on CentOS/RHEL 9.

v0.7.0 [2025-09-22]

Major Changes

• Replace native Parquet conversion with a DuckLake database backend. (#546)

  • DuckLake is DuckDB’s new lakehouse format: data remains in Parquet files, but metadata is efficiently tracked in a
    separate DuckDB database.
  • DuckLake supports function-based partitioning, which allows data to be partitioned by year and month. This enables
    efficient file pruning on tp_timestamp without needing a separate tp_date filter. A tp_date column will still
    be present for compatibility, but it is no longer required for efficient query filtering.
  • Existing data will be automatically migrated the next time Tailpipe runs. Migration does not
    occur if progress output is disabled (--progress=false) or when using machine-readable output (json, line,
    csv).

  Note: For CentOS/RHEL users, the minimum supported version is now CentOS Stream 10 / RHEL 10 due to libstdc++ library compatibility.

• The connect command now returns the path to an initialisation SQL script instead of the database path. (#550)

  • The script sets up DuckDB with required extensions, attaches the Tailpipe database, and defines views with optional
    filters.
  • You can pass the generated script to DuckDB using the --init argument to immediately configure the session. For
    example:

    duckdb --init $(tailpipe connect)

    Note: The minimum supported DuckDB version is 1.4.0.

Bug Fixes

• Include partitions for local plugins in the tailpipe plugin list command. (#538)

Set ExitCodeMigrationUnsupported exit code for migration unsupported (#…

…573)

Revert "update go-duckdb"

This reverts commit 8909e58.

Bug fixes

• Fix issue where --to was not respected for zero granularity data. (#483)
• Fix issue where the relative time passed to from/to args were getting parsed incorrectly. (#485)
• Fix issue where Tailpipe was crashing if the collection state file had nil trunk states from the previous collection. (#489)
• Fix .inspect output to show the plugin name for custom tables. (#360)
• Fix query JSON outputs to be consistent with DuckDB. (#432)

Dependencies

• Upgrade go-viper/mapstructure/v2 and oauth2 packages to remediate high and moderate vulnerabilities.

Bug fixes

• Update core version to v0.2.9 - fix issue where collection state is not being saved for zero granularity collection. (#251)

What's new

• Add --to flag for collect, allowing collection of standalone time ranges. (#238)
• Add --overwrite flag for collect, allowing recollection of existing data. (#454)

Bug fixes

• Fix issue where collection state end-objects are cleared when collection is complete, meaning no further data will be collected for that day. (#250)

Behaviour Change

When passing a from time to a collection, the existing partition data is no longer cleared before the collection starts. This means that data will not by default be recollected for time ranges that have already been collected. To recollect data for a time range, pass the new --overwrite flag to the collect command.

What's new

• Added tp_index property to partition HCL. Use this to specify the source column for the tp_index. (#414)
• Updated collection to apply the configured tp_index, or default if no tp_index is specified in the config.
• Added --reindex arg to compact. When set, compact will reindex the partition using configured tp_index value. (#413)
  • Removed the partition argument from compact and replaced it with a positional argument.
  • Updated compact cleanup to delete empty folders.
• collect now always validates required columns are present. (Previously this was only done for custom tables.) (#411)

What's new

• Enabled support for collecting only today's logs during log collection. (#394)
• Show available table names in autocomplete for DuckDB meta queries in interactive prompt. (#357)
• .inspect meta-command now shows tp_ columns at the end. (#401)

Bug fixes

• Update MinCorePluginVersion to v0.2.5.
• Fix issue where the core plugin was incorrectly throttling the downloads if no temp size limit was specified. (#204)

What's new

• Add support for memory and temp storage limits for CLI and plugins. (#396, #397)
  • memory_max_mb controls CLI memory usage and conversion worker count and memory allocation.
  • plugin_memory_max_mb controls a per-plugin soft memory cap.
  • temp_dir_max_mb limits size of temp data written to disk during a conversion.
  • conversion worker count is now based on memory limit, if set.
  • JSONL to Parquet conversion is now executed in multiple passes, limiting the number of distinct partition keys per conversion.
• Detect and report when a plugin crashes. (#341)
• Update show source output to include source properties. (#388)

Bug fixes

• Fix issue where tailpipe was mentioning steampipe in one of the error messages. (#389)

What's new

• Update MinCorePluginVersion to v0.2.2.
• Update tailpipe-plugin-sdk to v0.4.0.

Bug fixes

• Fix source file error for custom tables when using S3 or other external sources. (#188)

Bug fixes

• Fix partition filter argument. (#375)

What's new

• Add support for custom tables. ((#225))
• Add location to format list/show. (#283)
• Add plugin to source list/show. (#337)
• Improve display logic of conversion errors, since we log full error just display first line of error.

Bug fixes

• Plugin list command now correctly cases keys in JSON output
• Add support to querydisplay.ColumnValueAsString for UUID/Decimal in format received from DuckDB
• Add comma separators to numeric output in query results. (#685)
• Fix multiple backtick escaping - if more than 2 properties are back-ticked, the third onwards not being escaped.

Bug fixes

• Table introspection in json now displays descriptions correctly. (#323):
• Fix resolution of the format before calling Collect when the format plugin is different from the table plugin.(#319):

What's new

• Add support for plugins with custom formats
  A format block can be defined in config and plugins can provide formats types and presets . Format are supported by the new Nginx and Apache plugins.
• Add format list and format show commands. (#235)
• Update the plugin show command to add exported formats and correctly display partitions, etc. (#257)
• Improve the error reporting in the collection UI (#247).
• Update all SQL in code and messaging to be lower case.