github.com/openclaw/discrawl

0.15.5 - 2026-09-20

• Add explicit public Discord invite-count collection, scoped history import, and read-only status in a separate metrics database, preserving partial observations and existing archives. Thanks @hannesrudolph.
• Keep empty DM date-window hints consistent with the suggested follow-up command, excluding uncatalogued conversations only when removing the final date filter. Thanks @rnavarro.
• Refresh SQLite to v1.59.0 with its matching libc v1.75.7, go-humanize to v1.1.0, and terminfo to v1.2.0; retain the Go 1.27.0 minimum and current published Crawlkit v0.16.4.
• Update the optional Korean helper's native Kiwi library and model to 0.24.0, preserving the separate helper process and documented lexical-index rebuild procedure.
• Refresh CodeQL, GoReleaser, and pinned Go container images; align the Docker source smoke with Go 1.27.1.

0.15.4 - 2026-09-20

• Update Crawlkit to v0.16.4 so normal snapshot updates accept equivalent attachment schemas with reordered columns, preserving locally retained history without requiring forced replacement.

0.15.3 - 2026-09-19

• Hydrate Desktop-imported channels and their ancestors during targeted bot syncs so publication can use authoritative types, parents, and permissions without resetting stored history or weakening privacy filters.

0.15.2 - 2026-09-19

• Exclude private-thread crawl cursors from public-only snapshots while preserving public-thread cursors, full backups, and local sync state.
• Fix large message listings and TUI reply/mention hydration failing with SQLite's parameter limit, preserving all requested rows and display names.
• Add opt-in D1/R2 publication to the existing backup workflow, using one filtered export with resumable uploads and guarded archive adoption. Thanks @obviyus.
• Recover missing source members and messages through manual Discord reconciliation before adopting an existing hosted archive, preserving permission checks and existing source records. Thanks @obviyus.
• Update Crawlkit to v0.16.3, retaining the existing Go minimum and SQLite runtime pairing.

0.15.1 - 2026-09-13

• Explain empty search, messages, and dms results with stderr diagnostics for restrictive filters and missing local data, while preserving stdout, JSON output, and exit codes. Thanks @rnavarro.
• Avoid repeated requests to unavailable Discord channels for seven days during routine sync, while full and targeted syncs retry immediately, unavailable-only backlogs keep discovering channels, and doctor reports marker health using the same timestamp policy. Thanks @rnavarro.
• Preserve cancellation for interrupted SQLite work and closed transactions, while retaining the underlying diagnostic.
• Keep the generated documentation identity stable when building from a renamed checkout.
• Update CrawlKit to v0.16.2 and the optional Chinese tokenizer to gse v1.1.0 with cedar v0.50.0, retaining the Go minimum and SQLite runtime pairing.

0.15.0 - 2026-09-11

Highlights: Keep embeddings current during continuous capture, recover restart gaps on demand, and inspect live archive freshness.

• Add opt-in tail --embed-live to keep embeddings current during Discord capture, with coordinated queue ownership, provider batch and timeout settings, and background-worker health in status output. Thanks @hannesrudolph.
• Add opt-in tail --repair-on-start to recover restart gaps immediately after Gateway connection, using the existing serialized repair and writer ownership. Thanks @hannesrudolph.
• Exclude nested channels and threads beneath excluded categories during catalog discovery and live capture. Thanks @hannesrudolph.
• Include the last captured Gateway event time as optional last_tail_event_at in local status --json, separately from the last completed sync. Thanks @hannesrudolph.
• Speed per-channel changed-message queries with an additive update-cursor index; document synchronous first-writable-open costs and backup-first evaluation before broader rollout. Thanks @hannesrudolph and @vincentkoc.
• Allow explicit OS keyring credentials for embeddings, with safe initialization retries during capture and unchanged environment-based defaults. Thanks @hannesrudolph.
• Generate deterministic, aggregate-only Discord field notes alongside daily published activity reports, with paired Markdown/JSON artifacts, separate markers preserving legacy narrative notes, explicit timestamp and coverage limits, and filtered-publication cleanup. Thanks @vincentkoc.
• Use CrawlKit v0.16.1 to bound live-embedding shutdown when cancellation interrupts retry or completion persistence and cleanup storage is unavailable. Thanks @hannesrudolph.
• Refresh terminal-width, Go system, text and cryptography dependencies, Go analysis tools, and CodeQL while retaining the existing Go minimum and SQLite runtime pairing.

0.14.1 - 2026-09-09

• Accept the application's legacy migrated attachment column order in the backup-first repair helper while retaining exact schema and repair safeguards.

• Add a manually gated, backup-first maintenance workflow for narrowly scoped attachment text repair, with strict snapshot validation before installing a new Actions database cache.

• Keep extracted attachment text within its byte limit without splitting valid UTF-8 characters, and identify invalid admitted attachment columns in contents-free export errors without changing stored data or weakening snapshot validation.

• Limit disposable Discord backup clones to current main and required descendants, without fetching tagged history.

• Provide checked-out source and workflow run identity to Discord archive publications.

• Update CrawlKit to v0.15.0 while retaining the existing Go minimum and preferred toolchain.

• Preserve stable published shard paths for released subscribers, roll back handled pre-manifest installation failures, and report post-manifest cleanup failures without committing or pushing a partial result.

• Add optional manifest-bound producer.json receipts for explicitly identified publishers, preserving unrelated files and checking the binding before publication push retries. Receipts are unsigned publisher assertions, not authenticated provenance.

• Keep publication Git arguments bounded with literal NUL-delimited paths and one path-only commit, preserving unrelated staged work.

• Report incomplete selected channel or category permissions in publication preflight, including inherited thread permissions, without changing public-only export selection.

• Accept exact manifest-owned table generation paths alongside legacy snapshot shards without broadening publication cleanup or Git staging.

• Remove previously published embedding and media files when omitted, and commit only exact archive outputs and explicitly generated reports while preserving unrelated staged work.

• Exclude deleted messages from cloud row publication and SQLite bundles, using the existing replacement protocol and matching live-row counts.

• Preserve cached channel permissions during targeted sync and exclude unobserved permissions from public-only snapshots.

• Exclude nested unselected guild metadata from filtered snapshots while retaining local raw observations.

• Keep local Wiretap DMs out of embedding enqueue, rebuild and provider requests, including jobs queued by older versions.

• Keep inferred Git fingerprints separate from snapshot integrity metadata when importing legacy snapshots, including historical restores, while preserving incremental merge checkpoints.

• Exclude local-only direct messages from published README activity reports and add report --published for standalone shared reports, while preserving full local reports and filtered-publication safeguards.

• Remove prior manifest-declared embedding files when export is disabled or its model changes, preserving undeclared embedding/media siblings, unrelated staged work, and local vectors.

• Validate imported embedding rows against their manifest identity and vector dimensions; reject malformed or non-finite vectors transactionally and skip missing, deleted, and local-only message targets while preserving existing DM vectors.

0.14.0 - 2026-09-07

Highlights: Search Korean, Japanese, Chinese, and Arabic word forms with optional local language analyzers.

• Add opt-in multilingual lexical indexes with independent relevance ranking, safe analyzer-change rebuilds, and discrawl lexical rebuild; keep the default binary free of helper dictionaries and preserve existing search behavior when languages are disabled. Thanks @vkehfdl1.
• Reject unsafe snapshot shard paths with CrawlKit v0.14.9 while preserving the SQLite-required libc runtime.

0.13.4 - 2026-09-05

Highlights: Recover missing channel history and cached messages, and keep routine sync discovering new channels and archived threads.

• Verify supposedly complete channels with no stored messages, recover their full history, and preserve retryability across bounded cancellation cleanup or process interruption. Thanks @rnavarro.
• Recover unresolved Desktop cache entries across upgrades and input modes without overwriting newer edits, duplicating events, or rescanning completed full-cache files. Thanks @Nicolas0315.
• Discover newly archived threads during incremental sync with per-endpoint cursors and resume incomplete thread histories without rescanning the full archive. Thanks @obviyus.
• Bootstrap newly discovered channels with one newest message page during routine latest-only sync, preserving a backfill cursor for later full-history recovery. Thanks @obviyus.
• Keep Wiretap dry runs from creating archives or runtime directories; optional coverage reads an existing archive without migrations.
• Stop guild and archived-thread pagination with a cursor error when Discord repeats a page instead of hanging sync. Thanks @SebTardif.
• Reject repeated or missing message-page cursors without losing the last usable backfill checkpoint. Thanks @SebTardif.
• Bound release-check HTTP requests to 30 seconds with Crawlkit v0.14.8 so an unresponsive server cannot hang update checks indefinitely.
• Require Go 1.27.0 and prefer Go 1.27.1 for builds; refresh SQLite to v1.58.0, crypto and terminal dependencies, and container, analyzer, security-scan, and docs-build tooling.

v0.13.3 - 2026-08-17

Fixes

• Add persistent exact snapshot updates for dedicated Git-only readers so rows omitted by a newer filtered snapshot are removed without changing the merge-by-default behavior of richer local archives.
• Skip Discord members with a missing user object during pagination and record conversion so public sync no longer panics on a nil User. Thanks @SebTardif.

v0.13.2 - 2026-08-14

Maintenance

• Require Go 1.26.6 and refresh the pinned Go container images for the latest standard-library security fixes.

0.13.1 - 2026-08-09

Highlight: tail --verbose now tells you why a message was not archived.

Fixes

• Trace live Discord message receipt, handling, scope filtering, and archive completion in verbose tail logs. Each event reports gateway_received, handler_started, ignored with a reason, or archived, so an out-of-scope message is no longer indistinguishable from one the Gateway never delivered. archived is emitted only after the canonical write, event append and cursor update all succeed. Traces carry event type, stage and IDs only — never message content or author metadata.

Maintenance

• Update Crawlkit to v0.14.6, Kong to v1.16.1, the modernc SQLite stack to v1.56.0, and refresh Go analysis tooling.

0.13.0 - 2026-08-03

Changes

• Surface stored message-to-channel catalog integrity in diagnostics, annotate output whose channel metadata is missing, and warn when zero-row SQL results may hide orphaned messages. Thanks @htydev.

Maintenance

• Update Crawlkit to v0.14.5 and go-colorful to v1.4.1.

Documentation

• Rewrite the README as a concise front door to installation, archive sources, search, and the existing documentation site.

0.12.0 - 2026-08-02

Changes

• Queue live tail, failure replay, and repair messages for embedding behind explicit tail --with-embeddings opt-in, and support OpenAI projection dimensions (#159, based on #154 — thanks @hannesrudolph).

Maintenance

• Update Crawlkit to v0.14.4, Alpine to 3.24, and refresh the SQLite, libc, terminal width, and terminal detection dependency stack.
• Update CodeQL Action to v4.37.4 and actions/stale to v11 while preserving the existing security-scan and stale-item policies.
• Let the unified release workflow own version tags and organization-managed signing, notarization, publication, and Homebrew delivery.

0.11.10 - 2026-07-27

Maintenance

• Move official releases to the shared signed, notarized, independently verified GitHub Actions pipeline while preserving archive names, contents, checksums, and Homebrew delivery.

0.11.9 - 2026-07-26

Changes

• Scope Discord history and live tail collection with category allowlists and channel id/kind exclusions, capture new and updated threads immediately, and support wall-clock repair offsets. Thanks @hannesrudolph.
• Resolve requested channels directly before archived-thread catalog discovery so targeted sync handles multi-guild ownership with one lookup per channel and rejects DMs. Thanks @robinbraemer.

Full changelog: v0.11.8...v0.11.9

0.11.8 - 2026-07-20

Highlights

• Preserve member and guild history across syncs and Git-share merges with source-attributed tombstones, explicit restore semantics, and revision-aware reconciliation.
• Keep large archives responsive and attachment retrieval secure with bounded coverage aggregation and strict Discord CDN redirect validation.

Fixes

• Prevent global coverage reports from timing out on large archives by aggregating messages in one sequential scan and allowing a longer bounded deadline.
• Preserve guild and member history with source-attributed tombstones, explicit restore semantics, omission-safe member refreshes, and revision-aware Git-share merges.
• Keep attachment fetches compatible with three allowed Discord CDN redirects and injected HTTP transports while preserving final-response host validation.
• Reject malformed tombstone timestamps before routine incremental snapshot imports mutate the archive.

Maintenance

• Update golang.org/x/crypto to v0.54.0 and modernc.org/libc to v1.74.3, while confirming Crawlkit v0.14.3 remains current.

Full changelog: v0.11.7...v0.11.8

Highlights

• Ship official macOS binaries notarized by Apple, with hardened runtime, trusted timestamps, stable Foundation identity requirements, and online ticket verification.
• Keep incomplete macOS artifacts out of releases by preserving GoReleaser outputs until signing, notarization, and every post-notarization check succeeds.

Maintenance

• Require the private runtime notary profile for official releases and verify notarization tickets again from published macOS archives.
• Update Crawlkit to v0.14.3 and its compatible SQLite and terminal-detection stack.

Full changelog: v0.11.6...v0.11.7

0.11.6 - 2026-07-17

Highlights

• Keep live Discord tails recoverable across handler timeouts and process restarts with a private durable spool and exact create, update, and delete replay. Thanks @hannesrudolph.
• Make CLI discovery and setup smoother with descriptive root and nested help, side-effect-free -h, and actionable missing-configuration guidance. Thanks @0xdevalias.
• Harden archive boundaries by blocking attachment redirects off Discord CDN hosts and rejecting malformed tombstone timestamps before import mutations. Thanks @GrantTheAssistant.
• Report accurate versions for source-installed binaries.

Changes

• Add bounded tail failure replay through discrawl tail --replay-failures-only and --replay-limit, with privacy-safe diagnostics. Thanks @hannesrudolph.
• Improve root, command, and nested command help with descriptions and side-effect-free -h handling. Thanks @0xdevalias.

Fixes

• Keep Discord Gateway tails fail-closed while durably spooling and replaying timed-out message creates, updates, and deletes. Thanks @hannesrudolph.
• Reject attachment redirects whose final URL leaves Discord's allowlisted CDN hosts. Thanks @GrantTheAssistant.
• Reject malformed message tombstone timestamps before either SQLite snapshot import path mutates the archive. Thanks @GrantTheAssistant.
• Show setup guidance instead of a raw missing-file error when configuration-dependent commands cannot find config.toml. Thanks @0xdevalias.
• Report module build metadata for source-installed binaries instead of a stale hard-coded release version.

Maintenance

• Update Crawlkit to v0.14.2 and Kong to v1.16.0, alongside current x/sys, x/text, and TruffleHog releases.
• Harden GoReleaser macOS archive verification before signed publication.
• Sync canonical AutoReview tooling and strengthen repository secret scanning. Thanks @vincentkoc.

Full changelog: v0.11.5...v0.11.6

Maintenance

• Add Developer ID signing and release verification for official macOS binaries under OpenClaw Foundation Team ID FWJYW4S8P8, while keeping local and cross-platform snapshot builds credential-free.
• Update CrawlKit to v0.13.4 and Go to 1.26.5, including the standard-library crypto/tls security fixes.

Changes

• Add read-only publish --check scope preflight with fail-closed metadata readiness, source hints, predicted channel/message counts, and concrete repair guidance.
• Add shared channel resolution for channels, search, messages, and message sync with stable id precedence and actionable ambiguity candidates.
• Add read-only diagnostics output for SQLite integrity, WAL size, archive freshness, and authoritative Discrawl writer-lock ownership.
• Add read-only archive coverage reporting with per-guild/channel bounds, named-versus-synthetic channel counts, persisted wiretap skip counters, and watch-mode deltas via wiretap --stats.
• Preserve fetched attachment media metadata when duplicate attachment snapshots refresh the singleton attachment row. Thanks @agent-eli.
• Add a local-only failure ledger with row-level write context, retry/resolution tracking, JSON queries, and known-failure coverage counts.
• Add a local-first maintainer archive workflow guide covering health, coverage, wiretap, stable queries, and privacy-safe publish preflight. Thanks @joshka.

Fixes

• Prevent routine Git snapshot updates from clearing locally crawled cache rows: changed shards now merge monotonically, while removed shards, schema changes, historical restores, and exact reconciliation require explicit --force; upgrade Crawlkit to v0.13.2.

Maintenance

• Refresh stable Go modules and CI/release tooling, including SQLite, go-toml, Go analyzers, GitHub Actions, GoReleaser, and TruffleHog.

Fixes

• Drain large Cloudflare D1 table resets and split memory-bound ingest batches so discrawl cloud publish can refresh large Discord archives.

Fixes

• Keep repeated Discord attachment snapshots from aborting scheduled archive syncs when the same attachment id appears again.

Changes

• Add immutable Git snapshot tags and non-mutating historical restores with publish --tag and update --ref.
• Restore the missing v0.10.0 release history for the first Cloudflare remote archive release. Thanks @joshka.
• Expose the release changelog directly in the documentation site navigation. Thanks @joshka.

Fixes

• Preserve private share-repository permissions and unpublished local branches while moving Git history, ref, and FTS query mechanics onto CrawlKit; refresh Go dependencies.
• Refresh Discord member roles daily for published archives, and make sync --with-members bypass cached freshness when a refresh is required. Thanks @hannesrudolph.
• Keep incremental share imports compatible with crawlkit's safe changed-tail replacement plan instead of falling back to a full archive rebuild.
• Accept absolute Windows SQLite paths through the shared crawlkit store opener.

Changes

• Add optional turbovec semantic-search scoring via [search.embeddings].vector_backend, while keeping exact cosine as the default backend. Thanks @vincentkoc.
• Added the Homebrew install command to the discrawl.sh landing hero and agent docs index, with a one-row desktop layout and copy button.
• Update crawlkit through v0.12.0.
• Mirror the non-DM local SQLite archive into the Worker-backed R2 object store during discrawl cloud publish, alongside the D1 row ingest used for live queries.
• Compress the sanitized SQLite mirror as a gzip chunk bundle with an explicit privacy/count manifest before uploading to R2.

Fixes

• Kept resumed sync --full backfills from moving channel latest-message checkpoints backward, avoiding duplicate head recrawls on large interrupted channels. Thanks @hannesrudolph.
• Made messages --sync fail fast with an omit---sync hint when a live tail process owns the sync lock, while plain messages reads continue without waiting. Thanks @jeanmonet.

Changes

• Add read-only Cloudflare remote archive mode with [remote] config, subscribe-cloud, GitHub-backed remote login, remote status, remote archives, and cloud-mode status --json without creating a local SQLite database.
• Route cloud-mode search and filtered messages reads to Worker named queries so subscribers can inspect live D1 data without local SQLite.
• Add discrawl cloud publish to export non-DM local SQLite rows to the Cloudflare remote archive without changing Git snapshot publishing.

Changes

• Add cached release checks with discrawl check-update and passive terminal notices when a newer Discrawl release is available.

Fixes

Changes

• Media-enabled discrawl publish now migrates shared attachment media to gzip-compressed files while still importing older raw-media snapshots.
• Semantic search now scores lightweight embedding rows first and hydrates full message details only for the winning results.

Fixes

• Bounded gzip media restore and hash verification so malformed shared snapshots cannot decompress unbounded data.
• Cancelled concurrent message-sync workers when a peer hits a fatal channel error.
• Rejected inaccessible explicit guild targets during init --guild and sync --guild/--guilds instead of silently treating them as successful empty syncs.
• Hardened embedding snapshot imports against unsafe manifest paths, symlink escapes, and unbounded gzip input.
• Limited FTS search fallback to missing or unsupported FTS infrastructure errors so unrelated query failures are reported.
• Re-sniffed previously skipped wiretap cache files instead of treating unchanged skipped fingerprints as permanently unchanged.

Changes

• Added attachment media caching with discrawl attachments, attachments fetch, sync --with-media, and Git snapshot backup/restore for cached non-DM media files.
• Documented media backup flow, including CDN fetch failures, local cache behavior, and Git snapshot publishing.
• Docker: add a local image with /data persistence and CI smoke coverage.
• Moved stable store SQL for sync state, messages, attachments, embedding jobs, members, and status reads/writes to sqlc-generated typed wrappers while leaving dynamic FTS, semantic search, report, share, and user SQL handwritten.

Fixes

• Kept large Git snapshot imports and FTS rebuilds from exhausting memory on small hosts by using file-backed SQLite temp storage and a bounded import cache. (#65) Thanks @hxy91819.

Changes

Fixes

• Kept Git snapshot imports incremental when metadata, attachment, mention, or event tables changed alongside the message tail, avoiding full archive replays for routine updates.

Changes

• discrawl publish can now narrow Git snapshots with --public-only, --include-channels, and --exclude-channels, while the local archive can still sync from the full bot-visible dataset.
• New installs now use OS-native config/runtime paths via crawlkit, while existing ~/.discrawl installs keep working until users deliberately migrate.
• Moved top-level CLI parsing onto Kong while preserving Discrawl's existing command dispatch and archive lock policy.

Maintenance

• Moved Discrawl's platform path policy back into crawlkit so config, data, cache, log, and share directory defaults stay shared across crawler apps.

Fixes

• help search, search --help, messages --help, and sql --help now print focused command help without opening config, stores, or triggering Git snapshot auto-update.
• discrawl sync now warns once for newly discovered Discord Missing Access / Unknown Channel skips, then keeps repeat unavailable-channel skips out of normal logs while preserving summary counts.

Changes

• Added discrawl tui, a terminal archive browser for stored guild messages and local @me wiretap DMs using the shared crawlkit pane browser.
• Added crawlkit-backed metadata --json, status --json, and doctor --json control surfaces for launchers, automation, and CI checks.
• Published the generated documentation site at discrawl.sh, including command pages, install/setup docs, configuration, security notes, guides, a contact page, and social cards.
• Moved the Go module and release metadata to github.com/openclaw/discrawl.

Fixes

• Kept documented command-local search flags working after the query, such as discrawl search "term" --limit 5. Thanks @PrinceOfEgypt.
• Made the terminal browser more useful and accurate: default guild scoping, newest-message startup, compact panes, selected-message detail panes, count-header sorting, local/remote status labels, right-click actions, Discord message URLs, row labels, direct-message pane labels, mention rendering, inline mention resolution, attachment details, and reply-context hydration without broad thread scans.
• Kept read-only commands such as search, messages, and safe sql usable while tail or another writer holds the sync lock. Thanks @PrinceOfEgypt.
• Kept tui --help, status, and terminal-browser reads safe for fresh or missing local databases without triggering Git snapshot auto-update.
• Kept local-only snapshot rows filtered during shared archive imports and forwarded snapshot import progress through the crawlkit import path.
• Made stale Git snapshot imports plan shard deltas from crawlkit file fingerprints or Git object identity, so routine shared-archive refreshes import changed message tail shards instead of rebuilding every table and FTS index.
• Included progress percentages in message-sync logs.
• Fixed GoReleaser version stamping after the module path move.

Documentation

• Documented the crawlkit-backed config/status/control, snapshot, mirror, sync-state, output, and shared TUI surfaces now used on main.
• Clarified that Discord bot sync, desktop wiretap parsing, DM privacy filters, schema ownership, FTS/ranking, embeddings, and analytics remain app-owned.
• Aligned terminal-browser docs with the gitcrawl-style shared TUI model: channel/person/thread groups, message rows, detail/thread panes, sorting, mouse selection, right-click actions, and local/remote status chrome.
• Refreshed the repo-local discrawl agent skill for local Discord archive, freshness, query, boundary, TUI, verification, and read-only SQL workflows.

Maintenance

• Migrated runtime paths, SQLite opening, archive mirror/export/import helpers, output/status wiring, and TUI plumbing onto the shared crawlkit infrastructure.
• Moved reusable embedding providers and vector helpers onto crawlkit while keeping Discrawl-owned storage, FTS, queueing, and privacy filters local.
• Updated crawlkit through v0.4.1, switched imports to github.com/openclaw/crawlkit, and added CI smoke coverage for the crawlkit control surface and merge behavior.
• Added CodeQL, verified secret scanning, protected automation owners, stale issue automation, .editorconfig, and .gitattributes.
• Added release workflow automation that dispatches the Homebrew tap formula update after GoReleaser publishes a tag.