github.com/moov-io/watchman

ADDITIONS

• feat: add Prometheus duration histogram for HTTP API requests by @naemono in #811
  • Exposes watchman_http_request_duration_seconds (HistogramVec by method, route, code) on the existing admin /metrics endpoint
  • Route labels use the registered path template (e.g. /v2/ingest/{fileType}), not the raw request path

IMPROVEMENTS

• fix: resolve open CodeQL findings on master by @adamdecaf in #800
  • Avoid log injection in the ingest API by keeping user-controlled fileType on the structured field only
  • Pin third-party Actions in release.yml by commit SHA

BUILD

• fix(deps): update module github.com/openrouterteam/go-sdk to v0.7.40 by @renovate in #810
• fix(deps): update all by @renovate in #808
• chore(deps): update all (major) by @renovate in #786
• fix(deps): update all by @renovate in #807
• fix(deps): update module github.com/moov-io/base to v0.63.0 by @renovate in #806
• chore(deps): update registry.access.redhat.com/ubi9/ubi-minimal docker tag to v9.8-1786323074 by @renovate in #805
• fix(deps): update module github.com/openrouterteam/go-sdk to v0.7.32 by @renovate in #804
• chore(deps): bump github.com/moov-io/base to v0.62.1 by @adamdecaf in #803
• build(deps-dev): bump json from 2.20.0 to 2.21.2 in /docs by @dependabot in #802
• fix(deps): update module github.com/openrouterteam/go-sdk to v0.7.26 by @renovate in #799
• fix(deps): update module github.com/openrouterteam/go-sdk to v0.7.25 by @renovate in #798
• fix(deps): update opentelemetry-go monorepo to v1.45.0 by @renovate in #796
• fix(deps): update all by @renovate in #797
• fix(deps): update all by @renovate in #795
• fix(deps): update module github.com/openrouterteam/go-sdk to v0.7.18 by @renovate in #794

New Contributors

• @naemono made their first contribution in #811

Full Changelog: v0.65.2...v0.66.0

IMPROVEMENTS

• fix: reduce refresh memory peak after corpus swap by @adamdecaf in #791
  • Search corpus owns the entity slice; latestStats keeps metadata only
  • After index swap, run GC + FreeOSMemory so heap returns to single-corpus steady when Update returns (instead of lingering near ~2.5× until a later GC)
  • Local bench (8 lists / 39,837 entities): post-swap heap 2.48× → 1.00× steady; ~294 MB less retained at swap vs prior behavior
  • Live prod dumps (US lists: ofac/csl/non_sdn/fincen_311): ~3.5 GB RSS / ~100 MB Go heap — cgroup sizing is driven by RSS (libpostal in image), not Go corpus alone

BUILD

• fix(deps): update module github.com/openrouterteam/go-sdk to v0.7.10 by @renovate in #788
• fix(deps): update module github.com/openrouterteam/go-sdk to v0.7.17 by @renovate in #789

Full Changelog: v0.65.1...v0.65.2

IMPROVEMENTS

• fix: keep serving data when scheduled refresh fails by @adamdecaf in #787

What's Changed

• fix(deps): update all by @renovate[bot] in #780
• fix(deps): update all by @renovate[bot] in #782
• fix(deps): update module github.com/openrouterteam/go-sdk to v0.6.0 by @renovate[bot] in #783
• fix(deps): update module github.com/openrouterteam/go-sdk to v0.6.2 by @renovate[bot] in #784
• build(deps): bump google.golang.org/grpc from 1.81.1 to 1.82.1 by @dependabot[bot] in #781
• fix(deps): update all by @renovate[bot] in #785

Full Changelog: v0.65.0...v0.65.1

IMPROVEMENTS

• perf: candidate index, admission control, and hot-path allocation cuts by @adamdecaf in #779

What's Changed

• chore(deps): update all (major) by @renovate[bot] in #777
• fix(deps): update all by @renovate[bot] in #776
• fix(search): clean up errors from Client by @adamdecaf in #778

Full Changelog: v0.64.0...v0.65.0

ADDITIONS

• build: Compile for linux/arm64 as well by @elliotcourant in #772
• feat: add US Terrorist Exclusion List (TEL) by @cacheflow in #774
• feat: Allow data to be refreshed via an API call by @elliotcourant in #771

What's Changed

• build(deps-dev): bump nokogiri from 1.19.3 to 1.19.4 in /docs by @dependabot[bot] in #768
• build(deps-dev): bump faraday from 2.14.2 to 2.14.3 in /docs by @dependabot[bot] in #769
• build(deps): bump concurrent-ruby from 1.3.6 to 1.3.7 in /docs by @dependabot[bot] in #770
• chore(deps): update all (major) by @renovate[bot] in #766
• fix(deps): update all by @renovate[bot] in #765

New Contributors

• @cacheflow made their first contribution in #774

Full Changelog: v0.63.3...v0.64.0

Full Changelog: v0.63.2...v0.63.3

What's Changed

• perf: warm up libpostal concurrently with the initial data load by @akamick86 in #763

Full Changelog: v0.63.1...v0.63.2

IMPROVEMENTS

• fix: init libpostal before accepting requests by @adamdecaf in #758

What's Changed

• chore(deps): update all by @renovate[bot] in #757
• chore(deps): update all to v0.14.0 by @renovate[bot] in #759
• chore(deps): update all to v0.14.0 by @renovate[bot] in #760
• chore(deps): update all to v0.14.0 by @renovate[bot] in #761

Full Changelog: v0.63.0...v0.63.1

ADDITIONS

• feat(download): watchman ignored download errors by @bpross in #748

IMPROVEMENTS

• address: switch to moov-io/gopostal by @adamdecaf in #750
• perf: skip non-matching OFAC government ID regexes during preparation by @akamick86 in #752
• perf: skip language detection when it cannot change stopword removal by @akamick86 in #753

What's Changed

• fix(deps): update opentelemetry-go monorepo to v1.44.0 by @renovate[bot] in #749
• fix(deps): update all by @renovate[bot] in #751
• chore(config): migrate Renovate config - autoclosed by @renovate[bot] in #754

New Contributors

• @akamick86 made their first contribution in #752
• @bpross made their first contribution in #748

Full Changelog: v0.62.0...v0.63.0

NOTICE

The switch to using Senzing's libpostal data increases memory usage to over 3GB. Adjust your deployments accordingly.

ADDITIONS

• feat: add MCP server by @adamdecaf in #724

IMPROVEMENTS

• Make final score penalties configurable by @pvpacheco in #718
• feat: switch libpostal data to Senzing updated dataset by @adamdecaf in #735
• stringscore: add Soundex phonetic matching algorithm by @SamOluwadamilare300 in #743

What's Changed

• fix(deps): update opentelemetry-go monorepo to v1.41.0 by @renovate[bot] in #714
• fix(deps): update all by @renovate[bot] in #716
• fix(deps): update module go.opentelemetry.io/otel/trace to v1.42.0 by @renovate[bot] in #717
• chore(deps): update all (major) by @renovate[bot] in #715
• fix(deps): update opentelemetry-go monorepo to v1.43.0 by @renovate[bot] in #727
• feat: add MCPS message signing to MCP server by @razashariff in #730
• build(deps-dev): bump activesupport from 7.2.0 to 7.2.3.1 in /docs by @dependabot[bot] in #722
• build(deps-dev): bump json from 2.18.1 to 2.19.2 in /docs by @dependabot[bot] in #721
• chore(deps): update all by @renovate[bot] in #719
• build(deps): bump addressable from 2.8.7 to 2.9.0 in /docs by @dependabot[bot] in #728
• build(deps): bump github.com/jackc/pgx/v5 from 5.8.0 to 5.9.0 by @dependabot[bot] in #731
• fix(deps): update all by @renovate[bot] in #732
• feat(mcp): AgentPass agent-identity verification for MCP endpoint by @razashariff in #733
• fix(deps): update all by @renovate[bot] in #734
• build(deps): bump github.com/jackc/pgx/v5 from 5.9.0 to 5.9.2 by @dependabot[bot] in #736
• chore(deps): update all by @renovate[bot] in #737
• feat(mcp): stamp agent identity into search_entities responses by @MorganaFuture in #739
• fix(docs): Updating API spec by @elliotcourant in #742
• build(deps-dev): bump nokogiri from 1.19.1 to 1.19.3 in /docs by @dependabot[bot] in #741
• fix(deps): update all by @renovate[bot] in #738
• build(deps-dev): bump faraday from 2.14.1 to 2.14.2 in /docs by @dependabot[bot] in #744
• chore(deps): update all by @renovate[bot] in #745
• fix(deps): update all by @renovate[bot] in #746
• fix(deps): update all by @renovate[bot] in #747

New Contributors

• @pvpacheco made their first contribution in #718
• @razashariff made their first contribution in #730
• @elliotcourant made their first contribution in #742
• @SamOluwadamilare300 made their first contribution in #743

Full Changelog: v0.61.1...v0.62.0

IMPROVEMENTS

• docs: type param on search is no longer required
• download: error when unknown lists are provided in the config
• search: error requests when extra/unused query params are supplied
• search: stop requiring type API query param
• sources/csl_us: clean "and any subsidiary" from entity names
• sources/csl_us: read CSV from trade.gov
• sources/csl_us: set ReuseRecord on reader

BUILD

• build: setup CSL US benchmarks
• sources/csl_us: setup benchmarks, use smaller test file
• sources/ofac: add benchmark for FindEntity

IMPROVEMENTS

• docs: include source data in openapi spec
• ingest: log how many File templates are read
• search: read/set source (SouceList) query param
• sources/display: don't set ?source=api-request

This release of Watchman adds support for ingesting CSV files as datasets, improved address similarity scoring, returning the original query in search results, and consolidating the web interface in the server binary / docker image along with several bug fixes.

ADDITIONS

• download: read DOWNLOAD_TIMEOUT from environment, default to 45s
• ingest: support arbitrary CSV via configuration as datasets
• search: return the original query
• sources/display: add DetailsURL for getting linkable Entity pages

IMPROVEMENTS

• api: add test for ErrorResponse and JsonResponse
• build: consolidate webui into main binary / docker image
• fix: set INITIAL_DATA_DIRECTORY better in tests
• ingest: normalize Entity after parse
• ofactest: use a "tiny ofac" record set, except for benchmarks
• search: grab read lock before ProcessSliceFn
• search: hide SimilarityScore fields when zero
• search: normalize addresses to improve search results

BUILD

• build: only run benchmarks on non-github authored commits
• build(deps): bump golang.org/x/net from 0.35.0 to 0.38.0

BREAKING CHANGE

Watchman has undergone a complete overhaul of it's API endpoints, public models, search methodology, performance, and much more. We've redesigned Watchman to offer a unified endpoint and models to search sanctioned entities from any supported list. We encourage you to try out the new Watchman and report back any issues, comments, concerns you have.

This release contains a lot of changes so make sure to test, validate and verify your usage. Let us know in the #watchman slack room how the new version works for you. We are planning to make continual scoring improvements.

IMPROVEMENTS

• search: unified endpoint and model for all supported lists
• search: better performance for concurrently processing sanction lists
• search: integrate libpostal for address parsing and normalization