github.com/moov-io/watchman

IMPROVEMENTS

• docs: type param on search is no longer required
• download: error when unknown lists are provided in the config
• search: error requests when extra/unused query params are supplied
• search: stop requiring type API query param
• sources/csl_us: clean "and any subsidiary" from entity names
• sources/csl_us: read CSV from trade.gov
• sources/csl_us: set ReuseRecord on reader

BUILD

• build: setup CSL US benchmarks
• sources/csl_us: setup benchmarks, use smaller test file
• sources/ofac: add benchmark for FindEntity

IMPROVEMENTS

• docs: include source data in openapi spec
• ingest: log how many File templates are read
• search: read/set source (SouceList) query param
• sources/display: don't set ?source=api-request

This release of Watchman adds support for ingesting CSV files as datasets, improved address similarity scoring, returning the original query in search results, and consolidating the web interface in the server binary / docker image along with several bug fixes.

ADDITIONS

• download: read DOWNLOAD_TIMEOUT from environment, default to 45s
• ingest: support arbitrary CSV via configuration as datasets
• search: return the original query
• sources/display: add DetailsURL for getting linkable Entity pages

IMPROVEMENTS

• api: add test for ErrorResponse and JsonResponse
• build: consolidate webui into main binary / docker image
• fix: set INITIAL_DATA_DIRECTORY better in tests
• ingest: normalize Entity after parse
• ofactest: use a "tiny ofac" record set, except for benchmarks
• search: grab read lock before ProcessSliceFn
• search: hide SimilarityScore fields when zero
• search: normalize addresses to improve search results

BUILD

• build: only run benchmarks on non-github authored commits
• build(deps): bump golang.org/x/net from 0.35.0 to 0.38.0

BREAKING CHANGE

Watchman has undergone a complete overhaul of it's API endpoints, public models, search methodology, performance, and much more. We've redesigned Watchman to offer a unified endpoint and models to search sanctioned entities from any supported list. We encourage you to try out the new Watchman and report back any issues, comments, concerns you have.

This release contains a lot of changes so make sure to test, validate and verify your usage. Let us know in the #watchman slack room how the new version works for you. We are planning to make continual scoring improvements.

IMPROVEMENTS

• search: unified endpoint and model for all supported lists
• search: better performance for concurrently processing sanction lists
• search: integrate libpostal for address parsing and normalization

IMPROVEMENTS

• postalpool: use msgpack over JSON for binary pool

BUILD

• build(deps-dev): bump nokogiri from 1.16.7 to 1.18.3 in /docs

IMPROVEMENTS

• address: telemetry for all cases of ParseAddress
• ofactest: fix sample rate
• stringscore: make benchmark more consistent

IMPROVEMENTS

• build: fixup benchmark ResetTimer calls
• norm: add BenchmarkPhoneNumber, track ./internal/norm/ benchmarks in graph
• ofactest: start on sampling OFAC data
• search: leverage query's .PreparedFields.Name in compareName
• search: normalize phone numbers
• search: precompute strings.Fields

IMPROVEMENTS

• search: precompute normalized name, altnames
• search: remove ccache

IMPROVEMENTS

• fix: normalize Country once
• minmaxmed: optimize implementation
• postalpool: better pooling by default, offer configurable knobs

BUILD

• fix(deps): update opentelemetry-go monorepo to v1.34.0

IMPROVEMENTS

• ast: quick sanity check test
• country: centralize normalization
• docs: add arabic phonetics
• groupsize: cleanup benchmarks
• largest: make Items generic
• postalpool: force healthchecks to make network calls
• search: return watchman version in /v2/listinfo
• search: support ?debug query param to return base64 encoded matching data
• search: trim Address.Format()

BUILD

• build: require basic code coverage, use default linters
• build: stop publishing :latest docker tag

IMPROVEMENTS

• cmd/server: set automaxprocs on startup
• ofactest: include Client for calling Sanctions List Search application
• search: benchmark Similarity with debug and without
• search: weight addresses more, consider address as signal to not downrank

IMPROVEMENTS

• search: factor in contact info, weight only name match less

IMPROVEMENTS

• groupsize: cleanup old evaluations concurrently, better locking schematics

IMPROVEMENTS

• groupsize: better locking behavior

BREAKING CHANGE

Watchman has undergone a complete overhaul of it's API endpoints, public models, search methodology, performance, and much more. We've redesigned Watchman to offer a unified endpoint and models to search sanctioned entities from any supported list. We encourage you to try out the new Watchman and report back any issues, comments, concerns you have.

IMPROVEMENTS

• search: unified endpoint and model for all supported lists
• search: better performance for concurrently processing sanction lists
• search: integrate libpostal for address parsing and normalization

IMPROVEMENTS

• cmd/server: prevent NaN from escaping bestPairsJaroWinkler

IMPROVEMENTS

• cmd/server: add more configuration for disabling default lists
• cmd/server: copy from largest results
• fix: fetch UK sanctions .ods from HTML

BUILD

• build: update golang.org/x/net

IMPROVEMENTS

• feat: Add environment configuration to disable Web UI
• feat: Add environment configuration to disable UK CSL download

BUILD

• build(deps): bump rexml from 3.3.6 to 3.3.9 in /docs