github.com/segmentio/ctlstore

Description

Update the go-sqlite3 dependency for Ctlstore to get to the latest upstream version from https://github.com/mattn/go-sqlite3.

Notably we depend on a Segment-specific fork that includes a patch which enables preupdate hooks for our code without requiring a build-time flag to be manually put into place by all users of the ctlstore library.

Details

This depends on segmentio/go-sqlite3#9 to allow the go get to succeed.

Once that is merged, we deleted the segment-v1.14.22 tag in that repo and recreated it as v1.14.22-segment.

% go get github.com/segmentio/[email protected]
go: github.com/segmentio/[email protected] (v1.14.23-0.20240208202956-73b5bef61db6) > requires github.com/segmentio/[email protected]: parsing go.mod:
	module declares its path as: github.com/mattn/go-sqlite3
	        but was required as: github.com/segmentio/go-sqlite3

Testing completed successfully:

• ensure CI runs
• verify on other services that include ctlstore client, when those services are updated. https://segment.atlassian.net/browse/IO-1695

• start up multiple reflectors
• Concurrent ldb downloads
• Add rotating reader and tests
• add convenience methods
• use go 1.13 friendly atomic interface
• add support for multiple LDB copies in InitContainer
• only create dbs if they don't exist
• reopen and close last rotated reader
• create a changelog for first ldb in multimode

• properly check errors while shoveling

• address another parallel test suite not copying test case var

• add reflector poll timeout configuration cli arg

• refactor supervisor to use sdk v2 and add checksum

• open another reader to get checksum

• make generate

• refactoring

• hex encoding

• don't calculate

• use metadata

• WIP test

• wip

• wip

• WIP

• WIP

• WIP

• fix install

• WIP

• wip

• wip

• chmod

• install s3cmd from alpine

• install aws-cli in alpine

• install shasum in alpine

• wip

• test if sha value matches

• wip

• fix shell syntax error

• add 5 attempts threshold

• skip checksum validation if null

• test unhappy path

• revert false negative

• address feedbacks

• fix script

• test unhappy path

• revert false negative

• using sha1

• update log to avoid confusion

• fix wording

Co-authored-by: Hongyu Zhou [email protected]

• add s5cmd to image
• add pigz to test faster decompress
• add downloading to script
• add concurrency parameter
• Add metrics to the download script

• update sqlite version

• fix merge

• fix int test

• update version

Co-authored-by: Taylor Brennan [email protected]

Bumping testify version for CVE

feat(testing): improve support for running parallel tests with ctlsto…