github.com/johannesboyne/gofakes3

Adds paginated bucket listing to the s3bolt backend.

Features

• s3bolt: ListBucket now honours page.Marker and page.MaxKeys instead of returning ErrInternalPageNotImplemented, so listing a bolt-backed bucket can be paginated (#114) — thanks @Aliexe-code

Bug fixes

• s3bolt: fix infinite pagination when listing with a delimiter — NextMarker now advances past the whole common-prefix group instead of pointing at the prefix itself, so delimited listings terminate and each common prefix is returned exactly once (closes #66)

Delimited and flat multi-page listings are covered by new regression tests.

Maintenance release: correctness fixes, S3 compatibility improvements, and dependency updates.

Note on tag naming: starting with this release, tags use the Go-required v prefix (see #116). A matching v1.0.0 tag was also added so go get github.com/johannesboyne/[email protected] resolves correctly.

Bug fixes

• Fix CopyObject panic when X-Amz-Copy-Source is fully URL-encoded (#119) — thanks @rjpr
• Fix chunked-upload reader byte accounting and clean EOF on the terminating chunk (#120) — thanks @kailin4u
• Return 206 Partial Content for ranged GET requests (#121) — thanks @famulu
• Include IsTruncated=false in multipart list results, fixing nil-deref panics in AWS SDK consumers (#123, closes #122) — thanks @ryan0x44
• Return 206 for ranged HEAD requests (follow-up to #121)

Features

• Honour response-* query parameter header overrides on GetObject (#113) — thanks @zhangzqs

Dependencies

• Bump aws-sdk-go-v2/service/s3 to 1.97.3 and aws/protocol/eventstream to 1.7.8 (#117, #118)

Docs

• Add install section to the README (#115) — thanks @mittalyashu

What's Changed

• Fix http status code as 404 if bucket is not found by @grauwoelfchen in #4
• Move command gofakes3 to simplify vendoring by @kulti in #5
• Print using port by @kulti in #6
• compatibility changes for alpakka by @oker1 in #9
• Last-Modified header compatibility by @kulti in #10
• Fix 7 by @johannesboyne in #8
• Feature/pluggable backends by @shabbyrobe in #11
• Feature/spec errors by @shabbyrobe in #12
• Feature/prefix handling retry by @shabbyrobe in #14
• Stdlib-based routing by @shabbyrobe in #16
• Add code coverage from codecov.io by @johannesboyne in #18
• Content-MD5 validation by @shabbyrobe in #17
• Multi-delete support by @shabbyrobe in #19
• Feature/multipart uploads by @shabbyrobe in #20
• Fix "multiple calls to WriteHeader" bug by @shabbyrobe in #24
• Pluggable logger by @shabbyrobe in #22
• GetObject Range header support by @shabbyrobe in #25
• afero.Fs-based backends by @shabbyrobe in #21
• Calculate correct ETag on CreateObject by @shabbyrobe in #27
• Bucket rewriting switch by @shabbyrobe in #23
• Versioned object support by @shabbyrobe in #28
• Pagination for list bucket by @shabbyrobe in #30
• Add missing CORS headers by @shabbyrobe in #31
• Incorrect range handling for some edge cases by @jwoffindin in #37
• Fix MultiBucket CreateBucket by @shabbyrobe in #38
• Fix: removed omit tag for IsTruncated by @mars4myshare in #39
• PutObject support for zero-length files by @shabbyrobe in #42
• Support multipart uploads from browser by @korya in #43
• Support CopyObject operation by @korya in #44
• Add get bucket location by @ikozinov in #45
• Bugfix: do not overwrite the Last-Modified value with the current time by @cure in #46
• move requestID to guarantee 64-bit alignment to fix panic on 32-bit a… by @jrhy in #48
• Support for chunk upload signatures by @leoleoasd in #50
• Remove etag quotes when completing a multipart upload. by @tasdomas in #52
• unescape 'X-Amz-Copy-Source' value before using by @ilkinulas in #53
• backend/s3bolt: use "go.etcd.io/bbolt" by @igungor in #55
• Add Content-Type to metadataHeaders by @deansheather in #56
• unmarshall to boltobject before listing by @ocakhasan in #57
• Add test for unmarshall to bolt object before listing. by @ocakhasan in #58
• Implement conditional caching by @DerGut in #59
• Add Content-Disposition to metadata headers by @hlubek in #62
• Auto bucket creation option by @shabbyrobe in #65
• Investigate and fix issue 64 by @johannesboyne in #67
• Add example for aws-sdk-go-v2 by @johannesboyne in #68
• Correct V2 example by @elgohr in #70
• add multi delete for versioned buckets, omit delete markers on list operation by @kucukaslan in #69
• Fix auto bucket creation option by @aidilfbk in #73
• Bump github.com/aws/aws-sdk-go from 1.17.4 to 1.33.0 by @dependabot[bot] in #76
• Avoid error when execute 'mc cp' on single bucket backend. by @ideal in #74
• fixed typo in word "wanted" by @guettli in #77
• Add Content-Encoding to metadata headers by @csw in #79
• Add If-Modified-Since support by @DarthSim in #80
• Fs backend fixes by @shabbyrobe in #81
• Support for both path mode and host mode simultaneously by @shabbyrobe in #82
• Add MultipartBackend interface to support overriding the multipart upload behaviour by @ChrisSchinnerl in #83
• Extend Backend interface with CopyObject method by @ChrisSchinnerl in #84
• fix: CommonPrefixes follow S3 API behavior on edge cases in s3afero by @DenKoren in #86
• Fix startAfter when object is not present by @Alfus in #88
• Data race: lock on abort/complete of multipart upload by @rodaine in #90
• Fix timestamp formatting. by @tomclegg in #91
• docs: update README.md with host bucket example by @armsnyder in #92
• Add rudimentary Location return support for CompleteMultipartUpload by @daniel-sullivan in #95
• Fix multipart upload api mismatch with aws s3 api by @error1707 in #96
• Update gocovmerge by @daniel-sullivan in #97
• Fix calculation of etag in complete mpu by @error1707 in #98
• Updates for force delete and prefix and delim bug by @casibbald in #100
• Optionally allow all methods and headers in CORS preflight by @shabbyrobe in #102
• use github actions instead of circle ci by @miparnisari in #105
• Conditional Write (IfMatch and IfNoneMatch) support for PutObject by @Diggs in #107
• Allow arbitrary slashes in object keys by @doomguy-stg in #108
• remove dependency to sdk v1 by @miparnisari in #109

New Contributors

• @grauwoelfchen made their first contribution in #4
• @kulti made their first contribution in #5
• @oker1 made their first contribution in #9
• @johannesboyne made their first contribution in #8
• @shabbyrobe made their first contribution in #11
• @jwoffindin made their first contribution in #37
• @mars4myshare made their first contribution in #39
• @korya made their first contribution in #43
• @ikozinov made their first contribution in #45
• @cure made their first contribution in #46
• @jrhy made their first contribution in #48
• @leoleoasd made their first contribution in #50
• @tasdomas made their first contribution in #52
• @ilkinulas made their first contribution in #53
• @igungor made their first contribution in #55
• @deansheather made their first contribution in #56
• @ocakhasan made their first contribution in #57
• @DerGut made their first contribution in #59
• @hlubek made their first contribution in #62
• @elgohr made their first contribution in #70
• @kucukaslan made their first contribution in #69
• @aidilfbk made their first contribution in #73
• @dependabot[bot] made their first contribution in #76
• @ideal made their first contribution in #74
• @guettli made their first contribution in #77
• @csw made their first contribution in #79
• @DarthSim made their first contribution in #80
• @ChrisSchinnerl made their first contribution in #83
• @DenKoren made their first contribution in #86
• @Alfus made their first contribution in #88
• @rodaine made their first contribution in #90
• @tomclegg made their first contribution in #91
• @armsnyder made their first contribution in #92
• @daniel-sullivan made their first contribution in #95
• @error1707 made their first contribution in #96
• @casibbald made their first contribution in #100
• @miparnisari made their first contribution in #105
• @Diggs made their first contribution in #107
• @doomguy-stg made their first contribution in #108

Full Changelog: 0.0.2...1.0.0