github.com/codenotary/immudb

Bug Fixes

• include AtTx in StreamZScan response
• password reader 'inappropriate ioctl for device' from stdin (#658)
• fix StreamVerifiedSet and Get and add an (integration) test for them
• fix inclusion proofs in StreamVerifiedSet and Get
• embedded: use mutex to sync ops at tx lru-cache
• embedded: fix data races
• embedded/store: ensure waitees get notified when store is restarted
• embedded/store: remove checking for closed store when fetching any vlog
• embedded/store: continue indexing once index is replaced with compacted index
• embedded/store: set delay with duration in ms
• embedded/store: fix indexing sync and error retrieval
• embedded/store: ensure watchers get notified when indexing is up-to-date
• embedded/store: sync ReadTx operation
• embedded/tbtree: set lastSnapshot once flushed is completed
• embedded/tbtree: insertion delay while compacting not affecting compaction
• embedded/tbtree: release lock when compaction thld was not reached
• pkg/auth: add missing stream write methods to permissions
• pkg/client: fix minor leftover
• pkg/client: fix security issue: if client local state became corrupted an error is returned
• pkg/client: ensure dual proof verification is made when there is a previously verified state
• pkg/database: wrap seekKey with prefix only when seekKey is non-empty
• pkg/server: use latest snapshot when listing users

Changes

• refactor code quality issues
• improve serverside stream error handling
• remove fake proveSinceTxBs key send in streamVerifiableSet
• polish streams methods and add comments
• renaming stream methods
• updating copyright
• renaming stream methods, add stubs and stream service factory
• in server store creation max value entry is fixed to 32Mb
• set stream supports multiple key values
• mocked server uses the inner immudb grpc server and can be gracefully stopped
• fixed minimum chunk size at 4096 bytes
• add max tx values length guard and remove code duplication
• fix binary notation
• move stream service to a proper package
• add video streamer command
• increase stream coverage and add a guard if key is present on a stream but no value is found
• embedded: fix some typos with comments
• embedded: remove unused cbuffer package
• embedded: log indexing notifications
• embedded: descriptive logs on indexing and already closed errors
• embedded: add logs into relevant operations
• embedded: add logger
• embedded: compaction and snapshot handling
• embedded/appendable: thread-safe multi-appendable
• embedded/appendable: sync before copying appendable content
• embedded/appendable: multi-appendable fine-grained locking
• embedded/store: remove conditional locking before dumping index
• embedded/store: general improvements on snapshot management
• embedded/store: leverage fine-grained locking when reading tx data
• embedded/store: stop indexing while commiting with callback
• embedded/store: use buffered channel instead of a circular buffer
• embedded/store: remove duplicated logging
• embedded/store: set max file size to 2Gb (#649)
• embedded/store: lock-less readTx
• embedded/store: set a limit on indexing iteration
• embedded/store: log number of transactions yet to be indexed
• embedded/tbtree: optimize seek position
• embedded/tbtree: revert seek key setting
• embedded/tbtree: optimize seek position
• embedded/tbtree: terminate reader if prefix won't match any more
• embedded/tbtree: sync before dumping
• embedded/tbtree: sync key-history log during compaction
• embedded/watchers: broadcasting optimisation
• embedded/watchers: minor renaming
• embedded/watchers: accept non-continuous notification
• pkg/client: add stream service factory on client and increase stream coverage
• pkg/client: add GetKeyValuesFromFiles helper method
• pkg/client: add a guard to check for min chunk size
• pkg/client: remove local files tests
• pkg/client: maps server error on client package
• pkg/client: integration test is skipped if immudb server is not present
• pkg/database: return error while waiting for index to be up to date
• pkg/database: ensure scan runs over fully up-to-date snapshot
• pkg/database: return error while waiting for index to be up to date
• pkg/database: use in-mem current snapshot in execAll operation
• pkg/database: illegal state guard is added to verifiable get and getTx methods
• pkg/database: leverage lightweight waiting features of embedded store
• pkg/server: add a guard to check for min chunk size
• pkg/server: add server error mapper interceptor
• pkg/server: add small delay for indexing to be completed
• pkg/server: max recv msg size is set to 32M
• pkg/server: revert quit chan exposure
• pkg/server: exposes Quit chan
• pkg/stream: add more corner cases guards
• pkg/stream: add some comments to mesasge receiver
• pkg/stream: remove duplicated code
• pkg/stream: renamed stream test package
• pkg/stream: add a guard to detect ErrNotEnoughDataOnStream on client side
• pkg/stream: remove bufio.reader when not needed
• pkg/stream: remove bufio and add ventryreceiver unit test
• pkg/stream: add ErrNotEnoughDataOnStream error and ImmuServiceReceiver_StreamMock
• pkg/stream/streamtest: add dummy file generator
• tools: fix copyright
• tools/stream: get stream content directly from immudb
• tools/stream/benchmark: add stream benchmark command
• tools/stream/benchmark/streamb: add SinceTx value to getStream

Code Refactoring

• stream kvreceiver expose Next method to iterate over key values
• stream receiver implements reader interface
• use of explicit messages for stream request
• pkg/stream: use ParseValue func in zreceiver and remove the redundant readSmallMsg func
• pkg/stream: refactor receiver to increase simplicity

Features

• Remove unnecessary dependencies (#665)
• add support for user, password and database flags in immuclient (#659)
• increase default store max value length to 32MB
• add client->server stream handler
• refactors and implement server->client stream handler
• Add StreamVerifiedSet and StreamVerifiedGet
• add flusher to stream data to client
• chunk size is passed as argument in client and server
• add Stream Scan and client stream ServiceFactory
• embedded/store: integrate watchers to support indexing synchronicity
• embedded/store: configurable compaction threshold to set the min number of snapshots for a compaction to be done
• embedded/store: expose insertion delay while compacting
• embedded/store: tx header cache to speed up indexing
• embedded/tbtree: automatically set seekKey based on prefixKey when it's not set
• embedded/tbtree: configurable insertion delay while compaction is in progress
• embedded/watchers: lightweight watching center
• embedded/watchers: fetchable current state
• pkg/client: handle illegal state error
• pkg/database: non-blocking index compaction
• pkg/database: non-blocking, no history compaction
• pkg/database: default scan parameters using up-to-date snapshot
• pkg/server: add signature on stream verifiable methods and tests
• pkg/stream: add exec all stream

<a name="v0.9.1"></a>

Bug Fixes

• cmd/sservice: fix services management and add permissions guard
• cmd/sservice: fix group creation linux cross command
• embedded/history: read history log file to set initial offset
• embedded/store: copy key inside TxEntry constructor
• embedded/store: mutex on txlog
• embedded/store: continued indexing
• embedded/store: fix indexing sync (#621)
• embedded/tbtree: determine entry by provided seekKey
• embedded/tbtree: fix key history ordering (#619)
• embedded/tbtree: prevNode nil comparisson
• embedded/tbtree: use minkey for desc scan
• pkg/client: fix verifiedGetAt
• pkg/client/auditor: hide auditor password in logs
• pkg/client/cache: return an error if no state is found
• pkg/database: check key does not exists in latest state
• pkg/server: set default settings within DefaultStoreOptions method

Changes

• update acknowledgments
• cmd/sservice: minor fixes
• embeddded/tbtree: reduce mem allocs
• embedded: expose store opts
• embedded: refactor TxEntry
• embedded/store: adapt after History changes
• embedded/store: move TxReader code to its own file
• embedded/store: renamed reader as KeyReader
• embedded/store: use conditional locking in indexing thread
• embedded/store: minor KeyReader renaming
• embedded/store: validates targetTx is consistent with proof len
• embedded/store: sync access to commit and tx logs
• embedded/store/options.go: increase DefaultMaxKeyLen
• embedded/tbtree: offset map per branch
• embedded/tbtree: return ErrOffsetOutOfRange if invalid offset was provided
• embedded/tbtree: reduce mem consumption
• embedded/tbtree: history log file
• embedded/tbtree: configurable max key length
• embedded/tbtree: change history file extension
• pkg: unit testing index cleanup, use selected db
• pkg: current db included in signed state
• pkg/api: minor changes in TxScan message
• pkg/api: history limit as int32
• pkg/api: include server uuid and db name into state message
• pkg/client: validate returned entries from metadata
• pkg/client: bound reference if atTx is provided in VerifiedSetReferenceAt
• pkg/client: use indexing specified in GetRequest
• pkg/client: strip prefix from returned keys in txById and verifiedTxById
• pkg/client: add state service lock and unlock capabilities
• pkg/client: set bound on SetReference and ZAdd
• pkg/database: unsafe read tx inside CommitWith callback
• pkg/database: catch NoMoreEntries error and return empty list on scan and zscan operations
• pkg/database: return empty list if offset is out of range
• pkg/database: initial implementation of ExecAll with CommitWith
• pkg/server: naming conventions
• pkg/server: server mock wrapping default server implementation
• pkg/server: include uuid and db as result of verifiable operations
• pkg/server: initialize mts options
• pkg/server: expose store opts
• pkg/server: use server wrapper to enable post processing of results
• pkg/server: set default max value lenght to 1Mb
• pkg/server: change server default options. Max key value to 10kb

Features

• cmd/immuadmin: db index cleanup
• embedded: history with offset and limit, key updates counting
• embedded/appendable: flush and seek to start before copying
• embedded/appendable: implements Copy function
• embedded/appendable: check no closed and flush before copying
• embedded/store: commitWith callback receiving assigned txID
• embedded/store: TxScan asc/desc order
• embedded/store: index cleanup
• embedded/store: allow increasing max value size after creation time
• embedded/tbtree: full dump of current snapshot
• embedded/tbtree: complete history implementation
• embedded/tbtree: HistoryReader to iterate over key updates
• embedded/tbtree: full dump using copy on history log
• pkg: index cleanup service
• pkg/client: add state file locker
• pkg/client: add verifiedGetSince
• pkg/client: implementation of TxScan operation
• pkg/database: history with offset and limit
• pkg/database: TxScan implementation
• pkg/database: support for free and bound references
• pkg/database: KeyRequest retrieves key at a specific tx or since a given tx
• pkg/server: sign new state within verifiable operations
• pkg/server: use exposed synced mode

<a name="v0.9.0"></a>

Bug Fixes

• remove badger metrics and fix stats command
• cmd/immuadmin/command: fix immuadmin stats (#592)
• pkg/database: enable scan on fresh snapshot
• pkg/server: shutdown handlers and metrics server are moved in start method

Changes

• removing audit-signature and add serverSigningPubKey
• remove print tree method
• restore inmemory_cache test
• cmd/immuadmin: temporary disable stats functionality
• pkg/api: upgrade rest endpoints
• pkg/client: implement missing methods in immuclient mock
• pkg/server: temporary remove proactive corruption checker (#595)

Features

• add signature verification with a submitted public key

<a name="v0.9.0-RC2"></a>