github.com/nakabonne/tstorage

Release v0.3.6 with changes since v0.3.5

• Release v0.3.6 (#63) - by GitHub
• Fix pipeline to setup Go1.20 properly - by nakabonne
• Move int64 to beginning of struct to ensure alignment (#61) - by GitHub
• Bump Go to 1.20 (#62) - by GitHub
• Bump coverage action from v1 to v2 (#50) - by GitHub
• Remove unneeded cmd package - by nakabonne

Release v0.3.5 with changes since v0.3.4

• Release v0.3.5 (#49) - by GitHub
• Allow WAL segment to have invalid rows (#48) - by GitHub

Release v0.3.4 with changes since v0.3.3

• Release v0.3.4 (#46) - by GitHub
• Ignore invalid disk partitions when starting (#45) - by GitHub

Release v0.3.3 with changes since v0.3.2

• Release v0.3.3 (#44) - by GitHub
• Consider data points within expired partitions as ErrNoDataPoints - by nakabonne

This is a minor release with a bug that panics when recovering WAL.

Release v0.3.2 with changes since v0.3.1

• Release v0.3.2 (#42) - by GitHub
• Avoid to panic when recoverying WAL segments (#41) - by GitHub

A minor release with bug fixes on WAL recovery.

Release v0.3.1 with changes since v0.3.0

• Release v0.3.1 (#40) - by GitHub
• Allow WAL to have invalid row (#39) - by GitHub
• Remove Used by section - by nakabonne
• Bump runtime to 1.17 in the test environment - by nakabonne

Notable change - WAL support

This release comes with WAL (Write Ahead Log) support, which means a certain level of durability is guaranteed because the data on the memory partition is appended to the log file on the disk.
WAL is enabled by default in on-disk mode. This slightly affects the write performance, so if you want to disable it, give -1 to WithWALBufferedSize() like:

	storage, err := tstorage.NewStorage(
		tstorage.WithDataPath("./data"),
		tstorage.WithWALBufferedSize(-1),
	)

Changelog

• Release v0.3.0 (#35) - by GitHub
• Bump pipecd/actions-gh-release to v2.1.0 - by nakabonne
• Ensure to remove the oldest segment when flushing (#34) - by GitHub
• Add support for Write Ahead Log (#28) - by GitHub
• Bump Go from 1.16 to 1.17 (#33) - by GitHub

This is a minor update with a couple of bug fixes.

• f951f22 Release v0.2.2 (#31) - by @github
• 2ad4f71 Merge pull request #30 from fedorlitau/fix-slice-range-error - by @github
• b92d1c9 Fix "slice bounds out of range" error in memoryMetric.selectPoints() - by @fedor Litau
• 9034eef Avoid panic when inserting out-of-order points in spite of one partition - by @nakabonne

Notable changes

Add support for the WithRetention option (#22)

Data points will get automatically removed from the disk after a specified period of time after a disk partition was created.

	storage, err := tstorage.NewStorage(
		tstorage.WithDataPath("./data"),
		tstorage.WithRetention(time.Hour*24)
	)

Support writes outside of head partition (#23) by @dpgil

Basically, all data points get ingested into the head partition. But sometimes data points get across the partition boundary, which should get into the next partition. From this version, tstorage doesn't discard them.

Changelog

• fda1a6f Release v0.2.1 (#27) - by @github
• 11d3d74 Remove dependency on deprecated ioutil package - by @nakabonne
• f651d67 Add WithRetention to automatically remove expired data (#22) - by @github
• 2fd043f Handle the case that fakeEncoder doesn't have actual function - by @nakabonne
• 6b08cbe Add notes on used by section - by @nakabonne
• afe10c8 Update notes on data point chunks - by @nakabonne
• 832a382 Support writes outside of head partition (#23) - by @github
• a6fdddc Merge pull request #24 from bartmika/main - by @github
• c9a8484 Included tstorage-server to the Used by section. - by @bartmika
• b1bbbaf Add interface to clean partition resources - by @nakabonne

tstorage v0.2.0 is here 🎉

This release mostly includes encoding improvements, which massively reduces the size of encoded data points. Facebook is publishing a paper called Gorilla: A Fast, Scalable, In-Memory Time Series Database, in which they introduced an encoding way that takes advantage of the characteristics of time series data. According to that, tstorage v0.2.0 encodes/decodes buffered data points right before flushing into a disk partition. For more details, see: https://nakabonne.dev/posts/write-tsdb-from-scratch/#encoding

Breaking changes

Please note that it cannot decode data points encoded by previous versions as the encoding scheme is totally different from the old one and doesn't consider backward compatibility at all.

This release also contains a couple of tiny bug fixes around disk partitions.

Changelog

• f13c10e Bump to v0.2.0 (#15) - by @github
• 74a487b Update RELEASE file to be valid - by @nakabonne
• 885b0ac Add RELEASE file - by @nakabonne
• e2304d6 Add action to release via git commit - by @nakabonne
• ec46d21 Guarantee it encodes points at random interval - by @nakabonne
• a832e87 Fix the way to take offset so that it can decode no matter the number of metrics is (#12) - by @github
• 167b85e Ensure to make data dir if it doesn't exist - by @nakabonne
• a855017 Ensure to ignore query that end is less than min - by @nakabonne
• c774fbb Add link to the blog post for more details - by @nakabonne
• 2eacb01 Add implementation of Gorilla encoding (#11) - by @github