What's Changed

• chore: Release by @PSeitz in #2686
• update release instructions by @PSeitz-dd in #2687
• feat: Regex query grammar by @Darkheir in #2677
• Update CHANGELOG.md for bugfixes by @PSeitz in #2674
• This PR changes the logic used on the ingestion of floats. by @fulmicoton in #2692
• Add fast field fallback for term query if not indexed by @PSeitz-dd in #2693
• Optimize ExistsQuery for a high number of dynamic columns by @PSeitz-dd in #2694
• Fix CI with rust 1.90 by @rdettai-sk in #2696
• fix compiler warnings by @PSeitz-dd in #2699
• clippy by @PSeitz in #2700
• perf: deduplicate queries by @PSeitz-dd in #2698
• add and/or queries benchmark by @PSeitz in #2701
• improve naming in buffered_union by @PSeitz-dd in #2705
• docs: correct grammar by @msmoiz in #2704
• refactor postings by @PSeitz in #2709
• Add DocSet::cost() by @PSeitz in #2707
• Replace AggregationsWithAccessor by @PSeitz in #2715
• Minor clarifications on the AggregationsWithAccessor refacto by @rdettai-sk in #2716
• Add Filtering for Term Aggregations by @PSeitz in #2717
• fix merge intermediate aggregation results by @PSeitz in #2719
• use FxHashMap for Aggregations Request by @PSeitz in #2722
• cleanup by @PSeitz in #2724
• Removing obsolete benchmark screenshot by @fulmicoton in #2730
• remove duplicated call to index_writer.commit in example by @Platane in #2732
• feat: added filter aggregation by @mdashti in #2711
• Optimize term aggregation with low cardinality + some refactoring by @fulmicoton-dd in #2740
• clippy by @PSeitz in #2741
• remove unused columnar api by @PSeitz-dd in #2742
• Optimization when posting list are saturated. by @fulmicoton in #2745
• split Term in Term and IndexingTerm by @PSeitz-dd in #2744
• Revert "remove unused columnar api (#2742)" by @fulmicoton in #2748
• fix: Handle phrase prefixed with star by @Darkheir in #2751
• Fix typos again by @kianmeng in #2753
• Implement collect_block for Collectors which wrap other Collectors by @stuhood in #2727
• Added some benchmark for top K by a fast field by @fulmicoton in #2754
• Lazy scorers by @fulmicoton in #2726
• docs: add usage example to README by @rustmailer in #2743
• fix unsafe warnings by @PSeitz in #2757
• add term->histogram benchmark by @PSeitz in #2758
• Term and IndexingTerm cleanup by @PSeitz in #2750
• Handle JSON fields and columnar in space_usage by @PSeitz-dd in #2761
• Make Language hashable by @philippemnoel in #2763
• Expose AddOperation and with_max_doc by @philippemnoel in #2762
• Make DeleteMeta pub by @philippemnoel in #2765
• Make some delete-related functions public by @philippemnoel in #2766
• move rand_distr to dev_dep by @PSeitz-dd in #2772
• fix coverage by @PSeitz-dd in #2769
• fix bug with minimum_should_match and AllScorer by @trinity-1686a in #2774
• fix: vint buffer can overflow by @rebasedming in #2778
• fix: boolean query incorrectly dropping documents when AllScorer is present by @mdashti in #2760
• fix: fixed integer overflow in ExpUnrolledLinkedList for large datasets by @mdashti in #2735
• Use a strict comparison in TopNComputer by @stuhood in #2777
• Add support for natural-order-with-none-highest in TopDocs::order_by by @stuhood in #2780
• optimize RangeDocSet for non-overlapping query ranges by @ChangRui-Ryan in #2783
• add seek_into_the_danger_zone on DocSet by @PSeitz in #2538
• add benchmark for boolean query with range sub query by @ChangRui-Ryan in #2787
• Addressing clippy comments by @fulmicoton in #2789
• Add an erased SortKeyComputer to sort on types which are not known until runtime by @stuhood in #2770
• document 1bit hole and some queries supporting running with just fastfield by @trinity-1686a in #2779
• fix flaky test by @PSeitz in #2784
• fix union performance regression by @PSeitz in #2790
• Making stemming optional. by @fulmicoton in #2791
• fix Column.first method parameter type by @ChangRui-Ryan in #2792
• one collector per agg request instead per bucket by @PSeitz-dd in #2759
• Bumped bitpacking version by @fulmicoton in #2797
• Minor refactoring in PostingsSerializer by @fulmicoton in #2801
• Bump lru to 0.16.3 per dependabot by @jollygreenlaser in #2799
• upgrade some dependancies by @trinity-1686a in #2802
• add benchmark for string search and get by @ChangRui-Ryan in #2795
• Stuhood.intersection termination upstream b by @fulmicoton in #2810
• Bugfix on intersection. by @fulmicoton in #2812
• Remove temp file by @fulmicoton in #2815
• Fix closing parenthesis on lenient parser by @evance-br in #2816
• Bump oneshot to 0.1.13 per dependabot by @jollygreenlaser in #2821
• Fix rfc3339 typos and add Claude Code skills by @fulmicoton-dd in #2823
• faster exclude queries by @PSeitz in #2825
• Add regex search and merge segments benchmark by @PSeitz in #2826
• add intermediate result accessors by @congx4 in #2829
• feat: add fast field support for Bytes type by @mdashti in #2830
• Export fields of the PercentileValuesVecEntry by @mdumandag in #2833
• fix(query-grammar): Fix regexes between parentheses by @Darkheir in #2818
• make serializer pub by @PSeitz in #2835
• Replace hyperloglogplus with Apache DataSketches HLL (lg_k=11) by @congx4 in #2837
• Use sketches-ddsketch fork with Java-compatible binary encoding by @congx4 in #2842
• Fixed integer overflow in segment sorting and merge policy truncation by @anaslimem in #2846
• Update time deps to defensively address a vulnerability. by @fulmicoton in #2850
• Fix format by @fulmicoton in #2852
• Composite agg merge by @fulmicoton in #2856
• fix: deduplicate doc counts in term aggregation for multi-valued fields by @nuri-yoo in #2854
• update CHANGELOG for tantivy 0.26 release by @PSeitz in #2857
• Fix clippy warnings: deprecated gen_range, manual div_ceil, legacy import in #2860
• Improve Union Performance for non-score unions by @PSeitz in #2863
• Update binggan requirement from 0.14.2 to 0.15.3 by @dependabot[bot] in #2870

New Contributors

• @rdettai-sk made their first contribution in #2696
• @msmoiz made their first contribution in #2704
• @Platane made their first contribution in #2732
• @mdashti made their first contribution in #2711
• @rustmailer made their first contribution in #2743
• @rebasedming made their first contribution in #2778
• @ChangRui-Ryan made their first contribution in #2783
• @jollygreenlaser made their first contribution in #2799
• @evance-br made their first contribution in #2816
• @congx4 made their first contribution in #2829
• @mdumandag made their first contribution in #2833
• @anaslimem made their first contribution in #2846

Full Changelog: 0.25.0...0.26.1

Performance

• Fix quadratic runtime in nested term and composite aggregations: memory accounting scanned all parent buckets on every collect instead of just the current parent (@PSeitz @fulmicoton)

Bugfixes

• Fix memory consumption accounting in nested term aggregation to only scan the active parent bucket (@PSeitz)
• Fix memory consumption accounting in composite aggregation to only scan the active parent bucket (@PSeitz)

• Fix TopNComputer for reverse order. #2672(@stuhood @PSeitz)

Affected queries are order_by_fast_field and order_by_u64_field for Order::Asc

Tantivy 0.24.1

• fix: Set rust-version to 1.81

Tantivy 0.24

Tantivy 0.24 will be backwards compatible with indices created with v0.22 and v0.21. The new minimum rust version will be 1.75. Tantivy 0.23 will be skipped.

Bugfixes

• fix potential endless loop in merge #2457(@PSeitz)
• fix bug that causes out-of-order sstable key. #2445(@fulmicoton)
• fix ReferenceValue API flaw #2372(@PSeitz)
• fix OwnedBytes debug panic #2512(@b41sh)
• catch panics during merges #2582(@rdettai)
• switch from u32 to usize in bitpacker. This enables multivalued columns larger than 4GB, which crashed during merge before. #2581 #2586(@fulmicoton-dd @PSeitz)

Breaking API Changes

• remove index sorting #2434(@PSeitz)

Features/Improvements

• Aggregation

  • Support for cardinality aggregation #2337 #2446 (@raphaelcoeffic @PSeitz)
  • Support for extended stats aggregation #2247(@giovannicuccu)
  • Add Key::I64 and Key::U64 variants in aggregation to avoid f64 precision issues #2468(@PSeitz)
  • Faster term aggregation fetch terms #2447(@PSeitz)
  • Improve custom order deserialization #2451(@PSeitz)
  • Change AggregationLimits behavior #2495(@PSeitz)
  • lower contention on AggregationLimits #2394(@PSeitz)
  • fix postcard compatibility for top_hits, add postcard test #2346(@PSeitz)
  • reduce top hits memory consumption #2426(@PSeitz)
  • check unsupported parameters top_hits #2351(@PSeitz)
  • Change AggregationLimits to AggregationLimitsGuard #2495(@PSeitz)
  • add support for counting non integer in aggregation #2547(@trinity-1686a)

• Range Queries

  • Support fast field range queries on json fields #2456(@PSeitz)
  • Add support for str fast field range query #2460 #2452 #2453(@PSeitz)
  • modify fastfield range query heuristic #2375(@trinity-1686a)
  • add FastFieldRangeQuery for explicit range queries on fast field (for RangeQuery it is autodetected) #2477(@PSeitz)

• add format backwards-compatibility tests #2485(@PSeitz)

• add columnar format compatibility tests #2433(@PSeitz)

• Improved snippet ranges algorithm #2474(@gezihuzi)

• make find_field_with_default return json fields without path #2476(@trinity-1686a)

• Make BooleanQuery support minimum_number_should_match #2405(@LebranceBW)

• Make NUM_MERGE_THREADS configurable #2535(@Barre)

• RegexPhraseQuery
  RegexPhraseQuery supports phrase queries with regex. E.g. query "b.* b.* wolf" matches "big bad wolf". Slop is supported as well: "b.* wolf"~2 matches "big bad wolf" #2516(@PSeitz)

• Optional Index in Multivalue Columnar Index
  For mostly empty multivalued indices there was a large overhead during creation when iterating all docids (merge case).
  This is alleviated by placing an optional index in the multivalued index to mark documents that have values.
  This will slightly increase space and access time. #2439(@PSeitz)

• Store DateTime as nanoseconds in doc store DateTime in the doc store was truncated to microseconds previously. This removes this truncation, while still keeping backwards compatibility. #2486(@PSeitz)

• Performace/Memory

  • lift clauses in LogicalAst for optimized ast during execution #2449(@PSeitz)
  • Use Vec instead of BTreeMap to back OwnedValue object #2364(@fulmicoton)
  • Replace TantivyDocument with CompactDoc. CompactDoc is much smaller and provides similar performance. #2402(@PSeitz)
  • Recycling buffer in PrefixPhraseScorer #2443(@fulmicoton)

• Json Type

  • JSON supports now all values on the root level. Previously an object was required. This enables support for flat mixed types. allow more JSON values, fix i64 special case #2383(@PSeitz)
  • add json path constructor to term #2367(@PSeitz)

• QueryParser

  • fix de-escaping too much in query parser #2427(@trinity-1686a)
  • improve query parser #2416(@trinity-1686a)
  • Support field grouping title:(return AND "pink panther") #2333(@trinity-1686a)
  • allow term starting with wildcard #2568(@trinity-1686a)

• Exist queries match subpath fields #2558(@rdettai)

• add access benchmark for columnar #2432(@PSeitz)

• extend indexwriter proptests #2342(@PSeitz)

• add bench & test for columnar merging #2428(@PSeitz)

• Change in Executor API #2391(@fulmicoton)

• Removed usage of num_cpus #2387(@fulmicoton)

• use bingang for agg and stacker benchmark #2378#2492(@PSeitz)

• cleanup top level exports #2382(@PSeitz)

• make convert_to_fast_value_and_append_to_json_term pub #2370(@PSeitz)

• remove JsonTermWriter #2238(@PSeitz)

• validate sort by field type #2336(@PSeitz)

• Fix trait bound of StoreReader::iter #2360(@adamreichold)

• remove read_postings_no_deletes #2526(@PSeitz)

• Fix: bump required rust version to 1.81

• Fix TopNComputer for reverse order. #2672(@stuhood @PSeitz)

Affected queries are order_by_fast_field and order_by_u64_field for Order::Asc

Bugfixes

• Range queries on fast fields with less values on that field than documents had an invalid end condition, leading to missing results. #2226(@appaquet @PSeitz)
• Increase the minimum memory budget from 3MB to 15MB to avoid single doc segments (API fix). #2176(@PSeitz)

• Align numerical type priority order on the search side. #2088 (@fmassot)
• Fix is_child_of function not considering the root facet. #2086 (@adamreichhold)

• Fix building on windows with mmap #2070 (@ChillFish8)

• Hotfix: positions computation. #1629 (@fmassot, @fulmicoton, @PSeitz)

• Major Bugfix on multivalued fastfield. #1151
• Demux operation (@PSeitz)

• Bugfix in the filesum check. (@evanxg852000) #1127
• Bugfix in positions when the index is sorted by a field. (@appaquet) #1125

• Major bugfix. Deleting documents was broken when the index was sorted by a field. (@appaquet, @fulmicoton) #1101

• Major bugfix. DocStore still panics when a deleted doc is at the beginning of a block. (@appaquet) #1088

• Major bugfix. DocStore panics when first block is deleted. (@appaquet) #1077

• API Changes. Using Range instead of (start, end) in the API and internals (FileSlice, OwnedBytes, Snippets, ...) This change is breaking but migration is trivial.
• Added an Histogram collector. (@fulmicoton) #994
• Added support for Option<TCollector>. (@fulmicoton)
• DocAddress is now a struct (@scampi) #987
• Bugfix consistent tie break handling in facet's topk (@hardikpnsp) #357
• Date field support for range queries (@rihardsk) #516
• Added lz4-flex as the default compression scheme in tantivy (@PSeitz) #1009
• Renamed a lot of symbols to avoid all uppercasing on acronyms, as per new clippy recommendation. For instance, RAMDirectory -> RamDirectory. (@fulmicoton)
• Simplified positions index format (@fulmicoton) #1022
• Moved bitpacking to bitpacker subcrate and add BlockedBitpacker, which bitpacks blocks of 128 elements (@PSeitz) #1030
• Added support for more-like-this query in tantivy (@evanxg852000) #1011
• Added support for sorting an index, e.g presorting documents in an index by a timestamp field. This can heavily improve performance for certain scenarios, by utilizing the sorted data (Top-n optimizations)(@PSeitz). #1026
• Add iterator over documents in doc store (@PSeitz). #1044
• Fix log merge policy (@PSeitz). #1043
• Add detection to avoid small doc store blocks on merge (@PSeitz). #1054
• Make doc store compression dynamic (@PSeitz). #1060
• Switch to json for footer version handling (@PSeitz). #1060
• Updated TermMerger implementation to rely on the union feature of the FST (@scampi) #469
• Add boolean marking whether position is required in the query_terms API call (@fulmicoton). #1070

• Remove dependency to atomicwrites #833 .Implemented by @fulmicoton upon suggestion and research from @asafigan).
• Migrated tantivy error from the now deprecated failure crate to thiserror #760. (@hirevo)
• API Change. Accessing the typed value off a Schema::Value now returns an Option instead of panicking if the type does not match.
• Large API Change in the Directory API. Tantivy used to assume that all files could be somehow memory mapped. After this change, Directory return a FileSlice that can be reduced and eventually read into an OwnedBytes object. Long and blocking io operation are still required by they do not span over the entire file.
• Added support for Brotli compression in the DocStore. (@ppodolsky)
• Added helper for building intersections and unions in BooleanQuery (@guilload)
• Bugfix in Query::explain
• Removed dependency on notify #924. Replaced with FileWatcher struct that polls meta file every 500ms in background thread. (@halvorboe @guilload)
• Added FilterCollector, which wraps another collector and filters docs using a predicate over a fast field (@barrotsteindev)
• Simplified the encoding of the skip reader struct. BlockWAND max tf is now encoded over a single byte. (@fulmicoton)
• FilterCollector now supports all Fast Field value types (@barrotsteindev)
• FastField are not all loaded when opening the segment reader. (@fulmicoton)
• Added an API to merge segments, see tantivy::merge_segments #1005. (@evanxg852000)

This version breaks compatibility and requires users to reindex everything.

Bugfix. Acquiring a facet reader on a segment that does not contain any doc with this facet returns None. (#896)

Made Query and Collector Send + Sync. Updated misc dependency versions.

Tantivy 0.13 introduce a change in the index format that will require you to reindex your index (BlockWAND information are added in the skiplist). The index size increase is minor as this information is only added for full blocks. If you have a massive index for which reindexing is not an option, please contact me so that we can discuss possible solutions.

• Bugfix in FuzzyTermQuery not matching terms by prefix when it should (@Peachball)
• Relaxed constraints on the custom/tweak score functions. At the segment level, they can be mut, and they are not required to be Sync + Send.
• MMapDirectory::open does not return a Result anymore.
• Change in the DocSet and Scorer API. (@fulmicoton). A freshly created DocSet point directly to their first doc. A sentinel value called TERMINATED marks the end of a DocSet. .advance() returns the new DocId. Scorer::skip(target) has been replaced by Scorer::seek(target) and returns the resulting DocId. As a result, iterating through DocSet now looks as follows

let mut doc = docset.doc();
while doc != TERMINATED {
   // ...
   doc = docset.advance();
}

The change made it possible to greatly simplify a lot of the docset's code.

• Misc internal optimization and introduction of the Scorer::for_each_pruning function. (@fulmicoton)
• Added an offset option to the Top(.*)Collectors. (@robyoung)
• Added Block WAND. Performance on TOP-K on term-unions should be greatly increased. (@fulmicoton, and special thanks to the PISA team for answering all my questions!)

• Removing static dispatch in tokenizers for simplicity. (#762)
• Added backward iteration for TermDictionary stream. (@halvorboe)
• Fixed a performance issue when searching for the posting lists of a missing term (@audunhalland)
• Added a configurable maximum number of docs (10M by default) for a segment to be considered for merge (@hntd187, landed by @halvorboe #713)
• Important Bugfix #777, causing tantivy to retain memory mapping. (diagnosed by @poljar)
• Added support for field boosting. (#547, @fulmicoton)

How to update?

Crates relying on custom tokenizer, or registering tokenizer in the manager will require some minor changes. Check https://github.com/quickwit-oss/tantivy/blob/main/examples/custom_tokenizer.rs to check for some code sample.

• Fixed DateTime as a fast field (#735)

• The future returned by IndexWriter::merge does not borrow self mutably anymore (#732)
• Exposing a constructor for WatchHandle (#731)

• Bug fix #729

• Added f64 field. Internally reuse u64 code the same way i64 does (@fdb-hiroshima)
• Various bugfixes in the query parser.
  • Better handling of hyphens in query parser. (#609)
  • Better handling of whitespaces.
• Closes #498 - add support for Elastic-style unbounded range queries for alphanumeric types eg. "title:>hello", "weight:>=70.5", "height:<200" (@petr-tik)
• API change around Box<BoxableTokenizer>. See detail in #629
• Avoid rebuilding Regex automaton whenever a regex query is reused. #639 (@brainlock)
• Add footer with some metadata to index files. #605 (@fdb-hiroshima)
• Add a method to check the compatibility of the footer in the index with the running version of tantivy (@petr-tik)
• TopDocs collector: ensure stable sorting on equal score. #671 (@brainlock)
• Added handling of pre-tokenized text fields (#642), which will enable users to load tokens created outside tantivy. See usage in examples/pre_tokenized_text. (@kkoziara)
• Fix crash when committing multiple times with deleted documents. #681 (@brainlock)

How to update?

• The index format is changed. You are required to reindex your data to use tantivy 0.11.
• Box<dyn BoxableTokenizer> has been replaced by a BoxedTokenizer struct.
• Regex are now compiled when the RegexQuery instance is built. As a result, it can now return an error and handling the Result is required.
• tantivy::version() now returns a Version object. This object implements ToString()

• Closes #656. Solving memory leak.

• Closes #544. A few users experienced problems with the directory watching system. Avoid watching the mmap directory until someone effectively creates a reader that uses this functionality.

Tantivy 0.10.0 index format is compatible with the index format in 0.9.0.

• Added an API to easily tweak or entirely replace the default score. See TopDocs::tweak_scoreand TopScore::custom_score (@fulmicoton)
• Added an ASCII folding filter (@drusellers)
• Bugfix in query.count in presence of deletes (@fulmicoton)
• Added .explain(...) in Query and Weight to (@fulmicoton)
• Added an efficient way to delete_all_documents in IndexWriter (@petr-tik). All segments are simply removed.

Minor

• Switched to Rust 2018 (@uvd)
• Small simplification of the code. Calling .freq() or .doc() when .advance() has never been called on segment postings should panic from now on.
• Tokens exceeding u16::max_value() - 4 chars are discarded silently instead of panicking.
• Fast fields are now preloaded when the SegmentReader is created.
• IndexMeta is now public. (@hntd187)
• IndexWriter add_document, delete_term. IndexWriter is Sync, making it possible to use it with a Arc<RwLock<IndexWriter>>. add_document and delete_term can only require a read lock. (@fulmicoton)
• Introducing Opstamp as an expressive type alias for u64. (@petr-tik)
• Stamper now relies on AtomicU64 on all platforms (@petr-tik)
• Bugfix - Files get deleted slightly earlier
• Compilation resources improved (@fdb-hiroshima)

How to update?

Your program should be usable as is.

Fast fields

Fast fields used to be accessed directly from the SegmentReader. The API changed, you are now required to acquire your fast field reader via the segment_reader.fast_fields(), and use one of the typed method:

• .u64(), .i64() if your field is single-valued ;
• .u64s(), .i64s() if your field is multi-valued ;
• .bytes() if your field is bytes fast field.

0.9.0 index format is not compatible with the previous index format.

• MAJOR BUGFIX : Some Mmap objects were being leaked, and would never get released. (@fulmicoton)
• Removed most unsafe (@fulmicoton)
• Indexer memory footprint improved. (VInt comp, inlining the first block. (@fulmicoton)
• Stemming in other language possible (@pentlander)
• Segments with no docs are deleted earlier (@barrotsteindev)
• Added grouped add and delete operations. They are guaranteed to happen together (i.e. they cannot be split by a commit). In addition, adds are guaranteed to happen on the same segment. (@elbow-jason)
• Removed INT_STORED and INT_INDEXED. It is now possible to use STORED and INDEXED for int fields. (@fulmicoton)
• Added DateTime field (@barrotsteindev)
• Added IndexReader. By default, index is reloaded automatically upon new commits (@fulmicoton)
• SIMD linear search within blocks (@fulmicoton)

How to update ?

tantivy 0.9 brought some API breaking change. To update from tantivy 0.8, you will need to go through the following steps.

• schema::INT_INDEXED and schema::INT_STORED should be replaced by schema::INDEXED and schema::INT_STORED.

• The index now does not hold the pool of searcher anymore. You are required to create an intermediary object called IndexReader for this.

  // create the reader. You typically need to create 1 reader for the entire
  // lifetime of you program.
  let reader = index.reader()?;
  
  // Acquire a searcher (previously `index.searcher()`) is now written:
  let searcher = reader.searcher();
  
  // With the default setting of the reader, you are not required to
  // call `index.load_searchers()` anymore.
  //
  // The IndexReader will pick up that change automatically, regardless
  // of whether the update was done in a different process or not.
  // If this behavior is not wanted, you can create your reader with
  // the `ReloadPolicy::Manual`, and manually decide when to reload the index
  // by calling `reader.reload()?`.
  
  

Fixing build for x86_64 platforms. (#496) No need to update from 0.8.1 if tantivy is building on your platform.

Hotfix of #476.

Merge was reflecting deletes before commit was passed. Thanks @barrotsteindev for reporting the bug.

No change in the index format

• API Breaking change in the collector API. (@jwolfe, @fulmicoton)
• Multithreaded search (@jwolfe, @fulmicoton)

No change in the index format

• Bugfix: NGramTokenizer panics on non ascii chars
• Added a space usage API

• Bugfix #324. GC removing was removing file that were still in useful
• Added support for parsing AllQuery and RangeQuery via QueryParser
  • AllQuery: *
  • RangeQuery:
    • Inclusive field:[startIncl to endIncl]
    • Exclusive field:{startExcl to endExcl}
    • Mixed field:[startIncl to endExcl} and vice versa
    • Unbounded field:[start to *], field:[* to end]

• bugfix #274
• bugfix #280
• bugfix #289

• bugfix #254 : tantivy failed if no documents in a segment contained a specific field.

• Bugfix race condition when deleting files. (#198)

• Prevent usage of AVX2 instructions (#201)

• Bugfix for non-indexed fields. (#199)

• Raise the limit of number of fields (previously 256 fields) (@fulmicoton)
• Removed u32 fields. They are replaced by u64 and i64 fields (#65) (@fulmicoton)
• Optimized skip in SegmentPostings (#130) (@lnicola)
• Replacing rustc_serialize by serde. Kudos to benchmark@KodrAus and @lnicola
• Using error-chain (@KodrAus)
• QueryParser: (@fulmicoton)
  • Explicit error returned when searched for a term that is not indexed
  • Searching for a int term via the query parser was broken (age:1)
  • Searching for a non-indexed field returns an explicit Error
  • Phrase query for non-tokenized field are not tokenized by the query parser.
• Faster/Better indexing (@fulmicoton)
  • using murmurhash2
  • faster merging
  • more memory efficient fast field writer (@lnicola )
  • better handling of collisions
  • lesser memory usage
• Added API, most notably to iterate over ranges of terms (@fulmicoton)
• Bugfix that was preventing to unmap segment files, on index drop (@fulmicoton)
• Made the doc! macro public (@fulmicoton)
• Added an alternative implementation of the streaming dictionary (@fulmicoton)

• Expose a method to trigger files garbage collection