ownedbytes

0.9.0 index format is not compatible with the previous index format.

• MAJOR BUGFIX : Some Mmap objects were being leaked, and would never get released. (@fulmicoton)
• Removed most unsafe (@fulmicoton)
• Indexer memory footprint improved. (VInt comp, inlining the first block. (@fulmicoton)
• Stemming in other language possible (@pentlander)
• Segments with no docs are deleted earlier (@barrotsteindev)
• Added grouped add and delete operations. They are guaranteed to happen together (i.e. they cannot be split by a commit). In addition, adds are guaranteed to happen on the same segment. (@elbow-jason)
• Removed INT_STORED and INT_INDEXED. It is now possible to use STORED and INDEXED for int fields. (@fulmicoton)
• Added DateTime field (@barrotsteindev)
• Added IndexReader. By default, index is reloaded automatically upon new commits (@fulmicoton)
• SIMD linear search within blocks (@fulmicoton)

How to update ?

tantivy 0.9 brought some API breaking change. To update from tantivy 0.8, you will need to go through the following steps.

• schema::INT_INDEXED and schema::INT_STORED should be replaced by schema::INDEXED and schema::INT_STORED.

• The index now does not hold the pool of searcher anymore. You are required to create an intermediary object called IndexReader for this.

  // create the reader. You typically need to create 1 reader for the entire
  // lifetime of you program.
  let reader = index.reader()?;
  
  // Acquire a searcher (previously `index.searcher()`) is now written:
  let searcher = reader.searcher();
  
  // With the default setting of the reader, you are not required to
  // call `index.load_searchers()` anymore.
  //
  // The IndexReader will pick up that change automatically, regardless
  // of whether the update was done in a different process or not.
  // If this behavior is not wanted, you can create your reader with
  // the `ReloadPolicy::Manual`, and manually decide when to reload the index
  // by calling `reader.reload()?`.
  
  

• Raise the limit of number of fields (previously 256 fields) (@fulmicoton)
• Removed u32 fields. They are replaced by u64 and i64 fields (#65) (@fulmicoton)
• Optimized skip in SegmentPostings (#130) (@lnicola)
• Replacing rustc_serialize by serde. Kudos to benchmark@KodrAus and @lnicola
• Using error-chain (@KodrAus)
• QueryParser: (@fulmicoton)
  • Explicit error returned when searched for a term that is not indexed
  • Searching for a int term via the query parser was broken (age:1)
  • Searching for a non-indexed field returns an explicit Error
  • Phrase query for non-tokenized field are not tokenized by the query parser.
• Faster/Better indexing (@fulmicoton)
  • using murmurhash2
  • faster merging
  • more memory efficient fast field writer (@lnicola )
  • better handling of collisions
  • lesser memory usage
• Added API, most notably to iterate over ranges of terms (@fulmicoton)
• Bugfix that was preventing to unmap segment files, on index drop (@fulmicoton)
• Made the doc! macro public (@fulmicoton)
• Added an alternative implementation of the streaming dictionary (@fulmicoton)