fancy-regex

Added

• Add BytesMode and the RegexInput trait so the matching and capture APIs can operate on strings or bytes, and allows to opt out of unicode handling if desired (#248)
  • Also allows searching within a range without slicing (#253)
  • Assertions can be overridden at runtime, if the regex builder allow_input_assertion_overrides option was used (#254)
  • Search can be performed in anchored mode, searching only at the start position specified (#263)
• Add experimental seek optimization to only invoke the VM at candidate positions where a match could occur (#246)
• Add RegexBuilder::disallow_empty_match_at_eof_after_newline to reject empty matches at the end of the haystack following a trailing newline, to match Oniguruma behavior (#247)
• Add RegexSet API for efficiently matching multiple patterns against the same text (#255)

Changed

• Case-insensitive backreference comparison of non-ASCII text now uses direct Unicode simple case folding instead of building a regex engine per comparison, which makes patterns like (?i)(\w+)\1 orders of magnitude faster on non-ASCII haystacks. The comparison is now strict fold equality over the captured text's byte-length window: a case-folded prefix of the window no longer counts as a match (previously a substring search could accept one and misalign the match end)
• Compilation performance: delegated regex-automata engines (including the whole pattern in the easy case) are now built from a directly-constructed Hir instead of a re-serialized pattern string, and RegexSet parses each pattern once for both of its set-wide engines instead of three times. This reduces compile time and peak compile memory, especially for RegexSet and small easy patterns
• Match performance: the backtracking VM's working memory (saves, backtrack stack, delegate slots) is pooled and reused across runs, making is_match/find/find_iter and RegexSet candidate verification allocation-free per call; RegexSet::find_input no longer allocates at failed candidate positions. Short and repeated matches improve substantially (up to ~30% for small patterns, ~10% for find_iter over many matches); single very long backtracking runs, where per-run setup is fully amortized anyway, can be a few percent slower
• Matches, CaptureMatches, Captures, and SubCaptureMatches are now generic over RegexInput, which is a breaking change for code that named these types explicitly (#248)
• Patterns no longer force Unicode mode during parsing, and inline (?u) / (?-u) flags are accepted when they agree with the builder configuration

Fixed

• Fix bug whereby inline flags were not overriding the builder options (#247)
• Fix bug whereby \G optimizations were applying where they shouldn't, giving wrong results (#256)
• Support Oniguruma quantifier parsing rules, whereby swapped ordering causes possessiveness, + only causes possessiveness after ?, * or + (#258)

Upgrade guide

If you previously stored i.e. Captures, you would need to change the type to Captures<str> to get the code to compile.

Added

• Add BytesMode and the RegexInput trait so the matching and capture APIs can operate on strings or bytes, and allows to opt out of unicode handling if desired (#248)
  • Also allows searching within a range without slicing (#253)
  • Assertions can be overridden at runtime, if the regex builder allow_input_assertion_overrides option was used (#254)
  • Search can be performed in anchored mode, searching only at the start position specified (#263)
• Add experimental seek optimization to only invoke the VM at candidate positions where a match could occur (#246)
• Add RegexBuilder::disallow_empty_match_at_eof_after_newline to reject empty matches at the end of the haystack following a trailing newline, to match Oniguruma behavior (#247)
• Add RegexSet API for efficiently matching multiple patterns against the same text (#255)

Changed

• Case-insensitive backreference comparison of non-ASCII text now uses direct Unicode simple case folding instead of building a regex engine per comparison, which makes patterns like (?i)(\w+)\1 orders of magnitude faster on non-ASCII haystacks. The comparison is now strict fold equality over the captured text's byte-length window: a case-folded prefix of the window no longer counts as a match (previously a substring search could accept one and misalign the match end)
• Compilation performance: delegated regex-automata engines (including the whole pattern in the easy case) are now built from a directly-constructed Hir instead of a re-serialized pattern string, and RegexSet parses each pattern once for both of its set-wide engines instead of three times. This reduces compile time and peak compile memory, especially for RegexSet and small easy patterns
• Match performance: the backtracking VM's working memory (saves, backtrack stack, delegate slots) is pooled and reused across runs, making is_match/find/find_iter and RegexSet candidate verification allocation-free per call; RegexSet::find_input no longer allocates at failed candidate positions. Short and repeated matches improve substantially (up to ~30% for small patterns, ~10% for find_iter over many matches); single very long backtracking runs, where per-run setup is fully amortized anyway, can be a few percent slower
• Matches, CaptureMatches, Captures, and SubCaptureMatches are now generic over RegexInput, which is a breaking change for code that named these types explicitly (#248)
• Patterns no longer force Unicode mode during parsing, and inline (?u) / (?-u) flags are accepted when they agree with the builder configuration

Fixed

• Fix bug whereby inline flags were not overriding the builder options (#247)
• Fix bug whereby \G optimizations were applying where they shouldn't, giving wrong results (#256)
• Support Oniguruma quantifier parsing rules, whereby swapped ordering causes possessiveness, + only causes possessiveness after ?, * or + (#258)

Upgrade guide

If you previously stored i.e. Captures, you would need to change the type to Captures<str> to get the code to compile.

Added

• Add support for \N to mean any character except newline, for Oniguruma compatibility (#206)
• Add support for (*FAIL) backtracking control verb (#210)
• Add support for more \p{...} and \P{...} aliases, for Oniguruma compatibility (#207)
• Add support for word boundaries and zero-length fancy patterns inside variable lookbehinds (at top level) (#216)
• Add support for \R to mean general newline, matching all common line break characters, treating \r\n atomically (#220)
• Add support for the regex crate's (?R) CRLF mode flag (#238)
• Add support for subroutine calls (including recursion up to 20 levels deep, matching Oniguruma behavior) \g<1> (#230)
• Add support for Oniguruma's absent repeater (?~abc) (#233)
• Add support for ignoring empty matches (#240)
• Add support for treating unnamed capture groups as non-capturing when named groups exist (#241)

Changed

• RegexBuilder can now build multiple patterns with the same options (#213)
• Parsing of capture group names is now more lenient (#241)

Fixed

• Fixed bug with parsing nested character classes containing unescaped ] (#211)

Added

• Add support for \N to mean any character except newline, for Oniguruma compatibility (#206)
• Add support for (*FAIL) backtracking control verb (#210)
• Add support for more \p{...} and \P{...} aliases, for Oniguruma compatibility (#207)
• Add support for word boundaries and zero-length fancy patterns inside variable lookbehinds (at top level) (#216)
• Add support for \R to mean general newline, matching all common line break characters, treating \r\n atomically (#220)
• Add support for the regex crate's (?R) CRLF mode flag (#238)
• Add support for subroutine calls (including recursion up to 20 levels deep, matching Oniguruma behavior) \g<1> (#230)
• Add support for Oniguruma's absent repeater (?~abc) (#233)
• Add support for ignoring empty matches (#240)
• Add support for treating unnamed capture groups as non-capturing when named groups exist (#241)

Changed

• RegexBuilder can now build multiple patterns with the same options (#213)
• Parsing of capture group names is now more lenient (#241)

Fixed

• Fixed bug with parsing nested character classes containing unescaped ] (#211)

Added

• Add support for "easy" variable-length lookbehinds (enabled via new feature flag) (#194, #196)
• Add support for more word boundaries: \b{start}, \b{end}, \b{start-half}, \b{end-half} (#193)
• Allow {...} repetition syntax after assertions (except in Oniguruma mode) (#193)
• Add Oniguruma mode flag to playground (#199)

Changed

• Rewrite Theory section of Readme in a more formal style (#187)
• Box CompileError in Error enum to reduce size (#204)
• Bail out early when \G fails to match for increased performance (#198)
• Handle ignore_whitespace case in parsing escape sequences (#193)
• Add tests to ensure that Regex is Send and Sync (#195)

Fixed

• Fix clippy lint warnings (#204)

Added

• Add support for "easy" variable-length lookbehinds (enabled via new feature flag) (#194, #196)
• Add support for more word boundaries: \b{start}, \b{end}, \b{start-half}, \b{end-half} (#193)
• Allow {...} repetition syntax after assertions (except in Oniguruma mode) (#193)
• Add Oniguruma mode flag to playground (#199)

Changed

• Rewrite Theory section of Readme in a more formal style (#187)
• Box CompileError in Error enum to reduce size (#204)
• Bail out early when \G fails to match for increased performance (#198)
• Handle ignore_whitespace case in parsing escape sequences (#193)
• Add tests to ensure that Regex is Send and Sync (#195)

Fixed

• Fix clippy lint warnings (#204)

Added

• Add an "oniguruma mode" flag to control whether \< and \> are treated as literals or word-boundary assertions. (#186)
• Add support for const-size backrefs in lookbehinds. (#182)

Changed

• A few small internal changes which allowed us to expose a web-based playground for fancy-regex. (#181)

Fixed

• Fix behavior of repetition on an empty match. (#179)
• Always return the original input pattern when as_str is called. (#185)
• Fix panic when matching non-ascii text as part of a case insensitive backref. (#189)
• The toy example wasn't always showing the correct analysis or compiled program when optimizations were applied. (#181)

Added

• Add an "oniguruma mode" flag to control whether \< and \> are treated as literals or word-boundary assertions. (#186)
• Add support for const-size backrefs in lookbehinds. (#182)

Changed

• A few small internal changes which allowed us to expose a web-based playground for fancy-regex. (#181)

Fixed

• Fix behavior of repetition on an empty match. (#179)
• Always return the original input pattern when as_str is called. (#185)
• Fix panic when matching non-ascii text as part of a case insensitive backref. (#189)
• The toy example wasn't always showing the correct analysis or compiled program when optimizations were applied. (#181)

Fixed

• Fixed a bug whereby sometimes a backreference to a non-existing capture group would compile successfully
  when it should fail, causing a panic in the VM when trying to match the regex. (#174)

Fixed

• Fixed a bug whereby sometimes a backreference to a non-existing capture group would compile successfully when it should fail, causing a panic in the VM when trying to match the regex. (#174)

Added

• Add an optimization step after the pattern is parsed but before it is analyzed.
  Currently it only optimizes one specific use-case - where the expression is easy except
  for a trailing positive lookahead whose contents are also easy. The optimization is to delegate to the regex crate instead of using the backtracking VM. (#171)

Changed

• Patterns which are anchored to the start of the text (i.e. with ^ when not in multiline mode) should now fail faster when there is no match, because fancy-regex no longer tries to match at other positions. (#170)
• The CompileError for an invalid (numbered) backref has been updated to mention which backref was invalid (#170)
• Removed dependency on derivative (#169)

Fixed

• Fixed a bug whereby sometimes a capture group containing a backref to itself would cause a compile error, when it is valid - this fixes a few Oniguruma test cases (#170)

Added

• Add an optimization step after the pattern is parsed but before it is analyzed. Currently it only optimizes one specific use-case - where the expression is easy except for a trailing positive lookahead whose contents are also easy. The optimization is to delegate to the regex crate instead of using the backtracking VM. (#171)

Changed

• Patterns which are anchored to the start of the text (i.e. with ^ when not in multiline mode) should now fail faster when there is no match, because fancy-regex no longer tries to match at other positions. (#170)
• The CompileError for an invalid (numbered) backref has been updated to mention which backref was invalid (#170)
• Removed dependency on derivative (#169)

Fixed

• Fixed a bug whereby sometimes a capture group containing a backref to itself would cause a compile error, when it is valid - this fixes a few Oniguruma test cases (#170)

Added

• Support \Z - anchor to the end of the text before any trailing newlines. (#148)
• Support \O - any character including newlines. (#158)
• The parser can now parse subroutine calls and relative backreferences (but execution is still unsupported). This is preparation for future work. Some new error variants have been added for features which can be parsed but are still otherwise unsupported.
• Backreferences can now be case insensitive. (#160)
• RegexBuilder: Add options for multi_line, ignore_whitespace, dot_matches_new_line (#165)

Fixed

• Fix infinite loop when backtracking limit is hit (#153)
• Fix RegexBuilder.case_insensitive not always applying when it should. (#163)
• The toy example has had various bugfixes, and unit tests added. (#152, #159)

Added

• Support \Z - anchor to the end of the text before any trailing newlines. (#148)
• Support \O - any character including newlines. (#158)
• The parser can now parse subroutine calls and relative backreferences (but execution is still unsupported). This is preparation for future work. Some new error variants have been added for features which can be parsed but are still otherwise unsupported.
• Backreferences can now be case insensitive. (#160)
• RegexBuilder: Add options for multi_line, ignore_whitespace, dot_matches_new_line (#165)

Fixed

• Fix infinite loop when backtracking limit is hit (#153)
• Fix RegexBuilder.case_insensitive not always applying when it should. (#163)
• The toy example has had various bugfixes, and unit tests added. (#152, #159)

Added

• Add split, splitn methods to Regex to split a string into substrings (#140)
• Add case_insensitive method to RegexBuilder to force case-insensitive mode (#132)

Changed

• Bump bit-set dependency to 0.8 (#139)

Added

• Add split, splitn methods to Regex to split a string into substrings (#140)
• Add case_insensitive method to RegexBuilder to force case-insensitive mode (#132)

Changed

• Bump bit-set dependency to 0.8 (#139)

Added

• Support for relative backreferences using \k<-1> (-1 references the
  previous group) (#121)
• Add try_replacen to Regex which returns a Result instead of panicking
  when matching errors (#130)

Changed

• Switch from regex crate to regex-automata and regex-syntax (lower level APIs)
  to simplify internals (#121)
• Note: Due to above change, more backtracking is done in fancy-regex itself
  instead of regex-automata, and you might get a BacktrackLimitExceeded with
  some patterns that you didn't get before. You can increase the backtrack limit
  using RegexBuilder::backtrack_limit to help with that.
• Allow escaping some letters in character classes, e.g. [\A] used to error
  but now matches the same as [A] (for compatibility with Oniguruma)
• MSRV (minimum supported Rust version) is now 1.66.1 (from 1.61.0)

Fixed

• Fix index out of bounds panic when parsing unclosed (?( (#125)

Added

• Support for relative backreferences using \k<-1> (-1 references the previous group) (#121)
• Add try_replacen to Regex which returns a Result instead of panicking when matching errors (#130)

Changed

• Switch from regex crate to regex-automata and regex-syntax (lower level APIs) to simplify internals (#121)
• Note: Due to above change, more backtracking is done in fancy-regex itself instead of regex-automata, and you might get a BacktrackLimitExceeded with some patterns that you didn't get before. You can increase the backtrack limit using RegexBuilder::backtrack_limit to help with that.
• Allow escaping some letters in character classes, e.g. [\A] used to error but now matches the same as [A] (for compatibility with Oniguruma)
• MSRV (minimum supported Rust version) is now 1.66.1 (from 1.61.0)

Fixed

• Fix index out of bounds panic when parsing unclosed (?( (#125)

Added

• Support for no_std (the std feature is enabled by default but can be
  disabled if desired) (#111)
• TryFrom &str and String impl for Regex (#115)

Changed

• Error and its components are now Clone (#116)
• MSRV (minimum supported Rust version) is now 1.61.0 (from 1.42.0)

Added

• Support for no_std (the std feature is enabled by default but can be disabled if desired) (#111)
• TryFrom &str and String impl for Regex (#115)

Changed

• Error and its components are now Clone (#116)
• MSRV (minimum supported Rust version) is now 1.61.0 (from 1.42.0)

Added

• Support for conditionals: using a regex like (?<test>a)?b(?(test)c|d) will try to match c after b if a matched in the capture group named test, otherwise d after b if a wasn't captured into the test group.

Changed

• Updated parse errors to show the position they occurred at.

Fixed

• Fix panic when backref is used within referenced group itself and group end index is not known yet (#103)

Added

• Support for \G (anchor to end of previous match): Using a regex like \G\w will match each letter of foo in foo bar but nothing else.

Added

• Support for \K (keep out): Using a regex like @\K\w+ will match things like @foo but the resulting match text will only include foo, keeping out the @.

Added

• Allow users to disable any of the unicode and perf-* features of the regex crate. Disabling these features can reduce compile time and/or binary size for use cases where these features are not needed. (All features remain enabled by default.)

Changed

• MSRV (minimum supported Rust version) is now 1.42.0 (from 1.41.1)

Fixed

• Fix panic on incomplete escape sequences in input regexes
• Disallow quantifers on lookarounds and other zero-width assertion expressions, e.g. the + in (?=hello)+

Added

• Regex now has replace methods like the regex crate:
  • replace - single replacement
  • replace_all - replace all non-overlapping matches
  • replacen - configurable number of replacements

Added

• Regex now implements Clone, Display, FromStr
• Captures now implements Index<usize> to access captures by number and Index<&str> to access by name

Added

• Methods find_iter and captures_iter to iterate over all non-overlapping matches for a string
• Method find_from_pos to find starting from a specific position

Changed

• MSRV (minimum supported Rust version) is now 1.41.1 (from 1.32.0)

Added

• escape function to escape special characters in a string so that it matches literally

Added

• Support for named groups and backrefs:
  • Capture with (?<name>...) or (?P<name>...)
  • Backref with \k<name> or (?P=name)
  • Captures::name to get matched group by name
  • Regex::capture_names to get capture names in regex
• Support for expanding matches using a replacement template string
  • Captures::expand for regex crate compatible syntax
  • See Expander for python-compatible syntax and advanced usage
• Match::range and some From impls for convenience

Changed

• Include string snippet in errors for unknown group and invalid escape to make it easier to identify the problem.

Added

• Support comments using (?# comment) syntax
• Support unicode escapes like \u21D2 and \U0001F60A

Changed

• Optimization: Delegate const-sized suffixes in more cases
• Optimization: Use captures_read_at when delegating to regex crate

Fixed

• Some regexes with fancy parts in the beginning/middle didn't match when they should have, e.g. ((?!x)(a|ab))c didn't match abc.

Added

• Add delegate_size_limit and delegate_dfa_size_limit to RegexBuilder to allow configuring these limits for regex crate.

Added

• Add limit for backtracking so that execution errors instead of running for a long time in case of catastrophic backtracking.
• Add RegexBuilder with backtrack_limit to configure the new backtrack limit per regex.
• Error now implements std::error::Error trait

Fixed

• Fix panic in backref matching with multibyte chars

Added

• More documentation and examples
• Support character class nesting and intersections (implemented in regex crate)
• Support atomic groups, both the the (?>foo) group syntax and the a++, a*+ and a?+ possessive syntax
• Support \b, \f, \t, \n, \r, \v
• Support look-behind with variable sized alternative
• Implement Debug for Regex
• More test coverage including running one of Oniguruma's test suites

Changed

• Change find to return a Match struct (breaking change)
• Change Captures API (breaking change):
  • Replace at and pos with get that returns a Match struct
  • Remove is_empty (use len)
• Allow unescaped ] and } as literals
• Allow unescaped { as literal when not after atom
• Allow escapes such as \< or \e inside character classes
• Allow up to 8 characters in \x{...} escape
• Allow escaping of space to make literal space
• Allow (a|)
• Reject invalid backreferences

Fixed

• Multiple fixes for alternatives in look-arounds
• Fix hex escape to not include letters after "F"
• Fix handling of unescaped ] in character classes
• Fix case insensitive character classes and other escapes
• Don't ignore spaces in character classes even with "comment mode"

Added

• Initial release