• Fixed bug in _trim_arity when pyparsing code is included in a PyInstaller, reported by maluwa.

• Fixed catastrophic regex backtracking in implementation of the quoted string expressions (dblQuotedString, sglQuotedString, and quotedString). Reported on the pyparsing wiki by webpentest, good catch! (Also tuned up some other expressions susceptible to the same backtracking problem, such as cStyleComment, cppStyleComment, etc.)

• Added support for assigning to ParseResults using slices.

• Fixed bug in ParseResults.toDict(), in which dict values were always converted to dicts, even if they were just unkeyed lists of tokens. Reported on SO by Gerald Thibault, thanks Gerald!

• Fixed bug in SkipTo when using failOn, reported by robyschek, thanks!

• Fixed bug in Each introduced in 2.1.0, reported by AND patch and unit test submitted by robyschek, well done!

• Removed use of functools.partial in replaceWith, as this creates an ambiguous signature for the generated parse action, which fails in PyPy. Reported by Evan Hubinger, thanks Evan!

• Added default behavior to QuotedString to convert embedded '\t', '\n', etc. characters to their whitespace counterparts. Found during Q&A exchange on SO with Maxim.

• Modified the internal _trim_arity method to distinguish between TypeError's raised while trying to determine parse action arity and those raised within the parse action itself. This will clear up those confusing "<lambda>() takes exactly 1 argument (0 given)" error messages when there is an actual TypeError in the body of the parse action. Thanks to all who have raised this issue in the past, and most recently to Michael Cohen, who sent in a proposed patch, and got me to finally tackle this problem.

• Added compatibility for pickle protocols 2-4 when pickling ParseResults. In Python 2.x, protocol 0 was the default, and protocol 2 did not work. In Python 3.x, protocol 3 is the default, so explicitly naming protocol 0 or 1 was required to pickle ParseResults. With this release, all protocols 0-4 are supported. Thanks for reporting this on StackOverflow, Arne Wolframm, and for providing a nice simple test case!

• Added optional 'stopOn' argument to ZeroOrMore and OneOrMore, to simplify breaking on stop tokens that would match the repetition expression.

  It is a common problem to fail to look ahead when matching repetitive tokens if the sentinel at the end also matches the repetition expression, as when parsing "BEGIN aaa bbb ccc END" with:

  "BEGIN" + OneOrMore(Word(alphas)) + "END"

  Since "END" matches the repetition expression "Word(alphas)", it will never get parsed as the terminating sentinel. Up until now, this has to be resolved by the user inserting their own negative lookahead:

  "BEGIN" + OneOrMore(~Literal("END") + Word(alphas)) + "END"

  Using stopOn, they can more easily write:

  "BEGIN" + OneOrMore(Word(alphas), stopOn="END") + "END"

  The stopOn argument can be a literal string or a pyparsing expression. Inspired by a question by Lamakaha on StackOverflow (and many previous questions with the same negative-lookahead resolution).

• Added expression names for many internal and builtin expressions, to reduce name and error message overhead during parsing.

• Converted helper lambdas to functions to refactor and add docstring support.

• Fixed ParseResults.asDict() to correctly convert nested ParseResults values to dicts.

• Cleaned up some examples, fixed typo in fourFn.py identified by aristotle2600 on reddit.

• Removed keepOriginalText helper method, which was deprecated ages ago. Superceded by originalTextFor.

• Same for the Upcase class, which was long ago deprecated and replaced with the upcaseTokens method.

• Simplified string representation of Forward class, to avoid memory and performance errors while building ParseException messages. Thanks, Will McGugan, Andrea Censi, and Martijn Vermaat for the bug reports and test code.

• Cleaned up additional issues from enhancing the error messages for Or and MatchFirst, handling Unicode values in expressions. Fixes Unicode encoding issues in Python 2, thanks to Evan Hubinger for the bug report.

• Fixed implementation of dir() for ParseResults - was leaving out all the defined methods and just adding the custom results names.

• Fixed bug in ignore() that was introduced in pyparsing 1.5.3, that would not accept a string literal as the ignore expression.

• Added new example parseTabularData.py to illustrate parsing of data formatted in columns, with detection of empty cells.

• Updated a number of examples to more current Python and pyparsing forms.

• Fixed a bug in Each when multiple Optional elements are present. Thanks for reporting this, whereswalden on SO.

• Fixed another bug in Each, when Optional elements have results names or parse actions, reported by Max Rothman - thank you, Max!

• Added optional parseAll argument to runTests, whether tests should require the entire input string to be parsed or not (similar to parseAll argument to parseString). Plus a little neaten-up of the output on Python 2 (no stray ()'s).

• Modified exception messages from MatchFirst and Or expressions. These were formerly misleading as they would only give the first or longest exception mismatch error message. Now the error message includes all the alternatives that were possible matches. Originally proposed by a pyparsing user, but I've lost the email thread - finally figured out a fairly clean way to do this.

• Fixed a bug in Or, when a parse action on an alternative raises an exception, other potentially matching alternatives were not always tried. Reported by TheVeryOmni on the pyparsing wiki, thanks!

• Fixed a bug to dump() introduced in 2.0.4, where list values were shown in duplicate.

• (&$(@#&$(@!!!! Some "print" statements snuck into pyparsing v2.0.4, breaking Python 3 compatibility! Fixed. Reported by jenshn, thanks!

• Added ParserElement.addCondition, to simplify adding parse actions that act primarily as filters. If the given condition evaluates False, pyparsing will raise a ParseException. The condition should be a method with the same method signature as a parse action, but should return a boolean. Suggested by Victor Porton, nice idea Victor, thanks!

• Slight mod to srange to accept unicode literals for the input string, such as "[а-яА-Я]" instead of "[\u0430-\u044f\u0410-\u042f]". Thanks to Alexandr Suchkov for the patch!

• Enhanced implementation of replaceWith.

• Fixed enhanced ParseResults.dump() method when the results consists only of an unnamed array of sub-structure results. Reported by Robin Siebler, thanks for your patience and persistence, Robin!

• Fixed bug in fourFn.py example code, where pi and e were defined using CaselessLiteral instead of CaselessKeyword. This was not a problem until adding a new function 'exp', and the leading 'e' of 'exp' was accidentally parsed as the mathematical constant 'e'. Nice catch, Tom Grydeland - thanks!

• Adopt new-fangled Python features, like decorators and ternary expressions, per suggestions from Williamzjc - thanks William! (Oh yeah, I'm not supporting Python 2.3 with this code any more...) Plus, some additional code fixes/cleanup - thanks again!

• Added ParserElement.runTests, a little test bench for quickly running an expression against a list of sample input strings. Basically, I got tired of writing the same test code over and over, and finally added it as a test point method on ParserElement.

• Added withClass helper method, a simplified version of withAttribute for the common but annoying case when defining a filter on a div's class - made difficult because 'class' is a Python reserved word.

• Fixed escaping behavior in QuotedString. Formerly, only quotation marks (or characters designated as quotation marks in the QuotedString constructor) would be escaped. Now all escaped characters will be escaped, and the escaping backslashes will be removed.

• Fixed regression in ParseResults.pop() - pop() was pretty much broken after I added improvements in 2.0.2. Reported by Iain Shelvington, thanks Iain!

• Fixed bug in And class when initializing using a generator.

• Enhanced ParseResults.dump() method to list out nested ParseResults that are unnamed arrays of sub-structures.

• Fixed UnboundLocalError under Python 3.4 in oneOf method, reported on Sourceforge by aldanor, thanks!

• Fixed bug in ParseResults init method, when returning non-ParseResults types from parse actions that implement eq. Raised during discussion on the pyparsing wiki with cyrfer.

• Extended "expr(name)" shortcut (same as "expr.setResultsName(name)") to accept "expr()" as a shortcut for "expr.copy()".

• Added "locatedExpr(expr)" helper, to decorate any returned tokens with their location within the input string. Adds the results names locn_start and locn_end to the output parse results.

• Added "pprint()" method to ParseResults, to simplify troubleshooting and prettified output. Now instead of importing the pprint module and then writing "pprint.pprint(result)", you can just write "result.pprint()". This method also accepts additional positional and keyword arguments (such as indent, width, etc.), which get passed through directly to the pprint method (see https://docs.python.org/2/library/pprint.html#pprint.pprint).

• Removed deprecation warnings when using '<<' for Forward expression assignment. '<<=' is still preferred, but '<<' will be retained for cases where '<<=' operator is not suitable (such as in defining lambda expressions).

• Expanded argument compatibility for classes and functions that take list arguments, to now accept generators as well.

• Extended list-like behavior of ParseResults, adding support for append and extend. NOTE: if you have existing applications using these names as results names, you will have to access them using dict-style syntax: res["append"] and res["extend"]

• ParseResults emulates the change in list vs. iterator semantics for methods like keys(), values(), and items(). Under Python 2.x, these methods will return lists, under Python 3.x, these methods will return iterators.

• ParseResults now has a method haskeys() which returns True or False depending on whether any results names have been defined. This simplifies testing for the existence of results names under Python 3.x, which returns keys() as an iterator, not a list.

• ParseResults now supports both list and dict semantics for pop(). If passed no argument or an integer argument, it will use list semantics and pop tokens from the list of parsed tokens. If passed a non-integer argument (most likely a string), it will use dict semantics and pop the corresponding value from any defined results names. A second default return value argument is supported, just as in dict.pop().

• Fixed bug in markInputline, thanks for reporting this, Matt Grant!

• Cleaned up my unit test environment, now runs with Python 2.6 and 3.3.

• Removed use of "nonlocal" that prevented using this version of pyparsing with Python 2.6 and 2.7. This will make it easier to install for packages that depend on pyparsing, under Python versions 2.6 and later. Those using older versions of Python will have to manually install pyparsing 1.5.7.

• Fixed implementation of <<= operator to return self; reported by Luc J. Bourhis, with patch fix by Mathias Mamsch - thanks, Luc and Mathias!

• Rather than release another combined Python 2.x/3.x release I've decided to start a new major version that is only compatible with Python 3.x (and consequently Python 2.7 as well due to backporting of key features). This version will be the main development path from now on, with little follow-on development on the 1.5.x path.

• Operator '<<' is now deprecated, in favor of operator '<<=' for attaching parsing expressions to Forward() expressions. This is being done to address precedence of operations problems with '<<'. Operator '<<' will be removed in a future version of pyparsing.

• NOTE: This is the last release of pyparsing that will try to maintain compatibility with Python versions < 2.6. The next release of pyparsing will be version 2.0.0, using new Python syntax that will not be compatible for Python version 2.5 or older.

• An awesome new example is included in this release, submitted by Luca DellOlio, for parsing ANTLR grammar definitions, nice work Luca!

• Fixed implementation of ParseResults.str to use Pythonic ''.join() instead of repeated string concatenation. This purportedly has been a performance issue under PyPy.

• Fixed bug in ParseResults.dir under Python 3, reported by Thomas Kluyver, thank you Thomas!

• Added ParserElement.inlineLiteralsUsing static method, to override pyparsing's default behavior of converting string literals to Literal instances, to use other classes (such as Suppress or CaselessLiteral).

• Added new operator '<<=', which will eventually replace '<<' for storing the contents of a Forward(). '<<=' does not have the same operator precedence problems that '<<' does.

• 'operatorPrecedence' is being renamed 'infixNotation' as a better description of what this helper function creates. 'operatorPrecedence' is deprecated, and will be dropped entirely in a future release.

• Added optional arguments lpar and rpar to operatorPrecedence, so that expressions that use it can override the default suppression of the grouping characters.

• Added support for using single argument builtin functions as parse actions. Now you can write 'expr.setParseAction(len)' and get back the length of the list of matched tokens. Supported builtins are: sum, len, sorted, reversed, list, tuple, set, any, all, min, and max. A script demonstrating this feature is included in the examples directory.

• Improved linking in generated docs, proposed on the pyparsing wiki by techtonik, thanks!

• Fixed a bug in the definition of 'alphas', which was based on the string.uppercase and string.lowercase "constants", which in fact aren't constant, but vary with locale settings. This could make parsers locale-sensitive in a subtle way. Thanks to Kef Schecter for his diligence in following through on reporting and monitoring this bugfix!

• Fixed a bug in the Py3 version of pyparsing, during exception handling with packrat parsing enabled, reported by Catherine Devlin - thanks Catherine!

• Fixed typo in ParseBaseException.dir, reported anonymously on the SourceForge bug tracker, thank you Pyparsing User With No Name.

• Fixed bug in srange when using '\x###' hex character codes.

• Added optional 'intExpr' argument to countedArray, so that you can define your own expression that will evaluate to an integer, to be used as the count for the following elements. Allows you to define a countedArray with the count given in hex, for example, by defining intExpr as "Word(hexnums).setParseAction(int(t[0],16))".

• Cleanup of parse action normalizing code, to be more version-tolerant, and robust in the face of future Python versions - much thanks to Raymond Hettinger for this rewrite!

• Removal of exception cacheing, addressing a memory leak condition in Python 3. Thanks to Michael Droettboom and the Cape Town PUG for their analysis and work on this problem!

• Fixed bug when using packrat parsing, where a previously parsed expression would duplicate subsequent tokens - reported by Frankie Ribery on stackoverflow, thanks!

• Added 'ungroup' helper method, to address token grouping done implicitly by And expressions, even if only one expression in the And actually returns any text - also inspired by stackoverflow discussion with Frankie Ribery!

• Fixed bug in srange, which accepted escaped hex characters of the form '\0x##', but should be '\x##'. Both forms will be supported for backwards compatibility.

• Enhancement to countedArray, accepting an optional expression to be used for matching the leading integer count - proposed by Mathias on the pyparsing mailing list, good idea!

• Added the Verilog parser to the provided set of examples, under the MIT license. While this frees up this parser for any use, if you find yourself using it in a commercial purpose, please consider making a charitable donation as described in the parser's header.

• Added the excludeChars argument to the Word class, to simplify defining a word composed of all characters in a large range except for one or two. Suggested by JesterEE on the pyparsing wiki.

• Added optional overlap parameter to scanString, to return overlapping matches found in the source text.

• Updated oneOf internal regular expression generation, with improved parse time performance.

• Slight performance improvement in transformString, removing empty strings from the list of string fragments built while scanning the source text, before calling ''.join. Especially useful when using transformString to strip out selected text.

• Enhanced form of using the "expr('name')" style of results naming, in lieu of calling setResultsName. If name ends with an '*', then this is equivalent to expr.setResultsName('name',listAllMatches=True).

• Fixed up internal list flattener to use iteration instead of recursion, to avoid stack overflow when transforming large files.

• Added other new examples: . protobuf parser - parses Google's protobuf language . btpyparse - a BibTex parser contributed by Matthew Brett, with test suite test_bibparse.py (thanks, Matthew!) . groupUsingListAllMatches.py - demo using trailing '*' for results names

• Typo in Python3 version of pyparsing, "builtin" should be "builtins". (sigh)

• Fixed builtins and file references in Python 3 code, thanks to Greg Watson, saulspatz, sminos, and Mark Summerfield for reporting their Python 3 experiences.

• Added new example, apicheck.py, as a sample of scanning a Tcl-like language for functions with incorrect number of arguments (difficult to track down in Tcl languages). This example uses some interesting methods for capturing exceptions while scanning through source code.

• Added new example deltaTime.py, that takes everyday time references like "an hour from now", "2 days ago", "next Sunday at 2pm".

• ======= NOTE: API CHANGE!!!!!!! =============== With this release, and henceforward, the pyparsing module is imported as "pyparsing" on both Python 2.x and Python 3.x versions.

• Fixed up setup.py to auto-detect Python version and install the correct version of pyparsing - suggested by Alex Martelli, thanks, Alex! (and my apologies to all those who struggled with those spurious installation errors caused by my earlier fumblings!)

• Fixed bug on Python3 when using parseFile, getting bytes instead of a str from the input file.

• Fixed subtle bug in originalTextFor, if followed by significant whitespace (like a newline) - discovered by Francis Vidal, thanks!

• Fixed very sneaky bug in Each, in which Optional elements were not completely recognized as optional - found by Tal Weiss, thanks for your patience.

• Fixed off-by-1 bug in line() method when the first line of the input text was an empty line. Thanks to John Krukoff for submitting a patch!

• Fixed bug in transformString if grammar contains Group expressions, thanks to patch submitted by barnabas79, nice work!

• Fixed bug in originalTextFor in which trailing comments or otherwised ignored text got slurped in with the matched expression. Thanks to michael_ramirez44 on the pyparsing wiki for reporting this just in time to get into this release!

• Added better support for summing ParseResults, see the new example, parseResultsSumExample.py.

• Added support for composing a Regex using a compiled RE object; thanks to my new colleague, Mike Thornton!

• In version 1.5.2, I changed the way exceptions are raised in order to simplify the stacktraces reported during parsing. An anonymous user posted a bug report on SF that this behavior makes it difficult to debug some complex parsers, or parsers nested within parsers. In this release I've added a class attribute ParserElement.verbose_stacktrace, with a default value of False. If you set this to True, pyparsing will report stacktraces using the pre-1.5.2 behavior.

• New examples:

  . pymicko.py, a MicroC compiler submitted by Zarko Zivanov. (Note: this example is separately licensed under the GPLv3, and requires Python 2.6 or higher.) Thank you, Zarko!

  . oc.py, a subset C parser, using the BNF from the 1996 Obfuscated C Contest.

  . stateMachine2.py, a modified version of stateMachine.py submitted by Matt Anderson, that is compatible with Python versions 2.7 and above - thanks so much, Matt!

  . select_parser.py, a parser for reading SQLite SELECT statements, as specified at https://www.sqlite.org/lang_select.html this goes into much more detail than the simple SQL parser included in pyparsing's source code

  . excelExpr.py, a simplistic first-cut at a parser for Excel expressions, which I originally posted on comp.lang.python in January, 2010; beware, this parser omits many common Excel cases (addition of numbers represented as strings, references to named ranges)

  . cpp_enum_parser.py, a nice little parser posted my Mark Tolonen on comp.lang.python in August, 2009 (redistributed here with Mark's permission). Thanks a bunch, Mark!

  . partial_gene_match.py, a sample I posted to Stackoverflow.com, implementing a special variation on Literal that does "close" matching, up to a given number of allowed mismatches. The application was to find matching gene sequences, with allowance for one or two mismatches.

  . tagCapture.py, a sample showing how to use a Forward placeholder to enforce matching of text parsed in a previous expression.

  . matchPreviousDemo.py, simple demo showing how the matchPreviousLiteral helper method is used to match a previously parsed token.

• Added pyparsing_py3.py module, so that Python 3 users can use pyparsing by changing their pyparsing import statement to:

  import pyparsing_py3
  

  Thanks for help from Patrick Laban and his friend Geremy Condra on the pyparsing wiki.

• Removed slots declaration on ParseBaseException, for compatibility with IronPython 2.0.1. Raised by David Lawler on the pyparsing wiki, thanks David!

• Fixed bug in SkipTo/failOn handling - caught by eagle eye cpennington on the pyparsing wiki!

• Fixed second bug in SkipTo when using the ignore constructor argument, reported by Catherine Devlin, thanks!

• Fixed obscure bug reported by Eike Welk when using a class as a ParseAction with an errant getitem method.

• Simplified exception stack traces when reporting parse exceptions back to caller of parseString or parseFile - thanks to a tip from Peter Otten on comp.lang.python.

• Changed behavior of scanString to avoid infinitely looping on expressions that match zero-length strings. Prompted by a question posted by ellisonbg on the wiki.

• Enhanced classes that take a list of expressions (And, Or, MatchFirst, and Each) to accept generator expressions also. This can be useful when generating lists of alternative expressions, as in this case, where the user wanted to match any repetitions of '+', '*', '#', or '.', but not mixtures of them (that is, match '+++', but not '+-+'):

  codes = "+*#."
  format = MatchFirst(Word(c) for c in codes)
  

  Based on a problem posed by Denis Spir on the Python tutor list.

• Added new example eval_arith.py, which extends the example simpleArith.py to actually evaluate the parsed expressions.

• Added new helper method originalTextFor, to replace the use of the current keepOriginalText parse action. Now instead of using the parse action, as in:

  fullName = Word(alphas) + Word(alphas)
  fullName.setParseAction(keepOriginalText)
  

  (in this example, we used keepOriginalText to restore any white space that may have been skipped between the first and last names) You can now write:

  fullName = originalTextFor(Word(alphas) + Word(alphas))
  

  The implementation of originalTextFor is simpler and faster than keepOriginalText, and does not depend on using the inspect or imp modules.

• Added optional parseAll argument to parseFile, to be consistent with parseAll argument to parseString. Posted by pboucher on the pyparsing wiki, thanks!

• Added failOn argument to SkipTo, so that grammars can define literal strings or pyparsing expressions which, if found in the skipped text, will cause SkipTo to fail. Useful to prevent SkipTo from reading past terminating expression. Instigated by question posed by Aki Niimura on the pyparsing wiki.

• Fixed bug in nestedExpr if multi-character expressions are given for nesting delimiters. Patch provided by new pyparsing user, Hans-Martin Gaudecker - thanks, H-M!

• Removed dependency on xml.sax.saxutils.escape, and included internal implementation instead - proposed by Mike Droettboom on the pyparsing mailing list, thanks Mike! Also fixed erroneous mapping in replaceHTMLEntity of " to ', now correctly maps to ". (Also added support for mapping ' to '.)

• Fixed typo in ParseResults.insert, found by Alejandro Dubrovsky, good catch!

• Added dir() methods to ParseBaseException and ParseResults, to support new dir() behavior in Py2.6 and Py3.0. If dir() is called on a ParseResults object, the returned list will include the base set of attribute names, plus any results names that are defined.

• Fixed bug in ParseResults.asXML(), in which the first named item within a ParseResults gets reported with an <ITEM> tag instead of with the correct results name.

• Fixed bug in '-' error stop, when '-' operator is used inside a Combine expression.

• Reverted generator expression to use list comprehension, for better compatibility with old versions of Python. Reported by jester/artixdesign on the SourceForge pyparsing discussion list.

• Fixed bug in parseString(parseAll=True), when the input string ends with a comment or whitespace.

• Fixed bug in LineStart and LineEnd that did not recognize any special whitespace chars defined using ParserElement.setDefault- WhitespaceChars, found while debugging an issue for Marek Kubica, thanks for the new test case, Marek!

• Made Forward class more tolerant of subclassing.

This version of pyparsing includes work on two long-standing FAQ's: support for forcing parsing of the complete input string (without having to explicitly append StringEnd() to the grammar), and a method to improve the mechanism of detecting where syntax errors occur in an input string with various optional and alternative paths. This release also includes a helper method to simplify definition of indentation-based grammars. With these changes (and the past few minor updates), I thought it was finally time to bump the minor rev number on pyparsing - so 1.5.0 is now available! Read on...

• AT LAST!!! You can now call parseString and have it raise an exception if the expression does not parse the entire input string. This has been an FAQ for a LONG time.

  The parseString method now includes an optional parseAll argument (default=False). If parseAll is set to True, then the given parse expression must parse the entire input string. (This is equivalent to adding StringEnd() to the end of the expression.) The default value is False to retain backward compatibility.

  Inspired by MANY requests over the years, most recently by ecir-hana on the pyparsing wiki!

• Added new operator '-' for composing grammar sequences. '-' behaves just like '+' in creating And expressions, but '-' is used to mark grammar structures that should stop parsing immediately and report a syntax error, rather than just backtracking to the last successful parse and trying another alternative. For instance, running the following code:

  port_definition = Keyword("port") + '=' + Word(nums) entity_definition = Keyword("entity") + "{" + Optional(port_definition) + "}"

  entity_definition.parseString("entity { port 100 }")

  pyparsing fails to detect the missing '=' in the port definition. But, since this expression is optional, pyparsing then proceeds to try to match the closing '}' of the entity_definition. Not finding it, pyparsing reports that there was no '}' after the '{' character. Instead, we would like pyparsing to parse the 'port' keyword, and if not followed by an equals sign and an integer, to signal this as a syntax error.

  This can now be done simply by changing the port_definition to:

  port_definition = Keyword("port") - '=' + Word(nums)

  Now after successfully parsing 'port', pyparsing must also find an equals sign and an integer, or it will raise a fatal syntax exception.

  By judicious insertion of '-' operators, a pyparsing developer can have their grammar report much more informative syntax error messages.

  Patches and suggestions proposed by several contributors on the pyparsing mailing list and wiki - special thanks to Eike Welk and Thomas/Poldy on the pyparsing wiki!

• Added indentedBlock helper method, to encapsulate the parse actions and indentation stack management needed to keep track of indentation levels. Use indentedBlock to define grammars for indentation-based grouping grammars, like Python's.

  indentedBlock takes up to 3 parameters:

  • blockStatementExpr - expression defining syntax of statement that is repeated within the indented block
  • indentStack - list created by caller to manage indentation stack (multiple indentedBlock expressions within a single grammar should share a common indentStack)
  • indent - boolean indicating whether block must be indented beyond the current level; set to False for block of left-most statements (default=True)

  A valid block must contain at least one indented statement.

• Fixed bug in nestedExpr in which ignored expressions needed to be set off with whitespace. Reported by Stefaan Himpe, nice catch!

• Expanded multiplication of an expression by a tuple, to accept tuple values of None: . expr*(n,None) or expr*(n,) is equivalent to exprn + ZeroOrMore(expr) (read as "at least n instances of expr") . expr(None,n) is equivalent to expr*(0,n) (read as "0 to n instances of expr") . expr*(None,None) is equivalent to ZeroOrMore(expr) . expr*(1,None) is equivalent to OneOrMore(expr)

  Note that expr*(None,n) does not raise an exception if more than n exprs exist in the input stream; that is, expr*(None,n) does not enforce a maximum number of expr occurrences. If this behavior is desired, then write expr*(None,n) + ~expr

• Added None as a possible operator for operatorPrecedence. None signifies "no operator", as in multiplying m times x in "y=mx+b".

• Fixed bug in Each, reported by Michael Ramirez, in which the order of terms in the Each affected the parsing of the results. Problem was due to premature grouping of the expressions in the overall Each during grammar construction, before the complete Each was defined. Thanks, Michael!

• Also fixed bug in Each in which Optional's with default values were not getting the defaults added to the results of the overall Each expression.

• Fixed a bug in Optional in which results names were not assigned if a default value was supplied.

• Cleaned up Py3K compatibility statements, including exception construction statements, and better equivalence between _ustr and basestring, and nonzero and bool.

• With help from Robert A. Clark, this version of pyparsing is compatible with Python 3.0a3. Thanks for the help, Robert!

• Added WordStart and WordEnd positional classes, to support expressions that must occur at the start or end of a word. Proposed by piranha on the pyparsing wiki, good idea!

• Added matchOnlyAtCol helper parser action, to simplify parsing log or data files that have optional fields that are column dependent. Inspired by a discussion thread with hubritic on comp.lang.python.

• Added withAttribute.ANY_VALUE as a match-all value when using withAttribute. Used to ensure that an attribute is present, without having to match on the actual attribute value.

• Added get() method to ParseResults, similar to dict.get(). Suggested by new pyparsing user, Alejandro Dubrovksy, thanks!

• Added '==' short-cut to see if a given string matches a pyparsing expression. For instance, you can now write:

  integer = Word(nums) if "123" == integer: # do something

  print [ x for x in "123 234 asld".split() if x==integer ]

  prints ['123', '234']

• Simplified the use of nestedExpr when using an expression for the opening or closing delimiters. Now the content expression will not have to explicitly negate closing delimiters. Found while working with dfinnie on GHOP Task #277, thanks!

• Fixed bug when defining ignorable expressions that are later enclosed in a wrapper expression (such as ZeroOrMore, OneOrMore, etc.) - found while working with Prabhu Gurumurthy, thanks Prahbu!

• Fixed bug in withAttribute in which keys were automatically converted to lowercase, making it impossible to match XML attributes with uppercase characters in them. Using with- Attribute requires that you reference attributes in all lowercase if parsing HTML, and in correct case when parsing XML.

• Changed '<<' operator on Forward to return None, since this is really used as a pseudo-assignment operator, not as a left-shift operator. By returning None, it is easier to catch faulty statements such as a << b | c, where precedence of operations causes the '|' operation to be performed after inserting b into a, so no alternation is actually implemented. The correct form is a << (b | c). With this change, an error will be reported instead of silently clipping the alternative term. (Note: this may break some existing code, but if it does, the code had a silent bug in it anyway.) Proposed by wcbarksdale on the pyparsing wiki, thanks!

• Several unit tests were added to pyparsing's regression suite, courtesy of the Google Highly-Open Participation Contest. Thanks to all who administered and took part in this event!

• Added new helper method nestedExpr to easily create expressions that parse lists of data in nested parentheses, braces, brackets, etc.

• Added withAttribute parse action helper, to simplify creating filtering parse actions to attach to expressions returned by makeHTMLTags and makeXMLTags. Use withAttribute to qualify a starting tag with one or more required attribute values, to avoid false matches on common tags such as <TD> or <DIV>.

• Added new examples nested.py and withAttribute.py to demonstrate the new features.

• Added performance speedup to grammars using operatorPrecedence, instigated by Stefan Reichör - thanks for the feedback, Stefan!

• Fixed bug/typo when deleting an element from a ParseResults by using the element's results name.

• Fixed whitespace-skipping bug in wrapper classes (such as Group, Suppress, Combine, etc.) and when using setDebug(), reported by new pyparsing user dazzawazza on SourceForge, nice job!

• Added restriction to prevent defining Word or CharsNotIn expressions with minimum length of 0 (should use Optional if this is desired), and enhanced docstrings to reflect this limitation. Issue was raised by Joey Tallieu, who submitted a patch with a slightly different solution. Thanks for taking the initiative, Joey, and please keep submitting your ideas!

• Fixed bug in makeHTMLTags that did not detect HTML tag attributes with no '= value' portion (such as "<td nowrap>"), reported by hamidh on the pyparsing wiki - thanks!

• Fixed minor bug in makeHTMLTags and makeXMLTags, which did not accept whitespace in closing tags.

• NEW NOTATION SHORTCUT: ParserElement now accepts results names using a notational shortcut, following the expression with the results name in parentheses. So this:

  stats = "AVE:" + realNum.setResultsName("average") +
  "MIN:" + realNum.setResultsName("min") +
  "MAX:" + realNum.setResultsName("max")

  can now be written as this:

  stats = "AVE:" + realNum("average") +
  "MIN:" + realNum("min") +
  "MAX:" + realNum("max")

  The intent behind this change is to make it simpler to define results names for significant fields within the expression, while keeping the grammar syntax clean and uncluttered.

• Fixed bug when packrat parsing is enabled, with cached ParseResults being updated by subsequent parsing. Reported on the pyparsing wiki by Kambiz, thanks!

• Fixed bug in operatorPrecedence for unary operators with left associativity, if multiple operators were given for the same term.

• Fixed bug in example simpleBool.py, corrected precedence of "and" vs. "or" operations.

• Fixed bug in Dict class, in which keys were converted to strings whether they needed to be or not. Have narrowed this logic to convert keys to strings only if the keys are ints (which would confuse getitem behavior for list indexing vs. key lookup).

• Added ParserElement method setBreak(), which will invoke the pdb module's set_trace() function when this expression is about to be parsed.

• Fixed bug in StringEnd in which reading off the end of the input string raises an exception - should match. Resolved while answering a question for Shawn on the pyparsing wiki.

• Simplified constructor for ParseFatalException, to support common exception construction idiom: raise ParseFatalException, "unexpected text: 'Spanish Inquisition'"

• Added method getTokensEndLoc(), to be called from within a parse action, for those parse actions that need both the starting and ending location of the parsed tokens within the input text.

• Enhanced behavior of keepOriginalText so that named parse fields are preserved, even though tokens are replaced with the original input text matched by the current expression. Also, cleaned up the stack traversal to be more robust. Suggested by Tim Arnold - thanks, Tim!

• Fixed subtle bug in which countedArray (and similar dynamic expressions configured in parse actions) failed to match within Or, Each, FollowedBy, or NotAny. Reported by Ralf Vosseler, thanks for your patience, Ralf!

• Fixed Unicode bug in upcaseTokens and downcaseTokens parse actions, scanString, and default debugging actions; reported (and patch submitted) by Nikolai Zamkovoi, spasibo!

• Fixed bug when saving a tuple as a named result. The returned token list gave the proper tuple value, but accessing the result by name only gave the first element of the tuple. Reported by Poromenos, nice catch!

• Fixed bug in makeHTMLTags/makeXMLTags, which failed to match tag attributes with namespaces.

• Fixed bug in SkipTo when setting include=True, to have the skipped-to tokens correctly included in the returned data. Reported by gunars on the pyparsing wiki, thanks!

• Fixed typobug in OnceOnly.reset method, omitted self argument. Submitted by eike welk, thanks for the lint-picking!

• Added performance enhancement to Forward class, suggested by akkartik on the pyparsing Wiki discussion, nice work!

• Added optional asKeyword to Word constructor, to indicate that the given word pattern should be matched only as a keyword, that is, it should only match if it is within word boundaries.

• Added S-expression parser to examples directory.

• Added macro substitution example to examples directory.

• Added holaMundo.py example, excerpted from Marco Alfonso's blog - muchas gracias, Marco!

• Modified internal cyclic references in ParseResults to use weakrefs; this should help reduce the memory footprint of large parsing programs, at some cost to performance (3-5%). Suggested by bca48150 on the pyparsing wiki, thanks!

• Enhanced the documentation describing the vagaries and idiosyncrasies of parsing strings with embedded tabs, and the impact on: . parse actions . scanString . col and line helper functions (Suggested by eike welk in response to some unexplained inconsistencies between parsed location and offsets in the input string.)

• Cleaned up internal decorators to preserve function names, docstrings, etc.

• Improved support for Unicode strings that would be returned using srange. Added greetingInKorean.py example, for a Korean version of "Hello, World!" using Unicode. (Thanks, June Kim!)

• Added 'hexnums' string constant (nums+"ABCDEFabcdef") for defining hexadecimal value expressions.

• NOTE: ===THIS CHANGE MAY BREAK EXISTING CODE=== Modified tag and results definitions returned by makeHTMLTags(), to better support the looseness of HTML parsing. Tags to be parsed are now caseless, and keys generated for tag attributes are now converted to lower case.

  Formerly, makeXMLTags("XYZ") would return a tag with results name of "startXYZ", this has been changed to "startXyz". If this tag is matched against '<XYZ Abc="1" DEF="2" ghi="3">', the matched keys formerly would be "Abc", "DEF", and "ghi"; keys are now converted to lower case, giving keys of "abc", "def", and "ghi". These changes were made to try to address the lax case sensitivity agreement between start and end tags in many HTML pages.

  No changes were made to makeXMLTags(), which assumes more rigorous parsing rules.

  Also, cleaned up case-sensitivity bugs in closing tags, and switched to using Keyword instead of Literal class for tags. (Thanks, Steve Young, for getting me to look at these in more detail!)

• Added two helper parse actions, upcaseTokens and downcaseTokens, which will convert matched text to all uppercase or lowercase, respectively.

• Deprecated Upcase class, to be replaced by upcaseTokens parse action.

• Converted messages sent to stderr to use warnings module, such as when constructing a Literal with an empty string, one should use the Empty() class or the empty helper instead.

• Added ' ' (space) as an escapable character within a quoted string.

• Added helper expressions for common comment types, in addition to the existing cStyleComment (/.../) and htmlStyleComment (<!-- ... -->) . dblSlashComment = // ... (to end of line) . cppStyleComment = cStyleComment or dblSlashComment . javaStyleComment = cppStyleComment . pythonStyleComment = # ... (to end of line)

• Added definition for htmlComment to help support HTML scanning and parsing.

• Fixed bug in generating XML for Dict classes, in which trailing item was duplicated in the output XML.

• Fixed release bug in which scanExamples.py was omitted from release files.

• Fixed bug in transformString() when parse actions are not defined on the outermost parser element.

• Added example urlExtractor.py, as another example of using scanString and parse actions.

• Fixed minor bug in scanString(), so that start location is at the start of the matched tokens, not at the start of the whitespace before the matched tokens.

• Inclusion of HTML documentation, generated using Epydoc. Reformatted some doc strings to better generate readable docs. (Beautiful work, Ed Loper, thanks for Epydoc!)

• Minor performance speedup, 5-15%

• And on a process note, I've used the unittest module to define a series of unit tests, to help avoid the embarrassment of the version 1.1 snafu.