rhai_codegen

Starting from this version, the official preferred method of registering an API for a custom type is via the #[derive(CustomType)] macro. The old API is still available for types that reside in external crates (and thus cannot implement CustomType).

Starting from this version, the new FuncRegistration API is preferred for registering native Rust functions into a Module. The old API is still available but deprecated.

Starting from this version, fuzzing via Google OSS-Fuzz is used to flush out hidden bugs and edge cases. This should result in higher code quality, better stability and improved security. And indeed, a large number of bugs have been discovered from this and fixed.

Potentially breaking changes

• ImmutableString now derefs to &str instead of &SmartString. Normally this should not be a breaking change.
• Traits implemented by ImmutableString are cleaned up. Normally this should not be a breaking change.
• EvalContext::new, FloatWrapper and ConditionalExpr are now gated under internals.
• Previously, Rhai follows Unicode's definition for whitespace, which allows many exotic whitespace characters in scripts. Starting from this version, whitespace follows WhatWG's definition of five ASCII characters (TAB, SPACE, CR, LF and FF), which is the same as Rust. All other Unicode whitespace characters (not inside strings) are not considered whitespace by Rhai. If a script used to contain non-ASCII whitespace characters, it now fails to parse with a syntax error.

New features

• Great thanks to @silvergasp for setting up fuzzing.
• #[derive(CustomType)] is now available, driven by procedural macros in rhai_codegen (thanks @MavethGH #817).
• A new FuncRegistration API is added to assist in registering native Rust functions into modules with various settings. Some of the original Module::set_fn... API is now deprecated.
• Functions defined in plugin modules can now be marked as volatile which prevents it from being optimized away even under OptimizationLevel::Full.
• Added Engine::max_functions and Engine::set_max_functions to limit the maximum number of functions allowed in a script. This is to guard against DOS attacks -- e.g. a simple closure || (two characters) is a function. When max_function is exceeded during script compilation, a new parse error, ParseErrorType::TooManyFunctions, is returned.
• Engine::get_interned_string is made public instead of gated under internals.

Deprecated API's

• rhai::config::hashing::set_ahash_seed, rhai::config::hashing::get_ahash_seed and the RHAI_AHASH_SEED environment variable are deprecated in favor of rhai::config::hashing::set_hashing_seed, rhai::config::hashing::get_hashing_seed and RHAI_HASHING_SEED.
• AST::clear_doc is deprecated.
• Much of the Module::update_XXX API is deprecated in favor of using the FuncRegistration API.
• Module::gen_fn_signatures is deprecated in favor of Module::gen_fn_signatures_with_mapper.

Fixes to bugs found via fuzzing

• Fixed crash when parsing multi-segment interpolated string longer than maximum.
• Fixed crash when parsing unterminated comment.
• Fixed crash when parsing deeply-nested right-associated operators such as **.
• Fixed crash when parsing combo-chaining expressions such as (a.b).c.
• Fixed crash when calling functions that have Dynamic parameters with more than 16 parameters.
• Fixed crash when indexing into an empty array with negative index.
• Indexing into an array with a negative index that is larger than the length of the array now throws an out-of-bounds error (similar to positive indices) instead of defaulting to the first element.
• Fixed edge-case crash in timestamp functions.
• Fixed crash when indenting a block doc-comment with Unicode multi-byte space characters.
• Fixed improper parsing of numbers with too many decimal points.
• Fixed exponential running time when raising a decimal number to a very large power (> 1 million) -- it now returns an overflow error.
• Shared values that contain reference loops no longer cause a stack overflow when printing.
• sleep no longer panics on NaN.
• switch on ranges now work properly.

Other bug fixes

• Arrays in object maps now serialize to JSON correctly via to_json() when the serde feature is not enabled.
• Engine::format_map_as_json now serializes arrays correctly.
• Engine::gen_fn_signatures(false) now properly skips functions in the standard library.
• TypeBuilder::with_name now properly sets the display-name of the type for use in generating metadata.

Enhancements

• Avoid cloning values unless needed when performing constants propagation in optimization.
• Added to_int method for characters.
• Token::FloatConstant and Token::DecimalConstant now carry the original text representation for use in, say, a token mapper.
• Dynamic::is_fnptr is made a public API.
• Scope::get_value_ref and Scope::get_value_mut are added.
• TypeBuilder::with_name now takes any &str instead of just &'static str.
• Engine::gen_fn_signatures now formats the function signatures using pretty-print names of custom types.

This version, in particular, fixes a plugin macro hygiene error for the nightly compiler:

error[E0425]: cannot find value `args` in this scope

Compiler version

• Minimum compiler version is now 1.57 due to smartstring dependency.

Bug fixes

• Fixed macro hygiene error with nightly compiler.
• Invalid property or method access such as a.b::c.d or a.b::func() no longer panics but properly returns a syntax error.
• Scope::is_constant now returns the correct value.
• Exporting a variable that contains a local function pointer (including anonymous function or closure) now raises a runtime error.
• Full optimization is now skipped for method calls.

New features

• Type aliases in plugin modules are now used as friendly names for custom types. This makes plugin modules more self-contained when they are used to define a custom type's API.

Enhancements

• Variable definitions are optimized so that shadowed variables are reused as much as possible to reduce memory consumption.
• FnAccess and FnNamespace now implement Ord and PartialOrd.
• The event_handler_map example is enhanced to prevent shadowing of the state object map.
• Separation of constants in function calls is removed as its performance benefit is dubious.
• A function sleep is added to block the current thread by a specified number of seconds.
• Scope::set_alias is added to export a variable under a particular alias name.
• starts_with and ends_with are added for strings.
• Variables in modules registered via register_global_module can now be accessed in the global namespace.
• Dynamic::into_read_only is added to convert a Dynamic value into constant.
• Module now holds a collection of custom types with an API.

This version adds a debugging interface, which can be used to integrate a debugger.

Based on popular demand, an option is added to throw exceptions when invalid properties are accessed on object maps (default is to return ()).

Also based on popular demand, the REPL tool now uses a slightly-enhanced version of rustyline for line editing and history.

Bug fixes

• In Scope::clone_visible, constants are now properly cloned as constants.
• Variables introduced inside try blocks are now properly cleaned up upon an exception.
• Off-by-one error in character positions after a comment line is now fixed.
• Globally-defined constants are now encapsulated correctly inside a loaded module and no longer spill across call boundaries.
• Type names display is fixed.
• Exceptions thrown inside function calls now unwrap correctly when catch-ed.
• Error messages for certain invalid property accesses are fixed.

Script-breaking changes

• For consistency with the import statement, the export statement no longer exports multiple variables.
• Appending a BLOB to a string (via +, +=, append or string interpolation) now treats the BLOB as a UTF-8 encoded string.
• Appending a string/character to a BLOB (via += or append) now adds the string/character as a UTF-8 encoded byte stream.

New features

• A debugging interface is added.
• A new bin tool, rhai-dbg (aka The Rhai Debugger), is added to showcase the debugging interface.
• A new package, DebuggingPackage, is added which contains the back_trace function to get the current call stack anywhere in a script.
• Engine::set_fail_on_invalid_map_property is added to control whether to raise an error (new EvalAltResult::ErrorPropertyNotFound) when invalid properties are accessed on object maps.
• Engine::set_allow_shadowing is added to allow/disallow variables shadowing, with new errors EvalAltResult::ErrorVariableExists and ParseErrorType::VariableExists.
• Engine::on_def_var allows registering a closure which can decide whether a variable definition is allow to continue, during compilation or runtime, or should fail with an error (ParseErrorType::ForbiddenVariable or EvalAltResult::ErrorForbiddenVariable).
• A new syntax for defining custom packages is introduced that removes the need to specify the Rhai crate name (internally uses the $crate meta variable).

Enhancements

• Default features for dependencies (such as ahash/std and num-traits/std) are no longer required.
• The no_module feature now eliminates large sections of code via feature gates.
• Debug display of AST is improved.
• NativeCallContext::call_level() is added to give the current nesting level of function calls.
• A new feature, bin-features, pulls in all the required features for bin tools.
• AST position display is improved:
  • Expr::start_position is added to give the beginning of the expression (not the operator's position).
  • StmtBlock and Stmt::Block now keep the position of the closing } as well.
• EvalAltResult::unwrap_inner is added to access the base error inside multiple layers of wrappings (e.g. EvalAltResult::ErrorInFunction).
• Yet another new syntax is introduced for def_package! that further simplifies the old syntax.
• A new method to_blob is added to convert a string into a BLOB as UTF-8 encoded bytes.
• A new method to_array is added to convert a BLOB into array of integers.

REPL tool changes

The REPL bin tool, rhai-rpl, has been enhanced.

Build changes

• The rustyline feature is now required in order to build rhai-repl.
• Therefore, rhai-repl is no longer automatically built when using a simple cargo build with default features.

Line editor

• rhai-repl now uses a modified version of rustyline as a line editor with history.
• Ctrl-Enter can now be used to enter multiple lines without having to attach the \ continuation character the end of each line.
• Bracketed paste is supported, even on Windows (version 10 or above), so pasting code directly into rhai-repl is made much more convenient.

New commands

• strict to turn on/off Strict Variables Mode.
• optimize to turn on/off script optimization.
• history to print lines history.
• !!, !num, !text and !?text to recall a history line.
• keys to print all key bindings.

This is primarily a bug-fix version which fixes a large number of bugs.

Bug fixes

• Expressions such as x = x + 1 no longer panics.
• Padding arrays with another array via pad no longer loops indefinitely.
• chop for arrays and BLOB's now works properly.
• set_bit for bit-flags with negative index now works correctly.
• Misnamed params field name in the JSON output of Engine::gen_fn_metadata_to_json is fixed (was incorrectly named type).
• Fixes a potential unsafe violation in for loop.
• Missing to_hex, to_octal and to_binary for i128 and u128 are added.
• remove for arrays and BLOB's now treat negative index correctly.
• parse_int now works properly for negative numbers.
• Engine::gen_fn_signatures now generates signatures for external packages registered via Engine::register_global_module.
• \r\n pairs are now recognized correctly for doc-comments.

Enhancements

• Formatting of return types in functions metadata info is improved.
• Use SmartString for Scope variable names and remove unsafe lifetime casting.
• Functions in the standard library now have doc-comments (which can be obtained via Engine::gen_fn_metadata_to_json).
• get and set methods are added to arrays, BLOB's, object maps and strings.

This version adds support for integer ranges via the .. and ..= operators. Many standard API's are extended with range parameters where appropriate.

Script-breaking changes

• is is (pun intended) now a reserved keyword to prepare for possible future type checking expressions (e.g. x is "string").

Breaking changes

• LogicPackage is removed from CorePackage.
• Bit-field functions are moved into a new BitFieldPackage (used to be in LogicPackage) which makes more sense.

Bug fixes

• Custom syntax now works properly inside binary expressions and with method calls.
• Hex numbers with the high-bit set now parse correctly into negative integer numbers.
• Constructing a literal array or object map now checks for size limits for each item instead of at the very end when it is already too late.
• Non-INT integer types are now treated exactly as custom types under only_i64 and only_i32.
• Calling pad on an array now checks for total size over limit after each item added.

New features

• Added support for integer ranges via the .. and ..= operators.
• Added EvalAltResult::ErrorCustomSyntax to catch errors in custom syntax, which should not happen unless an AST is compiled on one Engine but evaluated on another unrelated Engine.

Enhancements

• BLOB's are refined to display in a more compact hex format.
• A new syntax is introduced for def_package! that will replace the old syntax in future versions.
• Added NativeCallContext::call_fn to easily call a function.
• Doc-comments on plugin module functions are extracted into the functions' metadata.

Deprecated API's

• Expression::get_variable_name is deprecated in favor of the new Expression::get_string_value.
• The old syntax of def_package! is deprecated in favor of the new syntax.

This version adds native support for BLOB's (byte arrays), as well as a number of configuration settings to fine-tun language features.

Compiler requirement

• Minimum compiler version is now 1.51.

Bug fixes

• from_dynamic now supports deserializing Option.

New features

• BLOB (essentially a byte array) is added as a supported primitive value type parallel to arrays.
• New options for Engine which allows disabling if-expressions, switch-expressions, statement expressions, anonymous functions and/or looping (i.e. while, loop, do and for statements):
  • Engine::set_allow_if_expression
  • Engine::set_allow_switch_expression
  • Engine::set_allow_statement_expression
  • Engine::set_allow_anonymous_fn
  • Engine::set_allow_looping
• New strict variables mode for Engine (enabled via Engine::set_strict_variables) to throw parse errors on undefined variable usage. Two new parse error variants, ParseErrorType::VariableNotFound and ParseErrorType::ModuleNotFound, are added.

Enhancements

• Two double quotes ("") in a string literal now maps to "; two back-ticks (``) in a literal string now maps to `.
• Added Engine::register_type_with_name_raw to register a custom type based on a fully-qualified type path.
• Added into_array and into_typed_array for Dynamic.
• Added FnPtr::call and FnPtr::call_within_context to simplify calling a function pointer.
• A function's hashes are included in its JSON metadata to assist in debugging. Each function's baseHash field in the JSON object should map directly to the pre-calculated hash in the function call.
• Expression now derefs to Expr.

Deprecated and Gated API's

• NativeCallContext::new is deprecated because it is simpler to call a function pointer via FnPtr::call.
• AST::merge_filtered and AST::combine_filtered are no longer exported under no_function.
• AST::new and AST::new_with_source are moved under internals.
• FnPtr::call_dynamic is deprecated in favor of FnPtr::call_raw.

Bug fixes (potentially script-breaking)

• As originally intended, function calls with a bang (!) now operates directly on the caller's scope, allowing variables inside the scope to be mutated.
• As originally intended, Engine::XXX_with_scope API's now properly propagate constants within the provided scope also to functions in the script.
• Printing of integral floating-point numbers is fixed (used to only prints 0.0).
• func!() calls now work properly under no_closure.
• Fixed parsing of unary negation such that expressions like if foo { ... } -x parses correctly.

New features

• #[cfg(...)] attributes can now be put directly on plugin functions or function defined in a plugin module.
• A custom syntax parser can now return a symbol starting with $$ to inform the implementation function which syntax variant was actually parsed.
• AST::iter_literal_variables is added to extract all top-level literal constant/variable definitions from a script without running it.
• Engine::call_fn_dynamic is deprecated and Engine::call_fn_raw is added which allows keeping new variables in the custom scope.

Enhancements

• Array methods now avoid cloning as much as possible (although most predicates will involve cloning anyway if passed a closure).
• Array methods that take function pointers (e.g. closures) now optionally take the function name as a string.
• Array adds the dedup method.
• Array adds a sort method with no parameters which sorts homogeneous arrays of built-in comparable types (e.g. INT).
• Inlining is disabled for error-path functions because errors are exceptional and scripts usually fail completely when an error is encountered.
• The optimize module is completely eliminated under no_optimize, which should yield smaller code size.
• NativeCallContext::position is added to return the position of the function call.
• Scope::clone_visible is added that copies only the last instance of each variable, omitting all shadowed variables.

Deprecated API's

• NativeCallContext::call_fn_dynamic_raw is deprecated and NativeCallContext::call_fn_raw is added.
• From<EvalAltResult> for Result<T, Box<EvalAltResult> > is deprecated so it will no longer be possible to do EvalAltResult::ErrorXXXXX.into() to convert to a Result; instead, Err(EvalAltResult:ErrorXXXXX.into()) must be used. Code is clearer if errors are explicitly wrapped in Err.

Bug fixes

• Custom syntax starting with a disabled standard keyword now works properly.
• When calling Engine::call_fn, new variables defined during evaluation of the body script are removed and no longer spill into the function call.
• NamespaceRef::new is fixed.

Enhancements

Engine API

• Engine::consume_XXX methods are renamed to Engine::run_XXX to make meanings clearer. The consume_XXX API is deprecated.
• Engine::register_type_XXX are now available even under no_object.
• Added Engine::on_parse_token to allow remapping certain tokens during parsing.
• Added Engine::const_empty_string to merge empty strings into a single instance.

Custom Syntax

• $symbol$ is supported in custom syntax to match any symbol.
• Custom syntax with $block$, } or ; as the last symbol are now self-terminating (i.e. no need to attach a terminating ;).

Dynamic Values

• Dynamic::as_string and Dynamic::as_immutable_string are deprecated and replaced by into_string and into_immutable_string respectively.
• Added a number of constants to Dynamic.
• Added a number of constants and fromXXX constant methods to Dynamic.
• Added sin, cos and tan for Decimal values.

Decimal Values

• parse_float(), PI() and E() now defer to Decimal under no_float if decimal is turned on.
• Added log10() for Decimal.
• ln for Decimal is now checked and won't panic.

String Values

• SmartString now uses LazyCompact instead of Compact to minimize allocations.
• Added pop for strings.
• Added ImmutableString::ptr_eq to test if two strings point to the same allocation.
• The serde feature of SmartString is turned on under metadata to make Map serializable.

Scope API

• Scope::set_value now takes anything that implements Into<Cow<str> >.
• Added Scope::is_constant to check if a variable is constant.
• Added Scope::set_or_push to add a new variable only if one doesn't already exist.

AST API

• Added ASTNode::position.
• ReturnType is removed in favor of option flags for Stmt::Return.
• Stmt::Break and Stmt::Continue are merged into Stmt::BreakLoop via an option flag.
• StaticVec is changed to keep three items inline instead of four.

The official version 1.0.

Almost the same version as 0.20.3 but with deprecated API's removed.

Bug fixes

• Fixed infinite loop in certain script optimizations.
• Building for no-std no longer requires patching smartstring.
• Parsing a lone return or throw without a semicolon at the end of a block no longer raises an error.

Breaking changes

• All deprecated API's (e.g. the RegisterFn and RegisterResultFn traits) are removed.
• Module::set_id is split into Module::set_id and Module::clear_id pair.
• begin, end, each, then, unless are no longer reserved keywords.

Enhancements

• New methods is_odd, is_even for integers, and is_zero for all numbers.
• From<BTreeSet> and From<HashSet> are added for Dynamic, which create object maps with () values.