github.com/zclconf/go-cty

• stdlib: ContainsFunc now allows its second argument to be null, to test whether the given collection contains any null elements.
• stdlib: MergeFunc no longer panics if all of its arguments are null values of the same object type with at least one attribute.

cty now requires Go 1.25 or later.

• cty.Value.Equals now has a special case where if a null value is compared with a non-null value then only top-level marks from the non-null value will transfer to the boolean result.

  This is a limited introduction of the idea that only the parts of a nested data structure that were actually relevant to the comparison should transfer to the result. The more general form of that idea might follow in a later release, but that would require some more severe refactoring of this method's implementation that would be far riskier and so this is a pragmatic compromise to support just the relatively-common case of comparing with null in callers like HCL where an equality test is the canonical way to test a value for "null-ness".

• cty.IndexStep.Apply now works for traversing through a set.

  Although cty.Value.Index does not allow looking up a set element due to set elements not having indices, we often use cty.Path to describe a specific location in a nested structure and have a convention of handling traversal through a set as a cty.IndexStep whose "key" is the set element's value.

  To make that work a little better with code that uses cty.Path.Apply on such paths, cty.IndexStep now has a special case where if the given value is a set then it checks whether the index step's key is a member of the set and returns that value if so. If unknown values mean that it's not decidable whether there is a matching element then the result is an unknown value of the set's element type, so that traversal can continue and presumably eventually return an unknown value of the appropriate leaf type.

cty now requires Go 1.23 or later.

• cty.Value.Elements offers a modern iter.Seq2-based equivalent of cty.Value.ElementIterator.

• cty.DeepValues offers a modern iter.Seq2-based equivalent of cty.Walk.

• cty.Value.WrangleMarksDeep allows inspecting and modifying individual marks throughout a possibly-nested data structure.

  Having now got some experience using marks more extensively in some callers, it's become clear that it's often necessary for different subsystems to be able to collaborate using independent marks without upsetting each other's assumptions. Today that tends to be achieved using hand-written transforms either with cty.Transform or cty.Value.UnmarkDeepWithPaths/cty.Value.MarkWithPaths, both of which can be pretty expensive even in the common case where there are no marks present at all.

  This new function allows inspecting and transforming marks with far less overhead, by creating new values only for parts of a structure that actually need to change and by reusing (rather than recreating) the "payloads" of the values being modified when we know that only the marks have changed.

• cty.ValueMarksOfType and cty.ValueMarksOfTypeDeep make it easier to use type-based rather than value-based mark schemes, where different values of a common type are used to track a specific kind of relationship with multiple external values.

• cty.Value.HasMarkDeep provides a "deep" version of the existing cty.Value.HasMark, searching throughout a possibly-nested structure for any values that have the given mark.

• cty.Value.UnmarkDeep and cty.Value.UnmarkDeepWithPaths are now implemented in terms of cty.Value.WrangleMarksDeep, so they benefit from its reduced overhead. In particular they avoid reconstructing a data structure that contains no marked values at all.

• cty.Value.MarkWithPaths now has a fast path when it's given a zero-length PathValueMarks, in which case it just returns the value it was given with no modifications.

• cty.UnknownAsNull now accepts marked values and preserves the given marks in its result. Previously it had no direct support for marks and so would either panic or return incorrect results when given marked values.

• convert: Now generates more specific error messages in various cases of type conversion failure, giving additional information about the type that was given as compared to the type that was wanted by the caller.

• json: ImpliedType now returns an error if a JSON object contains two properties of the same name. As a compatibility concession it allows duplicates whose values have the same implied type, since it was unintentionally possible to combine ImpliedType and Unmarshal successfully in that case before, but this is not an endorsement of using duplicate property names since that makes the input ambiguous in any case. (#199)
• function/stdlib: ElementFunc no longer crashes when asked for a negative index into a tuple. This fixes a miss in the negative index support added back in v1.15.0. (#200)

• cty: Value.HasElement now treats unknown set elements consistently with how much of the rest of cty treats them.
• function/stdlib: FormatFunc and FormatListFunc now handle unknown and null values of unknown type as arguments, rather than letting the function system's short-circuit behavior take care of it. This allows cty.DynamicVal and cty.NullVal(cty.DynamicPseudoType) to be treated consistently with other values, returning results consistent with the documented behavior, rather than forcing the function to immediately return cty.DynamicVal.

• convert: When converting between two different capsule types, will now try to use the "conversion from" implementation from the target type if the source type doesn't have a suitable "conversion to" implementation. (#194)
• convert: When converting to a map whose element type is an object type with optional attributes, will no longer construct a broken result when a final map is empty. (#198)

• function: Function calls can now return more mark-related information when called with unknown values when neither AllowMarks nor AllowUnknown are set for a particular parameter. (#196)

• function/stdlib: The element function now accepts negative indices, extending the illusion of an infinitely-long list into the negative direction too.
• cty: The various "transform" functions were previously incorrectly propagating marks from a parent object down to attribute values when calling the caller-provided transform functions. The marks will now no longer be propagated downwards, which is consistent with the treatment of collection and tuple elements. If your transform function needs to take into account context about marks of upstream containers then you can maintain a stack of active marks in your Transformer implementation, pushing in Enter and popping in Exit. (#185)

• msgpack: Now uses string encoding instead of float encoding for a whole number that is too large to fit in any of MessagePack's integer types.
• function/stdlib: Type conversion functions (constructed with MakeToFunc) can now convert null values of unknown type into null values of the target type, rather than returning an unknown value in that case.
• json: Will now correctly reject attempts to encode cty.DynamicVal, whereas before it would just produce an invalid JSON document without any error. (This is invalid because JSON encoding cannot support unknown values at all; cty.DynamicVal is a special case of unknown value where even the type isn't known.)

• msgpack: Fixed edge-case bug that could cause loss of floating point precision when round-tripping due to incorrectly using a MessagePack integer to represent a large non-integral number. #176
• cty: Fixed some false-negative numeric equality test results by comparing numbers as integers when possible. #176

• convert: Converting from an unknown map value to an object type now correctly handles the situation where the map element type disagrees with an optional attribute of the target type, since when a map value is unknown we don't yet know which keys it has and thus cannot predict what subset of the elements will get converted as attributes in the resulting object. (#175)

• cty: It's now valid to use the Refine method on cty.DynamicVal, although all refinements will be silently discarded. This replaces the original behavior of panicking when trying to refine cty.DynamicVal.

• cty: Value.Range will now return a clearer panic message if called on a marked value. The "value range" concept is only applicable to unmarked values because not all of the ValueRange functions are able to propagate marks into their return values, due to returning Go primitive types instead of new cty.Value results.

  Callers that use marks must, as usual, take care to unmark them before exporting values into "normal" Go types, and then explicitly re-apply the marks to their result as appropriate. Applications that make no use of value marks, and library callers that exclude marked values from what they support, can safely ignore this requirement.

This release updates the supported version of Unicode from Unicode 13 to Unicode 15. This is a backwards-compatible change that means that cty supports normalization and segmentation of strings containing new Unicode characters. The algorithms for normalization and segmentation themselves are unchanged.

If you use cty in an application that cares about consistent Unicode support, you should upgrade to Go 1.21 at the same time as updating to cty v1.14, because that will then also update the Unicode tables embedded in the Go standard library (used for case folding, etc).

• cty: The cty.String type will now normalize incoming string values using the Unicode 15 normalization rules.
• function/stdlib: The various string functions which split strings into individual characters as part of their work will now use the Unicode 15 version of the text segmentation algorithm to do so.

• msgpack: As a compromise to avoid unbounded memory usage for a situation that some callers won't take advantage of anyway, the MessagePack decoder has a maximum length limit on encoded unknown value refinements. For consistency, the encoder will now truncate string prefix refinements if necessary to avoid making the encoded refinements too long. (#167)

  This is consistent with the documented conventions for serializing refinements -- that we can potentially lose detail through serialization -- but in this case we are still able to preserve shorter string prefixes, whereas other serializations tend to just discard refinement information altogether.

• cty: IndexStep.Apply will no longer panic if given a marked collection to traverse through. (#160).

• function: If a function parameter that doesn't declare AllowDynamicType: true recieves a cty.DynamicVal, the function system would previously just skip calling the function's Type callback and treat the result type as unknown. However, the Call method was then still calling a function's Impl callback anyway, which violated the usual contract that Type acts as a guard for Impl so Impl doesn't have to repeat type-checking already done in Type: it's only valid to call Impl if Type was previosly called and it succeeded.

  The function system will now skip calling Impl if it skips calling Type, immediately returning cty.DynamicVal in that case. Individual functions can opt out of this behavior by marking one or more of their parameters as AllowDynamicType: true and then handling that situation manually inside the Type and Impl callbacks.

  As a result of this problem, some of the function/stdlib functions were not correctly handling cty.DynamicVal arguments after being extended to support refinements in the v1.13.0 release, causing unexpected errors or panics when calling them. Those functions are fixed indirectly by this change, since their callbacks will no longer run at all in those cases, as was true before they were extended to support refinements.

Upgrade Notes

This release introduces a new concept called Refinements, which allow cty to constrain the range of an unknown value beyond just a type constraint and then make deductions about validity or result range based on those refinements.

These changes are consistent with the backward-compatibility policy but you may see some changed results in your unit tests of operations involving unknown values. If the new results don't seem like valid refinements of what was previously being returned in the v1.12 series, please open an issue to discuss that.

If the new results have a range that is a valid subset of the old results then that is expected behavior and you should update your tests as part of upgrading.

Other changes in this release

• Refinements: cty will can track a refined range for some unknown values and will take those into account when evaluating certain operations, thereby allowing a "more known" result than before. (#153)

• function/stdlib: The FormatDate and TimeAdd functions in previous releases were accidentally more liberal than intended in their interpretation of timestamp strings documented as requiring RFC3339. (#152)

  Those functions are now corrected to use a stricter RFC3339 parser, meaning that they will now reject some inputs that were previously accepted but were not valid per the RFC3339 syntax rules. The documentation for these functions already specified that RFC3339 syntax was required and so this is a fix to a defect rather than a breaking change, but calling applications which embed these functions may wish to pass on an upgrade note about this behavior difference in their own releaase notes after upgrading.