github.com/open-policy-agent/opa

This release contains improvements to monitoring and an assortment of fixes and improvements.

Monitoring

Surface unauthorized request count from OPA HTTP API authz handler via Status API

Currently when OPA's HTTP server rejects requests per the authz policy, this is not accounted for via the management APIs. This change adds that count in the metric registry that is part of the Status API for more visibility.

(#3378) authored by @ashutosh-narkar.

Surface more decision log errors via Status API

Previously in 5732, we updated the decision log plugin to surface errors via the Status API. However, in that change certain events like encoder errors and log drops due to buffer size limits had no metrics associated with them. This change adds more metrics for these events so that they can be surfaced via the Status API.

(#5637) authored by @ashutosh-narkar.

Include truncated HTTP response in logs

This change updates the client debug log to include the full HTTP response in case of non-200 status codes. Recording the response in the logs can help to provide more information to debug error scenarios.

(#2961) authored by @ashutosh-narkar reported by @gshively11.

Topdown and Rego

• Wasm: Add native support for object.union_n built-in function (authored by @Azanul)

Fixes

• ast: Properly set the reported location of unused variables in strict-mode errors. (#5662) authored by @boranx
• fmt: report wrong arity for built-in functions. (#5646) authored by @Trolloldem
• topdown: http.send(): Ensuring intra-query caching consistency. (#5736) authored by @johanfylling
• Performance improvements to decision logging. Specifically, by removing superfluous json encoding roundtrip and double work in AST conversion of to-be-logged events. (authored by @srenatus)

Docs, Website, and Ecosystem

• Fix typo in documentation (authored by @eternaltyro)
• Update TLS authentication docs (authored by @charlieegan3)
• Clarification in docs about checksums of Windows executables (authored by @Ronnie-personal)
• docs: Small fix to context placement in integration (authored by @craigpastro)
• docs/website: Fix floating navbar anchor issue (5774) authored by @charlieegan3 reported by @kristiansvalland

Miscellaneous

• Update -debug images to use Chainguard images (5544) (authored by @charlieegan3)
• Various third-party dependencies were updated.

This is a bug fix release that addresses a regression in 0.50.1. This regression impacts policies with rules that, as its else-value, assign a comprehension containing variables. Such rules would cause the compilation of the policy to fail with a rego_unsafe_var_error error.

E.g. the following policy would fail to compile with a policy.rego:5: rego_unsafe_var_error: var x is unsafe error:

package example

p {
	false
} else := [x | x := 1]

Fixes

• ast: Fixing bug where comprehensions in rule else-heads weren't rewritten correctly (#5771) authored by @johanfylling reported by @davidmdm

This is a bug fix release addressing the following issues:

Fixes

• ast/compile: Guard recursive module equality check. (#5756) authored by @philipaconrad. Resolves a performance regression when using large bundles.
• ast: Relaxing strict-mode check for unused args in else-branching functions (#5758) authored by @johanfylling reported by @ethanjli.

Miscellaneous

• Use normalized policy paths as compiler module keys and store IDs (authored by @ashutosh-narkar). Resolves an issue with bundle loading on Windows.

This release contains a mix of new features, bugfixes, security fixes, optimizations and build updates related to OPA's published images.

New Built-in Functions: JSON Schema Verification and Validation

These new built-in functions add functionality to verify and validate JSON Schema (#5486) (co-authored by @jkulvich and @johanfylling).

• json.verify_schema: Checks that the input is a valid JSON schema object
• json.match_schema: Checks that the document matches the JSON schema

See the documentation for all details.

Annotations scoped to package carries across modules

package scoped schema annotations are now applied across modules instead of only local to the module where it's declared (#5251) (authored by @johanfylling). This change may cause compile-time errors and behavioural changes to type checking when the schemas annotation is used, and to rules calling the rego.metadata.chain() built-in function:

• Existing projects with the same package declared in multiple files will trigger a rego_type_error: package annotation redeclared error if two or more of these are annotated with the package scope.
• If using the package scope, the schemas annotation will be applied to type checking also for rules declared in another file than the annotation declaration, as long as the package is the same.
• The chain of metadata returned by the rego.metadata.chain() built-in function will now contain an entry for the package even if the annotations are declared in another file, if the scope is package.

Remote bundle URL shorthand for run command

To load a remote bundle using opa run, the set directive can be provided multiple times as shown below:

 $ opa run -s --set "services.default.url=https://example.com" \
              --set "bundles.example.service=default" \
              --set "bundles.example.resource=/bundles/bundle.tar.gz" \
              --set "bundles.example.persist=true"

The following command can be used as a shorthand to easily start OPA with a remote bundle (#5674) (authored by @anderseknert):

$ opa run -s https://example.com/bundles/bundle.tar.gz

Performance Improvements for json.patch Built-in Function

Performance improvements in json.patch were achieved with the introduction of a new EditTree data structure, which is built for applying in-place modifications to an ast.Term, and can render the final result of all edits efficiently by applying all patches in a JSON-Patch sequence rapidly, and then collapsing all edits at the end with minimal wasted ast.Term copying (authored by @philipaconrad). For more details and benchmarks refer #5494 and #5390.

Surface decision log errors via status API

Errors encountered during decision log uploads will now be surfaced via the Status API in addition to being logged. This functionality should give users greater visibility into any issues OPA may face while processing, uploading logs etc (#5637) (authored by @ashutosh-narkar).

See the documentation for more details.

OPA Published Images Update

All published OPA images now run with a non-root uid/gid. The uid:gid is set to 1000:1000 for all images. As a result there is no longer a need for the -rootless image variant and hence it will be not be published as part of future releases. This change is in line with container security best practices. OPA can still be run with root privileges by explicitly setting the user, either with the --user argument for docker run, or by specifying the securityContext in the Kubernetes Pod specification.

Runtime, Tooling, SDK

• server: Support compression of response payloads if HTTP client supports it (#5310) authored by @AdrianArnautu
• bundle: Ensure the bundle resulting from merging a set of bundles does not contain nil data (#5703) authored by @anderseknert
• repl: Use lowercase for repl commands only and keep any provided arguments as-is (#5229) authored by @Trolloldem
• metrics: New endpoint /metrics/alloc_bytes to show OPA's memory utilization (#5715) authored by @anderseknert
• server: When using OPA TLS authorization, authz policy authors will now have access to the client certificates presented as part of the TLS connection. This new data will be available under the key client_certificates (#5538) authored by @charlieegan3
• server: Use streaming implementation of json.Decode rather than using an intermediate buffer for the incoming request (#5661) authored by @anderseknert

Topdown and Rego

• ast: Extend compiler strict mode check to include unused arguments (#5602) authored by @boranx. This change may cause compile-time errors for policies that have unused arguments in the scope when the strict mode is enabled. These variables could be replaced with _ (wildcard) or get cleaned up if they are not intended to be used in the body of the functions.
• ast: Respect inlined schemas annotations even if --schema flag isn't used (#5506) authored by @johanfylling
• ast: Force type-checker to respect allow_net capability when fetching remote schemas (#5670) authored by @johanfylling
• ast/parse: Provide custom parsing options that allow location information of AST nodes to be included in their JSON representation. This location information can be used by tools that work with the OPA AST (#3143) authored by @charlieegan3

Docs

• docs/policy-reference: Fix typo in policy reference doc (#5654) authored by @alvarogomez93
• docs/extensions: Fix sample code provided in the custom built-in implementation example (#5666) authored by @Ronnie-personal
• docs/bundles: Clarify delta bundle behavior when it contains an empty list of patch operations (#5629) authored by @charlieegan3
• docs/http-api-authz: Update the HTTP API authz tutorial with steps related to proper bundle creation (#5682) authored by @lamoboos223
• Fix broken 'future keywords' url link (#5686) authored by @neelanjan00

Website + Ecosystem

• Ecosystem:

  • Styra Load (#5659) authored by @charlieegan3

• Website:

  • Update OPA documentation search to use Algolia v3 (#5706) authored by @Parsifal-M
  • Drop Google Universal Analytics (UA) code as part of Google Analytics 4 migration (authored by @chalin)

Miscellaneous

• Dependency bumps, notably:
  • golang from 1.20.1 to 1.20.2
  • github.com/containerd/containerd from 1.6.16 to 1.6.19
  • github.com/golang/protobuf from 1.5.2 to 1.5.3
  • golang.org/x/net from 0.5.0 to 0.8.0
  • google.golang.org/grpc from 1.52.3 to 1.53.0
  • OpenTelemetry-related dependencies (#5701)

This release migrates the ORAS Go library from v1.2.2 to v2. The earlier version of the library had a dependency on the docker package. That version of the docker package had some reported vulnerabilities such as CVE-2022-41716, CVE-2022-41720. The ORAS Go library v2 removes the dependency on the docker package.

This is a bug fix release addressing the following Golang security issues:

Golang security fix CVE-2022-41723

A maliciously crafted HTTP/2 stream could cause excessive CPU consumption in the HPACK decoder, sufficient to cause a denial of service from a small number of small requests.

Golang security fix CVE-2022-41724

Large handshake records may cause panics in crypto/tls. Both clients and servers may send large TLS handshake records which cause servers and clients, respectively, to panic when attempting to construct responses.

Golang security fix CVE-2022-41722

A path traversal vulnerability exists in filepath.Clean on Windows. On Windows, the filepath.Clean function could transform an invalid path such as "a/../c:/b" into the valid path "c:\b". This transformation of a relative (if invalid) path into an absolute path could enable a directory traversal attack. After fix, the filepath.Clean function transforms this path into the relative (but still invalid) path ".\c:\b".