1.75.0 - 2024-06-03

Added

• Pro: Semgrep can now track taint through tuple/list (un)packing intra-procedurally (i.e., within a single function). For example:

  t = ["ok", "taint"]
  x, y = t
  sink(x) # OK, no finding
  sink(y) # tainted, finding
  ``` (code-6935)
  

• Optional type matching is supported in the Pro engine for Python. For example, in Python, Optional[str], str | None, and Union[str, None] represent the same type but in different type expressions. The optional type match support enables matching between these expressions, allowing any optional type expression to match any other optional type expression when used with metavariable-type filtering. It's important to note that syntactic pattern matching still distinguishes between these types. (code-6939)

• Add support for pnpm v9 (pnpm)

• Added a new rule option decorators_order_matters, which allows users to make decorators/ non-keyword attributes matching stricter. The default matching for attributes is order-agnostic, but if this rule option is set to true, non-keyword attributes (e.g. decorators in Python) will be matched in order, while keyword attributes (e.g. static, inline, etc) are not affected.

  An example usage will be a rule to detect any decorator that is outside of the route() decorator in Flask, since any decorator outside of the route() decorator takes no effect.

  bad: another.func() takes no effect

  @another.func("func") @app.route("route") def f(): pass

  ok: route() is the outermost decorator

  @app.route("route") @another.func("func") def f(): pass (saf-435)

Fixed

• Pro: taint-mode: Fixed issue causing findings to be missed (false negatives) when a global or class field was tainted, and then used in a sink after two or more function calls.

  For example:

  class Test {
      string bad;
  
      void test() {
          bad = "taint";
          foo();
      }
  
      void foo() {
          bar();
      }
  
      void bar() {
          sink(bad); // finding no longer missed
      }
  } (saf-1059)
  

• [Mostly applicable to Pro Engine] Typed metavariables will now match against the inferred type of a binding even if a constant is propagated for that binding, if we are unable to infer a type from the constant. Previously, we would simply fail to match in this case. (saf-1060)

• Removed the URLs at the end of the log when semgrep ci --dryrun is ran because dry run doesn't interact with the app so the URLs don't make sense. (saf-924)

1.74.0 - 2024-05-23

Fixed

• One part of interfile tainting was missing a constant propagation phase, which causes semgrep to miss some true positives in some cases during interfile analysis.

  This fix adds the missing constant propagation. (saf-1032)

• Semgrep now matches YAML tags (e.g. !number in !number 42) correctly rather than ignoring them. (saf-1046)

• Upgraded Semgrep's Dockerfile parser. This brings in various fixes from tree-sitter-dockerfile including minimal support for heredoc templates, support for variables in keys of LABEL instructions, support for multiple parameters for ADD and COPY instructions, tolerance for blanks after the backslash of a line continuation. As a result of supporting variables in LABEL keys, the multiple key/value pairs found in LABEL instructions are now treated as if they each had they own LABEL instruction. It allows a pattern LABEL a=b to match LABEL a=b c=d without the need for an ellipsis (LABEL a=b ...). Another consequence is that the pattern LABEL a=b c=d can no longer match LABEL c=d a=b but it will match a LABEL a=b instruction immediately followed by a separate LABEL c=d. (upgrade-dockerfile-parser)

1.73.0 - 2024-05-16

Added

• Added new AWS validator syntax for Secrets (scrt-278)

Fixed

• Fix couldn't find metavar $MT in the match results error, which may occur when we capture FQN with the metavariable and use metavariable-type filter on it. (code-7042)
• Fixes the crash (during scan) caused by improper handling of unicode characters present in the source code. (gh-8421)
• [Pro Engine Only] Tainted values are now tracked through instantiation of React functional components via JSX. (jsx-taint)

1.72.0 - 2024-05-08

Fixed

• Dockerfile support: Avoid a silent parsing error that was possibly accompanied with a segfault when parsing Dockerfiles that lack a trailing newline character. (gh-10084)

• Fixed bug that was preventing the use of metavariable-pattern with the aliengrep engine of the generic mode. (gh-10222)

• Added support for function declarations on object literals in the dataflow analysis.

  For example, previously taint rules would not have matched the following javascript code but now would.

  let tainted = source()
  let o = {
      someFuncDecl(x) {
          sink(tainted)
      }
  }
  ``` (saf-1001)
  

• Osemgrep only:

  When rules have metavariable-type, they don't show up in the SARIF output. This change fixes that.

  Also right now dataflow traces are always shown in SARIF even when --dataflow-traces is not passed. This change also fixes that. (saf-1020)

• Fixed bug in rule parsing preventing patternless SCA rules from being validated. (saf-1030)

1.71.0 - 2024-05-03

Added

• Pro: const-prop: Previously inter-procedural const-prop could only infer whether a function returned an arbitrary string constant. Now it will be able to infer whether a function returns a concrete constant value, e.g.:

  def bar():
    return "bar"
  
  def test():
    x = bar()
    foo(x) # now also matches pattern `foo("bar")`, previously only `foo("...")`
  ``` (flow-61)
  

• Python: const-prop: Semgrep will now recognize "..." * N expression as arbitrary constant string literals (thus matching the pattern "..."). (flow-75)

Changed

• The --beta-testing-secrets-enabled option, deprecated for several months, is now removed. Use --secrets as its replacement. (gh-9987)

Fixed

• When using semgrep --test --json, we now report in the config_missing_fixtests field in the JSON output not just rule files containing a fix: without a corresponding ".fixed" test file; we now also report rule files using a fix-regex: but without a corresponding a .fixed test file, and the fix: or fix-regex: can be in any rule in the file (not just the first rule). (fixtest)

• Fixes matching for go struct field tags metadata.

  For example given the program:

  type Rectangle struct {
      Top    int `json:"top"`
      Left   int `json:"left"`
      Width  int `json:"width"`
      Height int `json:"height"`
  }
  

  The pattern,

  type Rectangle struct {
      ...
      $NAME $TYPE $TAGS
      ...
  }
  

  will now match each field and the $TAGS metavariable will be bound when used in susequent patterns. (saf-949)

• Matching: Patterns of statements ending in ellipsis metavariables, such as

  x = 1
  $...STMTS
  

  will now properly extend the match range to accommodate whatever is captured by the ellipsis metavariable ($...STMTS). (saf-961)

• The SARIF output format should have the tag "security" when the "cwe" section is present in the rule. Moreover, duplicate tags should be de-duped.

  Osemgrep wasn't doing this before, but with this fix, now it does. (saf-991)

• Fixed bug in mix.lock parser where it was possible to fail on a python None error. Added handler for arbitrary exceptions during lockfile parsing. (sc-1466)

• Moved --historical-secrets to the "Pro Engine" option group, instead of "Output formats", where it was previously (in error). (scrt-570)

1.70.0 - 2024-04-24

Added

• Added guidance for resolving API token issues in CI environments. (gh-10133)

• The osemgrep show command supports 2 new options: dump-ast dump-pattern. See osemgrep show --help for more information. (osemgrep_show)

• Added additional output flags which allow you to write output to multiple files in multiple formats.

  For example, the comand semgrep ci --text --json-output=result.json --sarif-output=result.sarif.json Displays text output on stdout, writes the output that would be generated by passing the --json flag to result.json, and writes the output that would be generated by passing the --sarif to result.sarif.json. (saf-341)

• Added an experimental feature for users to use osemgrep to format SARIF output.

  When both the flags --sarif and --use-osemgrep-sarif are specified, semgrep will use the ocaml implementation to format SARIF.

  This flag is experimental and can be removed any time. Users must not rely on it being available. (saf-978)

Changed

• The main regex engine is now PCRE2 (was PCRE). While the syntax is mostly compatible, there are some minor instances where updates to rules may be needed, since PCRE2 is slightly more strict in some cases. For example, while we previously accepted [\w-.], such a pattern would now need to be written [\w.-] or [\w\-.] since PCRE2 rejects the first as having an invalid range. (scrt-467)

Fixed

• Semgrep LS now waits longer for users to login (gh-10109)

• When semgrep ci finishes scanning and uploads findings, it tells the app to mark the scan as completed.

  For large findings, this may take a while and marking the scan as completed may timeout. When a scan is not marked as completed, the app may show that the repo is still processing, and confuses the user.

  This change increases the timeout (previously 20 minutes) to 30 minutes. (saf-980)

• Fix semgrep ci --oss-only when secrets product is enabled. (scrt-223)

1.69.0 - 2024-04-16

Added

• Tracing: remove support for SEMGREP_OTEL_ENDPOINT and replace with --trace-endpoint <url>. This change is for an internal feature for debugging performance. (saf-885)

Changed

• Passing --debug to Semgrep will not print much, unless a set of tags is specified via LOG_TAGS. You can get all debug logs with LOG_TAGS=everything. We do not want --debug's output to be enourmous, as it tends not to be useful and yet cause some problems. Note that --debug is mainly intended for Semgrep developers, please ask for help if needed. (gh-10044)
• • The environment variables used to select the debug-level log messages are now prefixed with SEMGREP_ (or PYTEST_SEMGREP_) to avoid namespace pollution and undesired cross-application side effects. The supported environment variables are now SEMGREP_LOG_TAGS and PYTEST_SEMGREP_LOG_TAGS. (gh-10087)
• The implicit tag to show all debug-level log messages changes from everything to all. All debug-level messages shown by default are now tagged and selectable with a default tag. (gh-10089)

Fixed

• In generic mode (default, spacegrep engine), matching a pattern that ends with an ellipsis now favors the longest match rather than the shortest match when multiple matches are possible. For example, for a given target program a a b, the pattern a ... b will match a b as before but the pattern a ... will now match the longer a a b rather than a b. (gh-10039)
• Fixed the inter-file diff scan issue where the removal of pre-existing findings didn't work properly when adding a new file or renaming an existing file. (saf-897)

1.68.0 - 2024-04-08

Added

• Scan un-changed lockfiles in diff-aware scans (gh-9899)
• Languages: Added the QL language (used by CodeQL) to Semgrep (saf-947)
• SwiftPM parser will now report package url and reference. (sc-1218)
• Add support for Elixir (Mix) SCA parsing for pro engine users. (sc-1303)

Fixed

• Output for sarif format includes dataflow traces. (gh-10004)
• The environment variable LOG_LEVEL (as well as PYTEST_LOG_LEVEL) is no longer consulted by Semgrep to determine the log level. Only SEMGREP_LOG_LEVEL is consulted. PYTEST_SEMGREP_LOG_LEVEL is also consulted in the current implementation but should not be used outside of Semgrep's Pytest tests. This is to avoid accidentally affecting Semgrep when inheriting the LOG_LEVEL destined to another application. (gh-10044)
• Fixed swiftpm parser to no longer limit the amount of found packages in manifest file. (sc-1364)
• Fixed incorrect ecosystem being used for Elixir. Hex should be used instead of Mix. (sc-elixir)
• Fixed the match_based_ids of lockfile-only findings to differentiate between findings in cases where one rule produces multiple findings in one lockfile (sca-mid)
• Secrets historical scans: fixed a bug where historical scans could run on differential scans. (scrt-545)

1.67.0 - 2024-03-28

Added

• --historical-secrets flag for running Semgrep Secrets regex rules on git history (requires Semgrep Secrets). This flag is not yet implemented for --experimental. (scrt-531)

Changed

• Files with the .phtml extension are now treated as PHP files. (gh-10009)

• [IMPORTANT] Logged in users running semgrep ci will now run the pro engine by default! All semgrep ci scans will run with our proprietary languages (Apex and Elixir), as well as cross-function taint within a single file, and other single file pro optimizations we have developed. This is equivalent to semgrep ci --pro-intrafile. Users will likely see improved results if they are running semgrep ci and did not already have additional configuration to enable pro analysis.

  The current default engine does not include cross-file analysis. To scan with cross-file analysis, turn on the app toggle or pass in the flag --pro. We recommend this unless you have very large repos (talk to our support to get help enabling cross-file analysis on monorepos!)

  To revert back to our OSS analysis, pass the flag --oss-only (or use --pro-languages to continue to receive our proprietary languages).

  Reminder: because we release first to our canary image, this change will only immediately affect you if you are using semgrep/semgrep:canary. If you are using semgrep/semgrep:latest, it will affect you when we bump canary to latest. (saf-845)

Fixed

• Fixed a parsing error in Kotlin when there's a newline between the class name and the primary constructor.

  This could not parse before

  class C
  constructor(arg:Int){}
  

  because of the newline between the class name and the constructor.

  Now it's fixed. (saf-899)

1.66.2 - 2024-03-26

Added

• osemgrep now respects HTTP_PROXY and HTTPS_PROXY when making network requests (cdx-253)

Changed

• [IMPORTANT] The public rollout of inter-file differential scanning has been temporarily reverted for further polishing of the feature. We will reintroduce it in a later version. (saf-268)

Fixed

• Autofix on variable definitions should now handle the semicolon in Java, C++, and C#. (saf-928)

1.66.1 - 2024-03-25

Fixed

• Autofix on variable definitions should now handle the semicolon in Rust, Cairo, Solidity, Dart. (autofix_vardef)
• [IMPORTANT] we restored bash, jq, and curl in our semgrep docker image as some users were relying on it. We might remove them in the futur but in the mean time we restored the packages and if we remove them we will announce it more loudly. We also created a new page giving more information about our policy for our docker images: https://semgrep.dev/docs/semgrep-ci/packages-in-semgrep-docker/ (docker_bash)
• Fixed autofix application on lines containing multi-byte characters. (multibyte)

1.66.0 - 2024-03-19

Added

• Added information about interfile pre-processing to --max-memory help. (gh-9932)
• We've implemented basic support for the yield keyword in Python. The Pro engine now detects taint findings from taint sources returned by the yield keyword. (saf-281)

Changed

• osemgrep --remote will no longer clone into a tmp folder, but instead the CWD (cdx-remote)

• [IMPORTANT] Inter-file differential scanning is now enabled for all Pro users.

  Inter-file differential scanning is now enabled for all Pro users. While it may take longer than intra-file differential scanning, which is the current default for pro users, it offers deeper analysis of dataflow paths compared to intra-file differential scanning. Additionally, it is significantly faster than non-differential inter-file scanning, with scan times reduced to approximately 1/10 of the non-differential inter-file scan. Users who enable the pro engine and engage in differential PR scans on GitHub or GitLab may experience the impact of this update. If needed, users can revert to the previous intra-file differential scan behavior by configuring the --no-interfile-diff-scan command-line option. (saf-268)

Fixed

• The official semgrep docker image does not contain anymore the bash, jq, and curl utilities, to reduce its attack surface. (saf-861)

1.65.0 - 2024-03-11

Changed

• Removed the extract-mode rules experimental feature. (extract_mode)

1.64.0 - 2024-03-07

Changed

• Removed the AST caching experimental feature (--experimental --ast-caching in osemgrep and -parsing_cache_dir in semgrep-core). (ast_caching)
• Removed the Registry caching experimental feature (--experimental --registry-caching) in osemgrep. (registry_caching)

Fixed

• Clean any credentials from project URL before using it, to prevent leakage. (saf-876)
• ci: Updated logic for informational message printed when no rules are sent to correctly display when secrets is enabled (in additional to when code is). (scrt-455)

1.63.0 - 2024-02-27

Added

• Dataflow: Added support for nested record patterns such as { body: { param } } in the LHS of an assignment. Now given { body: { param } } = tainted Semgrep will correctly mark param as tainted. (flow-68)
• Matching: metavariable-regex can now match on metavariables of interpolated strings which use variables that have known values. (saf-865)
• Add support for parsing Swift Package Manager manifest and lockfiles (sc-1217)

Fixed

• fix: taint signatures do not capture changes to parameters' fields (flow-70)
• Scan summary links printed after semgrep ci scans now reflect a custom SEMGREP_APP_URL, if one is set. (saf-353)

1.62.0 - 2024-02-22

Added

• Pro: Adds support for python constructors to taint analysis.

  If interfile naming resolves that a python constructor is called taint will now track these objects with less heuristics. Without interfile analysis these changes have no effect on the behavior of tainting. The overall result is that in the following program the oss analysis would match both calls to sink while the interfile analysis would only match the second call to sink.

  class A:
      untainted = "not"
      tainted = "not"
      def __init__(self, x):
      	self.tainted = x
  
  a = A("tainted")
  # OK:
  sink(a.untainted)
  # MATCH:
  sink(a.tainted)
  ``` (ea-272)
  

• Pro: taint-mode: Added basic support for "index sensitivity", that is, Semgrep will track taint on individual indexes of a data structure when these are constant values (integers or strings), and the code uses the built-in syntax for array indexing in the corresponding language (typically E[i]). For example, in the Python code below Semgrep Pro will not report a finding on sink(x) or sink(x[1]) because it will know that only x[42] is tainted:

  x[1] = safe
  x[42] = source()
  sink(x) // no more finding
  sink(x[1]) // no more finding
  sink(x[42]) // finding
  sink(x[i]) // finding
  

  There is still a finding for sink(x[i]) when i is not constant. (flow-7)

Changed

• taint-mode: Added exact: false sinks so that one can specify that anything inside a code region is a sink, e.g. if (...) { ... }. This used to be the semantics of sink specifications until Semgrep 1.1.0, when we made sink matching more precise by default. Now we allow reverting to the old semantics.

  In addition, when exact: true (the default), we simplified the heuristic used to support traditional sink(...)-like specs together with the option taint_assume_safe_functions: true, now we will consider that if the spec formula is not a patterns with a focus-metavarible, then we must look for taint in the arguments of a function call. (flow-1)

• The project name for repos scanned locally will now be local_scan/<repo_name> instead of simply <repo_name>. This will clarify the origin of those findings. Also, the "View Results" URL displayed for findings now includes the repository and branch names. (saf-856)

Fixed

• taint-mode: experimental: For now Semgrep CLI taint traces are not adapted to support multiple labels, so Semgrep picks one arbitrary label to report, which sometimes it's not the desired one. As a temporary workaround, Semgrep will look at the requires of the sink, and if it has the shape A and ..., then it will pick A as the preferred label and report its trace. (flow-65)
• Fixed trailing newline parsing in pyproject.toml and poetry.lock files. (gh-9777)
• Fixed an issue that led to incorrect autofix application in certain cases where multiple fixes were applied to the same line. (saf-863)
• The tokens for type parameters brackets are now stored in the generic AST allowing to correctly autofix those constructs. (tparams)

1.61.1 - 2024-02-14

Added

• Added performance metrics using OpenTelemetry for better visualization. Users wishing to understand the performance of their Semgrep scans or to help optimize Semgrep can configure the backend collector created in libs/tracing/unix/Tracing.ml.

  This is experimental and both the implementation and flags are likely to change. (ea-320)

• Created a new environment variable SEMGREP_REPO_DISPLAY_NAME for use in semgrep CI. Currently, this does nothing. The goal is to provide a way to override the display name of a repo in the Semgrep App. (gh-8953)

• The OCaml/C executable (semgrep-core or osemgrep) is now passed through the strip utility, which reduces its size by 10-25% depending on the platform. Contribution by Filipe Pina (@fopina). (gh-9471)

Changed

• "Missing plugin" errors (i.e., rules that cannot be run without --pro) will now be grouped and reported as a single warning. (ea-842)

1.60.1 - 2024-02-09

Added

• Rule syntax: Metavariables by the name of $_ are now anonymous, meaning that they do not unify within a single pattern or across patterns, and essentially just unconditionally specify some expression.

  For instance, the pattern foo($_, $_) may match the code foo(1, 2).

  This will change the behavior of existing rules that use the metavariable $_, if they rely on unification still happening. This can be fixed by simply giving the metavariable a real name like $A. (ea-837)

• Added infrastructure for semgrep supply chain in semgrep-core. Not fully functional yet. (ssc-port)

Changed

• Dataflow: Simplified the IL translation for Python with statements to let symbolic propagation assume that with foo() as x: ... entails x = foo(), so that e.g. Session().execute("...") matches:

  with Session() as s:
      s.execute("SELECT * from T") (CODE-6633)
  

Fixed

• Output: Semgrep CLI now no longer sometimes interpolated metavariables twice, if the message that was substituted for a metavariable itself contained a valid metavariable to be interpolated (ea-838)

1.60.0 - 2024-02-08

Added

• Rule syntax: Metavariables by the name of $_ are now anonymous, meaning that they do not unify within a single pattern or across patterns, and essentially just unconditionally specify some expression.

  For instance, the pattern foo($_, $_) may match the code foo(1, 2).

  This will change the behavior of existing rules that use the metavariable $_, if they rely on unification still happening. This can be fixed by simply giving the metavariable a real name like $A. (ea-837)

• Added infrastructure for semgrep supply chain in semgrep-core. Not fully functional yet. (ssc-port)

Fixed

• Output: Semgrep CLI now no longer sometimes interpolated metavariables twice, if the message that was substituted for a metavariable itself contained a valid metavariable to be interpolated (ea-838)

1.59.1 - 2024-02-02

Added

• taint-mode: Pro: Semgrep can now track taint via static class fields and global variables, such as in the following example:

  static char* x;
  
  void foo() {
      x = "tainted";
  }
  
  void bar() {
      sink(x);
  }
  
  void main() {
      foo();
      bar();
  }
  ``` (pa-3378)
  
  
  

Fixed

• Pro: Make inter-file analysis more tolerant to small bugs, resorting to graceful degradation and continuing with the scan, rather than crashing. (pa-3387)

1.59.0 - 2024-01-30

Added

• Swift: Now supports typed metavariables, such as ($X : ty). (pa-3370)

Changed

• Add Elixir to Pro languages list in help information. (gh-9609)

• Removed sg alias to avoid naming conflicts with the shadow-utils sg command for Linux systems. (gh-9642)

• Prevent unnecessary computation when running scans without verbose logging enabled (gh-9661)

• Deprecated option taint_match_on introduced in 1.51.0, it is being renamed to taint_focus_on. Note that taint_match_on was experimental, and taint_focus_on is experimental too. Option taint_match_on will continue to work but it will be completely removed at some point after 1.63.0. (pa-3272)

• Added information on product-related flags to help output, especially for Semgrep Secrets. (pa-3383)

• taint-mode: Improve inference of best matches for exact-sources, exact-sanitizers, and sinks. Now we also avoid FPs in cases such as:

  dangerouslySetInnerHTML = {
    // ok:
    {__html: props ? DOMPurify.sanitize(props.text) : ''} // no more FPs!
  }
  

  where props is tainted and the sink specification is:

  patterns:
    - pattern: |
       dangerouslySetInnerHTML={{__html: $X}}
    - focus-metavariable: $X
  

  Previously Semgrep wrongly considered the individual subexpressions of the conditional as sinks, including the props in props ? ..., thus producing a false positive. Now it will only consider the conditional expression as a whole as the sink. (rules-6457)

• Removed an internal legacy syntax for secrets rules (mode: semgrep_internal_postprocessor). (scrt-320)

Fixed

• Autofix: Fixes that span multiple lines will now try to align inserted fixed lines with each other. (gh-3070)

• Matching: Try blocks with catch clauses can now match try blocks that have extraneous catch clauses, as long as it matches a subset. For instance, the pattern

  try:
    ...
  catch A:
    ...
  

  can now match

  try:
    ...
  catch A:
    ...
  catch B:
    ...
  ``` (gh-3362)
  

• Previously, some people got the error:

  Encountered error when running rules: Other syntax error at line NO FILE INFO YET:-1:
  Invalid_argument: String.sub / Bytes.sub
  

  Semgrep should now report this error properly with a file name and line number and handle it gracefully. (gh-9628)

• Fixed Dockerfile parsing bug where multiline comments were parsed incorrectly. (gh-9628-2)

• The language server will now properly respect findings that have been ignored via the app (lsp-fingerprints)

• taint-mode: Pro: Semgrep will now propagate taint via instance variables when calling methods within the same class, making this example work:

  class Test {
  
    private String str;
  
    public setStr() {
      this.str = "tainted";
    }
  
    public useStr() {
      //ruleid: test
      sink(this.str);
    }
  
    public test() {
      setStr();
      useStr();
    }
  
  }
  ``` (pa-3372)
  

• taint-mode: Pro: Taint traces will now reflect when taint is propagated via class fields, such as in this example:

  class Test {
  
    private String str;
  
    public setStr() {
      this.str = "tainted";
    }
  
    public useStr() {
      //ruleid: test
      sink(this.str);
    }
  
    public test() {
      setStr();
      useStr();
    }
  
  }
  

  Previously Semgrep will report that taint originated at this.str = "tainted", but it would not tell you how the control flow got there. Now the taint trace will indicate that we get there by calling setStr() inside test(). (pa-3373)

• Addressed an issue related to matching top-level identifiers with meta-variable qualified patterns in C++, such as matching ::foo with ::$A::$B. This problem was specific to Pro Engine-enabled scans. (pa-3375)

1.58.0 - 2024-01-23

Added

• Added a severity icon (e.g. "❯❯❱") and corresponding color to our CLI text output for findings of known severity. (grow-97)

• Naming has better support for if statements. In particular, for languages with block scope, shadowed variables inside if-else blocks that are tainted won't "leak" outside of those blocks.

  This helps with features related to naming, such as tainting.

  For example, previously in Go, the x in sink(x) will report that x is tainted, even though the x that is tainted is the one inside the scope of the if block.

  func f() {
    x := "safe";
    if (c) {
      x := "tainted";
    }
    // x should not be tainted
    sink(x);
  }
  

  This is now fixed. (pa-3185)

• OSemgrep can now scan remote git repositories. Pass --experimental --pro --remote http[s]://<website>/.../<repo>.git to use this feature (pa-remote)

Changed

• Rules stored under an "hidden" directory (e.g., dir/.hidden/myrule.yml) are now processed when using --config <dir>. We used to skip dot files under dir, but keeping rules/.semgrep.yml, but not path/.github/foo.yml, but keeping src/.semgrep/bad_pattern.yml but not ./.pre-commit-config.yaml, ... This was mainly because we used to fetch rules from ~/.semgrep/ implicitely when --config was not given, but this feature was removed, so now we can keep it simple. (hidden_rules)
• Removed support for writing rules using jsonnet. This feature will be restored once we finish the port to OCaml of the semgrep CLI. (jsonnet)
• The primitive object construct expression will no longer match the new expression pattern. For example, the pattern new $TYPE will now only match new int, not int(). (pa-3336)
• The placement new expression will no longer match the new expression without placement. For instance, the pattern new ($STORAGE) $TYPE will now only match new (storage) int and not new int. (pa-3338)

Fixed

• Java: You can now use metavariable ellipses properly in function arguments, as statements, and as expressions.

  For instance, you may write the pattern

  public $F($...ARGS) { ... }
  ``` (gh-9260)
  

• Nosemgrep: Fixed a bug where Semgrep would err upon reading a nosemgrep comment with multiple rule IDs. (gh-9463)

• Fixed bugs in gitignore/semgrepignore globbing implementation affecting --experimental. (gh-9544)

• Fixed rule IDs, descriptions, findings, and autofix text not wrapping as expected. Use newline instead of horiziontal separator for findings with a shared file but for different rules per design spec. (grow-97)

• Keep track of the origin of return; statements in the dataflow IL so that recently added (Pro-only) at-exit: true sinks work properly on them. (pa-3337)

• C++: Improve translation of delete expressions to the dataflow IL so that recently added (Pro-only) at-exit: true sinks work on them. Previously delete expression at "exit" positions were not being properly recognized as such. (pa-3339)

• cli: fix python runtime error with 0 width wrapped printing (pa-3366)

• Fixed a bug where Gemfile.lock files with multiple GEM sections would not be parsed correctly. (sc-1230)

1.57.0 - 2024-01-18

Added

• Added a severity icon (e.g. "❯❯❱") and corresponding color to our CLI text output for findings of known severity. (grow-97)

• Naming has better support for if statements. In particular, for languages with block scope, shadowed variables inside if-else blocks that are tainted won't "leak" outside of those blocks.

  This helps with features related to naming, such as tainting.

  For example, previously in Go, the x in sink(x) will report that x is tainted, even though the x that is tainted is the one inside the scope of the if block.

  func f() {
    x := "safe";
    if (c) {
      x := "tainted";
    }
    // x should not be tainted
    sink(x);
  }
  

  This is now fixed. (pa-3185)

• OSemgrep can now scan remote git repositories. Pass --experimental --pro --remote http[s]://<website>/.../<repo>.git to use this feature (pa-remote)

Changed

• Rules stored under an "hidden" directory (e.g., dir/.hidden/myrule.yml) are now processed when using --config <dir>. We used to skip dot files under dir, but keeping rules/.semgrep.yml, but not path/.github/foo.yml, but keeping src/.semgrep/bad_pattern.yml but not ./.pre-commit-config.yaml, ... This was mainly because we used to fetch rules from ~/.semgrep/ implicitely when --config was not given, but this feature was removed, so now we can keep it simple. (hidden_rules)
• The primitive object construct expression will no longer match the new expression pattern. For example, the pattern new $TYPE will now only match new int, not int(). (pa-3336)
• The placement new expression will no longer match the new expression without placement. For instance, the pattern new ($STORAGE) $TYPE will now only match new (storage) int and not new int. (pa-3338)

Fixed

• Java: You can now use metavariable ellipses properly in function arguments, as statements, and as expressions.

  For instance, you may write the pattern

  public $F($...ARGS) { ... }
  ``` (gh-9260)
  

• Fixed bugs in gitignore/semgrepignore globbing implementation affecting --experimental. (gh-9544)

• Fixed rule IDs, descriptions, findings, and autofix text not wrapping as expected. Use newline instead of horiziontal separator for findings with a shared file but for different rules per design spec. (grow-97)

• Keep track of the origin of return; statements in the dataflow IL so that recently added (Pro-only) at-exit: true sinks work properly on them. (pa-3337)

• C++: Improve translation of delete expressions to the dataflow IL so that recently added (Pro-only) at-exit: true sinks work on them. Previously delete expression at "exit" positions were not being properly recognized as such. (pa-3339)

• Fixed a bug where Gemfile.lock files with multiple GEM sections would not be parsed correctly. (sc-1230)

1.56.0 - 2024-01-10

Added

• Added a new field that breaks down the number of findings per product in the metrics that are sent out by the CLI. This will help Semgrep understand users better. (pa-3312)

1.55.2 - 2024-01-05

Fixed

• taint-mode: Semgrep was missing some sources occurring inside type expressions, for example:

  char *p = new char[source(x)];
  sink(x);
  

  Now, if x is tainted by side-effect, Semgrep will check x inside the type expression char[...] and record it as tainting, and generate a finding for sink(x). (pa-3313)

• taint-mode: C/C++: Sanitization by side-effect was not working correctly for ptr->fld l-values. In particular, if ptr is tainted, and then ptr->fld is sanitized, Semgrep will now correctly consider ptr->fld as clean. (pa-3328)

1.55.1 - 2024-01-04

Fixed

• Honor temporary folder specified via the TMPDIR environment variable (or equivalent on Windows) in some instances where it used to be hardcoded as /tmp. (gh-9534)
• Fix pipfile manifest parser error (sc-1084)

Changed

• The rule option commutative_compop has been renamed to symmetric_eq. It is deprecated and will be removed after the 1.60.0 release. (gh-9496)

1.54.3 - 2023-12-22

Added

• Pro only: taint-mode: Added experimental at-exit: true option for sinks, that makes a sink spec only apply on the "exit" instructions/statements of a function. That is, the instructions after which the control-flow exits the function. This is useful for writing rules to find "leaks", such as checking that file descriptors are being closed within the same function where they were opened.

  For example, given this taint rule:

  pattern-sources:
    - by-side-effect: true
      patterns:
        - pattern: $FILE = open(...)
        - focus-metavariable: $FILE
  pattern-sanitizers:
    - by-side-effect: true
      patterns:
        - pattern: $FILE.close(...)
        - focus-metavariable: $FILE
  pattern-sinks:
    - at-exit: true
      pattern: |
        def $FUN(...):
          ...
  

  Semgrep will report a finding in the code below since at print(content), after which the control flow reaches the exit of the function, the file has not yet been closed:

  def test():
      file = open("test.txt")
      content = file.read()
      print(content) # FINDING
  ``` (pa-3266)

1.54.2 - 2023-12-21

Added

• metrics: added more granular information about pro engine configurations to help differentiate scans using different engine capabilities. For instance, maintainers are now able to distinguish intraprocedural scans without secrets validation from intraprocedural scans with secrets validation. This allows us to have a better understanding of usage and more accurately identify product-specific issues (e.g., to see if something only affects secrets scans). (ea-297)

Fixed

• Revise error message when running semgrep ci without being logged in to clarify that --config is used with semgrep scan. (gh-9485)

1.54.1 - 2023-12-20

No significant changes.

1.54.0 - 2023-12-19

Added

• Pro only: taint-mode: In a function/method call, it is now possible to arbitrarily propagate taint between arguments and the callee. For example in C, one can propagate taint from the second argument of strcat to the first, that is, strcat($TO, $FROM). Another example, in C++ one can propagate taint from the left operand of >> to the right one, that is, $FROM >> $TO. (pa-3131)
• Semgrep IDE integrations will now cache workspace targets, so a full traversal of a workspace is no longer needed on every scan (pdx-148)

Changed

• OCaml: switch to using the tree-sitter based parser instead of the menhir parser, which has a more complete AST, especially for objects and classes. (ocaml)

Fixed

• solidity: support ellipsis in for loops header in the init part. (gh-9431)

• taint-mode: Fixed recently added by-side-effect: only option for taint sources, so that it does not incorrectly taint expressions that are not l-values, e.g. given this taint source:

  pattern-sources:
    - by-side-effect: only
      patterns:
        - pattern: delete $VAR;
        - focus-metavariable: $VAR
  

  The get(*from) expression should not become tainted since it's not an l-value:

  delete get(*from);
  ``` (pa-2980)
  

• In C++, the string literal now has a type of char *. It won't match with the string type. For instance,

  - metavariable-type:
      metavariable: $EXPR
      type: string
  

  will only match

  string f;
  // MATCH
  int x = f.length();
  

  but not

  const char *s;
  // OK
  s = "foo";
  ``` (pa-3236)
  

• taint-mode: Semgrep will now treat lambdas' parameters as fresh, so a taint rule that finds double-delete's should not be triggered on the code below:

  for (ListNode *node : list) {
  	list.erase(node, [](ListNode *p) {
  		delete p;
  	});
  }
  ``` (pa-3298)
  

• Fixed bug where empty tables in pyproject.toml files would fail to parse (sc-1196)

1.53.0 - 2023-12-12

Added

• Users can now ignore findings locally in Semgrep IDE Extensions, per workspace, and this will persist between restarts (pdx-154)
• A new subcommand 'semgrep test', which is an alias for 'semgrep scan --test'. This means that if you were running semgrep on a test directory, you will now have to use 'semgrep scan test' otherwise it will be interpreted as the new 'semgrep test' subcommand. (subcommand_test)

Changed

• Handling qualified identifiers in constant propagation

  We've added support for qualified identifiers in constant propagation. Notably, this enables the following matches (with the pro engine):

  rules:
    - id: cpp-const-field
      languages:
        - cpp
      message: testing
      severity: INFO
      pattern: std::cout<<1
  

  #include<iostream>
  #include "a.h"
  
  namespace B {
  class Bar {
      public:
          static const int one = 1;
  };
  }
  
  int main() {
      // ruleid: cpp-const-field
      std::cout<<1<<std::endl;
  
      // ruleid: cpp-const-field
      std::cout<<A::Foo::one<<std::endl;
  
      // ruleid: cpp-const-field
      std::cout<<B::Bar::one<<std::endl;
  }
  ``` (gh-9354)
  
  
  

Fixed

• Updated the parser used for Rust. The largest change relates to how macros are parsed. (rust)

1.52.0 - 2023-12-05

Added

• Java: Semgrep will now recognize String.format(...) expressions as constant strings when all their arguments are constant, but it will still not know what exact string it is. For example, code String.format("Abc %s", "123") will match pattern "..." but it will not match pattern "Abc 123". (pa-3284)

Changed

• Inter-file diff scan will be gradually introduced to a small percentage of users through a slow rollout process. Users who enable the pro engine and engage in differential PR scans on Github or Gitlab may experience the impact of this update. (ea-268)
• secrets: now performs more aggressive deduplication for instances where an invalid and valid match are reported at the same range. Instead of reporting both, we now report only the valid match when they are otherwise visually identical. (scrt-271)

Fixed

• In expression-based languages, definitions are also expressions.

  This change allows dataflow to properly handle definition expressions.

  For example, the pattern 0 == 0 will match x == 0 in

  def f(c) do
    x = (y = 0)
    x == 0
  end
  

  because now dataflow is able to handle the expression y = 0. (pa-3262)

• In version 1.14.0 (pa-2477) we made sink-matching more precise when the sink specification was like:

  pattern-sinks:
    - patterns:
       - pattern: sink($X, ...)
       - focus-metavariable: $X
  

  Where the sink specification most likely has the intent to specify the first argument of sink as a sink, and sink(ok1 if tainted else ok2) should NOT produce a finding, because tainted is not really what is being passed to the sink function.

  But we only intercepted the most simple pattern above, and more complex sink specifications that had the same intent were not properly recognized.

  Now we have generalized that pattern to cover more complex cases like:

  patterns:
   - pattern-either:
     - patterns:
       - pattern-inside: |
           def foo(...):
             ...
       - pattern: sink1($X)
     - patterns:
       - pattern: sink2($X)
       - pattern-not: bar(...)
   - focus-metavariable: $X
  ``` (pa-3284)
  

• Updated the parser used for Rust (rust)

1.51.0 - 2023-11-29

Added

• taint-mode: Added experimental rule option taint_match_on: source that makes Semgrep report taint findings on the taint source rather than on the sink. (pa-3272)

Changed

• Elixir got moved to Pro. (elixir_pro)
• The 'fix_regex' field has been removed from the semgrep JSON output. Instead, the 'fix' field contains the content the result of the fix_regex. (fix_regex)
• taint-mode: Tweaked experimental option taint_only_propagate_through_assignments so that when it is enabled, tainted.field and tainted(args) will no longer propagate taint. (pa-2193)

Fixed

• Fixed Kotlin parse error.

  Previously, code like this would throw a parse error

  fun f1(context : Context) {
      Foo(context).elem = var1
  }
  

  due to not recognizing Foo(context).elem = ... as valid. Now calls are recognized as valid in the left hand of assignments. (ea-104)

• Python: async statements are now translated into the Dataflow IL so Semgrep will be able to report findings e.g. inside async with ... statements. (gh-9182)

• In gitlab output, use correct url attached to rule instead of generating it. This fixes url for supply chain findings. (gitlab)

• • The language server will no longer crash on startup for intellij (language-server)
• • The language server no longer crashes when installed through pip on Mac platforms (language-server-macos)

• taint-mode: When we encountered an assignment lval := expr where expr returned no taints, we automatically cleaned lval. This was correct in the early days of taint-mode, before we introduced taint by side-effect, but it is wrong now. The LHS lval may be tainted by side-effect, in which case we cannot clean it just because expr returns no taint. Now that we introduced by-side-effect: only it is also possible for expr to taint lval by side-effect and return no immediate taint.

  This kind of source should now work as expected:

  - by-side-effect: true
    patterns:
      - pattern: |
          $X = source()
      - focus-metavariable: $X
  ``` (pa-3164)
  

• taint-mode: Fixed a bug in the recently added by-side-effect: only option causing that when matching l-values of the form l.x and l[i], the l occurence would unexpectedly become tainted too. This led to FPs in some typestate rules like those checking for double-lock or double-free.

  Now a source such as:

  - by-side-effect: only
    patterns:
    - pattern: lock($L)
    - focus-metavariable: $L
  

  will not produce FPs on code such as:

  lock(obj.l)
  unlock(obj.l)
  lock(obj.l)
  ``` (pa-3282)
  

• taint-mode: Removed a hack that made lval = new ... assignments to not clean the lval despite the RHS was not tainted. This caused FPs in double-free rules. For example, given this source:

  pattern-sources:
    - by-side-effect: only
      patterns:
        - pattern: delete $VAR;
        - focus-metavariable: $VAR
  

  And the code below:

  while (nondet) {
    int *v = new int;
    delete v; // FP
  }
  

  The delete v statement was reported as a double-free, because Semgrep did not consider that v = new int would clean the taint in v. (pa-3283)

1.50.0 - 2023-11-17

No significant changes.

1.49.0 - 2023-11-15

Added

• Added support in Ruby, Julia, and Rust to match implicit return statement inside functions.

  For example:

  return 0
  

  can now match 0 in

  function f()
    0
  end
  

  This matching is enabled by default and can be disabled with the rule option implicit_return. (gh-8408)

• Pro engine supports constant propagation of numbers defined via macro in C++ (gh-9221)

• taint-mode: The by-side-effect option for taint sources (only) now accepts a third value only (besides true and false). Setting by-side-effect: only will define a taint source that only propagates by side effect. This option should allow (ab)using taint-mode for writing some typestate rules.

  For example, this taint rule:

  pattern-sources:
    - by-side-effect: only
      patterns:
      - pattern: lock($L)
      - focus-metavariable: $L
  pattern-sanitizers:
    - by-side-effect: true
      patterns:
      - pattern: unlock($L)
      - focus-metavariable: $L
  pattern-sinks:
    - pattern: lock($L)
  

  will match the second lock(x) in this code:

  lock(x) # no finding
  lock(x) # finding
  

  The first lock(x) will not result in any finding, because the occurrence of x in itself will not be tainted. Only after the function call we will record that x is tainted (as a side-effect of lock). The second lock(x) will result in a finding because the x has been tainted by the previous lock(x). (pa-2980)

Changed

• In the metrics sent we now record the languages for which we invoked the interfile engine. This will enable us to measure the performance impact and error rates of new interfile languages. (For scans which don't send metrics, there is no change.) See the PRIAVCY.md for more information. (ea-251)

• Removed support for named snippets (org_name:rule_id) from semgrep scan which were removed from semgrep.dev a few months ago. (gh-9203)

• Added support for --config <code|secrets> to semgrep scan. When using code or secrets, the environment variable SEMGREP_REPO_NAME must be set.

  For example,

  $ SEMGREP_REPO_NAME=test_repo semgrep --config secrets
  

  Internally, semgrep scan --config <product> now uses the same endpoint as the semgrep ci to fetch the scan configuration. (gh-9205)

• Improved handling of unused lambdas to reduce false positives

  Previously, we used to insert the CFGs of unused lambdas at the declaration site. However, this approach triggered some false positives. For example, consider the following code:

  void incorrect(int *p) {
    auto f1 = [&p]() {
      source(p);
    };
    auto f2 = [&p]() {
      sink(p);
    };
  }
  

  In this code, there's no actual control flow between the source and sink, and the lambdas are never even called. But when we inserted their CFGs at the declaration site, it incorrectly indicated a taint finding. To prevent these types of false positives while still scanning the body of unused lambdas, we now insert their CFGs in parallel at the end of their parent function, right after all other statements and just before the end node. (pa-3089)

• Bumped timeout (per-rule and per-file) from 2s to 5s. Recently we lowered it from 30s down to 2s, but based on what we have observed so far, we believe 5s is a better timeout for the time being. (timeout)

Fixed

• Fixed a bug where enabling the secret beta causes the default scan mode to be set to OSS, even when the Pro flag is turned on in the web UI. (ea-248)

• Semgrep does not stop a scan anymore for parsing errors due to unconventional exceptions (e.g., Failure "not a program") in some parsers. Instead, such errors are reported as "Other syntax error". (lang-13)

• Fix regression for the unused lambda change in react-nextjs-router-push test

  A lambda expression defined in a return expression is also treated as used at the location of the return expression. (pa-3089)

• Updated the Rust parser with miscellaneous improvements. In particular, Semgrep can now parse yield expressions in Rust. (rust)

• taint-mode: If an expressions is tainted by multiple labels A and B, with B requiring A, the expression will now get boths labels A and B. (taint-labels)

1.48.0 - 2023-11-06

[!NOTE] Starting from version 1.46.0, Semgrep is first released in the following ecosystems: - pypy - brew - returntocorp/semgrep:canary (Docker) If no issues are detected after a few days, the Semgrep team then promotes the :canary Docker tag to :latest when everything looks fine.

Added

• Matching: Matches with the same range but bindings in different locations will now no longer deduplicate.

  For instance, the pattern $FUNC(..., $A, ...) would produce only one match on the target file:

  foo(true, true)
  

  because you would have two matches to the range of the call, and both bindings of $A would be to true.

  Now, the deduplication logic sees that the bindings of $A are in different places, and thus should not be considered the same, and produce two matches. (pa-3230)

Fixed

• Fixed out of bounds list access error in Cargo.lock parser (sc-1072)
• Secrets: metadata overrides specified in validators were incorrectly applied on top of one another (on a per-rule basis), so that only the last was applied. Each update is now correctly applied independently to each finding based on the rule's validators. (scrt-231)

1.46.0 - 2023-10-24

[!NOTE] Starting this release (1.46.0) Semgrep is first released in the following ecosystems: - pypy - brew - returntocorp/semgrep:canary (Docker) If no issues are detected after a few days, the Semgrep team then promotes the :canary Docker tag to :latest when everything looks fine.

Added

• semgrep install-semgrep-pro now takes an optional --custom-binary flag to install the specified semgrep-core-proprietary binary rather than downloading it. (custom-pro-binary)

Fixed

• pyproject.toml parser now handles optional newlines right after section headers. (gh-10879)

• Updated the parsers for poetry.lock, pipfile.lock, and requirements.txt to ignore case sensitivity from package names. This matches their respective specifications. Test cases were added to account for this change. (gh-8984)

• Reduced the limits for the prefilter optimization so that rules that cause computing the prefilter to blow up will abort more quickly. This improves performance by 2-3 seconds for each of the slowest rules. May cause a slowdown if a rule that previously could be filtered out no longer will be, but based on testing this is unlikely. (gh-9040)

• Fixed issue where conditional expressions aren't handled properly in expression based language.

  Rust example:

  Before:

  fn expr_stmt_if(c) {
    y = 0;
    x = if c { y = 1 };
  
    // Before: this matches when it shouldn't because y is not always 1.
    // After: this does not match, which is the correct behavior.
    y == 1;
  }
  ``` (pa-3205)
  

• Fixed type error in creation of DependencyParserError object in the pnpm-lock.yaml parser (sc-1115)

1.45.0 - 2023-10-18

Changed

• Previously, to ignore a finding from a rule foo.bar.my-rule, nosemgrep ignored a finding only if its fully qualified name was used: nosemgrep: foo.bar.my-rule. Now, nosemgrep can also accept just the rule ID: nosemgrep: my-rule. (#8979)

• [Breaking Change] Improved Matching of C++ Constructors (pa-3114)

  • In this update, the Semgrep team has enhanced Semgrep's ability to match C++ constructors more accurately.
  • C++ introduces a syntactic ambiguity between function and variable definitions, particularly with constructors. The C++ compiler determines how to interpret an expression based on contextual information, such as whether the immediate parent scope is a function or a class, and whether the identifiers within the parentheses represent variables or types.
  • Due to this complexity, static analyzers face challenges in precisely parsing these expressions without additional information.
  • This commit introduces several workarounds to provide a better solution for handling this ambiguity: - By default, when parsing a target file, Semgrep will consider an expression like foo bar(x, y, z); defined within the body of a function as a variable definition with a constructor. This is because variable initialization is a more common use case within the body of a function. - Users can specify rule options that annotate, in patterns where the expression can be interpreted in both ways, which interpretation should take precedence. For instance, foo bar(x, y, z); will be parsed as a function definition when the as_fundef option is used and as a variable definition with a constructor when the as_vardef_with_ctor option is applied. It's worth noting that an expression like foo bar(1, y, z); will be parsed as a variable definition without any additional annotation since 1 cannot be a type.
  • Here's an example rule and its corresponding target file to illustrate these changes:

  rules:
    - id: cpp-match-func-def
      message: Semgrep found a match
      options:
        cpp_parsing_pref: as_fundef
      languages:
        - cpp
      severity: WARNING
      pattern-either:
        - pattern: foo $X($Y);
        - pattern: foo $X($Y, $Z);
  
    - id: cpp-match-ctor
      message: Semgrep found a match
      options:
        cpp_parsing_pref: as_vardef_with_ctor
      languages:
        - cpp
      severity: WARNING
      patterns:
        - pattern: foo $X(...);
        - pattern-not: foo $X(3, ...);
  
    - id: cpp-match-ctor-3
      message: Semgrep found a match
      languages:
        - cpp
      severity: WARNING
      pattern: foo $X(3, ...);
  

  class Test {
  
    // ruleid: cpp-match-func-def
    foo bar(x);
    // ruleid: cpp-match-func-def
    foo bar(x, y);
  
    void test() {
      // ruleid: cpp-match-ctor
      foo bar(1);
      // ruleid: cpp-match-ctor
      foo bar(1, 2);
  
      // ruleid: cpp-match-ctor
      foo bar(x);
      // ruleid: cpp-match-ctor
      foo bar(x, y);
  
      // ruleid: cpp-match-ctor
      foo bar(x, 2);
      // ruleid: cpp-match-ctor
      foo bar(1, y);
  
      // ruleid: cpp-match-ctor-3
      foo bar(3);
      // ruleid: cpp-match-ctor-3
      foo bar(3, 4);
      // ruleid: cpp-match-ctor-3
      foo bar(3, y);
    }
  };
  

Fixed

• Semgrep Docker image: Reduction of the docker image size by using --no-cache when apk upgrading. Thanks to Peter Dave Hello for the contribution.

• Fixed a bug with pre-filtering introduced in 1.42.0 that caused significant slowdowns, particularly for Kotlin repos. Kotlin repos running default pro rules may see a 30 minute speedup. (ea-208)

• Taint analysis: track ptr->field l-values in C++

  • In C++, we now track tainted field access via pointer dereference. For instance, consider the following code snippet:

  void test_intra_001() {
    TestObject *obj = new TestObject();
  
    obj->a = taint_source();
    obj->b = SAFE_STR;
  
    // ok: cpp-tainted-field-ptr
    sink(obj->b, __LINE__);
    // ruleid: cpp-tainted-field-ptr
    sink(obj->a, __LINE__);
  }
  

  This can be matched by the rule (gh-1058):

  rules:
    - id: cpp-tainted-field-ptr
      languages:
        - cpp
      message: testing flows though C++ ptrs
      severity: INFO
      mode: taint
      pattern-sources:
        - pattern: taint_source()
      pattern-sinks:
        - patterns:
            - pattern: sink($X, ...)
            - focus-metavariable:
                - $X
  

• Do not crash anymore with an Invalid_arg exception when the terminal has very few columns (e.g., in some precommit context). (#8792)

• Add --supply-chain flag to semgrep ci --help documentation (#8975)

• Avoid catastrophic Invalid_argument: index out of bounds errors when reporting the location of findings (#9011)

• IntelliJ and VSCode extensions: The Semgrep Language Server (LSP) no longer freezes while scanning long files.

• Pre-filtering is now less aggressive and tries not to skip files that could be matched by a rule due to constant-propagation. Previously, a rule searching for the string "foobar" would skip a file that did not contain exactly "foobar", but that contained e.g. "foo" + "bar". (#8767)

• semgrep ci does not crash anymore when ran from git repositories coming from Azure projects with whitespaces in the name. (#8971)

• The --test flag now processes test target files even if they do not match the paths: directive of a rule. This is especially useful for rules using the include: which is now disabled in a test context. (#8192)

1.44.0 - 2023-10-11

Added

• A new --matching-explanations CLI flag has been added, to get matching explanations. This was internally used by the Semgrep Playground to help debug rules, but is now available also directly from the CLI. (explanations)

• Using C++ tree-sitter as a failsafe pattern parser for C (gh-8905)

• Allowing multiple type fields in metavariable-type rule syntax

  Users have the flexibility to utilize multiple type fields to match the type of metavariables. For instance:

  metavariable-type: metavariable: $X types: - typeA - typeB

  This approach is also supported in rule 2.0. (gh-8913)

• Support for parsing pubspec (Dart/Flutter) lockfiles (gh-8925)

• Added support for matching template type arguments using metavariables in C++. Users can now successfully match code snippets like:

  #include <memory>
  using namespace std;
  
  void foo() {
      int *i = 0;
  
      // ruleid: match-with-template
      shared_ptr<int> p;
  }
  

  with the pattern:

  shared_ptr<$TY> $LOCAL_VAR;
  ``` (pa-3102)
  
  
  

Fixed

• Avoid fatal "missing plugin" exceptions when scanning some Apex rules for which no Apex pattern is used by the rule such as a pattern-regex: and nothing else. (gh-8945)

• Semgrep can now parse optional assignments in Swift (e.g. a.b? = 1). (lang-1)

• Sequential tainting is now supported in Elixir.

  def f() do
    x = "tainted"
    y = x
  
    # This now matches.
    sink(y)
  end
  ``` (pa-3130)
  

• Target files that disappeared before the scan or that have special byte characters in their filename do not cause the whole scan to crash anymore. The file is skipped instead. (pa-3144)

• go.mod parsing now correctly allows arbitrary newlines and whitespace between dependencies (sc-1076)

• fix: Improve typed metavariable matching against expressions consisting of names only. (type-inference)

1.43.0 - 2023-10-03

Added

• Dart: Full Semgrep support for Dart has been added, whereas previously most Semgrep constructs (and Semgrep itself) would not work correctly. (pa-2968)

Changed

• We have reduced the default timeout (per-rule and per-file) to 2s (down from 30s). Typically, running a rule on a file should take a fraction of a second. When a rule takes more than a couple of seconds is often because the rule is not optimally written, or because the file is unusually large (a minified file or machine- generated code), so waiting 30s for it does not tend to bring any value. Plus, by cutting it earlier, we may prevent a potential OOM crash when running the rule is very memory intensive. (pa-3155)

Fixed

• The language server will no longer surface committed findings when a user types but does not save (pdx-ls-git)

1.42.0 - 2023-09-29

Added

• Rule-writing: Capture group metavariables used in regexes in a metavariable-regex can now introduce their bindings into the scope of the pattern, similarly to metavariable-pattern.

  For instance, in the pattern: patterns:

  • pattern: | foo($BAR)
  • metavariable-regex: metavariable: $BAR regex: "(?<X>.*)end"
  • focus-metavariable: $X

  the rule will match the contents of what is inside of the foo to the regex that binds anything before an "end" to the metavariable $X. This metavariable can then be focused at a later time, or processed somewhere above this pattern. (pa-3011)

• Try-catch-else-finally is now supported in taint analysis.

  This change also includes some updates to our analysis. Previously we assumed that any statement inside the try clause may throw an exception, but now only function calls are assumed to possibly throw exceptions.

  Throw statements always throw an exception as it was before.

  This kind of statement is supported in languages including Python, Ruby, and Julia.

  Python example:

  def f(tainted_input):
    try:
      a = 0
      b = 0
      c = tainted_input
      d = tainted_input
    except RuntimeError:
      a = tainted_input
      c = sanitize(c)
    else:
      b = tainted_input
    finally:
      d = sanitize(d)
  
    # a is not tainted because exception wasn't assumed to be thrown
    sink(a)
    # b is tainted through the else clause
    sink(b)
    # c is tainted at the beginning, but it was not sanitized
    # because an exception was not thrown
    sink(c)
    # d is tainted at the beginning, but it was sanitized
    # because the finally clause is always executed
    sink(d)
  ``` (pa-3054)
  

• Semgrep can now derive facts about constants from equality tests.

  For example, pattern foobar(&nullptr) will not match here:

  int* ptr = nullptr;
  
  do_something(ptr);
  
  if (ptr == nullptr) {
      return;
  }
  
  foobar(&ptr); // OK
  

  But it will match here:

  if (ptr != nullptr) {
      return;
  }
  
  foobar(&ptr); // finding
  ``` (pa-3091)
  

• Metavariable-type rule support for C, C++

  Users now can use metavariable-type rules in both C and C++. For instance, the provided code snippet:

  #include <fstream>
  
  using namespace std;
  
  void test_001() {
      ifstream in;
      // ruleid: match-simple-metavar-type
      in.get(str, 2);
  
      mystream my;
      // ok: type mismatch
      my.get(str, 2);
  }
  

  can be matched by the following rule:

  rules:
    - id: match-simple-metavar-type
      patterns:
        - pattern: $X.get($SRC, ...)
        - metavariable-type:
            metavariable: $X
            type: ifstream
      message: Semgrep found a match
      languages:
        - cpp
      severity: WARNING
  ``` (pa-3106)
  

• C/C++: If conditions such as if (int x = f()) are now correctly translated into the Dataflow IL, so Semgrep can report a finding in the example below:

  if (const char *tainted_or_null = source("PATH"))
  {
      // ruleid:
      sink(tainted_or_null);
  }
  ``` (pa-3107)
  
  
  

Changed

• The _comment field in the JSON output of semgrep scan has been removed. (_comment)
• Use config=auto by default for the scan command when other options are not specified (grow-50)
• Use subprocess.run to get contributions instead of StreamingSemgrepCore so crashes don't affect the actual scan. (os-967)

Fixed

• The CLI autocompletion code has been removed. It was not currently working and nobody reported it, which probably means nobody was using it. (autocomplete)

• The --core-opts flag has been removed. (core_opts)

• fix: metavariable-type now correctly matches non-primitive types in php (gh-8781)

• fixed the regression in --registry-caching and add better error message to tell the user he needs also --experimental. (gh-8828)

• Support labeled let bindings within Swift case statements

  Correctly parsing labeled let bindings within Swift case statements. For instance, the code snippet:

  switch self {
    case .bar(_, _, x: let y):
      return y
  }
  

  now successfully matches the pattern:

  switch self {case .$X(..., $Y: $Z): ...}
  ``` (pa-3120)
  

• Add parsing support for various rare Swift constructs (swift-parsing)

Changed

• Rule validation no longer fails if a rule contains additional unknown fields. This makes it so older versions of semgrep do not fail rules that contain extra functionality. When writing a custom rule, the min-version field should be used to identify rules that should not be run, meaning that the additional functionality present in the min-version of Semgrep is necessary in running the rule. (#8712)
• Limit collection of the contributions from git log to the last 30 days of commits.

Fixed

• semgrep ci now shows a more specific error message if a scan cannot complete due to a user disabling all rules on semgrep.dev (#8716)
• Docker: For the nonroot Docker build stage, moved semgrep-core to /home/semgrep/bin and updated $PATH env variable with the new location. This avoids permissions issues when running and installing Pro Engine while using the nonroot Docker image. (#8685)
• Ruby: Fixed a bug where patterns such as <id> ... do ... end would not match properly. (#8714)
• Swift: Implemented key path expression parsing in Swift. The following example should now be correctly matched by the \$X.isActive pattern:

  employee.filter(\.isActive)
  

  Note that when the implicit type is used, the metavariable X binds to the backslash character instead of the type name. (#8694)
• C++: Translate for (T var : E) loops into the Dataflow IL as for-each loops, so that Semgrep reports no finding in the following code:

    for (int *p : set) {
      sink(p); // no finding
      source(p);
    }
  

  Since each p is (in principle) a different object, even if source(p) taints the current p, that should not affect the next one. (#8749)
• Ruby: Fixed patterns which involve command calls with blocks and Semgrep ellipses, when there are newlines around. For instance, the pattern

  $METHOD ... do
    ...
  end
  

  now parses properly. (#8758)

• Fixed a bug in which Semgrep miscategorized direct dependencies that were erroneously identified as transitive in Node.js v9, lockfile version 3 and above.

Added

• Dot files, for example .vscode and .vimrc, are now displayed in the skip report when using --verbose and --develop.
• Added CLI output for secrets-related findings. (#8666)
• Rules are now skipped with an [INFO]-level error message if they can't run due to an unavailable plugin such as the Semgrep Pro Engine. This enables Semgrep to finish its scan without failing due to some unavailable plugin. The intended use is for a public rule registry to provide all kinds of rules including some that require particular plugins. (#8668)
• Allow Semgrep CI users to run a SAST scan through the Semgrep Code product using --code command-line option. This works the same as --supply-chain (#8679)
• Semgrep VSCode Extension: Semgrep Language Server now does not show findings that have been ignored through Semgrep Code.
• taint-mode: Semgrep now tracks taint via globals or class attributes that are effectively final (as in Java), for example:

  class Test {
    private String x = source();
  
    void test() {
      sink(x); // finding here !
    }
  }
  

  Semgrep recognizes that x must be tainted because it is a private class attribute that is initialized to source(), and it is not re-defined anywhere else. This also works if x is initialized in the constructor (if there is only one constructor), or in a static block. (#8652)
• Constant propagation: Semgrep can now identify private class attributes as constants that are assigned just once in a class constructor, for example: https://semgrep.dev/playground/s/R1re. (#8624)
• Added -dump_contributions flag to semgrep-core and include contributions when posting findings to Scan API.
• There is a new semgrep show command to display information about Semgrep, for example semgrep show supported-languages. The goal is to clean up semgrep scan which is currently abused to not scan but also display Semgrep information, for example, semgrep scan --show-supported-languages. See semgrep show --help for more information.
• Add exception handling for dump_contributions core command in pysemgrep.

Changed

• Further improvements to timeouts and logging for semgrep ci (#8665)

Fixed

• Semgrep VSCode Extension: Semgrep Language Server no longer duplicates some findings.
• GitLab SAST output has now been updated to accommodate the new SAST schema as of GitLab 16.x, which means that findings in GitLab will now properly display descriptions of the findings. (#8657)
• Julia: Ellipses can now properly match when used in conjunction with single statements. This is achieved by preserving a single statement as a block. (#8643) For instance, the pattern

  ...
  foo()

can now properly match a target of

foo()

• Matching: Numeric capture group metavariables of the form $1, $2, and so on, no longer unify implicitly. In the case of numeric capture group metavariables, which can occur implicitly in the rule-writing process, it is not intended that all metavariables must be combined to create a match. For this reason, Semgrep no longer requires that these these unnamed metavariables are taken as a whole ("unified") to create a match. (#8644)
• Ruby: The CFG (context-free grammar) now supports case statements in Ruby, which does not fall through.
• Constant propagation now handles implicit number-to-string conversions in Java and JS/TS. A Java expression such as "foo" + 123 now matches the string pattern "foo123".

Added

• Matching: Qualified names written as patterns can now match valid instances of identifiers which lie underneath a wildcard import (#8514). For instance, in Python, we could write the pattern A.B.C.x, and match the usage in the program:

from A.B import *
foo(C.x)

• Ruby: Replaced old Ruby parser with the latest tree-sitter Ruby parser, meaning that there could be small edge cases of differences in how Semgrep matches Ruby programs. (#8539)

Fixed

• Previously, requests to Semgrep servers could result in gateway timeout errors (504 errors) without attempting a retry, resulting in a failure. Now, 504 errors are automatically retried first rather than failed. (#8629)
• The error message for skipped rules due to incompatible min-version or max-version constraints has been improved. (#8634)
• When metavariable-type cannot be evaluated then it defaults to "false", that is, it filters out the range. The following rule:

      patterns:
        - pattern: private int $X;
        - metavariable-type:
            metavariable: $Y
            type: int
  

  now produces no matches because $Y is not bound to anything. (#8566)
• Julia: using and import now match separately, instead of before, where if you wrote using $X, you would also match to imports. (#8567)
• Semgrep VSCode Extension: Previously, findings could disappear as a user explored or highlighted them. This is now fixed. (#8633)

1.38.3 - 2023-09-02

No significant changes.

1.38.2 - 2023-09-01

Fixed

• restore access to the --text option (gh-8610)

1.38.1 - 2023-09-01

Fixed

• restored access to the --output flag (gh-8602)
• Adds an error message clarifying the removal of python -m semgrep. This change originated in https://github.com/returntocorp/semgrep/pull/8504. (gh-8605)

1.38.0 - 2023-08-31

Added

• The CLI now returns the commit timestamp when using semgrep ci (cli-timestamp)
• Add support for min-version and max-version fields for each rule, specifying a range of compatible Semgrep versions. If a rule is incompatible with the version of Semgrep being used, it is reported in the JSON output at the "info" level which doesn't cause an exit failure. (gh-8496)
• Dependency data is now also sent to the /results endpoint of semgrep app. It is still sent to the /complete endpoint. (sc-async)

Changed

• Adjust the count printed at the conclusion summary to match the top summary (only printing the count of rules actually run by semgrep and not just the number of rules received from the server). (counts)
• The option to omit --config and to look for the presence of a .semgrep.yml or .semgrep/.semgrep.yml in the current directory has been removed. You now have to explicitly use --config. (dotsemgrep)
• The deprecated --enable-metrics and --disable-metrics flags have finally been removed. Use --metrics=on or --metrics=off instead (or --metrics=auto). (enable_metrics)
• The semgrep_main.py module has been renamed to run_scan.py and its invoke_semgrep() function renamed to run_scan_and_return_json(). External tools (e.g., semgrep wrappers) using directly those functions should be updated. Note that this function will soon disappear as part of a migration effort converting Python code to OCaml. Thus, those tools should instead wrap the semgrep CLI and rely on semgrep_output_v1.atd for a more stable official API. (internals)

Fixed

• Running just semgrep now displays the help message. Semgrep does not try anymore to look for a .semgrep.yml config file or .semgrep/ in the current directory, which used to cause issues when running from your home directory which can contain the .semgrep/settings.yml file (which is actually not a semgrep rule). (gh-4457)

• Fixed CLI output to display matches from different rules with the same message. (gh-8557)

• Semgrep PyPI package can now be pip install-ed on aarch64 libmusl platforms (e.g. Alpine) (gh-8565)

• Updated --max-memory help description to make it more clear/concise. To say "Defaults to 0 for all CLI scans." implies a different default for non-CLI scans, where in practicality the default is 0 for all scans except when using Pro Engine, where the default is 5000. (max_memory_help)

• Julia: Fixed a bug where let end blocks were not being parsed correctly, causing their contents to not strictly match while inside of a block.

  For instance, let ... end would not count as being inside of the let, and would match everything. (pa-3029)

• Fixed bug where dependencies in (pnpm-lock.yaml at version 6.0 or above) files were not parsed. (sc-1033)

1.37.0 - 2023-08-25

Added

• semgrep scan is now more resilient to failures when fetching config from semgrep.dev. If it can't fetch a config from semgrep.dev it will use backup infrastructure to fetch the most recent successful config for that customers environment. (gh-8459)
• C#: Added experimental NuGet ecosystem parser (gh-8484)
• metavariable-comparison: You can now use "in" and "not in" for strings in the same sense as in Python, for substring checking. (pa-2979)
• Julia: Added the deep expression operator, so now you can write patterns like foo(<... 42 ...>) to find instances of calls to foo that contain 42 somewhere inside of it. (pa-3018)
• semgrep ci displays enabled products when scans are created and/or when the scan config is generated from Semgrep Cloud Platform. Additionally, if no products are enabled then a friendly error is raised. (scp-432)

Changed

• The --dump-ast flag now requires the additional --experimental flag and does not require to pass a --config flag anymore. Example of use: semgrep --experimental --lang python --dump-ast foo.py (dumpast)
• The 'semgrep shouldafound' command has been removed. It was not really used and it might be better to offer such a functionality in the IDE instead of in the CLI. (shouldafound)

Fixed

• Parsing: Some parsing errors involving tree-sitter inserting fake "missing" nodes were previously unreported. They are now reported as errors although the parse tree is preserved, including the phony node inserted by tree-sitter. This should not result in different Semgrep findings. It results only in more reports of partial parsing. See the original issue at https://github.com/returntocorp/ocaml-tree-sitter-core/issues/8 for technical details. (gh-8190)

• fix(extract): correctly map metavariable locations into source file (gh-8416)

• fix(julia): correctly parse BitOr and BitAnd (gh-8449)

• Implement missing pcre-ocaml stub (pcre_get_stringnumber_stub_bc) in JavaScript (gh-8520)

• Julia: Fixed a bug where parenthesized expressions would sometimes not match in constructs like metavariable-comparison. (pa-2991)

• Fixed a regression introduced three years ago in 0.9.0, when optimizing the evaluation of ... (ellipsis) to be faster. We made ... only match deeply (inside an if for example) if nothing matched non-deeply, thus causing that this pattern:

  foo()
  ...
  bar($A)
  

  would only produce a match rather than two on this code:

  foo()
  if cond:
      bar(x)
  bar(y)
  

  Semgrep matched from foo() to bar(y) and because of that it did not try to match inside the if, thus there was no match from foo() to bar(x). However, if we commented out bar(y), then Semgrep did match bar(x).

  Semgrep now produces the two expected matches. (pa-2992)

• Julia: Type information from declarations can now be used in metavariable-type. For instance, the program:

  x :: Int64 = 2
  

  will now allow uses of x to match to the type Int64. (pa-3001)

• Julia: Metavariables should now be able to appear anywhere that identifiers can.

  For instance, they were not able to appear as the argument to a do block. Now, we can write patterns like:

  map($Y) do $X
    ...
  end
  ``` (pa-3007)
  

• Java: Fixed naming bug affecting Java and other OO languages that allowed a method parameter to shadow a class attribute, e.g. in:

  class Test {
  
      private int x;
  
      public void test2(int x) {
          foo(this.x);
      }
  
  }
  

  Semgrep was considering that this.x referred to the parameter x of test2 rather than to the class attribute x. (pa-3010)

• Fixed bug where packages in build.gradle files had their names incorrectly parsed without their group ID (sc-1012)

1.36.0 - 2023-08-14

Added

• Added general machinery to support languages with case insensitive identifiers and generalized php to use these case insensitive identifiers.

  For example, in php the pattern MyClass() will now match calls with different capitalization such as myclass() and Myclass(). (gh-8356)

Fixed

• Convert all '@r2c.dev' email addresses to '@semgrep.com'. (gh-8437)
• Semgrep LSP now compiled with tls, should no longer crash with not compiled with tls error (ls-conduit)
• Fixed multiprocess testing crash due to new osemgrep entrypoint (pa-2963)
• Pro: JS/TS: taint-mode: Fix bug introduced in 1.33.1 that had the side-effect of hurting performance of taint rules on JS/TS repos that used destructuring in functions formal parameters. (pro-119)

1.35.0 - 2023-08-09

Added

• Maven Dep Tree parsing now surfaces children dependencies per package (sc-996)

Fixed

• fix(promql): make aggregation labels not depend on order

  "sum by (..., b, a, c, ...) (X)" should match "sum by (a,b,c) (X)" (gh-8399)

1.34.1 - 2023-07-28

Added

• feat(eval): add "parse_promql_duration" function to convert a promql duration into milliseconds. This makes it possible to write comparisons like this:

  - metavariable-comparison:
      metavariable: $RANGE
      comparison: parse_promql_duration(str($RANGE)) > parse_promql_duration("1d")
  ``` (gh-8381)
  
  
  

Fixed

• fix(yaml): fix captures for sequences that contain mappings (gh-8388)

1.34.0 - 2023-07-27

Added

• Added support for naming propagation when the left-hand side (lhs) of a variable definition is an identifier pattern

  In certain languages like Rust, the variable definition is parsed as a pattern assignment, for example:

  let x: SomeType = SomeFunction();
  

  This commit ensures that the annotated type is propagated to the identifier pattern on the left-hand side (lhs) of the assignment, thus ensuring proper naming behavior. (gh-8365)

• feat(metavar type): Metavariable type support for Julia

  Metavariable type is supported for Julia. (gh-8367)

• New --legacy flag to force the use of the old Python implementation of Semgrep (also known as 'pysemgrep'). Note that by default most semgrep commands are still using the Python implementation (except 'semgrep interactive'), so in practice you don't need to add this flag, but as we port more commands to OCaml, the new --legacy flag might be useful if you find some regressions. (legacy)

• Matching: Added the ability to use metavariables in parameters to match more sophisticated kinds of parameters.

  In particular, metavariables should now be able to match self parameters, such as in Rust.

  So fn $F($X, ...) { ... } should match fn $F(self) { }. (pa-2937)

• taint-mode: Added experimental control: true option to pattern-sources, e.g.:

      pattern-sources:
        - control: true
          pattern: source(...)
  

  Such sources taint the "control flow" (or the program counter) so that it is possible to implement reachability queries that do not require the flow of any data. Thus, Semgrep reports a finding in the code below, because after source() the flow of control will reach sink(), even if no data is flowing between both:

  def test():
    source()
    foo()
    bar()
    #ruleid: test
    sink()
  ``` (pa-2958)
  

• taint-mode: Taint sanitizers will be included in matching explanations. (pa-2975)

Changed

• Started using ATD to define the schema for data sent to the /complete endpoint of semgrep app (app-4255)
• Targets in a .yarn/ directory are now ignored by the default .semgrepignore patterns. (dotyarn)

Fixed

• Aliengrep mode: Fix whitespace bug preventing correct matching of parentheses. (gh-7990)
• yaml: exclude style markers from matched token in block scalars (gh-8348)
• Fixed stack overflow caused by symbolic propagation. (pa-2933)
• Rust: Macro calls which involve dereferencing and reference operators (such as foo!(&x) and foo!(*x)) now properly transmit taint (pa-2951)
• Semgrep no longer crashes when running --test (pa-2963)
• Exceptions raised during parsing of manifest files no longer interrupt general parser execution, which previously prevented lockfile parsing if a manifest failed to parse. (sc-exceptions)

1.33.2 - 2023-07-21

No significant changes.

1.33.1 - 2023-07-21

Added

• Rust: Added support for ellipsis patterns in attribute argument position. (e.g. #[get(...)]) (gh-8234)
• Promql: Initial language support (gh-8281)
• .h files will now run when C or C++ are selected as the language. (pa-123)
• .cjs and .mjs files will now run when javascript is selected as the language. (pa-124)
• Tainting: Parameters to functions in languages with pattern matching in function arguments, such as Rust and OCaml, now transmit taint when they are sources. This works with nested patterns too. For instance, in Rust:

  fn f ((x, (y, z)): t) {
    let x = 2;
  }
  

  tainting the sole argument to this function will result in all of the identifiers x, y, and z now being tainted. (pa-2919)
• Added rule option interfile: true, so this can be set under options: as it is the norm for rule options. This rule option shall replace setting interfile under metadata. Metadata is not mean to have any effect on how a rule is run. (pro-94)

Changed

• Updated semgrep-interfaces, changed api_scans_findings to ci_scan_results, removed gitlab_token field and added ignores and renamed_paths field to ci_scan_results. (app-4252)

Fixed

• Dockerfile language support: String matching is now done by contents, treating the strings foo, 'foo', or "foo" as equal. (gh-8229)

• Fixed error where we were not filtering the logging of a new third party library. (gh-8310)

• Julia: Fixed a bug where try-catch patterns would not match properly. Now, you can use an empty try-catch pattern, such as:

  try
    ...
  catch
    ...
  end
  

  to catch only Julia code which does not specify an identifier for the catch.

  Otherwise, if you want to match any kind of try-catch, you can specify an ellipsis for the catch identifier instead:

  try
    ...
  catch ...
    ...
  end
  

  and this will match any try-catch, including those that do not specify an identifier for the catch. It is strictly more general than the previous. (pa-2918)

• Rust: Fixed an issue where implicit returns did not allow taint to flow, and various other small translation issues that would affect taint. (pa-2936)

• Fixed bug in gradle.lockfile parser where we would error on empty= with nothing after it (sc-987)

1.32.0 - 2023-07-13

Added

• feat(docker): Create a semgrep user for our docker container so that people can run it as a non-root user (gh-8116)

• feat(typed metavar): Typed metavariable support for Rust

  Users can create TypedMetavar using Rust's type annotation syntax :. For example, the following rule works for matching HttpResponseBuilder type of variables:

  rules:
  - id: no-direct-response-write
    patterns:
    - pattern: '($BUILDER : HttpResponseBuilder).body(...)'
    - pattern-not: '($BUILDER : HttpResponseBuilder).body("...".to_string())'
    message: find dangerous codes
    severity: WARNING
    languages: [rust]
  ``` (gh-8200)
  
  
  

Fixed

• baseline scans reporting on existing findings (baseline-supply-chain)
• Fixed an issue leading to incorrect autofix results involving JS/TS async arrow functions (e.g. async () => {}, etc.). (gh-7353)
• Workaround for rootless containers as git operations may fail due to dubious ownership of /src (gh-8267)

1.31.2 - 2023-07-07

No significant changes.