Merged PRs

doltgresql

• 2975: Bug fix: infer bound var types correctly when they aren't fully specified
  When a client indicates that a bind var's type is unspecified, the Postgres-compatible server should infer its type. Doltgres was incorrectly throwing away all provided type information if any bind var type wasn't specified, which resulted in Doltgres choosing a different type than the client was using. This further resulted in incorrect data being written to the table, since Doltgres misunderstood the type and read extra data that wasn't intended for that bind var.
  The main fix is to only infer the type for the individual bind vars that aren't specified, not for all bind vars. Additionally, the buffer reading code was changed to error out loudly if it encountered a future issue where the amount of expected bytes isn't present.
  Related to #2973
• 2970: Use LockSubsystem.TryLock in pg_try_advisory_lock, remove stale reentrancy TODO
  pg_try_advisory_lock previously emulated a try-lock by calling LockSubsystem.Lock with a 1ms timeout and translating ErrLockTimeout into a false return value. This was a workaround for the lock subsystem not having a true non-blocking acquisition API.
  dolthub/go-mysql-server#3584 added LockSubsystem.TryLock, which attempts the lock acquisition exactly once and returns immediately. This change replaces the workaround with a direct call to it.
  Also removes a stale //TODO on pg_advisory_lock stating that LockSubsystem needed to be updated to support reentrant locks. LockSubsystem already supports reentrant locks: re-acquiring a held lock increments a lock count, and each unlock decrements it, so a lock is only fully released after being unlocked as many times as it was acquired. A new test in testing/go/lock_test.go verifies the reentrant behavior.
• 2969: Enable automatic GC, by default
  Enable automatic garbage collection for Doltgres. Ports the existing auto GC tests from Dolt.
  Depends on: dolthub/dolt#11343
• 2967: Bump google.golang.org/grpc from 1.79.3 to 1.82.1
  Bumps google.golang.org/grpc from 1.79.3 to 1.82.1. Release notes

  Sourced from google.golang.org/grpc's releases.

  Release 1.82.1

  Security

  • server: Stop reading from the connection when flooded by HTTP/2 frames. The default value for this limit is 100 frames, excluding DATA and HEADERS, and may be changed by setting environment variable GRPC_GO_EXPERIMENTAL_CONTROL_BUFFER_THROTTLE_LIMIT.
  • xds/rbac: Support Metadata and RequestedServerName permissions matcher fields. If present in a DENY rule, previously these would be ignored and fail-open.
  • xds/rbac: Fix panic when parsing unsupported fields in NotRule/NotId permissions.
  • xds/rbac: Support the deprecated source_ip principal identifier by treating it as equivalent to direct_remote_ip.

  Release 1.82.0

  Behavior Changes

  • server: Remove support for GRPC_GO_EXPERIMENTAL_DISABLE_STRICT_PATH_CHECKING environment varibale. Strict incoming RPC path validation (which has been the default since v1.79.3) can no longer be disabled. (#9112)
  • transport: Add environment variable to change the default max header list size from 16MB to 8KB. This may be enabled by setting GRPC_GO_EXPERIMENTAL_ENABLE_8KB_DEFAULT_HEADER_LIST_SIZE=true. This will be enabled by default in a subsequent release. (#9019)
  • balancer: Load Balancing policy registry is now case-sensitive. Set GRPC_GO_EXPERIMENTAL_CASE_SENSITIVE_BALANCER_REGISTRIES=false (and file an issue) to revert to case-insensitive behavior. (#9017)

  New Features

  • experimental/stats: Expose a new API, NewContextWithLabelCallback, to register a callback that is invoked when telemetry labels are added. (#8877)
    • Special Thanks: @​seth-epps
  • client: Return a portion of the response body in the error message, when the client receives an unexpected non-gRPC HTTP response, to make debugging easier. (#8929)
    • Special Thanks: @​chengxilo
  • server: Add environment variable GRPC_GO_SERVER_GOROUTINE_LABELS that controls setting runtime/pprof.Labels on goroutines spawned by the server. Set GRPC_GO_SERVER_GOROUTINE_LABELS=grpc.method=true to add the grpc.method label on goroutines spawned to handle incoming requests. (#9082)
    • Special Thanks: @​dfinkel

  Bug Fixes

  • xds/server: Fix a memory leak of HTTP filter instances occurring when route configurations are updated in-place during a Route Discovery Service (RDS) update. (#9138)
  • grpc: In the deprecated gzip Compressor (used via the deprecated WithCompressor dial option), enforce the MaxRecvMsgSize limit on the decompressed message buffer, preventing excessive memory allocation from highly compressed payloads. (#9114)
    • Special Thanks: @​evilgensec
  • stats/opentelemetry: Record retry attempts, grpc.previous-rpc-attempts, at the call level and not the attempt level. (#8923)
  • encoding: Ensure Close() is always called on readers returned from Compressor.Decompress if possible. (#9135)
  • channelz: Fix the LastMessageSentTimestamp and LastMessageReceivedTimestamp fields in SocketMetrics to ensure they contain correct timestamp values. (#9109)

  Release 1.81.1

  Security

  • xds/rbac: Fix a potential authorization bypass caused by incorrectly falling through URI/DNS SANs to Subject Distinguished Name (DN) when matching the authenticated principal name. With this fix, only the first non-empty identity source will be used, as per gRFC A41. (#9111)
    • Special Thanks: @​al4an444

  Bug Fixes

  • otel: Segregate client and server RPC information used for metrics and traces, to avoid one overwriting the other. (#9081)

  Release 1.81.0

  Behavior Changes

  • balancer/rls: Switch gauge metrics to asynchronous emission (once per collection cycle) to reduce telemetry noise and align with other gRPC language implementations. (#8808)

  Dependencies

  ... (truncated)

  Commits
  • ebd8f06 Change version to 1.82.1 (#9238)
  • 4ea465d Cherry-pick commits (#9236)
  • 9494a2c Change version to 1.82.1-dev (#9171)
  • bd23985 Change version to 1.82.0 (#9170)
  • 0f3086d Fix minor issues not covered by PR #9137 (#9147)
  • fef07fb internal: Split v3procservicepb import into pb and grpc for extproc (#9163)
  • 91dd64f transport: surface subsequent data when receiving non-gRPC header (#8929)
  • adc97de test/kokoro: add config for regional-td test (#9158)
  • 57c9ff1 xds: ensure full-string matching for RBAC Filter rules (#9148)
  • b58f32d server: Set a pprof label on new stream goroutines (#9082)
  • Additional commits viewable in compare view
  
  [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options
  You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/dolthub/doltgresql/network/alerts).
• 2959: Bug fix: display expressions for functional indexes
  Also includes additional tests for functional indexes with multiple expressions
• 2956: cache output functions
• 2947: skip driver tests on mac os for now
• 2941: Bug fixes for replication
  There were a handful of bugs that prevented replication and tests from running successfully:
  • replicas were immediately shutting down after server startup due to "CREATE DATABASE" statements being executed on them (they're read-only)
  • various procedures related to replica control were not exposed
  • root collections were not present on the cluster databases Dolt exposes
  • various setup and assertions made by the tests, which were ported directly from Dolt, needed to be adapted
• 2940: go.mod: bump eventsapi_schema to latest revision
• 2934: enable stats
• 2932: avoid append in wire conversion
  We're allocating a result string anyway, so the append(dest, val) optimization is useless.
• 2928: Implemented WITH RECURSIVE ... CYCLE
  Fixes:
  • #1976
• 2925: Get initial replication tests running
  Fixes an issue with the test runner that prevented tests from running because the initial database couldn't be created. Also unskips a couple of tests that already pass.
• 2923: Bump golang.org/x/crypto from 0.51.0 to 0.52.0
  Bumps golang.org/x/crypto from 0.51.0 to 0.52.0. Commits
  • a1c0d99 go.mod: update golang.org/x dependencies
  • 3c7c869 ssh: fix deadlock on unexpected channel responses
  • 533fb3f ssh: fix source-address critical option bypass
  • abbc44d ssh: fix incorrect operator order
  • e052873 ssh: fix infinite loop on large channel writes due to integer overflow
  • b61cf85 ssh: enforce user presence verification for security keys
  • 9c2cd33 ssh: enforce strict limits on DSA key parameters
  • 8907318 ssh: reject RSA keys with excessively large moduli
  • ffd87b4 ssh: fix panic when authority callbacks are nil
  • 4e7a738 ssh: fix deadlock on unexpected global responses
  • Additional commits viewable in compare view
  
  [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options
  You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/dolthub/doltgresql/network/alerts).
• 2921: implement GREATEST and LEAST functions
• 2919: Fixed racy first-start database initialization
  Dolt's CREATE DATABASE is non-transactional and non-atomic, which means that a different session might see a partially initialized database. This is a much bigger problem for Doltgres because it creates a postgres database on first run if no database exists, and clients polling for readiness might see this database before it's fully initialized.
  To fix this, we create the initial database as a startup service, before we begin accepting connections.
  Additionally, we add more defensive readiness checks in go-sql-driver, where this race was most commonly observed. They should be redundant at this point.
  Depends on dolthub/dolt#11286
• 2916: Bump golang.org/x/crypto from 0.45.0 to 0.52.0 in /integration-tests/go-sql-server-driver
  Bumps golang.org/x/crypto from 0.45.0 to 0.52.0. Commits
  • a1c0d99 go.mod: update golang.org/x dependencies
  • 3c7c869 ssh: fix deadlock on unexpected channel responses
  • 533fb3f ssh: fix source-address critical option bypass
  • abbc44d ssh: fix incorrect operator order
  • e052873 ssh: fix infinite loop on large channel writes due to integer overflow
  • b61cf85 ssh: enforce user presence verification for security keys
  • 9c2cd33 ssh: enforce strict limits on DSA key parameters
  • 8907318 ssh: reject RSA keys with excessively large moduli
  • ffd87b4 ssh: fix panic when authority callbacks are nil
  • 4e7a738 ssh: fix deadlock on unexpected global responses
  • Additional commits viewable in compare view
  
  [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options
  You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/dolthub/doltgresql/network/alerts).
• 2913: Improved window function support
  Implements several window functions in the doltgres layer, instead of relying on the implementations in go-mysql-server. This allows us to cleanly match PostgreSQL's behavior and exact return types. Also enables more engine tests for window functions.
  Depends on:
  • dolthub/go-mysql-server#3614
  • dolthub/go-mysql-server#3621
• 2906: Add tests for dolt_remote() operations
• 2902: implement pg_proc table
  Depends on dolthub/go-mysql-server#3617

Closed Issues

• 1976: WITH RECURSIVE support
• 2378: Handle Doltgres types in least/greatest functions
• 2922: Document if supported: doltgresql master for rich update(set)s with postgres replica(s) for readonly traffic?