github.com/zmap/zgrab2

We're very excited to release v1 of ZGrab2! We feel the module/framework API has stabilized to the point we feel comfortable with making this our first major release.

What's Changed

• Retry Postgres scans without TLS if the TLS connection attempt errors by @UnknownEclipse in #573
• 66 add memcached module by @nicholasingino in #551
• Implement a DNS and per-IP Target Rate Limit by @phillip-stephens in #572
• Group CLI flags and make --help text for modules descriptive by @phillip-stephens in #571
• replace blocked cloudflare.com test with working domain by @phillip-stephens in #585
• Fix "Application Options" mutliple module parsing by @phillip-stephens in #578
• Fix integration tests on Github Runner by @phillip-stephens in #586
• feat(http): add SkipHost request option by @UnknownEclipse in #580
• feat(siemens): add custom readtimeout by flag by @zsbahtiar in #584
• Normalize scanning module variable usage by @Seanstoppable in #595
• Add port to ScanResponse output by @Seanstoppable in #596
• Make verbose a base flag by @Seanstoppable in #599
• Add additional tests for http by @Seanstoppable in #606
• Phillip/98 metadata file improvements by @phillip-stephens in #607
• Update ZFlags by @phillip-stephens in #608
• Add support for HTTP/2 and HTTP/2 over clear-text by @phillip-stephens in #581
• Return all the TLS handshake details we have even if handshake fails by @phillip-stephens in #613
• Added better docs for the Scanner interface by @phillip-stephens in #615
• feat(ssh): Improve SSH_MSG_KEXINIT composition by @TrueSkrillor in #616
• Fix Lint issues and add a test for memcached's SnakeToCamel by @phillip-stephens in #630
• bugfix: MySQL Empty Packet Check by @GQW19 in #627
• MySQL Length Check for Version 9 Packets by @GQW19 in #631
• memcached boundry check bug fix + Checks for magic byte and opcode validations by @merterdemir in #629
• Add some helpful info to the top of our README by @phillip-stephens in #634
• Add ManageSieve Module by @BMKprIT in #617
• initialize monitor map and avoid accessing before checking membership by @phillip-stephens in #647
• Fix failing IPP/Cups integration test by @phillip-stephens in #649
• Fix bug with Oracle when used with multiple module by @phillip-stephens in #648
• validate pptp magic cookie in responses by @phillip-stephens in #650

New Contributors

• @nicholasingino made their first contribution in #551
• @zsbahtiar made their first contribution in #584
• @GQW19 made their first contribution in #627
• @merterdemir made their first contribution in #629
• @BMKprIT made their first contribution in #617

Full Changelog: v0.2.0...v1.0.0

Primary Changes

• Validate the magic cookie in responses for the PPTP module to validate responses are from PPTP services in #650
• Fixes a bug where the multiple module could seg fault under specific conditions In #647
• Adds a ManageSieve module by @BMKprIT in #617 (Thanks for the contribution!)
• Adds bounds checking and other improvements to memcached by @merterdemir in #629
• Adds bounds checking to MySQL packets by @GQW19 in #631
• Improve SSH_MSG_KEXINIT composition by @TrueSkrillor in #616

Smaller Changes/Dependency Bumps

• Added better docs for the Scanner interface by @phillip-stephens in #615
• build(deps): bump golang.org/x/term from 0.35.0 to 0.36.0 by @dependabot[bot] in #622
• build(deps): bump golang.org/x/crypto from 0.42.0 to 0.43.0 by @dependabot[bot] in #621
• build(deps): bump golang.org/x/time from 0.13.0 to 0.14.0 by @dependabot[bot] in #623
• build(deps): bump golang.org/x/net from 0.44.0 to 0.46.0 by @dependabot[bot] in #619
• Fix Lint issues and add a test for memcached's SnakeToCamel by @phillip-stephens in #630
• build(deps): bump actions/upload-artifact from 4 to 5 by @dependabot[bot] in #625
• build(deps): bump actions/download-artifact from 5 to 6 by @dependabot[bot] in #626
• build(deps): bump golangci/golangci-lint-action from 8.0.0 to 9.0.0 by @dependabot[bot] in #636
• build(deps): bump golang.org/x/sys from 0.37.0 to 0.38.0 by @dependabot[bot] in #637
• Add some helpful info to the top of our README by @phillip-stephens in #634
• Add ManageSieve Module by @BMKprIT in #617
• build(deps): bump golang.org/x/crypto from 0.43.0 to 0.44.0 by @dependabot[bot] in #639
• build(deps): bump golang.org/x/net from 0.46.0 to 0.47.0 by @dependabot[bot] in #641
• build(deps): bump golang.org/x/crypto from 0.44.0 to 0.45.0 by @dependabot[bot] in #643
• build(deps): bump actions/checkout from 5 to 6 by @dependabot[bot] in #645
• build(deps): bump golangci/golangci-lint-action from 9.0.0 to 9.1.0 by @dependabot[bot] in #646
• Fix failing IPP/Cups integration test by @phillip-stephens in #649
• Fix bug with Oracle when used with multiple module by @phillip-stephens in #648

New Contributors

• @GQW19 made their first contribution in #627
• @merterdemir made their first contribution in #629
• @BMKprIT made their first contribution in #617

Full Changelog: v1.0.0-RC2...v1.0.0-RC3

This RC2 v1 introduces HTTP/2 support, additional info in metadata, adding the port scanned in Scan Output, as well as upgrading a number of dependencies and bug fixes.

New Features

• Adds support for scanning servers that only support HTTP/2 and h2c (http2 over plain-text)
• Adds the port scanned to each output line. This is helpful when port can be defined as a tool-wide CLI flag, passed in as an input line, or defined differently for different modules in multiple module scans and will help users disambiguate what was scanned
• Adds more info to the metadata, specifically exactly what ZGrab2 CLI args were used for a scan.

Bug Fixes

• A regression was introduced in v0.2.0 where if a TLS handshake failed after collecting some info, only an error would be returned. The old behavior was that any info from the TLS handshake would be returned with the error, yielding more handshake details on these failing connections.
• Fixes an issue where the Application Options field on the multiple module wasn't being parsed correctly.

What's Changed

• build(deps): bump golang.org/x/net from 0.41.0 to 0.42.0 by @dependabot[bot] in #575
• replace blocked cloudflare.com test with working domain by @phillip-stephens in #585
• Fix "Application Options" mutliple module parsing by @phillip-stephens in #578
• Fix integration tests on Github Runner by @phillip-stephens in #586
• feat(http): add SkipHost request option by @UnknownEclipse in #580
• feat(siemens): add custom readtimeout by flag by @zsbahtiar in #584
• build(deps): bump github.com/prometheus/client_golang from 1.22.0 to 1.23.0 by @dependabot[bot] in #587
• build(deps): bump golang.org/x/text from 0.27.0 to 0.28.0 by @dependabot[bot] in #593
• build(deps): bump golang.org/x/sys from 0.34.0 to 0.35.0 by @dependabot[bot] in #589
• build(deps): bump golang.org/x/crypto from 0.40.0 to 0.41.0 by @dependabot[bot] in #594
• build(deps): bump golang.org/x/net from 0.42.0 to 0.43.0 by @dependabot[bot] in #592
• Normalize scanning module variable usage by @Seanstoppable in #595
• build(deps): bump actions/download-artifact from 4 to 5 by @dependabot[bot] in #588
• build(deps): bump actions/checkout from 4 to 5 by @dependabot[bot] in #591
• Add port to ScanResponse output by @Seanstoppable in #596
• Bump minimum version for go to 1.24 by @phillip-stephens in #597
• build(deps): bump actions/setup-go from 5 to 6 by @dependabot[bot] in #605
• build(deps): bump golang.org/x/time from 0.12.0 to 0.13.0 by @dependabot[bot] in #600
• build(deps): bump github.com/prometheus/client_golang from 1.23.0 to 1.23.2 by @dependabot[bot] in #603
• build(deps): bump golang.org/x/sys from 0.35.0 to 0.36.0 by @dependabot[bot] in #601
• Make verbose a base flag by @Seanstoppable in #599
• build(deps): bump golang.org/x/text from 0.28.0 to 0.29.0 by @dependabot[bot] in #602
• build(deps): bump actions/attest-build-provenance from 2 to 3 by @dependabot[bot] in #604
• Add additional tests for http by @Seanstoppable in #606
• Phillip/98 metadata file improvements by @phillip-stephens in #607
• build(deps): bump golang.org/x/crypto from 0.41.0 to 0.42.0 by @dependabot[bot] in #610
• build(deps): bump golang.org/x/net from 0.43.0 to 0.44.0 by @dependabot[bot] in #609
• Update ZFlags by @phillip-stephens in #608
• Add support for HTTP/2 and HTTP/2 over clear-text by @phillip-stephens in #581
• build(deps): bump github.com/rogpeppe/go-internal from 1.10.0 to 1.14.1 by @dependabot[bot] in #612
• Return all the TLS handshake details we have even if handshake fails by @phillip-stephens in #613

New Contributors

• @zsbahtiar made their first contribution in #584

Full Changelog: v1.0.0-RC1...v1.0.0-RC2

Announcing v1.0 of ZGrab2 in prerelease!
Assuming there's no regressions or bugs reported in the next month or so, we can tag a release version.

Major Changes

• Adds a DNS rate limiter and per-IP rate limiter. These can be used to cap the number of DNS and connections, respectively, a server will receive. Look at the --help for --dns-rate-limit and --server-rate-limit for more details
• Adds a memcached scan module for scanning the memcached protocol. Thanks to @nicholasingino for this!

Improvements

• Adds CLI flag grouping and more detailed module descriptions in --help
• Adds non-TLS retry to Postgres scans to get servers that don't support TLS. Thanks @UnknownEclipse for this!

New Contributors

• @UnknownEclipse made their first contribution in #573
• @nicholasingino made their first contribution in #551

Full Changelog: v0.2.0...v1.0.0-RC1

v0.2.0 includes quite a number of changes and improvements over v0.1.8. There were new features added and overall code quality improvements. The most important changes are outlined below.

Breaking Changes

• Addition of dialerGroup to the Scan() module API. This will let ZGrab be used as a library and let clients bring their own dialers/network connections. This theoretically would let you run any application protocol over any transport you want. In practice, several protocols are more tightly coupled and need both a TCP dialer, they'll exchange some info, and then establish a TLS connection. STARTTLS is an example of this. More info in #506.
• Made the default behavior for smtp more reasonable where it can auto-detect support for EHLO vs. HELO in the banner as well as STARTTLS support. Adds CLI override flags to specify behavior if user wishes to override. Removes --send-ehlo, --send-start-tls and --send-helo since these now auto-detect. Adds --send-X-override flags for overriding. #509

Major Changes/Improvements

• Categorizes the CLI flags for modules to group them into Application Options that affect all modules, Basic Options for a module that they all share, and module-specific options. Will let users find the flag they need faster. See #479
• Fixed timeout behavior which would let scans run far longer than expected. Provides the user with --connect-timeout to control initial connection establishment and --target-timeout to bound the runtime of a scan against a given input target. See #537
• User can specify --local-addr and --local-port to bind 1+ IPs to for outgoing connections. See #519
• Changes IPv6 behavior to assume IPv4 and have IPv6 as an optional CLI flag
• Adds support for specifying 1+ DNS resolvers to use for lookups. Default behavior remains to use the OS configured defaults (/etc/resolv.conf on UNIX) See #556
• Add a ZMap-style blocklist. See #553
• Adds a ZMap-style per-second scan status print-out to give constant user feedback on how a scan is proceeding. See #528

New Modules

• AMQP 0.9.1
• PPTP
• MQTT
• socks5

All Changes

• Bump google.golang.org/protobuf from 1.30.0 to 1.33.0 by @dependabot in #414
• build: bump alpine from 3.19 to 3.20 by @dependabot in #442
• New Protocol: AMQP 0.9.1 by @developStorm in #426
• fixed typo in cleanup script by @phillip-stephens in #445
• docs: docker usage by @developStorm in #438
• Fix non-compilation in /lib/ssh/messages_test.go by @phillip-stephens in #446
• style: reformat all go files by @developStorm in #437
• Try to align golang around a single version by @Seanstoppable in #461
• Upgrade golang.org/x/crypto from 0.23.0 to 0.31.0 by @Seanstoppable in #476
• fix: failing ci tests by @developStorm in #478
• Group Basic/TLS Flag Groupings in Modules and interface{} -> any by @phillip-stephens in #479
• build(deps): bump github.com/prometheus/client_golang from 1.14.0 to 1.20.5 by @dependabot in #482
• build(deps): bump golang.org/x/crypto from 0.31.0 to 0.32.0 by @dependabot in #480
• Document the PORT field for CSV-format input by @dlenskiSB in #456
• Upgrade to go 1.23 by @thecsw in #466
• build(deps): bump alpine from 3.20 to 3.21 by @dependabot in #484
• build(deps): bump actions/attest-build-provenance from 1 to 2 by @dependabot in #483
• build(deps): bump docker/build-push-action from 5 to 6 by @dependabot in #453
• Update zgrab's python2 dependency to python3 by @phillip-stephens in #485
• Use bodyTextLen instead of readLen for FailHTTPToHTTPS logic by @Nirusu in #345
• Add support for PPTP by @AlexAQ972 in #470
• Make banner read options configurable via CLI by @svbatalov in #468
• Add support for MQTT v3.1.1 and v5.0 by @AlexAQ972 in #467
• Add support for socks5 by @AlexAQ972 in #471
• Fix ssh by @espegro in #335
• ssh: Port lib/ssh from x/crypto v0.7.0 by @TrueSkrillor in #399
• Bump ubuntu from 16.04 to 24.04 in /docker-runner by @dependabot in #430
• Update Golang minimum version/installation instructions by @phillip-stephens in #488
• Test Automation Performance Improvements + Streamlining by @phillip-stephens in #489
• build(deps): bump golang.org/x/sys from 0.29.0 to 0.30.0 by @dependabot in #490
• build(deps): bump golang.org/x/text from 0.21.0 to 0.22.0 by @dependabot in #491
• build(deps): bump golang.org/x/crypto from 0.32.0 to 0.33.0 by @dependabot in #492
• build(deps): bump golang.org/x/net from 0.34.0 to 0.35.0 by @dependabot in #493
• build(deps): bump github.com/prometheus/client_golang from 1.20.5 to 1.21.0 by @dependabot in #494
• Bump all dependencies by @phillip-stephens in #497
• Generate server certs for MQTT test container by @Seanstoppable in #505
• Don't disable http tee after first header, so that headers of re-directs can be captured and we don't panic by @phillip-stephens in #504
• Upgrade the prometheus dependency by @phillip-stephens in #508
• Fix issues with running subset of integration tests by @phillip-stephens in #510
• Improve SMTP usability by @phillip-stephens in #509
• Add back in EHLO, HELO, and STARTTLS Overrides to SMTP by @phillip-stephens in #512
• Update the zcrypto dependency to bring in TLS 1.3 support by @phillip-stephens in #507
• build(deps): bump github.com/zmap/zcrypto to 4f0ea0eaccac by @developStorm in #513
• modbus - avoid array indexing out-of-bounds by @phillip-stephens in #514
• Fix SHA256 typo by @hegga in #515
• New DialerGroupConfig interface for modules to given greater control over connection (L4/TLS/etc) establishment by @phillip-stephens in #506
• WIP - Add large HTTP file integration test and remove --maxbytes by @phillip-stephens in #516
• Fix redirects not getting followed by @phillip-stephens in #518
• Use context.Deadline in addition to specified timeouts for connections by @phillip-stephens in #517
• build(deps): bump golang.org/x/crypto from 0.36.0 to 0.37.0 by @dependabot in #521
• Throw an error is all we got was a bunch of empty hashes for JARM by @Seanstoppable in #525
• Improve ZGrab Docs/--help by @phillip-stephens in #524
• Let user specify local addr(s) and/or local port(s) to be used for making connections by @phillip-stephens in #519
• Remove min() re-declarations by @phillip-stephens in #527
• Add a per-second status similar to ZDNS by @phillip-stephens in #528
• build(deps): bump golang.org/x/net from 0.38.0 to 0.39.0 by @dependabot in #529
• build(deps): bump github.com/prometheus/client_golang from 1.21.1 to 1.22.0 by @dependabot in #530
• Move setting global TLS parameters to init() so it's run once and -race won't complain by @phillip-stephens in #532
• Remove unused single local addr by @phillip-stephens in #531
• Stabilize conn timeout unit tests by @phillip-stephens in #536
• Fix Multiple Module regression by @phillip-stephens in #534
• Make timeout behavior user-changeable and easy-to-understand by @phillip-stephens in #537
• Downgrade to using v7 of redis, since v8 is buggy right now by @phillip-stephens in #539
• Give user IP that domain resolves to by @phillip-stephens in #533
• Fix timeout issue with setting ReadDeadline after conn creation by @phillip-stephens in #538
• Parse CAs file only if the file is different from last parsed by @phillip-stephens in #540
• Give accurate status for io-timeout by @phillip-stephens in #541
• build(deps): bump golang.org/x/sys from 0.32.0 to 0.33.0 by @dependabot in #543
• build(deps): bump golang.org/x/net from 0.39.0 to 0.40.0 by @dependabot in #545
• Add several linters to CI and fix found issues by @phillip-stephens in #542
• Fixes a few issues with redirects by @Seanstoppable in #462
• [HTTP] base64 encode binary data by @phillip-stephens in #549
• Set http default port to 443 if using --use-https by @phillip-stephens in #548
• Improve IP capability detection performance and over VPNs by @phillip-stephens in #552
• Add a ZMap-style Blocklist by @phillip-stephens in #553
• Bugfix - allow user to set the --dns option by @phillip-stephens in #556
• Remove IP capability detection, enable IPv4 by default by @phillip-stephens in #557
• build(deps): bump golang.org/x/text from 0.25.0 to 0.26.0 by @dependabot in #561
• build(deps): bump golang.org/x/crypto from 0.38.0 to 0.39.0 by @dependabot in #562
• build(deps): bump golang.org/x/net from 0.40.0 to 0.41.0 by @dependabot in #563
• Don't error if you can't close the connection by @phillip-stephens in #559
• Fix for status reporting not being complete and bound time of DNS resolutions by @phillip-stephens in #558
• [bug-fix] - Fix not including the body of a re-directed HTTP request by @phillip-stephens in #564
• Have default make target build, not install by @phillip-stephens in #566

New Contributors

• @phillip-stephens made their first contribution in #445
• @dlenskiSB made their first contribution in #456
• @thecsw made their first contribution in #466
• @AlexAQ972 made their first contribution in #470
• @espegro made their first contribution in #335
• @TrueSkrillor made their first contribution in #399
• @hegga made their first contribution in #515

Full Changelog: v0.1.8...v0.2.0