github.com/TecharoHQ/anubis

Anubis v1.27.0 adds Windows Server support, automatically renames cookies based on settings to avoid infinite challenge loops, adds two new localizations, and more.

Breaking change: cookie names are dynamically created based on cookie settings

Anubis tries to avoid breaking changes as much as possible, but sometimes we have to make them for the sake of the users. This is technically a breaking change in something that is not part of the public API of Anubis; but some administrators rely heavily on cookie names in advanced configurations.

It seems that browsers store cookies disambiguated with their options. This means you can have multiple cookies named the same but with different options. Browsers will send these cookies to the server without the list of options. This means that changing any cookie settings requires you to change COOKIE_PREFIX, creating a new "cookie epoch" that will set things properly.

In order to be more robust, Anubis will automatically change cookie names based on the cookie settings. For example, the default configuration creates cookies named techaro.lol-anubis-auth-347ddb4a.

Without this change, changing any cookie setting without every client clearing their cookies causes challenges to become an infinite loop of thrashing, making it appear that Anubis "blocked" them.

If this becomes onerous in practice for administrators of HAProxy and other advanced setups that rely on cookie names, we will add an escape hatch in the policy file.

Windows Server support (beta)

Anubis now publishes .msi packages, allowing administrators to install and run Anubis on Windows Server. Please read the Windows Server page for more information.

This support is beta-grade as the Anubis team does not have a lot of experience with developing software for Windows Server. Feedback is more than welcome.

Please let us know how it works for you!

Pre-release docker images no longer populate the latest tag

Due to a misconfiguration of the GitHub Action docker/metadata-action, pre-release Docker images previously populated the :latest tag. This means that administrators that expected the :latest tag to result in a stable release of Anubis got a prerelease version suddenly when they ran automatic updates.

If administrators want to opt-in to the prerelease build track of Anubis for more frequent access to new features, they can use the :pre tag:

image: ghcr.io/techarohq/anubis:pre

Features

• Allow importing bot policy rules using wildcard matching (#1815).
• Add support for JSON sources to iplist2rule (#1833)
• Make the honeypot feature log detected addresses to the disk every minute when honeypot.ip_log_file is set. See the IP address logging section for more information.
• Add standard library rules for making Dillo less threatening to Anubis and document how to enable them.

Crawlers

• Allow Arquivo.pt, the Portuguese web archive, by default via its crawling network.
• Add (data)/bots/lyrenth.yaml snippet that denies Lyrenth's AIWebIndex crawler and AIWebIndex-Agent on-demand fetcher by user agent and by their published IP ranges. This is imported by (data)/bots/_deny-pathological.yaml.
• Updates Alibaba cloud IP list (#1813)
• Updates Googlebot IP list (#1812)
• Updates IP list for DuckDuckBot (#1810)
• Update Huawei Cloud IP list (#1814)

Fixes

• Fix bot policy imports to not require pedantically correct YAML formatting when using wildcard matching.
• JavaScript served by the fast challenge is loaded using defer instead of async (#1782).
• Amend default Lightpanda rule to match current behaviour, add smoke test to ensure it keeps working (#1822).
• Fix a panic when a request asks for the undetermined language tag, such as Accept-Language: und (#1776).
• Allow user agents that start with capital-G Git in (data)/clients/git.yaml.
• Enabled the Partitioned flag on cookies by default (#1701).
• Fix Windows MSI builds on prerelease tags such as v1.27.0-pre1.
• Bump AI-robots.txt to version 1.47.

i18n

• Add Basque (eu) localization.
• Update Bulgarian locale (#1708)
• Add Croatian (hr) localization.

Anubis v1.27.0 adds Windows Server support, automatically renames cookies based on settings to avoid infinite challenge loops, adds two new localizations, and more.

Breaking change: cookie names are dynamically created based on cookie settings

Anubis tries to avoid breaking changes as much as possible, but sometimes we have to make them for the sake of the users. This is technically a breaking change in something that is not part of the public API of Anubis; but some administrators rely heavily on cookie names in advanced configurations.

It seems that browsers store cookies disambiguated with their options. This means you can have multiple cookies named the same but with different options. Browsers will send these cookies to the server without the list of options. This means that changing any cookie settings requires you to change COOKIE_PREFIX, creating a new "cookie epoch" that will set things properly.

In order to be more robust, Anubis will automatically change cookie names based on the cookie settings. For example, the default configuration creates cookies named techaro.lol-anubis-auth-347ddb4a.

Without this change, changing any cookie setting without every client clearing their cookies causes challenges to become an infinite loop of thrashing, making it appear that Anubis "blocked" them.

If this becomes onerous in practice for administrators of HAProxy and other advanced setups that rely on cookie names, we will add an escape hatch in the policy file.

Windows Server support (beta)

Anubis now publishes .msi packages, allowing administrators to install and run Anubis on Windows Server. Please read the Windows Server page for more information.

This support is beta-grade as the Anubis team does not have a lot of experience with developing software for Windows Server. Feedback is more than welcome.

Please let us know how it works for you!

Pre-release docker images no longer populate the latest tag

Due to a misconfiguration of the GitHub Action docker/metadata-action, pre-release Docker images previously populated the :latest tag. This means that administrators that expected the :latest tag to result in a stable release of Anubis got a prerelease version suddenly when they ran automatic updates.

If administrators want to opt-in to the prerelease build track of Anubis for more frequent access to new features, they can use the :pre tag:

image: ghcr.io/techarohq/anubis:pre

Features

• Allow importing bot policy rules using wildcard matching (#1815).
• Add support for JSON sources to iplist2rule (#1833)
• Make the honeypot feature log detected addresses to the disk every minute when honeypot.ip_log_file is set. See the IP address logging section for more information.
• Add standard library rules for making Dillo less threatening to Anubis and document how to enable them.

Crawlers

• Allow Arquivo.pt, the Portuguese web archive, by default via its crawling network.
• Add (data)/bots/lyrenth.yaml snippet that denies Lyrenth's AIWebIndex crawler and AIWebIndex-Agent on-demand fetcher by user agent and by their published IP ranges. This is imported by (data)/bots/_deny-pathological.yaml.
• Updates Alibaba cloud IP list (#1813)
• Updates Googlebot IP list (#1812)
• Updates IP list for DuckDuckBot (#1810)
• Update Huawei Cloud IP list (#1814)

Fixes

• Fix bot policy imports to not require pedantically correct YAML formatting when using wildcard matching.
• JavaScript served by the fast challenge is loaded using defer instead of async (#1782).
• Amend default Lightpanda rule to match current behaviour, add smoke test to ensure it keeps working (#1822).
• Fix a panic when a request asks for the undetermined language tag, such as Accept-Language: und (#1776).
• Allow user agents that start with capital-G Git in (data)/clients/git.yaml.
• Enabled the Partitioned flag on cookies by default (#1701).
• Fix Windows MSI builds on prerelease tags such as v1.27.0-pre1.
• Bump AI-robots.txt to version 1.47.

i18n

• Add Basque (eu) localization.
• Update Bulgarian locale (#1708)
• Add Croatian (hr) localization.

This will become the stable release on Sunday.

• Add support for JSON sources to iplist2rule (#1833)
• Allow importing bot policy rules using wildcard matching (#1815).
• Fix bot policy imports to not require pedantically correct YAML formatting.
• JavaScript served by the fast challenge is loaded using defer instead of async (#1782).
• Pre-release tags no longer populate the latest tag in Docker. Pre-releases can be installed by using the pre tag.
• Updates Alibaba cloud IP list (#1813)
• Updates Googlebot IP list (#1812)
• Updates IP list for DuckDuckBot (#1810)
• Update Huawei Cloud IP list (#1814)
• Add standard library rules for making Dillo less threatening to Anubis and document how to enable them.
• Amend default Lightpanda rule to match current behaviour, add smoke test to ensure it keeps working (#1822).
• Fix a panic when a request asks for the undetermined language tag, such as Accept-Language: und (#1776).
• Allow user agents that start with capital-G Git in (data)/clients/git.yaml.
• Add Croatian locale

What's Changed

• fix(proofofwork): de-flake JavaScript loading in Google Chrome by @Xe in #1807
• fix(mkmsi): skip if there are no zips to make into an msi by @Xe in #1811
• chore(data/crawlers): update DuckDuckBot IP list by @dchandekstark in #1810
• chore(data/crawlers): update Googlebot IP list by @dchandekstark in #1812
• chore(data/crawlers): update alibaba-cloud IP list by @dchandekstark in #1813
• ci(docker): tag prereleases with "pre" by @Xe in #1808
• chore(data/crawlers): update Huawei Cloud IP list by @dchandekstark in #1814
• feat(config): allow wildcard matches in import statements by @Xe in #1816
• feat(data/small-internet-browsers): add rules to make Dillo less heavy by @Xe in #1820
• fix(config): support multiple yaml documents for wildcard imports by @Xe in #1819
• fix(localization): don't load manifest.json as a locale by @SihyeonJeon in #1821
• fix(data/headless-browsers): amend lightpanda block rule by @Xe in #1823
• fix(data/clients): allow capital G 'Git' in git client user agents by @Xe in #1824
• feat(localization): add Croatian locale support by @mgoljak in #1832
• feat(iplist2rule): Adds support for extracting addresses from JSON sources by @dchandekstark in #1833

New Contributors

• @SihyeonJeon made their first contribution in #1821
• @mgoljak made their first contribution in #1832

Full Changelog: v1.27.0-pre3...v1.27.0-pre4

• Make the honeypot feature log detected addresses to the disk every minute when honeypot.ip_log_file is set. See the IP address logging section for more information.
• Add Basque (eu) localization.
• Rename cookies based on cookie settings.
• Update Bulgarian locale (#1708)
• Enabled the Partitioned flag on cookies by default (#1701).

What's Changed

• chore(cmd/anubis): enable Partitioned flag by default by @Xe in #1797
• feat: add Basque localization by @erral in #1788
• fix(i18n): update Bulgarian language translation by @TrapStoner in #1708
• fix(i18n/vi): update translation accuracy by @honhatduy in #1748
• fix(lib): rename cookies based on settings by @Xe in #1798
• feat(honeypot): write honeypot IP addresses to a log file by @Xe in #1799
• docs(user): rewrite using ASD STE-100 by @Xe in #1802

New Contributors

• @erral made their first contribution in #1788
• @TrapStoner made their first contribution in #1708

Full Changelog: v1.27.0-pre2...v1.27.0-pre3

Anubis now supports running on Windows as a native service. To use it as a service, install the .msi from the release and follow the Windows directions. Currently Anubis supports Windows 10 / Windows Server 2016 and newer. Older versions of Windows currently not supported due to limits of the Go toolchain.

Please try the Windows release and give feedback! We want to make it as good as it can possibly be.

• Bump AI-robots.txt to version 1.47.
• Add (data)/bots/lyrenth.yaml snippet that denies Lyrenth's AIWebIndex crawler and AIWebIndex-Agent on-demand fetcher by user agent and by their published IP ranges. This is imported by (data)/bots/_deny-pathological.yaml.
• Add a Windows MSI installer and wire Anubis to run as a Windows service. See the Windows docs for more information.
• Fix Windows MSI builds on prerelease tags such as v1.27.0-pre1.
• Allow Arquivo.pt, the Portuguese web archive, by default via its crawling network.

What's Changed

• feat(data): add Lyrenth scraper by @Xe in #1787
• feat: native Windows service support by @Xe in #1793
• chore: Bump ai.robots.txt to v1.47 by @Dryusdan in #1780

Full Changelog: v1.26.2...v1.27.0-pre2

Automatically verify correct parsing of everything in (data). While doing post-release checks on v1.26.1, I discovered that I incorrectly merged (data)/services/updown.yaml in such a way that it became syntactically invalid. This has been mended and multiple layers of CI have been put into place to make sure that (data) entries are syntactically and semantically valid.

What's Changed

• chore(data): fix updown stdlib entry, add CI to catch future things by @Xe in #1771

Full Changelog: v1.26.1...v1.26.2

• Automatically verify correct parsing of everything in (data). While doing post-release checks on v1.26.1, I discovered that I incorrectly merged (data)/services/updown.yaml in such a way that it became syntactically invalid. This has been mended and multiple layers of CI have been put into place to make sure that (data) entries are syntactically and semantically valid.

• Fix support for semicolon-delimited query parameters that was dropped when moving from net/http/httputil#ReverseProxy.Director (deprecated) to net/http/httputil#ReverseProxy.Rewrite. This re-enables support for upstreams like gitweb (#1763). A functional test has been added to ensure this does not repeat.

Challenge page robustness

The challenge page can now survive transient failures, reduces the number of requests it makes to the Anubis app, and adds exponential backoff with retries to counteract an overwhelmed server being unable to serve any assets.

Previously if any request for JavaScript assets failed, the entire challenge attempt failed and users were forced to manually refresh the page, which is a bit of a bad user experience. This was made worse when the load balancer does not support HTTP/2, did not have resumable sessions enabled, and was implemented with Apache httpd pre-fork; making each asset fetch do its own TCP/TLS handshake. Under periods of heavy load such that TCP/TLS handshakes timed out, this made Anubis unable to fetch assets consistently or even made in-progress challenge attempts fail, which made challenges impossible to pass.

This has been fixed in a few ways:

• Fetch operations now retry with exponential backoff in hopes that they will eventually be able to get through when the server is less stressed.
• Attempting to fetch the main.mjs script now has fallback watchdog logic that periodically re-attempts to load the script.
• Worker source code is fetched once and then loaded into workers with a Blob instead of having each worker do an independent fetch of the worker source code.
• Individual workers can now die without making the entire challenge attempt fail. Surviving workers will cover the rest of the nonce space.
• When worker construction fails, already running proof of work workers are terminated instead of staying active as headless unmonitored infinite loops.
• Proof of work failures are now exposed as untranslated real errors. Browsers signal script load failures with real Events so the challenge failure page now shows a useful message.

What's Changed

• chore: update updown IPs by @hyperdefined in #1666
• fix(main): re-enable semicolon logic for proxied backends by @Xe in #1764
• feat: make proofofwork challenge more reliable under heavy server load by @Xe in #1766

Full Changelog: v1.26.0...v1.26.1

• Fix support for semicolon-delimited query parameters that was dropped when moving from net/http/httputil#ReverseProxy.Director (deprecated) to net/http/httputil#ReverseProxy.Rewrite. This re-enables support for upstreams like gitweb (#1763). A functional test has been added to ensure this does not repeat.

Challenge page robustness

The challenge page can now survive transient failures, reduces the number of requests it makes to the Anubis app, and adds exponential backoff with retries to counteract an overwhelmed server being unable to serve any assets.

Previously if any request for JavaScript assets failed, the entire challenge attempt failed and users were forced to manually refresh the page, which is a bit of a bad user experience. This was made worse when the load balancer does not support HTTP/2, did not have resumable sessions enabled, and was implemented with Apache httpd pre-fork; making each asset fetch do its own TCP/TLS handshake. Under periods of heavy load such that TCP/TLS handshakes timed out, this made Anubis unable to fetch assets consistently or even made in-progress challenge attempts fail, which made challenges impossible to pass.

This has been fixed in a few ways:

• Fetch operations now retry with exponential backoff in hopes that they will eventually be able to get through when the server is less stressed.
• Attempting to fetch the main.mjs script now has fallback watchdog logic that periodically re-attempts to load the script.
• Worker source code is fetched once and then loaded into workers with a Blob instead of having each worker do an independent fetch of the worker source code.
• Individual workers can now die without making the entire challenge attempt fail. Surviving workers will cover the rest of the nonce space.
• When worker construction fails, already running proof of work workers are terminated instead of staying active as headless unmonitored infinite loops.
• Proof of work failures are now exposed as untranslated real errors. Browsers signal script load failures with real Events so the challenge failure page now shows a useful message.

Sorry this took so long. A lot of work was done behind the scenes to do mass testing of Anubis on many versions of Google Chrome. After about 10 attempts failed, the current one seems to work, leading to the confidence of being able to cut this release.

This should support Chrome 69 and newer.

• Add option to disable the honeypot.
• Add (data)/clients/google-user-triggered-fetchers.yaml snippet that allows Google-owned user-triggered fetchers (Google Translate's website translation proxy, Google Read Aloud, Google Messages link previews) by their published IP ranges, fixing the infinite challenge loop for visitors using Google Translate (#444)
• Update QwantBot remote addresses range with latest value
• Migrate check-spelling workflow config to cspell
• Add Anubis Kubernetes Operator to the docs (#1675).
• Bump Playwright browser tooling to 1.61.1 and playwright-go to v0.6100.0.
• Add FreeBSD/Windows binaries to the yeetfile.
• Add systemd system extensions to the yeetfile.
• Set an explicit esbuild --target=chrome66 so modern syntax (e.g. optional chaining) is transpiled down. This lowers the minimum supported browser from Chrome 80 to Chrome 66.
• Patch GHSA-6wcg-mqvh-fcvg by containing subrequest logic to Anubis instances in subrequest mode.
• Implement robot9001 style delays on the honeypot feature so that the first hit takes 1 millisecond, the second takes 2, etc.
• Move metrics server configuration to the policy file.
• Expose pprof endpoints on the metrics listener to enable profiling Anubis in production.
• fix: prevent nil pointer panic in challenge validation when threshold rules match during PassChallenge (#1463)
• Instruct reverse proxies to not cache error pages.
• Fixed mixed tab/space indentation in Caddy documentation code block
• Improve error messages and fix broken REDIRECT_DOMAINS link in docs (#1193)
• Add Bulgarian locale (#1394)
• Fixed case-sensitivity mismatch in geoipchecker.go
• Use Go's native version stamping instead of a handrolled variant.
• Fix CEL internal errors when iterating headers/query map wrappers by implementing map iterators for HTTPHeaders and URLValues (#1465).
• Enable metrics serving via TLS, including mutual TLS (mTLS).
• Enable HTTP basic auth for the metrics server.
• Fix a bug in the dataset poisoning maze that could allow denial of service #1580.
• Add config option to add ASN to logs/metrics.
• Log weight when issuing challenge.
• Block x.ai's crawler for code review training.
• Gate pprof endpoints behind metrics.debug in the policy file.
• Limit naive honeypot r9k delay to one second.
• Fix an obscure case where adding query values to a subrequest match could cause an invalid rule match when using path based matching for protected resources.
• Anubis now requires Go 1.26 to build.
• Fix an edge case where load average expression values could nil pointer dereference when Anubis just started up.
• Fix an obscure case where Anubis in subrequest mode could allow redirects to invalid domains with strange instructions.
• Fix path_regex and CEL path rules not matching when using Traefik forwardAuth middleware. Anubis now checks X-Forwarded-Uri (Traefik) in addition to X-Original-URI (nginx) when resolving the request path in subrequest mode (#1628).
• Validate bounds in the CEL randInt helper so non-positive or platform-overflowing arguments surface a typed CEL error instead of an evaluator panic.
• Fix a race in the bbolt store where the asynchronous cleanup scheduled by an expired read could delete a value that had just been refreshed; the delete now only fires when the key still carries the same expired generation it observed.
• Marginally increase the performances of requests processing
• Marginally improve the performances of PoW validation
• Marginally improve the performances of challenges generation/display
• Significantly improve the performances of the gzip middleware
• Significantly improve the performances of the PoW validation
• Add trimpath option to artifact builds
• Add COOKIE_HTTP_ONLY option to set the HttpOnly flag on Anubis cookies
• Improve the performances of rules validation
• Only compute the JA4H fingerprint when a policy references the X-Http-Fingerprint-JA4H header, taking it off the hot path for configurations that don't use it (#834).
• Migrate the target reverse proxy off the deprecated httputil.ReverseProxy.Director to Rewrite for Go 1.26 compatibility, preserving the inbound Host and X-Forwarded-*/Forwarded headers.

What's Changed

• docs(admin/policy): document ReadWritePaths for logging to files by @Xe in #1469
• docs: fix mixed tab/space indentation in Caddy config example by @pzhlkj6612 in #1506
• ci: purge govulncheck, it's less signal than i hoped by @Xe in #1515
• feat(http): set "Cache-Control: no-store" on error responses by @gucci-on-fleek in #1474
• docs(known-instances): Add missing one. by @p0008874 in #1500
• chore(l10n): update French translation by @HarmfulBreeze in #1496
• docs: remove developer/code-quality by @gucci-on-fleek in #1475
• build(deps): bump the github-actions group across 1 directory with 11 updates by @dependabot[bot] in #1516
• build(deps): bump the npm group across 1 directory with 6 updates by @dependabot[bot] in #1512
• chore: gofix by @JasonLovesDoggo in #1466
• fix: nil deref when solve request lands on a different DIFFICULTY than the original req by @JasonLovesDoggo in #1467
• feat(docs): Add instructions on using Anubis with envoy-gateway by @fhoekstra in #1460
• feat(docs): Update HAProxy Advanced Variant documentation by @rozmansi in #1521
• feat(data): add Citoid to good bots list by @mvolz in #1524
• docs(faq): document that disabling JIT makes Anubis slow by @Xe in #1526
• fix(locales/ja): change the position of the バージョン (version) by @fu-sen in #1527
• fix: add cel iterator by @JasonLovesDoggo in #1465
• fix: Address issued raised in #1193 by @dwhitemv25 in #1519
• feat(anubis): enable pprof endpoints on the metrics server by @Xe in #1530
• feat(i18n): add Bulgarian language translation by @ramromcode in #1394
• chore: update vi.json by @honhatduy in #1459
• fix(locales/de): revise and update German translations by @michi-onl in #1457
• build(deps): bump actions-hub/kubectl from 1.35.2 to 1.35.3 in the github-actions group by @dependabot[bot] in #1532
• chore(sponsors): add embeDD GmbH by @Xe in #1539
• build(deps): bump the npm group with 5 updates by @dependabot[bot] in #1531
• docs(faq): document minimum versions of browsers Anubis supports by @Xe in #1540
• build(deps): bump the npm group across 1 directory with 5 updates by @dependabot[bot] in #1559
• build(deps): bump the github-actions group across 1 directory with 7 updates by @dependabot[bot] in #1560
• feat: move metrics server config to the policy file by @Xe in #1572
• feat(data): add Meta's web indexer used for AI purposes by @bnjbvr in #1573
• feat(metrics): enable TLS/mTLS serving support by @Xe in #1576
• feat(metrics): basic auth support by @Xe in #1579
• fix(honeypot): remove DoS vector by @Xe in #1581
• fix: Thoth geoip compare by @lenny87 in #1564
• fix(metrics): bind to the right network/bindhost by @Xe in #1606
• feat: add ASN data from Thoth to logs/metrics by @lillian-b in #1608
• feat: Log weight when issuing challenge by @tdgroot in #1611
• fix: patch GHSA-6wcg-mqvh-fcvg by @Xe in #1616
• build(deps-dev): bump the npm group across 1 directory with 6 updates by @dependabot[bot] in #1621
• build(deps): bump github.com/go-jose/go-jose/v3 from 3.0.4 to 3.0.5 by @dependabot[bot] in #1629
• fix(policy): correctly wire subrequest mode through CEL/path checkers by @Xe in #1630
• fix(honeypot/naive): apply robot9001 style delays by @Xe in #1632
• build(deps): bump the npm group across 1 directory with 6 updates by @dependabot[bot] in #1646
• perf(lib): iterate s.policy.Bots by index to drop per-call heap copy by @jvoisin in #1639
• fix: small security fixes by @Xe in #1651
• perf(internal/gzip): pool *gzip.Writer per middleware instance by @jvoisin in #1654
• perf: enable uuid randomness pool and minor cleanups by @jvoisin in #1652
• perf(challenge/proofofwork): stream sha256 into stack buffer in Validate by @jvoisin in #1653
• chore: use Go stdlib version stamping by @Xe in #1665
• chore: ban x.ai by @Xe in #1673
• feat: add HttpOnly cookie option by @tdgroot in #1679
• fix: add trimpath option to artifact builds by @tdgroot in #1682
• ci: remove spelling checks by @Xe in #1688
• perf(policy): only compute ja4h fingerprint when referenced by @jvoisin in #1689
• perf: don't look into the store for OpenGraph things if it is disabled by @jvoisin in #1690
• refactor(honeypot/naive): compute network hash once in incrementNetwork by @jvoisin in #1693
• perf: iterate s.policy.Bots by index to drop per-call heap copy by @jvoisin in #1691
• perf(internal/gzip): share writer pool across middleware reconstructions by @jvoisin in #1694
• perf(internal): cut allocations in computeXFFHeader by @jvoisin in #1692
• build(deps-dev): bump the npm group across 1 directory with 4 updates by @dependabot[bot] in #1677
• docs: use stable container in traefik deployment example by @RealOrangeOne in #1676
• chore: bump go version to 1.26 by @Xe in #1695
• chore: tag v1.26.0-pre1 by @Xe in #1705
• feat(qwantbot): update remote addresses by @mhardy-qwant in #1719
• ci: add cspell to replace check-spelling by @Xe in #1722
• chore: set up golangci-lint by @Xe in #1720
• chore: add golangci words to the spelling dictionary by @Xe in #1724
• build(deps): bump the npm group across 1 directory with 8 updates by @dependabot[bot] in #1723
• docs: add anubis-kubernetes-operator as another option by @eznix86 in #1675
• chore: update playwright dependencies by @JasonLovesDoggo in #1279
• build(deps): bump the gomod group across 1 directory with 16 updates by @dependabot[bot] in #1727
• build(deps): bump the github-actions group across 1 directory with 14 updates by @dependabot[bot] in #1725
• build(deps): bump the gomod group with 2 updates by @dependabot[bot] in #1730
• build(deps-dev): bump prettier from 3.8.5 to 3.9.1 in the npm group by @dependabot[bot] in #1729
• chore(data): fix google translate bug #444 by @Xe in #1734
• fix(web): lower chrome support floor to Chrome 66 by @Xe in #1738
• feat(yeetfile): build freebsd and windows binaries by @Xe in #1735
• chore: add chrome testing command by @Xe in #1740
• fix(crawlers): update MojeekBot remote_addresses subnet to match official spec by @bochard07 in #1746
• feat(config): add option to disable the honeypot by @Xe in #1759

New Contributors

• @pzhlkj6612 made their first contribution in #1506
• @HarmfulBreeze made their first contribution in #1496
• @fhoekstra made their first contribution in #1460
• @rozmansi made their first contribution in #1521
• @mvolz made their first contribution in #1524
• @fu-sen made their first contribution in #1527
• @dwhitemv25 made their first contribution in #1519
• @ramromcode made their first contribution in #1394
• @honhatduy made their first contribution in #1459
• @bnjbvr made their first contribution in #1573
• @lenny87 made their first contribution in #1564
• @jvoisin made their first contribution in #1639
• @RealOrangeOne made their first contribution in #1676
• @mhardy-qwant made their first contribution in #1719
• @eznix86 made their first contribution in #1675
• @bochard07 made their first contribution in #1746

Full Changelog: v1.25.0...v1.26.0

• Add option to disable the honeypot.
• Add (data)/clients/google-user-triggered-fetchers.yaml snippet that allows Google-owned user-triggered fetchers (Google Translate's website translation proxy, Google Read Aloud, Google Messages link previews) by their published IP ranges, fixing the infinite challenge loop for visitors using Google Translate (#444)
• Update QwantBot remote addresses range with latest value
• Migrate check-spelling workflow config to cspell
• Add Anubis Kubernetes Operator to the docs (#1675).
• Bump Playwright browser tooling to 1.61.1 and playwright-go to v0.6100.0.
• Add FreeBSD/Windows binaries to the yeetfile.
• Add systemd system extensions to the yeetfile.
• Set an explicit esbuild --target=chrome66 so modern syntax (e.g. optional chaining) is transpiled down. This lowers the minimum supported browser from Chrome 80 to Chrome 66.
• Patch GHSA-6wcg-mqvh-fcvg by containing subrequest logic to Anubis instances in subrequest mode.
• Implement robot9001 style delays on the honeypot feature so that the first hit takes 1 millisecond, the second takes 2, etc.
• Move metrics server configuration to the policy file.
• Expose pprof endpoints on the metrics listener to enable profiling Anubis in production.
• fix: prevent nil pointer panic in challenge validation when threshold rules match during PassChallenge (#1463)
• Instruct reverse proxies to not cache error pages.
• Fixed mixed tab/space indentation in Caddy documentation code block
• Improve error messages and fix broken REDIRECT_DOMAINS link in docs (#1193)
• Add Bulgarian locale (#1394)
• Fixed case-sensitivity mismatch in geoipchecker.go
• Use Go's native version stamping instead of a handrolled variant.
• Fix CEL internal errors when iterating headers/query map wrappers by implementing map iterators for HTTPHeaders and URLValues (#1465).
• Enable metrics serving via TLS, including mutual TLS (mTLS).
• Enable HTTP basic auth for the metrics server.
• Fix a bug in the dataset poisoning maze that could allow denial of service #1580.
• Add config option to add ASN to logs/metrics.
• Log weight when issuing challenge.
• Block x.ai's crawler for code review training.
• Gate pprof endpoints behind metrics.debug in the policy file.
• Limit naive honeypot r9k delay to one second.
• Fix an obscure case where adding query values to a subrequest match could cause an invalid rule match when using path based matching for protected resources.
• Anubis now requires Go 1.26 to build.
• Fix an edge case where load average expression values could nil pointer dereference when Anubis just started up.
• Fix an obscure case where Anubis in subrequest mode could allow redirects to invalid domains with strange instructions.
• Fix path_regex and CEL path rules not matching when using Traefik forwardAuth middleware. Anubis now checks X-Forwarded-Uri (Traefik) in addition to X-Original-URI (nginx) when resolving the request path in subrequest mode (#1628).
• Validate bounds in the CEL randInt helper so non-positive or platform-overflowing arguments surface a typed CEL error instead of an evaluator panic.
• Fix a race in the bbolt store where the asynchronous cleanup scheduled by an expired read could delete a value that had just been refreshed; the delete now only fires when the key still carries the same expired generation it observed.
• Marginally increase the performances of requests processing
• Marginally improve the performances of PoW validation
• Marginally improve the performances of challenges generation/display
• Significantly improve the performances of the gzip middleware
• Significantly improve the performances of the PoW validation
• Add trimpath option to artifact builds
• Add COOKIE_HTTP_ONLY option to set the HttpOnly flag on Anubis cookies
• Improve the performances of rules validation
• Only compute the JA4H fingerprint when a policy references the X-Http-Fingerprint-JA4H header, taking it off the hot path for configurations that don't use it (#834).
• Migrate the target reverse proxy off the deprecated httputil.ReverseProxy.Director to Rewrite for Go 1.26 compatibility, preserving the inbound Host and X-Forwarded-*/Forwarded headers.

Unless anyone screams, this will be what ships tomorrow.

• Add (data)/clients/google-user-triggered-fetchers.yaml snippet that allows Google-owned user-triggered fetchers (Google Translate's website translation proxy, Google Read Aloud, Google Messages link previews) by their published IP ranges, fixing the infinite challenge loop for visitors using Google Translate (#444)
• Update QwantBot remote addresses range with latest value
• Migrate check-spelling workflow config to cspell
• Add Anubis Kubernetes Operator to the docs (#1675).
• Bump Playwright browser tooling to 1.61.1 and playwright-go to v0.6100.0.
• Add FreeBSD/Windows binaries to the yeetfile.
• Add systemd system extensions to the yeetfile.
• Set an explicit esbuild --target=chrome66 so modern syntax (e.g. optional chaining) is transpiled down. This lowers the minimum supported browser from Chrome 80 to Chrome 66.

What's Changed

• feat(qwantbot): update remote addresses by @mhardy-qwant in #1719
• ci: add cspell to replace check-spelling by @Xe in #1722
• chore: set up golangci-lint by @Xe in #1720
• chore: add golangci words to the spelling dictionary by @Xe in #1724
• build(deps): bump the npm group across 1 directory with 8 updates by @dependabot[bot] in #1723
• docs: add anubis-kubernetes-operator as another option by @eznix86 in #1675
• chore: update playwright dependencies by @JasonLovesDoggo in #1279
• build(deps): bump the gomod group across 1 directory with 16 updates by @dependabot[bot] in #1727
• build(deps): bump the github-actions group across 1 directory with 14 updates by @dependabot[bot] in #1725
• build(deps): bump the gomod group with 2 updates by @dependabot[bot] in #1730
• build(deps-dev): bump prettier from 3.8.5 to 3.9.1 in the npm group by @dependabot[bot] in #1729
• chore(data): fix google translate bug #444 by @Xe in #1734
• fix(web): lower chrome support floor to Chrome 66 by @Xe in #1738
• feat(yeetfile): build freebsd and windows binaries by @Xe in #1735

New Contributors

• @mhardy-qwant made their first contribution in #1719
• @eznix86 made their first contribution in #1675

Full Changelog: v1.26.0-pre1...v1.26.0-pre2

Sorry it's been so long between releases. Recovering from surgery sucks.

This release's title will be Papalymo Totolymo.

• Patch GHSA-6wcg-mqvh-fcvg by containing subrequest logic to Anubis instances in subrequest mode.
• Implement robot9001 style delays on the honeypot feature so that the first hit takes 1 millisecond, the second takes 2, etc.
• Move metrics server configuration to the policy file.
• Expose pprof endpoints on the metrics listener to enable profiling Anubis in production.
• fix: prevent nil pointer panic in challenge validation when threshold rules match during PassChallenge (#1463)
• Instruct reverse proxies to not cache error pages.
• Fixed mixed tab/space indentation in Caddy documentation code block
• Improve error messages and fix broken REDIRECT_DOMAINS link in docs (#1193)
• Add Bulgarian locale (#1394)
• Fixed case-sensitivity mismatch in geoipchecker.go
• Use Go's native version stamping instead of a handrolled variant.
• Fix CEL internal errors when iterating headers/query map wrappers by implementing map iterators for HTTPHeaders and URLValues (#1465).
• Enable metrics serving via TLS, including mutual TLS (mTLS).
• Enable HTTP basic auth for the metrics server.
• Fix a bug in the dataset poisoning maze that could allow denial of service #1580.
• Add config option to add ASN to logs/metrics.
• Log weight when issuing challenge.
• Block x.ai's crawler for code review training.
• Gate pprof endpoints behind metrics.debug in the policy file.
• Limit naive honeypot r9k delay to one second.
• Fix an obscure case where adding query values to a subrequest match could cause an invalid rule match when using path based matching for protected resources.
• Anubis now requires Go 1.26 to build.
• Fix an edge case where load average expression values could nil pointer dereference when Anubis just started up.
• Fix an obscure case where Anubis in subrequest mode could allow redirects to invalid domains with strange instructions.
• Fix path_regex and CEL path rules not matching when using Traefik forwardAuth middleware. Anubis now checks X-Forwarded-Uri (Traefik) in addition to X-Original-URI (nginx) when resolving the request path in subrequest mode (#1628).
• Validate bounds in the CEL randInt helper so non-positive or platform-overflowing arguments surface a typed CEL error instead of an evaluator panic.
• Fix a race in the bbolt store where the asynchronous cleanup scheduled by an expired read could delete a value that had just been refreshed; the delete now only fires when the key still carries the same expired generation it observed.
• Marginally increase the performances of requests processing
• Marginally improve the performances of PoW validation
• Marginally improve the performances of challenges generation/display
• Significantly improve the performances of the gzip middleware
• Significantly improve the performances of the PoW validation
• Add trimpath option to artifact builds
• Add COOKIE_HTTP_ONLY option to set the HttpOnly flag on Anubis cookies
• Improve the performances of rules validation
• Only compute the JA4H fingerprint when a policy references the X-Http-Fingerprint-JA4H header, taking it off the hot path for configurations that don't use it (#834).
• Migrate the target reverse proxy off the deprecated httputil.ReverseProxy.Director to Rewrite for Go 1.26 compatibility, preserving the inbound Host and X-Forwarded-*/Forwarded headers.

What's Changed

• docs(admin/policy): document ReadWritePaths for logging to files by @Xe in #1469
• docs: fix mixed tab/space indentation in Caddy config example by @pzhlkj6612 in #1506
• ci: purge govulncheck, it's less signal than i hoped by @Xe in #1515
• feat(http): set "Cache-Control: no-store" on error responses by @gucci-on-fleek in #1474
• docs(known-instances): Add missing one. by @p0008874 in #1500
• chore(l10n): update French translation by @HarmfulBreeze in #1496
• docs: remove developer/code-quality by @gucci-on-fleek in #1475
• build(deps): bump the github-actions group across 1 directory with 11 updates by @dependabot[bot] in #1516
• build(deps): bump the npm group across 1 directory with 6 updates by @dependabot[bot] in #1512
• chore: gofix by @JasonLovesDoggo in #1466
• fix: nil deref when solve request lands on a different DIFFICULTY than the original req by @JasonLovesDoggo in #1467
• feat(docs): Add instructions on using Anubis with envoy-gateway by @fhoekstra in #1460
• feat(docs): Update HAProxy Advanced Variant documentation by @rozmansi in #1521
• feat(data): add Citoid to good bots list by @mvolz in #1524
• docs(faq): document that disabling JIT makes Anubis slow by @Xe in #1526
• fix(locales/ja): change the position of the バージョン (version) by @fu-sen in #1527
• fix: add cel iterator by @JasonLovesDoggo in #1465
• fix: Address issued raised in #1193 by @dwhitemv25 in #1519
• feat(anubis): enable pprof endpoints on the metrics server by @Xe in #1530
• feat(i18n): add Bulgarian language translation by @ramromcode in #1394
• chore: update vi.json by @honhatduy in #1459
• fix(locales/de): revise and update German translations by @michi-onl in #1457
• build(deps): bump actions-hub/kubectl from 1.35.2 to 1.35.3 in the github-actions group by @dependabot[bot] in #1532
• chore(sponsors): add embeDD GmbH by @Xe in #1539
• build(deps): bump the npm group with 5 updates by @dependabot[bot] in #1531
• docs(faq): document minimum versions of browsers Anubis supports by @Xe in #1540
• build(deps): bump the npm group across 1 directory with 5 updates by @dependabot[bot] in #1559
• build(deps): bump the github-actions group across 1 directory with 7 updates by @dependabot[bot] in #1560
• feat: move metrics server config to the policy file by @Xe in #1572
• feat(data): add Meta's web indexer used for AI purposes by @bnjbvr in #1573
• feat(metrics): enable TLS/mTLS serving support by @Xe in #1576
• feat(metrics): basic auth support by @Xe in #1579
• fix(honeypot): remove DoS vector by @Xe in #1581
• fix: Thoth geoip compare by @lenny87 in #1564
• fix(metrics): bind to the right network/bindhost by @Xe in #1606
• feat: add ASN data from Thoth to logs/metrics by @lillian-b in #1608
• feat: Log weight when issuing challenge by @tdgroot in #1611
• fix: patch GHSA-6wcg-mqvh-fcvg by @Xe in #1616
• build(deps-dev): bump the npm group across 1 directory with 6 updates by @dependabot[bot] in #1621
• build(deps): bump github.com/go-jose/go-jose/v3 from 3.0.4 to 3.0.5 by @dependabot[bot] in #1629
• fix(policy): correctly wire subrequest mode through CEL/path checkers by @Xe in #1630
• fix(honeypot/naive): apply robot9001 style delays by @Xe in #1632
• build(deps): bump the npm group across 1 directory with 6 updates by @dependabot[bot] in #1646
• perf(lib): iterate s.policy.Bots by index to drop per-call heap copy by @jvoisin in #1639
• fix: small security fixes by @Xe in #1651
• perf(internal/gzip): pool *gzip.Writer per middleware instance by @jvoisin in #1654
• perf: enable uuid randomness pool and minor cleanups by @jvoisin in #1652
• perf(challenge/proofofwork): stream sha256 into stack buffer in Validate by @jvoisin in #1653
• chore: use Go stdlib version stamping by @Xe in #1665
• chore: ban x.ai by @Xe in #1673
• feat: add HttpOnly cookie option by @tdgroot in #1679
• fix: add trimpath option to artifact builds by @tdgroot in #1682
• ci: remove spelling checks by @Xe in #1688
• perf(policy): only compute ja4h fingerprint when referenced by @jvoisin in #1689
• perf: don't look into the store for OpenGraph things if it is disabled by @jvoisin in #1690
• refactor(honeypot/naive): compute network hash once in incrementNetwork by @jvoisin in #1693
• perf: iterate s.policy.Bots by index to drop per-call heap copy by @jvoisin in #1691
• perf(internal/gzip): share writer pool across middleware reconstructions by @jvoisin in #1694
• perf(internal): cut allocations in computeXFFHeader by @jvoisin in #1692
• build(deps-dev): bump the npm group across 1 directory with 4 updates by @dependabot[bot] in #1677
• docs: use stable container in traefik deployment example by @RealOrangeOne in #1676
• chore: bump go version to 1.26 by @Xe in #1695
• chore: tag v1.26.0-pre1 by @Xe in #1705

New Contributors

• @pzhlkj6612 made their first contribution in #1506
• @HarmfulBreeze made their first contribution in #1496
• @fhoekstra made their first contribution in #1460
• @rozmansi made their first contribution in #1521
• @mvolz made their first contribution in #1524
• @fu-sen made their first contribution in #1527
• @dwhitemv25 made their first contribution in #1519
• @ramromcode made their first contribution in #1394
• @honhatduy made their first contribution in #1459
• @bnjbvr made their first contribution in #1573
• @lenny87 made their first contribution in #1564
• @jvoisin made their first contribution in #1639
• @RealOrangeOne made their first contribution in #1676

Full Changelog: v1.25.0...v1.26.0-pre1

Hey all,

I'm sure you've all been aware that things have been slowing down a little with Anubis development, and I want to apologize for that. A lot has been going on in my life lately (my blog will have a post out on Friday with more information), and as a result I haven't really had the energy to work on Anubis in publicly visible ways. There are things going on behind the scenes, but nothing is really shippable yet, sorry!

I've also been feeling some burnout in the wake of perennial waves of anger directed towards me. I'm handling it, I'll be fine, I've just had a lot going on in my life and it's been rough.

I've been missing the sense of wanderlust and discovery that comes with the artistic way I playfully develop software. I suspect that some of the stresses I've been through (setting up a complicated surgery in a country whose language you aren't fluent in is kind of an experience) have been sapping my energy. I'd gonna try to mess with things on my break, but realistically I'm probably just gonna be either watching Stargate SG-1 or doing unreasonable amounts of ocean fishing in Final Fantasy 14. Normally I'd love to keep the details about my medical state fairly private, but I'm more of a public figure now than I was this time last year so I don't really get the invisibility I'm used to for this.

I've also had a fair amount of negativity directed at me for simply being much more visible than the anonymous threat actors running the scrapers that are ruining everything, which though understandable has not helped.

Anyways, it all worked out and I'm about to be in the hospital for a week, so if things go really badly with this release please downgrade to the last version and/or upgrade to the main branch when the fix PR is inevitably merged. I hoped to have time to tame GPG and set up full release automation in the Anubis repo, but that didn't work out this time and that's okay.

If I can challenge you all to do something, go out there and try to actually create something new somehow. Combine ideas you've never mixed before. Be creative, be human, make something purely for yourself to scratch an itch that you've always had yet never gotten around to actually mending.

At the very least, try to be an example of how you want other people to act, even when you're in a situation where software written by someone else is configured to require a user agent to execute javascript to access a webpage.

Be well,

Xe

PS: if you're well-versed in FFXIV lore, the release title should give you an idea of the kind of stuff I've been going through mentally.

• Add iplist2rule tool that lets admins turn an IP address blocklist into an Anubis ruleset.
• Add Polish locale (#1292)
• Fix honeypot and imprint links missing BASE_PREFIX when deployed behind a path prefix (#1402)
• Add ANEXIA Sponsor logo to docs (#1409)
• Improve idle performance in memory storage
• Add HAProxy Configurations to Docs (#1424)