NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
Packagist · #704 most downloaded on Packagist
Determine if a page may be crawled from robots.txt and robots meta tags
Last release 7 months ago
25 Feb 2026
Release timing varies
gaps range from 4 weeks to 1.5 years
Most releases are documented
notes for 26 of 31 stable releases
Nothing withdrawn
no release was ever pulled
9 years old
31 releases · first in 2018
Send user agent header with file_get_contents requests to prevent Cloudflare and similar WAFs from blocking requests
Send user agent header with file_get_contents requests to prevent Cloudflare and similar WAFs from blocking requests
Bump stefanzweifel/git-auto-commit-action from 6 to 7 by @dependabot [bot] in #78
Full Changelog: 2.5.2...2.5.3
One column per quarter.
Full Changelog: https://github.com/spatie/robots-txt/compare/2.5.2...2.5.3
Bump aglipanci/laravel-pint-action from 2.5 to 2.6 by @dependabot [bot] in #73
Full Changelog: 2.5.1...2.5.2
Full Changelog: https://github.com/spatie/robots-txt/compare/2.5.1...2.5.2
Bump stefanzweifel/git-auto-commit-action from 5 to 6 by @dependabot in #68
Full Changelog: 2.5.0...2.5.1
Full Changelog: https://github.com/spatie/robots-txt/compare/2.5.0...2.5.1
Implement RobotsTxt->whyDisallows($path, $userAgent) to find why a path was disallowed by @gvlasov in #66
Full Changelog: 2.4.0...2.5.0
Full Changelog: https://github.com/spatie/robots-txt/compare/2.4.0...2.5.0
Bump aglipanci/laravel-pint-action from 2.4 to 2.5 by @dependabot in #63
Full Changelog: 2.3.1...2.4.0
Full Changelog: https://github.com/spatie/robots-txt/compare/2.3.1...2.4.0
Bump dependabot/fetch-metadata from 2.2.0 to 2.3.0 by @dependabot in #62
Full Changelog: 2.3.0...2.3.1
Full Changelog: https://github.com/spatie/robots-txt/compare/2.3.0...2.3.1
Update phpunit/phpunit requirement from ^9.0|^10.0 to ^11.5.2 by @dependabot in #59
Full Changelog: 2.2.3...2.3.0
Full Changelog: https://github.com/spatie/robots-txt/compare/2.2.3...2.3.0
avoid expensive queries that do not change certain results by @elcapo in #53
Full Changelog: https://github.com/spatie/robots-txt/compare/2.2.2...2.2.3
Bump actions/checkout from 2 to 4 by @dependabot in #47
Full Changelog: 2.2.1...2.2.2
Full Changelog: https://github.com/spatie/robots-txt/compare/2.2.1...2.2.2
Add missing use statement for InvalidArgumentException by @remcom in https://github.com/spatie/robots-txt/pull/44
Full Changelog: https://github.com/spatie/robots-txt/compare/2.2.0...2.2.1
optionally allow partial matches and global groups by @resohead in https://github.com/spatie/robots-txt/pull/43
Full Changelog: https://github.com/spatie/robots-txt/compare/2.1.0...2.2.0
Add some efficiencies to prevent unnecessary requests by @tsjason in https://github.com/spatie/robots-txt/pull/42
Full Changelog: https://github.com/spatie/robots-txt/compare/2.0.3...2.1.0
Fix deprecation message by @buismaarten in https://github.com/spatie/robots-txt/pull/41
Full Changelog: https://github.com/spatie/robots-txt/compare/2.0.2...2.0.3
Nothing published for this version
- added x-robots-tag: none
use php-cs-fixer & github workflow
- handle multiple user-agent
add support for PHP 8.0 + move to GitHub actions
fix find robots meta tag line if minified code
fix headers checking (nofollow, noindex) for custom userAgent
improvements around handling of wildcards, end-of-string, query string
- improve readability
fix parsing robotstxt urls with keywords
make robots.txt check case insensitive
prevent exception if the domain has no robots.txt
- initial release
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Your coding agent can read these notes before it upgrades. Set up the MCP server →