NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI, #3874 most downloaded on PyPI
Crawlee for Python
Latest stable
1.11.2
Last release 3 days ago
17 Nov 2026
Ships on a steady schedule
a new release about every 1 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
1 version withdrawn
withdrawn after publishing
3 years old
215 releases · first in 2024
Drag across the ruler to compare, or tap a release to open it.
Check out the Release blog post for more details.
AdaptivePlaywrightCrawler (#872) (5ba70b6) by @Pijukatel_snapshot_client for Snapshotter (#957) (ba4d384) by @MantisusPlaywrightCrawler (#959) (87cf446) by @PijukatelHeaderGenerator implementation by browserforge implementation (#960) (c2f8c93) by @Pijukatelallow_redirects override in CurlImpersonateHttpClient (#1017) (01d855a) by @2tunnelsfollow_redirects override in HttpxHttpClient (#1015) (88afda3) by @2tunnelstest_common_headers_and_user_agent (#1030) (58aa70e) by @Pijukatelincognito context to persistent context for Playwright (#985) (f01520d) by @MantisusSession cookies from dict to SessionCookies with CookieJar (#984) (6523b3a) by @MantisusEnqueueStrategy (#1019) (d2481ef) by @vdusekcli dependencies to optional dependencies (#1011) (4382959) by @MantisusYour coding agent can read these notes before it upgrades. Set up the MCP server →
AdaptivePlaywrightCrawler (#872) (5ba70b6) by @Pijukatel_snapshot_client for Snapshotter (#957) (ba4d384) by @Mantisus, closes #60PlaywrightCrawler (#959) (87cf446) by @Pijukatel, closes #953HeaderGenerator implementation by browserforge implementation (#960) (c2f8c93) by @Pijukatel, closes #937allow_redirects override in CurlImpersonateHttpClient (#1017) (01d855a) by @2tunnels, closes #1016follow_redirects override in HttpxHttpClient (#1015) (88afda3) by @2tunnels, closes #1013test_common_headers_and_user_agent (#1030) (58aa70e) by @Pijukatel, closes #1027incognito context to persistent context for Playwright (#985) (f01520d) by @Mantisus, closes #721, #963Session cookies from dict to SessionCookies with CookieJar (#984) (6523b3a) by @Mantisus, closes #710, #933EnqueueStrategy (#1019) (d2481ef) by @vdusekcli dependencies to optional dependencies (#1011) (4382959) by @Mantisus, closes #703, #1010Add support use_incognito_pages for browser_launch_options in PlaywrightCrawler (#941) (eae3a33) by @Mantisus
use_incognito_pages for browser_launch_options in PlaywrightCrawler (#941) (eae3a33) by @Mantisusenqueue_links for response with redirect (#956) (694102e) by @MantisusCurlImpersonateHttpClient cookies handler (#946) (ed415c4) by @Mantisususe_incognito_pages for browser_launch_options in PlaywrightCrawler (#941) (eae3a33) by @Mantisusenqueue_links for response with redirect (#956) (694102e) by @Mantisus, closes #955CurlImpersonateHttpClient cookies handler (#946) (ed415c4) by @MantisusAdd keep_alive flag to crawler.__init__ (#921) (7a82d0c) by @Pijukatel
keep_alive flag to crawler.__init__ (#921) (7a82d0c) by @Pijukatelblock_requests helper for PlaywrightCrawler (#919) (1030459) by @Mantisustransform_request_function for enqueue_links (#923) (6b15957) by @Mantisustime_remaining_secs property to MIGRATING event data (#940) (b44501b) by @fnesvedaLogisticalRegressionPredictor - rendering type predictor for adaptive crawling (#930) (8440499) by @Pijukatelkeep_alive flag to crawler.__init__ (#921) (7a82d0c) by @Pijukatel, closes #891block_requests helper for PlaywrightCrawler (#919) (1030459) by @Mantisus, closes #848transform_request_function for enqueue_links (#923) (6b15957) by @Mantisus, closes #894time_remaining_secs property to MIGRATING event data (#940) (b44501b) by @fnesvedaLogisticalRegressionPredictor - rendering type predictor for adaptive crawling (#930) (8440499) by @PijukatelHttpxHttpClient, fix proxy handling (#905) (d7ad480) by @Mantisus, closes #895BrowserPool and PlaywrightBrowserPlugin closure (#932) (997543d) by @MantisusAvoid use_state race conditions. Remove key argument to use_state (#868) (000b976) by @Pijukatel
Make result of RequestList.is_empty independent of fetch_next_request calls (#876) (d50249e) by @janbuchar
Check out the Release blog post for more details.
None as no proxy in tiered proxies (#760) (0fbd017) by @Pijukateluse_state context method (#682) (868b41e) by @MantisusAbstractHttpCrawler (#791) (0f23205) by @PijukatelPlaywrightCrawler (#789) (246cfc4) by @Pijukatelstop method to BasicCrawler (#807) (6d01af4) by @Pijukatelhtml_to_text helper function (#792) (2b9d970) by @PijukatelRequestManagerTandem, remove add_request from RequestList, accept any iterable in RequestList constructor (#777) (4172652) by @janbucharKeyValueStore (#805) (8bdf49d) by @Mantisusservice_locator (#691) (1d31c6c) by @vdusekverify in httpx client (#802) (074d083) by @Mantisuspage_options for PlaywrightBrowserPlugin (#796) (bd3bdd4) by @MantisusRequestQueue (#825) (fd6663f) by @Mantisusabort-on-error for successive runs (#834) (0cea673) by @MantisusServiceLocator restrictions (#837) (aa3667f) by @janbucharHttpCrawler, BeautifulSoupCrawler, ParselCrawler inheritance (#746) (9d3c269) by @Pijukateljson_ and order_no from Request (#788) (5381d13) by @MantisusPwPreNavContext to PwPreNavCrawlingContext (#827) (84b61a3) by @vdusekPlaywrightCrawler kwargs: browser_options, page_options (#831) (ffc6048) by @PijukatelNone as no proxy in tiered proxies (#760) (0fbd017) by @Pijukatel, closes #687use_state context method (#682) (868b41e) by @Mantisus, closes #191AbstractHttpCrawler (#791) (0f23205) by @Pijukatel, closes #635PlaywrightCrawler (#789) (246cfc4) by @Pijukatel, closes #684stop method to BasicCrawler (#807) (6d01af4) by @Pijukatel, closes #651html_to_text helper function (#792) (2b9d970) by @Pijukatel, closes #659RequestManagerTandem, remove add_request from RequestList, accept any iterable in RequestList constructor (#777) (4172652) by @janbucharKeyValueStore (#805) (8bdf49d) by @Mantisus, closes #804service_locator (#691) (1d31c6c) by @vdusek, closes #369, #539, #699verify in httpx client (#802) (074d083) by @Mantisus, closes #798page_options for PlaywrightBrowserPlugin (#796) (bd3bdd4) by @Mantisus, closes #755RequestQueue (#825) (fd6663f) by @Mantisus, closes #815abort-on-error for successive runs (#834) (0cea673) by @MantisusServiceLocator restrictions (#837) (aa3667f) by @janbuchar, closes #806HttpCrawler, BeautifulSoupCrawler, ParselCrawler inheritance (#746) (9d3c269) by @Pijukatel, closes #350json_ and order_no from Request (#788) (5381d13) by @Mantisus, closes #94PwPreNavContext to PwPreNavCrawlingContext (#827) (84b61a3) by @vdusekPlaywrightCrawler kwargs: browser_options, page_options (#831) (ffc6048) by @PijukatelImprove project bootstrapping (#538) (367899c) by @janbuchar
<2.10.0 and update timedelta validator, serializer type hints (#757) (c0050c0) by @PijukatelExpose browser_options and page_options to PlaywrightCrawler (#730) (dbe85b9) by @vdusek
Pydantic 2.10.0 issues (#716) (8d8b3fc) by @Pijukatel
Respect custom HTTP headers in PlaywrightCrawler (#685) (a84125f) by @Mantisus
PlaywrightCrawler (#685) (a84125f) by @MantisusPlaywrightCrawler (#685) (a84125f) by @MantisusAdd max_crawl_depth option to BasicCrawler (#637) (77deaa9) by @Prathamesh010
max_crawl_depth option to BasicCrawler (#637) (77deaa9) by @Prathamesh010BeautifulSoupParser type alias (#674) (b2cf88f) by @Pijukateltotal_size usage in memory size monitoring (#661) (c2a3239) by @janbucharHttpHeaders to module exports (#664) (f0c5ca7) by @vdusekValueError in request handler result processing (#666) (0a99d7f) by @janbucharBaseDatasetClient.iter_items type hints (#680) (a968b1b) by @Pijukatelmax_crawl_depth option to BasicCrawler (#637) (77deaa9) by @Prathamesh010, closes #460BeautifulSoupParser type alias (#674) (b2cf88f) by @Pijukateltotal_size usage in memory size monitoring (#661) (c2a3239) by @janbucharHttpHeaders to module exports (#664) (f0c5ca7) by @vdusek, closes #663ValueError in request handler result processing (#666) (0a99d7f) by @janbucharBaseDatasetClient.iter_items type hints (#680) (a968b1b) by @PijukatelCheck out the Upgrading guide to ensure a smooth update.
pre_navigation_hooks to PlaywrightCrawler (#631) (5dd5b60) by @Prathamesh010always_enqueue option to bypass URL deduplication (#621) (4e59fa4) by @Rutam21export_data method (#580) (6751635) by @deshanshProxyInfo port if httpx.URL port is None (#619) (8107a6f) by @steffansafeypre_navigation_hooks to PlaywrightCrawler (#631) (5dd5b60) by @Prathamesh010, closes #427always_enqueue option to bypass URL deduplication (#621) (4e59fa4) by @Rutam21, closes #547export_data method (#580) (6751635) by @deshansh, closes #526ProxyInfo port if httpx.URL port is None (#619) (8107a6f) by @steffansafey, closes #618Key-value store context helpers (#584) (fc15622) by @janbuchar
Improve Request.user_data serialization (#540) (de29c0e) by @janbuchar, closes #524
Add HTTP/2 support for HTTPX client (#513) (0eb0a33) by @vdusek, closes #512
Memory usage limit configuration via environment variables (#502) (c62e554) by @janbuchar
Expose basic crawling context (#501) (b484535) by @vdusek
Deduplicate requests by unique key before submitting them to the queue (#499) (6a3e0e7) by @janbuchar
Double incrementation of item_count (#443) (cd9adf1) by @cadlagtrader, closes #442
Curl http client selects chrome impersonation by default (#473) (82dc939) by @vdusek
Check out the Upgrading guide to ensure a smooth update.
ParselCrawler that adds support for Parsel (#348) (a3832e5) by @asymness, closes #335RequestQueue in the correct order (#411) (96fc33e) by @janbucharProlongRequestLockResponse (#458) (30ccc3a) by @janbuchar__all__ to top-level __init__.py file (#463) (353a1ce) by @janbucharDo not import curl impersonate in http clients init (#396) (3bb8009) by @vdusek
Add new curl impersonate HTTP client (#387) (9c06260) by @vdusek
Nothing published for this version
Add URL validation (#343) (1514538) by @vdusek
error_handler for context pipeline errors (#331) (7a66445) by @janbucharhref in enqueue_links (#346) (8a3174a) by @janbucharerror_handler for context pipeline errors (#331) (7a66445) by @janbuchar, closes #296href in enqueue_links (#346) (8a3174a) by @janbuchar, closes #337Expose crawler log (#316) (ae475fa) by @vdusek
PlaywrightCrawler (#325) (2e072b6) by @vdusekreportPrivateImportUsage errors (#313) (09d7203) by @vdusek, closes #283MemoryInfo (#245) (a030174) by @janbucharCheck out the Release blog post for more details.
Do not wait for consistency in request queue (#235) (03ff138) by @vdusek
BeautifulSoupCrawler enqueue_links (#231) (896501e) by @janbuchar, closes #230AutoscaledPool state management (#241) (fdea3d1) by @janbuchar, closes #236Maintain a global configuration instance (#207) (e003aa6) by @janbuchar
BasicCrawler (#198) (b5b3053) by @vdusekBasicCrawler.export_data helper, which replaces BasicCrawler.export_to (#222) (237ec78) by @janbucharBasicCrawler (#198) (b5b3053) by @vdusekBasicCrawler.export_data helper, which replaces BasicCrawler.export_to (#222) (237ec78) by @janbuchar, closes #211add_requests (#213) (8aa8c57) by @janbuchar, closes #202, #204BasicCrawler.run (#224) (337286e) by @janbuchar, closes #212Browser rotation and better browser abstraction (#177) (a42ae6f) by @vdusek
RequestQueue (#186) (f48c806) by @vdusektimedelta_ms accept string-encoded numbers (#190) (d8426ff) by @janbucharpsutil to v6 (#193) (eb91f51) by @renovateProxyConfiguration and its SDK counterpart (#201) (1a76124) by @janbucharget_info methods (#200) (332673c) by @janbucharRequestQueue (#186) (f48c806) by @vdusektimedelta_ms accept string-encoded numbers (#190) (d8426ff) by @janbucharpsutil to v6 (#193) (eb91f51) by @renovateProxyConfiguration and its SDK counterpart (#201) (1a76124) by @janbucharget_info methods (#200) (332673c) by @janbucharAnother internal release, adding statistics capturing, proxy configuration, and the initial version of browser management and PlaywrightCrawler.
PlaywrightCrawler.PlaywrightCrawler.Another internal release, adding mainly session management and BeautifulSoupCrawler.
BeautifulSoupCrawler.AutoscaledPool implementation (#55) (621ada2) by @janbucharSnapshotter (#20) (492ee38) by @vdusekBasicCrawler (#56) (6da971f) by @janbucharBeautifulSoupCrawler (#107) (4974dfa) by @janbucharadd_requests and enqueue_links context helpers (#120) (dc850a5) by @janbucharSessionPool in BasicCrawler (#128) (9fc4648) by @janbucharBeautifulSoupCrawler.AutoscaledPool implementation (#55) (621ada2) by @janbuchar, closes #19Snapshotter (#20) (492ee38) by @vdusekBasicCrawler (#56) (6da971f) by @janbuchar, closes #30BeautifulSoupCrawler (#107) (4974dfa) by @janbuchar, closes #31add_requests and enqueue_links context helpers (#120) (dc850a5) by @janbuchar, closes #5SessionPool in BasicCrawler (#128) (9fc4648) by @janbuchar, closes #110docutils to ^0.21.0 (#101) (534b613) by @renovateeval-type-backport to ^0.2.0 (#124) (c9e69a8) by @renovateSystemInfo events every second (#144) (f1359fa) by @vdusek<!-- generated by git-cliff -->
AutoscaledPool implementation ( #55 ) ( 621ada2 ) by @janbuchar , closes #19
Add Snapshotter ( #20 ) ( 492ee38 ) by @vdusek
Implement BasicCrawler ( #56 ) ( 6da971f ) by @janbuchar , closes #30
BeautifulSoupCrawler ( #107 ) ( 4974dfa ) by @janbuchar , closes #31
add_requests and enqueue_links context helpers ( #120 ) ( dc850a5 ) by @janbuchar , closes #5
Use SessionPool in BasicCrawler ( #128 ) ( 9fc4648 ) by @janbuchar , closes #110
Add base storage client and resource subclients ( #138 ) ( 44d6597 ) by @vdusek
deps: Update dependency docutils to ^0.21.0 ( #101 ) ( 534b613 ) by @renovate
deps: Update dependency eval-type-backport to ^0.2.0 ( #124 ) ( c9e69a8 ) by @renovate
Fire local SystemInfo events every second ( #144 ) ( f1359fa ) by @vdusek
Storage manager & purging the defaults ( #150 ) ( 851042f ) by @vdusek
1.11.3 - not yet released
🐛 Bug Fixes
1.11.2 (2026-11-10)
🐛 Bug Fixes
1.11.1 (2026-11-05)
🐛 Bug Fixes
⚡ Performance
1.11.0 (2026-10-15)
🚀 Features
🐛 Bug Fixes
⚡ Performance
1.10.4 (2026-10-06)
🐛 Bug Fixes
1.10.3 (2026-09-29)
🐛 Bug Fixes
1.10.2 (2026-09-22)
🐛 Bug Fixes
1.10.1 (2026-09-16)
🐛 Bug Fixes
1.10.0 (2026-08-31)
🚀 Features
🐛 Bug Fixes
1.9.3 (2026-08-24)
🐛 Bug Fixes
1.9.2 (2026-08-17)
🐛 Bug Fixes
⚡ Performance
1.9.1 (2026-08-06)
🐛 Bug Fixes
1.9.0 (2026-08-03)
🚀 Features
🐛 Bug Fixes
1.8.3 (2026-07-20)
🐛 Bug Fixes
⚡ Performance
1.8.2 (2026-07-13)
🐛 Bug Fixes
1.8.1 (2026-07-08)
🐛 Bug Fixes
1.8.0 (2026-07-02)
🚀 Features
🐛 Bug Fixes
1.7.3 (2026-06-22)
🐛 Bug Fixes
1.7.2 (2026-06-04)
🐛 Bug Fixes
1.7.1 (2026-05-26)
🐛 Bug Fixes
1.7.0 (2026-05-12)
🚀 Features
🐛 Bug Fixes
1.6.3 (2026-04-27)
🐛 Bug Fixes
1.6.2 (2026-04-08)
🐛 Bug Fixes
1.6.1 (2026-03-30)
🐛 Bug Fixes
⚡ Performance
1.6.0 (2026-03-20)
🚀 Features
🐛 Bug Fixes
1.5.0 (2026-03-06)
🚀 Features
🐛 Bug Fixes
1.4.0 (2026-02-17)
🚀 Features
🐛 Bug Fixes
⚡ Performance
1.3.2 (2026-02-09)
🐛 Bug Fixes
1.3.1 (2026-01-30)
🐛 Bug Fixes
1.3.0 (2026-01-20)
🚀 Features
🐛 Bug Fixes
1.2.1 (2025-12-16)
🐛 Bug Fixes
1.2.0 (2025-12-08)
🚀 Features
🐛 Bug Fixes
1.1.1 (2025-12-02)
🐛 Bug Fixes
1.1.0 (2025-11-18)
🚀 Features
🐛 Bug Fixes
1.0.4 (2025-10-24)
🐛 Bug Fixes
1.0.3 (2025-10-17)
🐛 Bug Fixes
1.0.2 (2025-10-08)
🐛 Bug Fixes
1.0.1 (2025-10-06)
🐛 Bug Fixes
1.0.0 (2025-09-29)
🚀 Features
🐛 Bug Fixes
🚜 Refactor
0.6.12 (2025-07-30)
🚀 Features
🐛 Bug Fixes
0.6.11 (2025-06-23)
🚀 Features
🐛 Bug Fixes
0.6.10 (2025-06-02)
🐛 Bug Fixes
0.6.9 (2025-05-02)
🚀 Features
🐛 Bug Fixes
0.6.8 (2025-04-25)
🚀 Features
🐛 Bug Fixes
0.6.7 (2025-04-17)
🚀 Features
🐛 Bug Fixes
0.6.6 (2025-04-03)
🚀 Features
🐛 Bug Fixes
0.6.5 (2025-03-13)
🐛 Bug Fixes
0.6.4 (2025-03-12)
🐛 Bug Fixes
0.6.3 (2025-03-07)
🚀 Features
🐛 Bug Fixes
0.6.2 (2025-03-05)
🚀 Features
0.6.1 (2025-03-03)
🐛 Bug Fixes
0.6.0 (2025-03-03)
🚀 Features
🐛 Bug Fixes
🚜 Refactor
0.5.4 (2025-02-05)
🚀 Features
🐛 Bug Fixes
0.5.3 (2025-01-31)
🚀 Features
🐛 Bug Fixes
0.5.2 (2025-01-17)
🐛 Bug Fixes
0.5.1 (2025-01-07)
🐛 Bug Fixes
0.5.0 (2025-01-02)
🚀 Features
🐛 Bug Fixes
🚜 Refactor
0.4.5 (2024-12-06)
🚀 Features
🐛 Bug Fixes
0.4.4 (2024-11-29)
🚀 Features
🐛 Bug Fixes
0.4.3 (2024-11-21)
🐛 Bug Fixes
0.4.2 (2024-11-20)
🐛 Bug Fixes
0.4.1 (2024-11-11)
🚀 Features
🐛 Bug Fixes
0.4.0 (2024-11-01)
🚀 Features
🐛 Bug Fixes
⚙️ Miscellaneous Tasks
0.3.9 (2024-10-23)
🚀 Features
🐛 Bug Fixes
0.3.8 (2024-10-02)
🚀 Features
🐛 Bug Fixes
0.3.7 (2024-09-25)
🐛 Bug Fixes
0.3.6 (2024-09-19)
🚀 Features
🐛 Bug Fixes
0.3.5 (2024-09-10)
🚀 Features
🐛 Bug Fixes
0.3.4 (2024-09-05)
🐛 Bug Fixes
0.3.3 (2024-09-05)
🐛 Bug Fixes
0.3.2 (2024-09-02)
🐛 Bug Fixes
0.3.1 (2024-08-30)
🚀 Features
0.3.0 (2024-08-27)
🚀 Features
🐛 Bug Fixes
🚜 Refactor
0.2.1 (2024-08-05)
🐛 Bug Fixes
0.2.0 (2024-08-05)
🚀 Features
0.1.2 (2024-07-30)
🚀 Features
🐛 Bug Fixes
0.1.1 (2024-07-19)
🚀 Features
🐛 Bug Fixes
0.1.0 (2024-07-08)
🚀 Features
🐛 Bug Fixes
0.0.7 (2024-06-27)
🐛 Bug Fixes
0.0.6 (2024-06-25)
🚀 Features
🐛 Bug Fixes
0.0.5 (2024-06-21)
🚀 Features
🐛 Bug Fixes
0.0.4 (2024-05-30)
🚀 Features
0.0.3 (2024-05-13)
🚀 Features
🐛 Bug Fixes
Nothing published for this version
Nothing published for this version
One column per month.