crawlee

🚀 Features

• Add URL validation (#343) (1514538) by @vdusek, closes #300

🐛 Bug Fixes

• Minor log fix (#341) (0688bf1) by @souravjain540
• Also use error_handler for context pipeline errors (#331) (7a66445) by @janbuchar, closes #296
• Strip whitespace from href in enqueue_links (#346) (8a3174a) by @janbuchar, closes #337
• Warn instead of crashing when an empty dataset is being exported (#342) (22b95d1) by @janbuchar, closes #334
• Avoid Github rate limiting in project bootstrapping test (#364) (992f07f) by @janbuchar
• Pass crawler configuration to storages (#375) (b2d3a52) by @janbuchar
• Purge request queue on repeated crawler runs (#377) (7ad3d69) by @janbuchar, closes #152

🚀 Features

• Expose crawler log (#316) (ae475fa) by @vdusek, closes #303
• Integrate proxies into PlaywrightCrawler (#325) (2e072b6) by @vdusek
• Blocking detection for playwright crawler (#328) (49ff6e2) by @vdusek, closes #239

🐛 Bug Fixes

• Pylance reportPrivateImportUsage errors (#313) (09d7203) by @vdusek, closes #283
• Set httpx logging to warning (#314) (1585def) by @vdusek, closes #302
• Byte size serialization in MemoryInfo (#245) (a030174) by @janbuchar
• Project bootstrapping in existing folder (#318) (c630818) by @janbuchar, closes #301

🚀 Features

• Project templates (#237) (c23c12c) by @janbuchar, closes #215

🐛 Bug Fixes

• CLI UX improvements (#271) (123d515) by @janbuchar, closes #267
• Error handling in CLI and templates documentation (#273) (61083c3) by @vdusek, closes #268

🐛 Bug Fixes

• Do not wait for consistency in request queue (#235) (03ff138) by @vdusek
• Selector handling in BeautifulSoupCrawler enqueue_links (#231) (896501e) by @janbuchar, closes #230
• Handle blocked request (#234) (f8ef79f) by @Mantisus
• Improve AutoscaledPool state management (#241) (fdea3d1) by @janbuchar, closes #236

🚀 Features

• Maintain a global configuration instance (#207) (e003aa6) by @janbuchar
• Add max requests per crawl to BasicCrawler (#198) (b5b3053) by @vdusek
• Add support decompress br response content (#226) (a3547b9) by @Mantisus
• BasicCrawler.export_data helper (#222) (237ec78) by @janbuchar, closes #211
• Automatic logging setup (#229) (a67b72f) by @janbuchar, closes #214

🐛 Bug Fixes

• Handling of relative URLs in add_requests (#213) (8aa8c57) by @janbuchar, closes #202, #204
• Graceful exit in BasicCrawler.run (#224) (337286e) by @janbuchar, closes #212

🚀 Features

• Browser rotation and better browser abstraction (#177) (a42ae6f) by @vdusek, closes #131
• Add emit persist state event to event manager (#181) (97f6c68) by @vdusek
• Batched request addition in RequestQueue (#186) (f48c806) by @vdusek
• Add storage helpers to crawler & context (#192) (f8f4066) by @vdusek, closes #98, #100, #172
• Handle all supported configuration options (#199) (23c901c) by @janbuchar, closes #84
• Add Playwright's enqueue links helper (#196) (849d73c) by @vdusek

🐛 Bug Fixes

• Tmp path in tests is working (#164) (382b6f4) by @vdusek, closes #159
• Add explicit err msgs for missing pckg extras during import (#165) (200ebfa) by @vdusek, closes #155
• Make timedelta_ms accept string-encoded numbers (#190) (d8426ff) by @janbuchar
• deps: Update dependency psutil to v6 (#193) (eb91f51) by @renovate[bot]
• Improve compatibility between ProxyConfiguration and its SDK counterpart (#201) (1a76124) by @janbuchar
• Correct return type of storage get_info methods (#200) (332673c) by @janbuchar
• Type error in statistics persist state (#206) (96ceef6) by @vdusek, closes #194

🚀 Features

• Capture statistics about the crawler run (#142) (eeebe9b) by @janbuchar, closes #97
• Proxy configuration (#156) (5c3753a) by @janbuchar, closes #136
• Add first version of browser pool and playwright crawler (#161) (2d2a050) by @vdusek

🚀 Features

• AutoscaledPool implementation (#55) (621ada2) by @janbuchar, closes #19
• Add Snapshotter (#20) (492ee38) by @vdusek
• Implement BasicCrawler (#56) (6da971f) by @janbuchar, closes #30
• BeautifulSoupCrawler (#107) (4974dfa) by @janbuchar, closes #31
• Add_requests and enqueue_links context helpers (#120) (dc850a5) by @janbuchar, closes #5
• Use SessionPool in BasicCrawler (#128) (9fc4648) by @janbuchar, closes #110
• Add base storage client and resource subclients (#138) (44d6597) by @vdusek

🐛 Bug Fixes

• deps: Update dependency docutils to ^0.21.0 (#101) (534b613) by @renovate[bot]
• deps: Update dependency eval-type-backport to ^0.2.0 (#124) (c9e69a8) by @renovate[bot]
• Fire local SystemInfo events every second (#144) (f1359fa) by @vdusek
• Storage manager & purging the defaults (#150) (851042f) by @vdusek

<!-- generated by git-cliff -->

🚀 Features ​

• AutoscaledPool implementation ( #55 ) ( 621ada2 ) by @janbuchar , closes #19

• Add Snapshotter ( #20 ) ( 492ee38 ) by @vdusek

• Implement BasicCrawler ( #56 ) ( 6da971f ) by @janbuchar , closes #30

• BeautifulSoupCrawler ( #107 ) ( 4974dfa ) by @janbuchar , closes #31

• Add_requests and enqueue_links context helpers ( #120 ) ( dc850a5 ) by @janbuchar , closes #5

• Use SessionPool in BasicCrawler ( #128 ) ( 9fc4648 ) by @janbuchar , closes #110

• Add base storage client and resource subclients ( #138 ) ( 44d6597 ) by @vdusek

🐛 Bug Fixes ​

• deps: Update dependency docutils to ^0.21.0 ( #101 ) ( 534b613 ) by @renovate[bot]

• deps: Update dependency eval-type-backport to ^0.2.0 ( #124 ) ( c9e69a8 ) by @renovate[bot]

• Fire local SystemInfo events every second ( #144 ) ( f1359fa ) by @vdusek

• Storage manager & purging the defaults ( #150 ) ( 851042f ) by @vdusek

• 1.10.4 - not yet released

• 🐛 Bug Fixes

• 1.10.3 (2026-09-29)

• 🐛 Bug Fixes

• 1.10.2 (2026-09-22)

• 🐛 Bug Fixes

• 1.10.1 (2026-09-16)

• 🐛 Bug Fixes

• 1.10.0 (2026-08-31)

• 🚀 Features

• 🐛 Bug Fixes

• 1.9.3 (2026-08-24)

• 🐛 Bug Fixes

• 1.9.2 (2026-08-17)

• 🐛 Bug Fixes

• ⚡ Performance

• 1.9.1 (2026-08-06)

• 🐛 Bug Fixes

• 1.9.0 (2026-08-03)

• 🚀 Features

• 🐛 Bug Fixes

• 1.8.3 (2026-07-20)

• 🐛 Bug Fixes

• ⚡ Performance

• 1.8.2 (2026-07-13)

• 🐛 Bug Fixes

• 1.8.1 (2026-07-08)

• 🐛 Bug Fixes

• 1.8.0 (2026-07-02)

• 🚀 Features

• 🐛 Bug Fixes

• 1.7.3 (2026-06-22)

• 🐛 Bug Fixes

• 1.7.2 (2026-06-04)

• 🐛 Bug Fixes

• 1.7.1 (2026-05-26)

• 🐛 Bug Fixes

• 1.7.0 (2026-05-12)

• 🚀 Features

• 🐛 Bug Fixes

• 1.6.3 (2026-04-27)

• 🐛 Bug Fixes

• 1.6.2 (2026-04-08)

• 🐛 Bug Fixes

• 1.6.1 (2026-03-30)

• 🐛 Bug Fixes

• ⚡ Performance

• 1.6.0 (2026-03-20)

• 🚀 Features

• 🐛 Bug Fixes

• 1.5.0 (2026-03-06)

• 🚀 Features

• 🐛 Bug Fixes

• 1.4.0 (2026-02-17)

• 🚀 Features

• 🐛 Bug Fixes

• ⚡ Performance

• 1.3.2 (2026-02-09)

• 🐛 Bug Fixes

• 1.3.1 (2026-01-30)

• 🐛 Bug Fixes

• 1.3.0 (2026-01-20)

• 🚀 Features

• 🐛 Bug Fixes

• 1.2.1 (2025-12-16)

• 🐛 Bug Fixes

• 1.2.0 (2025-12-08)

• 🚀 Features

• 🐛 Bug Fixes

• 1.1.1 (2025-12-02)

• 🐛 Bug Fixes

• 1.1.0 (2025-11-18)

• 🚀 Features

• 🐛 Bug Fixes

• 1.0.4 (2025-10-24)

• 🐛 Bug Fixes

• 1.0.3 (2025-10-17)

• 🐛 Bug Fixes

• 1.0.2 (2025-10-08)

• 🐛 Bug Fixes

• 1.0.1 (2025-10-06)

• 🐛 Bug Fixes

• 1.0.0 (2025-09-29)

• 🚀 Features

• 🐛 Bug Fixes

• 🚜 Refactor

• 0.6.12 (2025-07-30)

• 🚀 Features

• 🐛 Bug Fixes

• 0.6.11 (2025-06-23)

• 🚀 Features

• 🐛 Bug Fixes

• 0.6.10 (2025-06-02)

• 🐛 Bug Fixes

• 0.6.9 (2025-05-02)

• 🚀 Features

• 🐛 Bug Fixes

• 0.6.8 (2025-04-25)

• 🚀 Features

• 🐛 Bug Fixes

• 0.6.7 (2025-04-17)

• 🚀 Features

• 🐛 Bug Fixes

• 0.6.6 (2025-04-03)

• 🚀 Features

• 🐛 Bug Fixes

• 0.6.5 (2025-03-13)

• 🐛 Bug Fixes

• 0.6.4 (2025-03-12)

• 🐛 Bug Fixes

• 0.6.3 (2025-03-07)

• 🚀 Features

• 🐛 Bug Fixes

• 0.6.2 (2025-03-05)

• 🚀 Features

• 0.6.1 (2025-03-03)

• 🐛 Bug Fixes

• 0.6.0 (2025-03-03)

• 🚀 Features

• 🐛 Bug Fixes

• 🚜 Refactor

• 0.5.4 (2025-02-05)

• 🚀 Features

• 🐛 Bug Fixes

• 0.5.3 (2025-01-31)

• 🚀 Features

• 🐛 Bug Fixes

• 0.5.2 (2025-01-17)

• 🐛 Bug Fixes

• 0.5.1 (2025-01-07)

• 🐛 Bug Fixes

• 0.5.0 (2025-01-02)

• 🚀 Features

• 🐛 Bug Fixes

• 🚜 Refactor

• 0.4.5 (2024-12-06)

• 🚀 Features

• 🐛 Bug Fixes

• 0.4.4 (2024-11-29)

• 🚀 Features

• 🐛 Bug Fixes

• 0.4.3 (2024-11-21)

• 🐛 Bug Fixes

• 0.4.2 (2024-11-20)

• 🐛 Bug Fixes

• 0.4.1 (2024-11-11)

• 🚀 Features

• 🐛 Bug Fixes

• 0.4.0 (2024-11-01)

• 🚀 Features

• 🐛 Bug Fixes

• ⚙️ Miscellaneous Tasks

• 0.3.9 (2024-10-23)

• 🚀 Features

• 🐛 Bug Fixes

• 0.3.8 (2024-10-02)

• 🚀 Features

• 🐛 Bug Fixes

• 0.3.7 (2024-09-25)

• 🐛 Bug Fixes

• 0.3.6 (2024-09-19)

• 🚀 Features

• 🐛 Bug Fixes

• 0.3.5 (2024-09-10)

• 🚀 Features

• 🐛 Bug Fixes

• 0.3.4 (2024-09-05)

• 🐛 Bug Fixes

• 0.3.3 (2024-09-05)

• 🐛 Bug Fixes

• 0.3.2 (2024-09-02)

• 🐛 Bug Fixes

• 0.3.1 (2024-08-30)

• 🚀 Features

• 0.3.0 (2024-08-27)

• 🚀 Features

• 🐛 Bug Fixes

• 🚜 Refactor

• 0.2.1 (2024-08-05)

• 🐛 Bug Fixes

• 0.2.0 (2024-08-05)

• 🚀 Features

• 0.1.2 (2024-07-30)

• 🚀 Features

• 🐛 Bug Fixes

• 0.1.1 (2024-07-19)

• 🚀 Features

• 🐛 Bug Fixes

• 0.1.0 (2024-07-08)

• 🚀 Features

• 🐛 Bug Fixes

• 0.0.7 (2024-06-27)

• 🐛 Bug Fixes

• 0.0.6 (2024-06-25)

• 🚀 Features

• 🐛 Bug Fixes

• 0.0.5 (2024-06-21)

• 🚀 Features

• 🐛 Bug Fixes

• 0.0.4 (2024-05-30)

• 🚀 Features

• 0.0.3 (2024-05-13)

• 🚀 Features

• 🐛 Bug Fixes