crawlee

1.6.0 (2026-03-20)

🚀 Features

• Allow non-href links extract & enqueue (#1781) (6db365d) by @kozlice
• Add post_navigation_hooks to crawlers (#1795) (38ceda6) by @Mantisus
• Add page lifecycle hooks to BrowserPool (#1791) (6f2ac13) by @Mantisus
• Expose BrowserType and CrawleePage (#1798) (b50b9f2) by @Mantisus
• Expose use_state in BasicCrawler (#1799) (d121873) by @Mantisus

🐛 Bug Fixes

• redis: Do not remove handled request data from request queue (#1787) (3008c61) by @kozlice
• redis: Update actual Request state in request queue Redis storage client (#1789) (787231c) by @Mantisus

1.5.0 (2026-03-06)

🚀 Features

• Use specialized Playwright docker images in templates (#1757) (747c0cf) by @Pijukatel
• Add discover_valid_sitemaps utility (#1777) (872447b) by @Mantisus

🐛 Bug Fixes

• Prevent list modification during iteration in BrowserPool (#1703) (70309d9) by @vdusek
• Fix max_requests_per_crawl excluding failed requests (#1766) (d6bb0b4) by @Pijukatel
• playwright: Dispose of APIResponse body for send_request (#1771) (29d301b) by @kozlice
• Return None from add_request when storage client fails to enqueue request (#1775) (944753a) by @Mantisus
• Re-use pre-existing browser context in PlaywrightBrowserController (#1778) (4487543) by @Pijukatel

1.4.0 (2026-02-17)

🚀 Features

• Dynamic memory snapshots (#1715) (568a7b1) by @Pijukatel
• Add MySQL and MariaDB support for SqlStorageClient (#1749) (202b500) by @Mantisus

🐛 Bug Fixes

• Make log levels consistent in ServiceLocator (#1746) (4163413) by @janbuchar
• Fix PlaywrightCrawler unintentionally setting the global configuration (#1747) (fa58438) by @Pijukatel
• Fix Snapshotter handling of out of order samples (#1735) (387c712) by @Pijukatel

⚡ Performance

• Optimize metadata records processing in SqlStorageClient (#1551) (df1347a) by @Mantisus

1.3.2 (2026-02-09)

🐛 Bug Fixes

• Use max() instead of min() for request_max_duration statistic (#1701) (85c4335) by @vdusek
• Prevent mutation of default URL patterns list in block_requests (#1702) (fcf9adb) by @vdusek
• Keep None values for user_data in Request (#1707) (3c575bc) by @Mantisus
• Respect max_open_pages_per_browser limit for PlaywrightBrowserController on concurrent new_page calls (#1712) (2e5534b) by @Mantisus

1.3.1 (2026-01-30)

🐛 Bug Fixes

• Reset all counter in metadata with purge for RequestQueue (#1686) (ee09260) by @Mantisus
• Set default http3=False for ImpitHttpClient (#1685) (3f390f6) by @Mantisus
• Prevent get_request from permanently blocking requests (#1684) (da416f9) by @Mirza-Samad-Ahmed-Baig
• Do not share state between different crawlers unless requested (#1669) (64c246b) by @Pijukatel

1.3.0 (2026-01-20)

🚀 Features

• Expose AdaptivePlaywrightCrawlerStatisticState for AdaptivePlaywrightCrawler (#1635) (1bb4bcb) by @Mantisus

🐛 Bug Fixes

• Prevent race condition in concurrent storage creation (#1626) (7f17a43) by @Mantisus
• Create correct statistics for AdaptivePlaywrightCrawler on initialization with a custom parser (#1637) (bff7260) by @Mantisus
• Fix adding extra link for EnqueueLinksFunction with limit (#1674) (71d7867) by @Mantisus

1.2.1 (2025-12-16)

🐛 Bug Fixes

• Fix short error summary (#1605) (b751208) by @Pijukatel
• Freeze core Request fields (#1603) (ae6d86b) by @Mantisus
• Respect enqueue_strategy after redirects in enqueue_links (#1607) (700df91) by @Mantisus
• Protect Request from partial mutations on request handler failure (#1585) (a69caf8) by @Mantisus

1.2.0 (2025-12-08)

🚀 Features

• Add additional kwargs to Crawler's export_data (#1597) (5977f37) by @vdusek
• Add goto_options for PlaywrightCrawler (#1599) (0b82f3b) by @Mantisus

🐛 Bug Fixes

• Only apply requestHandlerTimeout to request handler (#1474) (0dfb6c2) by @janbuchar
• Handle the case when error_handler returns Request (#1595) (8a961a2) by @Mantisus
• Align Request.state transitions with Request lifecycle (#1601) (383225f) by @Mantisus

1.1.1 (2025-12-02)

🐛 Bug Fixes

• Unify separators in unique_key construction (#1569) (af46a37) by @vdusek
• Fix same-domain strategy ignoring public suffix (#1572) (3d018b2) by @Pijukatel
• Make context helpers work in FailedRequestHandler and ErrorHandler (#1570) (b830019) by @Pijukatel
• Fix non-ASCII character corruption in FileSystemStorageClient on systems without UTF-8 default encoding (#1580) (f179f86) by @Mantisus
• Respect <base> when enqueuing (#1590) (de517a1) by @Mantisus

1.1.0 (2025-11-18)

🚀 Features

• Add chrome BrowserType for PlaywrightCrawler to use the Chrome browser (#1487) (b06937b) by @Mantisus
• Add RedisStorageClient based on Redis v8.0+ (#1406) (d08d13d) by @Mantisus
• Add support for Python 3.14 (#1553) (89e9130) by @Mantisus
• Add transform_request_function parameter for SitemapRequestLoader (#1525) (dc90127) by @Mantisus

🐛 Bug Fixes

• Improve indexing of the request_queue_records table for SqlRequestQueueClient (#1527) (6509534) by @Mantisus
• Improve error handling for RobotsTxtFile.load (#1524) (596a311) by @Mantisus
• Fix crawler_runtime not being updated during run and only in the end (#1540) (0d6c3f6) by @Pijukatel
• Ensure persist state event emission when exiting EventManager context (#1562) (6a44f17) by @Pijukatel

1.0.4 (2025-10-24)

🐛 Bug Fixes

• Respect enqueue_strategy in enqueue_links (#1505) (6ee04bc) by @Mantisus
• Exclude incorrect links before checking robots.txt (#1502) (3273da5) by @Mantisus
• Resolve compatibility issue between SqlStorageClient and AdaptivePlaywrightCrawler (#1496) (ce172c4) by @Mantisus
• Fix BasicCrawler statistics persistence (#1490) (1eb1c19) by @Pijukatel
• Save context state in result for AdaptivePlaywrightCrawler after isolated processing in SubCrawler (#1488) (62b7c70) by @Mantisus

1.0.3 (2025-10-17)

🐛 Bug Fixes

• Add support for Pydantic v2.12 (#1471) (35c1108) by @Mantisus
• Fix database version warning message (#1485) (18a545e) by @Mantisus
• Fix reclaim_request in SqlRequestQueueClient to correctly update the request state (#1486) (1502469) by @Mantisus
• Fix KeyValueStore.auto_saved_value failing in some scenarios (#1438) (b35dee7) by @Pijukatel

1.0.2 (2025-10-08)

🐛 Bug Fixes

• Use Self type in the open() method of storage clients (#1462) (4ec6f6c) by @janbuchar
• Add storages name validation (#1457) (84de11a) by @Mantisus
• Pin pydantic version to <2.12.0 to avoid compatibility issues (#1467) (f11b86f) by @vdusek

1.0.1 (2025-10-06)

🐛 Bug Fixes

• Fix memory leak in PlaywrightCrawler on browser context creation (#1446) (bb181e5) by @Pijukatel
• Update templates to handle optional httpx client (#1440) (c087efd) by @Pijukatel

1.0.0 (2025-09-29)

• Check out the Release blog post for more details.
• Check out the Upgrading guide to ensure a smooth update.

🚀 Features

• Add utility for load and parse Sitemap and SitemapRequestLoader (#1169) (66599f8) by @Mantisus
• Add periodic status logging and status_message_callback parameter for customization (#1265) (b992fb2) by @Mantisus
• Add crawlee-cli option to skip project installation (#1294) (4d5aef0) by @Pijukatel
• Improve Crawlee CLI help text (#1297) (afbe10f) by @Pijukatel
• Add basic OpenTelemetry instrumentation (#1255) (a92d8b3) by @Pijukatel
• Add ImpitHttpClient http-client client using the impit library (#1151) (0d0d268) by @Mantisus
• Prevent overloading system memory when running locally (#1270) (30de3bd) by @janbuchar
• Expose PlaywrightPersistentBrowser class (#1314) (b5fa955) by @Mantisus
• Add impit option for Crawlee CLI (#1312) (508d7ce) by @Mantisus
• Persist RequestList state (#1274) (cc68014) by @janbuchar
• Persist DefaultRenderingTypePredictor state (#1340) (fad4c25) by @Mantisus
• Persist the SitemapRequestLoader state (#1347) (27ef9ad) by @Mantisus
• Add support for NDU storages (#1401) (5dbd212) by @vdusek
• Add RQ id, name, alias args to add_requests and enqueue_links methods (#1413) (1cae2bc) by @Mantisus
• Add SqlStorageClient based on sqlalchemy v2+ (#1339) (07c75a0) by @Mantisus

🐛 Bug Fixes

• Fix memory estimation not working on MacOS (#1330) (ab020eb) by @Pijukatel
• Fix retry count to not count the original request (#1328) (74fa1d9) by @Pijukatel
• [breaking] Remove unused "stats" field from RequestQueueMetadata (#1331) (0a63bef) by @vdusek
• Ignore unknown parameters passed in cookies (#1336) (50d3ef7) by @Mantisus
• Fix timeout for stream method in ImpitHttpClient (#1352) (54b693b) by @Mantisus
• Include reason in the session rotation warning logs (#1363) (d6d7a45) by @vdusek
• Improve crawler statistics logging (#1364) (1eb6da5) by @vdusek
• Do not add a request that is already in progress to MemoryRequestQueueClient (#1384) (3af326c) by @Mantisus
• Save RequestQueueState for FileSystemRequestQueueClient in default KVS (#1411) (6ee60a0) by @Mantisus
• Set default desired concurrency for non-browser crawlers to 10 (#1419) (1cc9401) by @vdusek

Refactor

• [breaking] Introduce new storage client system (#1194) (de1c03f) by @vdusek
• [breaking] Split BrowserType literal into two different literals based on context (#1070) (72b5698) by @Pijukatel
• [breaking] Change method HttpResponse.read from sync to async (#1296) (83fa8a4) by @Mantisus
• [breaking] Replace HttpxHttpClient with ImpitHttpClient as default HTTP client (#1307) (c803a97) by @Mantisus
• [breaking] Change Dataset unwind parameter to accept list of strings (#1357) (862a203) by @vdusek
• [breaking] Remove Request.id field (#1366) (32f3580) by @Pijukatel
• [breaking] Refactor storage creation and caching, configuration and services (#1386) (04649bd) by @Pijukatel

0.6.12 (2025-07-30)

🚀 Features

• Add retire_browser_after_page_count parameter for BrowserPool (#1266) (603aa2b) by @Mantisus

🐛 Bug Fixes

• Use perf_counter_ns for request duration tracking (#1260) (9e92f6b) by @Pijukatel, closes #1256
• Fix memory estimation not working on MacOS (#1330) (8558954) by @Pijukatel, closes #1329
• Fix retry count to not count the original request (#1328) (1aff3aa) by @Pijukatel, closes #1326
• Ignore unknown parameters passed in cookies (#1336) (0f2610c) by @Mantisus, closes #1333

0.6.11 (2025-06-23)

🚀 Features

• Add stream method for HttpClient (#1241) (95c68b0) by @Mantisus

🐛 Bug Fixes

• Fix ClientSnapshot overload calculation (#1228) (a4fc1b6) by @Pijukatel
• Use PSS instead of RSS to estimate children process memory usage on Linux (#1210) (436032f) by @Pijukatel
• Do not raise an error to check 'same-domain' if there is no hostname in the url (#1251) (a6c3aab) by @Mantisus

0.6.10 (2025-06-02)

🐛 Bug Fixes

• Allow config change on PlaywrightCrawler (#1186) (f17bf31) by @mylank
• Add payload to SendRequestFunction to support POST request (#1202) (e7449f2) by @Mantisus
• Fix match check for specified enqueue strategy for requests with redirect (#1199) (d84c30c) by @Mantisus
• Set WindowsSelectorEventLoopPolicy only for curl-impersonate template without playwright (#1209) (f3b839f) by @Mantisus
• Add support non-GET requests for PlaywrightCrawler (#1208) (dbb9f44) by @Mantisus
• Respect EnqueueLinksKwargs for extract_links function (#1213) (c9907d6) by @Mantisus

0.6.9 (2025-05-02)

🚀 Features

• Add an internal HttpClient to be used in send_request for PlaywrightCrawler using APIRequestContext bound to the browser context (#1134) (e794f49) by @Mantisus
• Make timeout error log cleaner (#1170) (78ea9d2) by @Pijukatel
• Add on_skipped_request decorator, to process links skipped according to robots.txt rules (#1166) (bd16f14) by @Mantisus

🐛 Bug Fixes

• Fix handle error without args in _get_error_message for ErrorTracker (#1181) (21944d9) by @Mantisus
• Temporarily add certifi<=2025.1.31 dependency (#1183) (25ff961) by @Pijukatel

0.6.8 (2025-04-25)

🚀 Features

• Handle unprocessed requests in add_requests_batched (#1159) (7851175) by @Pijukatel
• Add respect_robots_txt_file option (#1162) (c23f365) by @Mantisus

🐛 Bug Fixes

• Update UnprocessedRequest to match actual data (#1155) (a15a1f3) by @Pijukatel
• Fix the order in which cookies are saved to the SessionCookies and the handler is executed for PlaywrightCrawler (#1163) (82ff69a) by @Mantisus
• Call failed_request_handler for SessionError when session rotation count exceeds maximum (#1147) (b3637b6) by @Mantisus

0.6.7 (2025-04-17)

🚀 Features

• Add ErrorSnapshotter to ErrorTracker (#1125) (9666092) by @Pijukatel

🐛 Bug Fixes

• Improve validation errors in Crawlee CLI (#1140) (f2d33df) by @vdusek
• Disable logger propagation to prevent duplicate logs (#1156) (0b3648d) by @vdusek

0.6.6 (2025-04-03)

🚀 Features

• Add statistics_log_format parameter to BasicCrawler (#1061) (635ae4a) by @Mantisus
• Add Session binding capability via session_id in Request (#1086) (cda7b31) by @Mantisus
• Add requests argument to EnqueueLinksFunction (#1024) (fc8444c) by @Pijukatel

🐛 Bug Fixes

• Add port for same-origin strategy check (#1096) (9e24598) by @Mantisus
• Fix handling of loading empty metadata file for queue (#1042) (b00876e) by @Mantisus

0.6.5 (2025-03-13)

🐛 Bug Fixes

• Update to browserforge workaround (#1075) (2378cf8) by @Pijukatel

0.6.4 (2025-03-12)

🐛 Bug Fixes

• Add a check thread before set add_signal_handler (#1068) (6983bda) by @Mantisus
• Temporary workaround for browserforge import time code execution (#1073) (17d914f) by @Pijukatel

0.6.3 (2025-03-07)

🚀 Features

• Add project template with uv package manager (#1057) (9ec06e5) by @Mantisus
• Use fingerprint generator in PlaywrightCrawler by default (#1060) (09cec53) by @Pijukatel

🐛 Bug Fixes

• Update project templates for Poetry v2.x compatibility (#1049) (96dc2f9) by @Mantisus
• Remove tmp folder for PlaywrightCrawler in non-headless mode (#1046) (3a7f444) by @Mantisus

0.6.2 (2025-03-05)

🚀 Features

• Extend ErrorTracker with error grouping (#1014) (561de5c) by @Pijukatel

0.6.1 (2025-03-03)

🐛 Bug Fixes

• Add browserforge to mandatory dependencies (#1044) (ddfbde8) by @Pijukatel

0.6.0 (2025-03-03)

🚀 Features

• Integrate browserforge fingerprints (#829) (2b156b4) by @Pijukatel
• Add AdaptivePlaywrightCrawler (#872) (5ba70b6) by @Pijukatel
• Implement _snapshot_client for Snapshotter (#957) (ba4d384) by @Mantisus
• Add adaptive context helpers (#964) (e248f17) by @Pijukatel
• [breaking] Enable additional status codes arguments to PlaywrightCrawler (#959) (87cf446) by @Pijukatel
• Replace HeaderGenerator implementation by browserforge implementation (#960) (c2f8c93) by @Pijukatel

🐛 Bug Fixes

• Fix playwright template and dockerfile (#972) (c33b34d) by @janbuchar
• Fix installing dependencies via pip in project template (#977) (1e3b8eb) by @janbuchar
• Fix default migration storage (#1018) (6a0c4d9) by @Pijukatel
• Fix logger name for http based loggers (#1023) (bfb3944) by @Pijukatel
• Remove allow_redirects override in CurlImpersonateHttpClient (#1017) (01d855a) by @2tunnels
• Remove follow_redirects override in HttpxHttpClient (#1015) (88afda3) by @2tunnels
• Fix flaky test_common_headers_and_user_agent (#1030) (58aa70e) by @Pijukatel

Refactor

• [breaking] Remove unused config properties (#978) (4b7fe29) by @vdusek
• [breaking] Remove Base prefix from abstract class names (#980) (8ccb5d4) by @vdusek
• [breaking] Сhange default incognito context to persistent context for Playwright (#985) (f01520d) by @Mantisus
• [breaking] Change Session cookies from dict to SessionCookies with CookieJar (#984) (6523b3a) by @Mantisus
• [breaking] Replace enum with literal for EnqueueStrategy (#1019) (d2481ef) by @vdusek
• [breaking] Update status code handling (#1028) (6b59471) by @Mantisus
• [breaking] Move cli dependencies to optional dependencies (#1011) (4382959) by @Mantisus

0.5.4 (2025-02-05)

🚀 Features

• Add support use_incognito_pages for browser_launch_options in PlaywrightCrawler (#941) (eae3a33) by @Mantisus

🐛 Bug Fixes

• Fix session managment with retire (#947) (caee03f) by @Mantisus
• Fix templates - poetry-plugin-export version and camoufox template name (#952) (7addea6) by @Pijukatel
• Fix convert relative link to absolute in enqueue_links for response with redirect (#956) (694102e) by @Mantisus
• Fix CurlImpersonateHttpClient cookies handler (#946) (ed415c4) by @Mantisus

0.5.3 (2025-01-31)

🚀 Features

• Add keep_alive flag to crawler.__init__ (#921) (7a82d0c) by @Pijukatel
• Add block_requests helper for PlaywrightCrawler (#919) (1030459) by @Mantisus
• Return request handlers from decorator methods to allow further decoration (#934) (9ec0aae) by @mylank
• Add transform_request_function for enqueue_links (#923) (6b15957) by @Mantisus
• Add time_remaining_secs property to MIGRATING event data (#940) (b44501b) by @fnesveda
• Add LogisticalRegressionPredictor - rendering type predictor for adaptive crawling (#930) (8440499) by @Pijukatel

🐛 Bug Fixes

• Fix crawler not retrying user handler if there was timeout in the handler (#909) (f4090ef) by @Pijukatel
• Optimize memory consumption for HttpxHttpClient, fix proxy handling (#905) (d7ad480) by @Mantisus
• Fix BrowserPool and PlaywrightBrowserPlugin closure (#932) (997543d) by @Mantisus

0.5.2 (2025-01-17)

🐛 Bug Fixes

• Avoid use_state race conditions. Remove key argument to use_state (#868) (000b976) by @Pijukatel
• Restore proxy functionality for PlaywrightCrawler broken in v0.5 (#889) (908c944) by @Mantisus
• Fix the usage of Configuration (#899) (0f1cf6f) by @vdusek

0.5.1 (2025-01-07)

🐛 Bug Fixes

• Make result of RequestList.is_empty independent of fetch_next_request calls (#876) (d50249e) by @janbuchar

0.5.0 (2025-01-02)

🚀 Features

• Add possibility to use None as no proxy in tiered proxies (#760) (0fbd017) by @Pijukatel
• Add use_state context method (#682) (868b41e) by @Mantisus
• Add pre-navigation hooks router to AbstractHttpCrawler (#791) (0f23205) by @Pijukatel
• Add example of how to integrate Camoufox into PlaywrightCrawler (#789) (246cfc4) by @Pijukatel
• Expose event types, improve on/emit signature, allow parameterless listeners (#800) (c102c4c) by @janbuchar
• Add stop method to BasicCrawler (#807) (6d01af4) by @Pijukatel
• Add html_to_text helper function (#792) (2b9d970) by @Pijukatel
• [breaking] Implement RequestManagerTandem, remove add_request from RequestList, accept any iterable in RequestList constructor (#777) (4172652) by @janbuchar

🐛 Bug Fixes

• Fix circular import in KeyValueStore (#805) (8bdf49d) by @Mantisus
• [breaking] Refactor service usage to rely on service_locator (#691) (1d31c6c) by @vdusek
• Pass verify in httpx client (#802) (074d083) by @Mantisus
• Fix page_options for PlaywrightBrowserPlugin (#796) (bd3bdd4) by @Mantisus
• Fix event migrating handler in RequestQueue (#825) (fd6663f) by @Mantisus
• Respect user configuration for work with status codes (#812) (8daf4bd) by @Mantisus
• abort-on-error for successive runs (#834) (0cea673) by @Mantisus
• Relax ServiceLocator restrictions (#837) (aa3667f) by @janbuchar
• Fix typo in exports (#841) (8fa6ac9) by @janbuchar

Refactor

• [breaking] Refactor HttpCrawler, BeautifulSoupCrawler, ParselCrawler inheritance (#746) (9d3c269) by @Pijukatel
• [breaking] Remove json_ and order_no from Request (#788) (5381d13) by @Mantisus
• [breaking] Rename PwPreNavContext to PwPreNavCrawlingContext (#827) (84b61a3) by @vdusek
• [breaking] Rename PlaywrightCrawler kwargs: browser_options, page_options (#831) (ffc6048) by @Pijukatel
• [breaking] Update the crawlers & storage clients structure (#828) (0ba04d1) by @vdusek

0.4.5 (2024-12-06)

🚀 Features

• Improve project bootstrapping (#538) (367899c) by @janbuchar

🐛 Bug Fixes

• Add upper bound of HTTPX version (#775) (b59e34d) by @vdusek
• Fix incorrect use of desired concurrency ratio (#780) (d1f8bfb) by @Pijukatel
• Remove pydantic constraint <2.10.0 and update timedelta validator, serializer type hints (#757) (c0050c0) by @Pijukatel

0.4.4 (2024-11-29)

🚀 Features

• Expose browser_options and page_options to PlaywrightCrawler (#730) (dbe85b9) by @vdusek
• Add abort_on_error property (#731) (6dae03a) by @Mantisus

🐛 Bug Fixes

• Fix init of context managers and context handling in BasicCrawler (#714) (486fe6d) by @vdusek

0.4.3 (2024-11-21)

🐛 Bug Fixes

• Pydantic 2.10.0 issues (#716) (8d8b3fc) by @Pijukatel

0.4.2 (2024-11-20)

🐛 Bug Fixes

• Respect custom HTTP headers in PlaywrightCrawler (#685) (a84125f) by @Mantisus
• Fix serialization payload in Request. Fix Docs for Post Request (#683) (e8b4d2d) by @Mantisus
• Accept string payload in the Request constructor (#697) (19f5add) by @vdusek
• Fix snapshots handling (#692) (4016c0d) by @Pijukatel

0.4.1 (2024-11-11)

🚀 Features

• Add max_crawl_depth option to BasicCrawler (#637) (77deaa9) by @Prathamesh010
• Add BeautifulSoupParser type alias (#674) (b2cf88f) by @Pijukatel

🐛 Bug Fixes

• Fix total_size usage in memory size monitoring (#661) (c2a3239) by @janbuchar
• Add HttpHeaders to module exports (#664) (f0c5ca7) by @vdusek
• Fix unhandled ValueError in request handler result processing (#666) (0a99d7f) by @janbuchar
• Fix BaseDatasetClient.iter_items type hints (#680) (a968b1b) by @Pijukatel

0.4.0 (2024-11-01)

🚀 Features

• Add headers in unique key computation (#609) (6c4746f) by @Prathamesh010
• Add pre_navigation_hooks to PlaywrightCrawler (#631) (5dd5b60) by @Prathamesh010
• Add always_enqueue option to bypass URL deduplication (#621) (4e59fa4) by @Rutam21
• Split and add extra configuration to export_data method (#580) (6751635) by @deshansh

🐛 Bug Fixes

• Use strip in headers normalization (#614) (a15b21e) by @vdusek
• Merge payload and data fields of Request (#542) (d06fcef) by @vdusek
• Default ProxyInfo port if httpx.URL port is None (#619) (8107a6f) by @steffansafey

Chore

• Remove Request.query_params field (#639) (6ec0ec4) by @vdusek

0.3.9 (2024-10-23)

🚀 Features

• Key-value store context helpers (#584) (fc15622) by @janbuchar
• Added get_public_url method to KeyValueStore (#572, closes #514) (3a4ba8f) by @akshay11298

🐛 Bug Fixes

• Workaround for JSON value typing problems (#581, closes #563) (403496a) by @janbuchar

0.3.8 (2024-10-03)

🚀 Features

• Mask Playwright's "headless" headers (#545, closes #401) (d1445e4) by @vdusek
• Add new model for HttpHeaders (#544) (854f2c1) by @vdusek

🐛 Bug Fixes

• Call error_handler for SessionError (#557, closes #546) (e75ac4b) by @vdusek
• Extend from StrEnum in RequestState to fix serialization (#556, closes #551) (6bf35ba) by @vdusek
• Add equality check to UserData model (#562) (899a25c) by @janbuchar

0.3.7 (2024-09-25)

🐛 Bug Fixes

• Improve Request.user_data serialization (#540, closes #524) (de29c0e) by @janbuchar
• Adopt new version of curl-cffi (#543) (f6fcf48) by @vdusek

0.3.6 (2024-09-19)

🚀 Features

• Add HTTP/2 support for HTTPX client (#513, closes #512) (0eb0a33) by @vdusek
• Expose extended unique key when creating a new Request (#515) (1807f41) by @vdusek
• Add header generator and integrate it into HTTPX client (#530, closes #402) (b63f9f9) by @vdusek

🐛 Bug Fixes

• Use explicitly UTF-8 encoding in local storage (#533, closes #532) (a3a0ab2) by @vdusek

0.3.5 (2024-09-10)

🚀 Features

• Memory usage limit configuration via environment variables (#502) (c62e554) by @janbuchar

🐛 Bug Fixes

• Http clients detect 4xx as errors by default (#498, closes #496) (1895dca) by @vdusek
• Correctly handle log level configuration (#508) (7ea8fe6) by @janbuchar

0.3.4 (2024-09-05)

🐛 Bug Fixes

• Expose basic crawling context (#501) (b484535) by @vdusek

0.3.3 (2024-09-05)

🐛 Bug Fixes

• Deduplicate requests by unique key before submitting them to the queue (#499) (6a3e0e7) by @janbuchar

0.3.2 (2024-09-04)

🐛 Bug Fixes

• Double incrementation of item_count (#443, closes #442) (cd9adf1) by @cadlagtrader
• Field alias in BatchRequestsOperationResponse (#485) (126a862) by @janbuchar
• JSON handling with Parsel (#490, closes #488) (ebf5755) by @janbuchar

0.3.1 (2024-08-30)

🚀 Features

• Curl http client selects chrome impersonation by default (#473) (82dc939) by @vdusek

0.3.0 (2024-08-27)

• See upgrading guide.

🚀 Features

• Implement ParselCrawler that adds support for Parsel (#348, closes #335) (a3832e5) by @asymness
• Add support for filling a web form (#453, closes #305) (5a125b4) by @vdusek

🐛 Bug Fixes

• Remove indentation from statistics logging and print the data in tables (#322, closes #306) (359b515) by @TymeeK
• Remove redundant log, fix format (#408) (8d27e39) by @janbuchar
• Dequeue items from RequestQueue in the correct order (#411) (96fc33e) by @janbuchar
• Relative URLS supports & If not a URL, pass #417 (#431, closes #417) (ccd8145) by @black7375
• Typo in ProlongRequestLockResponse (#458) (30ccc3a) by @janbuchar
• Add missing all to top-level init.py file (#463) (353a1ce) by @janbuchar

Refactor

• [breaking] RequestQueue and service management rehaul (#429, closes #83, #174, #203, #423) (b155a9f) by @janbuchar
• [breaking] Declare private and public interface (#456) (d6738df) by @vdusek

0.2.1 (2024-08-05)

🐛 Bug Fixes

• Do not import curl impersonate in http clients init (#396) (3bb8009)