Highlights

• Ray plans to drop support for Pydantic V1 starting version 2.56.0. Please see this RFC for details.
• Ray Data now has support for bounded reading from Kafka and improved Iceberg support.

Ray Data

🎉 New Features

• Autoscaling: New utilization-based cluster autoscaler for Ray Data workloads (#59353, #59362, #59366). To use this new autoscaler set RAY_DATA_CLUSTER_AUTOSCALER=V2.
• Kafka Datasource: Add Kafka as a native datasource for data ingestion (#58592)
• Dataset summary API: Add Dataset.summary() API for quick dataset inspection (#58862)
• Iceberg support: Add Iceberg schema evolution, upsert, and overwrite support (#59210, #59335)
• Graceful error handling: Add should_continue_on_error for graceful error handling in batch inference (#59212)
• Datetime compute expressions: Add datetime compute expressions support (#58740)
• Grouped with_column expressions: Enable expressions for grouped with_column in Ray Data (#58231)
• Parallelized collation: Parallelize DefaultCollateFn, arrow_batch_to_tensors (#58821)

💫 Enhancements

• Optimized Autoscaler Step Size: Optimize autoscaler to support configurable step size for actor pool scaling (#58726)
• Improved Streaming Repartition: Improve streaming repartition performance (#58728)
• Actor init retry: Add actor retry if there's a failure in __init__ (#59105)
• Fused Repartition + MapBatches: Fuse StreamingRepartition with MapBatches operators to scale collate (#59108)
• Combined repartitions: Combine consecutive repartitions for efficiency (#59145)
• Prefetch buffering: Handle prefetch buffering in iter_batches (#58657)
• HashShuffle block breakdown: HashShuffleAggregator breaks down blocks on finalize (#58603)
• Backpressure tuning: Tune concurrency cap backpressure object store budget ratio (#58813)
• Non-string ApproximateTopK: Support non-string items for ApproximateTopK aggregator (#58659)
• Lance version support: Add version support to read_lance() (#58895)
• Dashboard metrics: Add time_to_first_batch and get_ref_bundles metrics to data dashboard (#58912)
• Iter prefetched bytes stats: Add iter_prefetched_bytes statistics tracking (#58900)
• Configurable batching for iter_batches: Add configurable batching for resolve_block_refs to speed up iter_batches (#58467)
• Improved dashboard metrics: Improve Ray Data dashboard metrics display (#58667)
• Histogram percentiles: Update Ray Data histograms to show percentiles in data dashboard (#58650)
• Deprecated API removal: Remove deprecated read_parquet_bulk API (#58970)
• Block shaping option: Add disable block shaping option to BlockOutputBuffer (#58757)
• Removed concurrency lock: Remove concurrency lock for better performance (#56798)

🔨 Fixes

• Fixes to Unique: Fix support of list types for Unique aggregator (#58916)
• Parquet NaN fix: Fix reading from written parquet for numpy with NaNs (#59172)
• Hash Shuffle empty block: Fix empty block sort in hash shuffle operator (#58836)
• Hive partitioning pushdown: Fix pushdown optimizations with Hive partitioning (#58723)
• Object Store usage reporting: Fix obj_store_mem_max_pending_output_per_task reporting (#58864)
• Pyarrow FileSystem serialization fix: Handle filesystem serialization issue in get_parquet_dataset (#57047)
• Azure UC SAS: Handle Azure UC user delegation SAS (#59393)
• Async UDF Thread Cleanup: Close threads from async UDF after actor died (#59261)
• Object Locality Default: Default return 0s for object locality instead of -1s (#58754)

📖 Documentation

• Added contributing guide to Ray Data documentation (#58589)
• Added download expression to key user journeys in documentation (#59417)
• Added Kafka user guide (#58881)
• Added unstructured data templates from Ray Summit 2025 (#57063)
• Improved instructions for reading Hugging Face datasets (#58492, #58832)
• Refined batch-format guidance in docs (#58971)
• Exposed vision_preprocess and vision_postprocess in VLM docs (#59012)
• Added upgrading huggingface_hub instruction (#59109)
• Added scaling out expensive collation functions doc (#58993)

Ray Serve

🎉 New Features

• Deployment topology visibility. Exposes deployment dependency graphs in Serve REST API, allowing users to visualize and understand the DAG structure of their applications. (#58355)
• External autoscaler integration. Adds external_scaler_enabled flag to application config, enabling third-party autoscalers to control replica counts. (#57727, #57698)
• Node rank and local rank support. Extends replica rank system to track node-level and per-node local ranks, enabling better distributed serving coordination for multi-node deployments. (#58477, #58479)
• Custom batch size function. Allows users to define custom functions for computing logical batch sizes in @serve.batch, useful when batch items have varying weights (e.g., token counts in LLM inference). (#59059)
• Stateful application-level autoscaling. Adds policy state persistence for custom autoscaling policies, allowing policies to maintain state across control-loop iterations. (#59118)
• New autoscaling, batching, and routing metrics. Adds Prometheus metrics for autoscaling decisions (ray_serve_deployment_target_replicas, ray_serve_autoscaling_decision_replicas), batching statistics, and router queue latency for improved observability. (#59220, #59232, #59233)

💫 Enhancements

• Smarter downscaling behavior. Prioritizes stopping most recently scaled-up replicas during downscale, preserving long-lived replicas that are optimally placed and fully warmed up. (#52929)
• Autoscaling performance optimizations. Short-circuits metric aggregation for single time series cases (O(n log n) → O(1)) and lazily evaluates expensive autoscaling context fields to reduce controller CPU usage. (#58962, #58963)
• Route matching cleanup. Removes redundant route matching logic from replicas since correct route values are now included in RequestMetadata. Also allows multiple methods (GET, PUT) corresponding to a route. (#58927)
• Deployment wrapper metadata preservation. Wrapper classes from decorators like @ingress now preserve original class metadata (__qualname__, __module__, __doc__, __annotations__). (#58478)
• Improved type annotations. Enhances generic type annotations on DeploymentHandle, DeploymentResponse, and DeploymentResponseGenerator for better IDE support and type inference. Adds .result() stub to DeploymentResponseGenerator to fix static typing errors. (#59363, #58522)

🔨 Fixes

• YAML serialization for autoscaling enums. Fixes RepresenterError when using serve build with AggregationFunction enum values in autoscaling config. (#58509)
• Autoscaling context timestamp fix. Correctly sets last_scale_up_time and last_scale_down_time on autoscaling context. (#59057)
• Deadlock in chained deployment responses. Fixes hang when awaiting intermediate DeploymentResponse objects in a chain of deployment calls from different event loops. (#59385)
• FastAPI class-based view inheritance. Fixes make_fastapi_class_based_view to properly handle inherited methods. (#59410)

📖 Documentation

• Async I/O best practices guide. New documentation covering async programming patterns and best practices for Ray Serve deployments. (#58909)
• Replica scheduling guide. New documentation covering compact scheduling, placement groups, custom resources, and guidance on when to use each feature. (#59114)

Ray Train

🎉 New Features

• Worker Placement with Label Selectors: Added label_selector to ScalingConfig. This allows users to control worker placement by targeting specific labeled nodes in the cluster. (#58845, #59414)
• Multihost JaxTrainer on GPU: Introduced support for JaxTrainer running on GPU machines. (#58322)
• Checkpoint Consistency Modes: Added CheckpointConsistencyMode to get_all_reported_checkpoints, providing options for handling checkpoint retrieval consistency. (#58271)
• Per-Dataset Execution Options: DataConfig now supports setting execution_options on a per-dataset basis for finer-grained control over data loading. (#58717)

💫 Enhancements

• Nested Metrics Support: Result.get_best_checkpoint now supports nested metrics, allowing for more flexible metric tracking and checkpoint selection. (#58537)
• Non-Blocking Checkpoint Retrieval: get_all_reported_checkpoints no longer blocks when only metrics are reported. (#58870)
• Improved Resource Cleanup: Implemented eager cleanup of data resources and placement groups upon training run failures or aborts, preventing resource leaks. (#58325, #58515)

🔨 Fixes

• MLflow Compatibility: Updated setup_mlflow API to ensure full compatibility with Ray Train V2. (#58705)
• Validation for Checkpoint Uploads: A ValueError is now raised if checkpoint_upload_fn fails to return a valid checkpoint. (#58863)

📖 Documentation

• New API Documentation: Added comprehensive documentation for the ray.train.get_all_reported_checkpoints method. (#58946)

Ray Tune

💫 Enhancements:

• Nested Metrics Support: Result.get_best_checkpoint now supports nested metrics, allowing for more flexible metric tracking and checkpoint selection. (#58537)

Ray LLM

💫 Enhancements

• Cloud filesystem restructuring with provider-specific implementations (#58469)
• Bump transformers to 4.57.3 (#58980)
• Ray Data LLM config refactor (#58298)
• Update vllm_engine.py to check for VLLM_USE_V1 attribute (#58820)
• Infer VLLM_RAY_PER_WORKER_GPUS from fractional placement-group bundles automatically (#58949)

🔨 Fixes

• Fix LLM DP release test configuration (#59090)

Ray RLlib

🎉 New Features

• DreamerV3: allow num_env_runners \> 0 (#58495)

💫 Enhancements

• 🔥 MetricsLogger tweaks+ Stats rewrite (#56838)
• move restart message into EnvRunner (#56750)
• make “Footsies” less verbose (optionally) (#58939)
• update an AlgorithmConfig deprecated argument with incorrect behavior/semantics (#59138)
• Examples/docs cleanup:
  • merge tuned examples into examples/ (#58893)
  • move old API examples (#59159)
  • move example run scripts (#59160)
  • remove Torch 2.x doc tied to removed benchmarks (#59173)
  • remove rllib/benchmark(s) folder from RLlib directory (#59158)
• Testing / CI & infra cleanup (part of a larger effort to organize + harden RLlib testing):
  • clean up tests folder layout in favor of /component/tests (#58890)
  • re-enable and fix nightly tests for APPO on Atari and MuJoCo (#58853)
  • re-enable all RLlib doctests (#58974)
  • add pytest reporting hook (pytest_runtest_makereport) across tests (#59003)
  • add/enable RLlib Py3.10 CI lane (#59226)
  • fix as-release-test silently failing (#59386)
  • fix recursive imports in old test-utils location (#59435)
  • Remove asv.conf.json (#58934)
  • Update requirement for byod_rllib.sh (#59157)

🔨 Fixes

• Fix custom model-config mismatch between EnvRunner and Learner (#58739)
• MultiAgentEnvRunner: prevent double-calling connectors (#58931)
• Error handling: log or raise when a case is not fully handled (#58889)
• Error handling: error out when data cannot be loaded (#59002)
• Assorted RLlib bugfixes (#59386)

📖 Documentation

• Update APPO paper reference to link to IMPACT paper (#58935)

Ray Core

🎉 New Features

• Support zero-copy serialization for read-only PyTorch tensors via RAY_ENABLE_ZERO_COPY_TORCH_TENSORS (#57639)
• Add .rayignore file support for controlling cluster uploads (#58500)
• Improve large-scale resource view synchronization through sync message batching (#57641)
• Autoscaler with cloud resource availability awareness (#58623)
• Token authentication UX improvements with new AuthenticationError exception (#58737)
• Support X-Ray-Authorization fallback header for auth token in dashboard (#58819)

💫 Enhancements

• Limit core worker gRPC reply threads to 2 by default via RAY_core_worker_num_server_call_thread (#58771)
• Make accessor node address and liveliness cache thread safe (#58947)
• Create OtlpGrpcMetricExporter wrapper to log export failures (#58929)
• Print detailed exception information when failing to report events (#58953)
• Simplify local/global GC logic (#58671)
• Surface correct error message when get_if_exists=True for actor lookup (#58628)
• Throw AuthenticationError from Python for token loading errors (#59031)
• Use secrets.token_hex(32) to generate auth tokens (#58818)
• Remove AUTH_MODE=token check in get-auth-token CLI (#58848)
• Introduce core chaos network release tests (#58868)

🔨 Fixes

• Fix grpc_authentication_server_interceptors streaming response handling (#59104)
• Fix handle leak in IsProcessAlive on Windows (#59106)
• Fix counter metric default branch for RAY_enable_open_telemetry (#59095)
• Fix leaking metric recorder in tests (#58952)
• Fix crash when using JVM HDFS by adding RAY_DISABLE_FAILURE_SIGNAL_HANDLER option (#58984)
• Fix heap corruption in RayletClient causing driver crash (use-after-free) (#58660)
• Use shared_ptr for pins_in_flight_ to prevent use-after-free (#58744)
• Remove deprecated add_command_alias (#58719)
• Remove cluster_full_of_actors_detected_* fields (unused in autoscaler v2) (#59052)

📖 Documentation

• Add token-auth.md documentation page (#58829)
• Update KubeRay authentication guide to use native Ray token authentication (#58729)

Dashboard

💫 Enhancements

• Add time_to_first_batch and get_ref_bundles metrics to data dashboard (#58912)
• Update Ray Data histograms to show percentiles grouped by operator (#58650)

Ray Wheels and Images

• Upgraded rich, cupy-cuda12x, and memray (#58983)
• Upgraded lxml to 6.0.2 (#58808)
• Upgraded requests from 2.32.3 to 2.32.5 (#58724)
• Added openlineage-python in the dependency set (#58724)

Thanks

Thank you to everyone who contributed to this release! @xinyuangui2, @harshit-anyscale, @Sparks0219, @israbbani, @siyuanfoundation, @robertnishihara, @thomasdesr, @spencer-p, @aslonnie, @ZacAttack, @soodoshll, @marosset, @simeetnayan81, @soffer-anyscale, @abrarsheikh, @400Ping, @richo-anyscale, @as-jding, @rueian, @kshanmol, @yancanmao, @zzchun, @coqian, @matthewdeng, @Future-Outlier, @YoussefEssDS, @ykdojo, @pseudo-rnd-thoughts, @lowdy1, @ArturNiederfahrenhorst, @myandpr, @komikndr, @machichima, @RisinT96, @curiosity-hyf, @alanwguo, @CaiZhanqi, @Aydin-ab, @MengjinYan, @suzuri-lollipop, @jeffreyjeffreywang, @rushikeshadhav, @alexeykudinkin, @meAmitPatil, @zcin, @teddygood, @elliot-barn, @dayshah, @srinathk10, @XLC127, @simonsays1980, @kevin85421, @bveeramani, @kunling-anyscale, @khluu, @andrew-anyscale, @KaisennHu, @kouroshHakha, @ryankert01, @pavitrabhalla, @jjyao, @dragongu, @SolitaryThinker, @justinrmiller, @wxwmd, @Haustle-v, @TimothySeah, @goutamvenkat-anyscale, @liulehui, @raulchen, @HassamSheikh, @Priya-753, @vaishdho1, @dancingactor, @daiping8, @eloaf, @JasonLi1909, @rayci-bot, @richardliaw, @SheldonTsen, @Yicheng-Lu-llll, @ktyxx, @pschmutz, @iamjustinhsu, @ahao-anyscale, @cem-anyscale, @eicherseiji, @edoakes, @rajeshg007, @arki05, @andrewsykim, @nrghosh, @ryanaoleary, @kyuds, @Daraan, @can-anyscale, @sampan-s-nayak, @xyuzh, @owenowenisme

• More robust handling for CVE-2025-62593: test for more browser-specific headers in dashboard browser rejection logic

Release Highlights

Ray Core:

• End of Life for Python 3.9 Support: Ray will no longer be releasing Python 3.9 wheels from now on.
• Token authentication: Ray now supports built-in token authentication across all components including the dashboard, CLI, API clients, and internal services. This provides an additional layer of security for production deployments to reduce the risk of unauthorized code execution. Token authentication is initially off by default. For more information, see: https://docs.ray.io/en/latest/ray-security/token-auth.html

Ray Data:

• We’ve added a number of improvements for Iceberg, including upserts, predicate and projection pushdown, and overwrite.
• We’ve added significant improvements to our expressions framework, including temporal, list, tensor, and struct datatype expressions.

Ray Libraries

Ray Data

🎉 New Features:

• Added predicate pushdown rule that pushes filter predicates past eligible operators (#58150, #58555)
• Iceberg support for upsert tables, schema updates, and overwrite operations (#58270)
• Iceberg support for predicate and projection pushdown (#58286)
• Iceberg write datafiles in write() then commit (#58601)
• Enhanced Unity Catalog integration (#57954)
• Namespaced expressions that expose PyArrow functions (#58465)
• Added version argument to read_delta_lake (#54976)
• Generator UDF support for map_groups (#58039)
• ApproximateTopK aggregator (#57950)
• Serialization framework for preprocessors (#58321)
• Support for temporal, list, tensor, and struct datatypes (#58225)

💫 Enhancements:

• Use approximate quantile for RobustScaler preprocessor (#58371)
• Map batches support for limit pushdown (#57880)
• Make all map operations zero-copy by default (#58285)
• Use tqdm_ray for progress reporting from workers (#58277)
• Improved concurrency cap backpressure tuning (#58163, #58023, #57996)
• Sample finalized partitions randomly to avoid lens effect (#58456)
• Allow file extensions starting with '.' (#58339)
• Set default file_extensions for read_parquet (#56481)
• URL decode values in parse_hive_path (#57625)
• Streaming partition enforces row_num per block (#57984)
• Streaming repartition combines small blocks (#58020)
• Lower DEFAULT_ACTOR_MAX_TASKS_IN_FLIGHT_TO_MAX_CONCURRENCY_FACTOR to 2 (#58262)
• Set udf-modifying-row-count default to false (#58264)
• Cache PyArrow schema operations (#58583)
• Explain optimized plans (#58074)
• Ranker interface (#58513)

🔨 Fixes:

• Fixed renamed columns to be appropriately dropped from output (#58040, #58071)
• Fixed handling of renames in projection pushdown (#58033, #58037)
• Fixed broken LogicalOperator abstraction barrier in predicate pushdown rule (#58683)
• Fixed file size ordering in download partitioning with multiple URI columns (#58517)
• Fixed HTTP streaming file download by using open_input_stream (#58542)
• Fixed expression mapping for Pandas (#57868)
• Fixed reading from zipped JSON (#58214)
• Fixed MCAP datasource import for better compatibility (#57964)
• Avoid slicing block when total_pending_rows < target (#58699)
• Clear queue for manually marked execution_finished operators (#58441)
• Add exception handling for invalid URIs in download operation (#58464)
• Fixed progress bar name display (#58451)

📖 Documentation:

• Documentation for Ray Data metrics (#58610)
• Simplify and add Ray Data LLM quickstart example (#58330)
• Convert rST-style to Google-style docstrings (#58523)

🏗 Architecture:

• Removed stats update thread (#57971)
• Refactor histogram metrics (#57851)
• Revisit OpResourceAllocator to make data flow explicit (#57788)
• Create unit test directory for fast, isolated tests (#58445)
• Dump verbose ResourceManager telemetry into ray-data.log (#58261)

Ray Train

🎉 New Features:

• Result::from_path implementation in v2 (#58216)

💫 Enhancements:

• Exit actor and log appropriately when poll_workers is in terminal state (#58287)
• Set JAX_PLATFORMS environment variable based on ScalingConfig (#57783)
• Default to disabling Ray Train collective util timeouts (#58229)
• Add SHUTTING_DOWN TrainControllerState and improve logging (#57882)
• Improved error message when calling training function utils outside Ray Train worker (#57863)
• FSDP2 template: Resume from previous epoch when checkpointing (#57938)
• Clean up checkpoint config and trainer param deprecations (#58022)
• Update failure policy log message (#58274)

📖 Documentation:

• Ray Train Metrics documentation page (#58235)
• Local mode user guide (#57751)
• Recommend tree_learner="data_parallel" in examples for distributed LightGBM training (#58709)

Ray Serve

🎉 New Features:

• Custom request routing with runtime environment support. Users can now define custom request router classes that are safely imported and serialized using the application's runtime environment, enabling advanced routing logic with custom dependencies. (#56855)
• Custom autoscaling policies with enhanced logging. Deployment-level and application-level autoscaling policies now display their custom policy names in logs, making it easier to debug and monitor autoscaling behavior. (#57878)
• Audio transcription support in vLLM backend. Ray Serve now supports transcription tasks through the vLLM engine, expanding multimodal capabilities. (#57194)
• Data parallel attention public API. Introduced a public API for data parallel attention, enabling efficient distributed attention mechanisms for large-scale inference workloads. (#58301)
• Route pattern tracking in proxy metrics. Proxy metrics now expose actual route patterns (e.g., /api/users/{user_id}) instead of just route prefixes, enabling granular endpoint monitoring without high cardinality issues. Performance impact is minimal (~1% RPS decrease). (#58180)
• Replica dependency graph construction. Added list_outbound_deployments() method to discover downstream deployment dependencies, enabling programmatic analysis of service topology for both stored and dynamically-obtained handles. (#58345, #58350)
• Multi-dimensional replica ranking. Introduced ReplicaRank schema with global, node-level, and local ranks to support advanced coordination scenarios like tensor parallelism and model sharding across nodes. (#58471, #58473)
• Proxy readiness verification. Added a check to ensure proxies are ready to serve traffic before serve.run() completes, improving deployment reliability. (#57723)
• IPv6 socket support. Ray Serve now supports IPv6 networking for socket communication. (#56147)

💫 Enhancements:

• Selective throughput optimization flag overrides. Users can now override individual flags set by RAY_SERVE_THROUGHPUT_OPTIMIZED without manually configuring all flags, improving flexibility for performance tuning. (#58057)
• OpenTelemetry metrics enabled by default. Ray now uses OpenTelemetry as the default metrics backend, with updated metric names (ray_serve_*) and improved observability infrastructure. (#56432)
• Cleaner long-poll communication. Removed actor handles from RunningReplicaInfo objects passed in long-poll updates, avoiding complex reference counting patterns. (#58174)
• Improved replica config handling. Excluded IMPLICIT_RESOURCE_PREFIX from ReplicaConfig.ray_actor_options to prevent internal resource annotations from leaking into user-visible configurations. (#58275)
• Custom autoscaling telemetry. Added telemetry tracking for custom autoscaling policy usage. (#58336)
• Proxy target group control. Added from_proxy_manager argument to get_target_groups() for finer control over returned routing targets. (#57620)

🔨 Fixes:

• Fixed default deployment name in async inference. Corrected the default deployment name which was changed to _TaskConsumerWrapper during async inference implementation. (#57664)
• Fixed proxy location handling in CLI and Python API. serve run now respects proxy_location from config files instead of hardcoding EveryNode, and serve.start() no longer defaults to HeadOnly when http_options are provided without an explicit location. (#57622)
• Fixed deprecated Stable Diffusion model in example. Updated documentation example to use a current model after stabilityai/stable-diffusion-2 was deprecated on Hugging Face. (#58609)

📖 Documentation:

• KV-cache offloading user guide. Added comprehensive documentation for KV-cache offloading in LLM deployments. (#58025)
• Model loading documentation. Documented best practices and options for loading models in Ray Serve. (#57922)
• Cross-node tensor/pipeline parallelism examples. Added examples and documentation for running TP/PP across multiple nodes. (#57715)
• Data parallel attention documentation. Created user guide for data parallel attention with architecture diagrams. (#58301, #58543)
• Custom autoscaling policy examples. Added missing imports and improved clarity in autoscaling policy examples. (#57896, #58170)
• Async inference documentation improvements. Added notes about task consumer replica configurations and fixed the end-to-end example. (#58493)
• Callback documentation. Added documentation for using callbacks in Ray Serve. (#58713)
• Monitoring and troubleshooting improvements. Enhanced monitoring section with links to Anyscale troubleshooting resources. (#58472)
• Minor documentation fixes. Fixed spelling errors and improved docstring alignment. (#58172, #58233)

🏗 Architecture refactoring:

• Replica rank management refactoring. Extracted generic RankManager class with type-safe ReplicaRank representation, creating a cleaner foundation for future multi-level rank support. (#58471, #58473)

Ray Tune

💫 Enhancements:

• Updated jobs test to use tune module (#57995)
• Add pydantic to Ray Tune requirements (#58354)

RLlib

🎉 New Features:

• Support for vectorize modes in SingleAgentEnvRunner.make_env (#58410)
• Support for composed spaces in Offline RL (#58594)
• Enhanced support for complex observations in SingleAgentEpisode (#57017)
• Prometheus metrics support for selected components (#57932)

💫 Enhancements:

• Improve test_single_agent_env_runner to prevent flaky tests (#58397)
• LINT improvements with enabled ruff imports across multiple modules (#56737, #56734, #56741, #56742, #56744, #56746)

🔨 Fixes:

• Resolve bug that fails to propagate model_config to MultiAgentRLModule instances (#58243)
• Fixed access to self._minibatch_size (#58595)
• Broken restore from remote - Add missing FileSystem argument (#58324)
• Fixed deterministic sampling and training documentation link (#58494)
• Corrected typo in pyspiel import error message (#54618)

📖 Documentation:

• Add reinforcement learning example illustrating GPU-to-GPU RDT and GRPO (#57961)

Ray Core

🎉 New Features:

• Token-based authentication across all Ray components (#58046, #58047, #58176, #58209, #58276, #58281, #58308, #58333, #58368, #58395, #58405, #58408, #58424, #58557, #57835, #58566, #58591)
• OpenTelemetry enabled by default for improved observability (#56432)
• Fallback strategy scheduling logic (#56369)
• TPU utility functions to support slice placement groups (#56723)
• Exponential backoff for retryable gRPCs (#56568)
• Option for in-flight RPC failure injection (#58512)
• Release test to simulate network transient errors via iptables (#58241)
• Nightly release test with cross-AZ fault injection (#57579)
• Owned object spill metrics (#57870)
• Monitoring in raylet for resource view (#58382)
• IPv6 support for sockets (#56147)

💫 Enhancements:

• Fault-tolerant RPCs: KillActor, CancelRemoteTask, NotifyGCSRestart, and ReleaseUnusedBundles (#57648, #57945, #57965)
• Use graceful actor shutdown when GCS polling detects actor ref deleted (#58605)
• Use graceful shutdown path when actor OUT_OF_SCOPE (del actor) (#57090)
• Improved actor kill logs (#58544)
• Scheduling detached actor with placement group not recommended (#57726)
• Better handling of detached actor restarts (#57931)
• Enhanced ray.get thread safety (#57911)
• Making concurrent ray.get requests for the same object thread-safe (#58606)
• Move request ID creation to worker to address plasma get perf regression (#58390)
• Make GlobalState lazy initialization thread-safe (#58182)
• Reporter agent can get PID via RPC to raylet (#57004)
• Add tee logging for subprocess exit codes in ray start --block (#57982)
• Add entrypoint log for jobs (#58300)
• Cleaner error message for exceeding list actors limit (#58255)
• Clean up NODE_DIED task error message (#58638)
• Improved histogram metrics midpoint calculation (#57948)
• Migrated from STATS to metric interface in RPC components (#57926)
• Kill STATS in core worker component (#58060)
• Kill STATS in object manager component (#57974)
• Improve scheduler_placement_time_s metric (#58217)
• Refactor OpenTelemetry environment variable handling (#57910)
• Add option to disable OpenTelemetry SDK error logs (#58257)
• Improved cgroups support (#57776, #57864, #57731, #58017, #58028, #58059, #58064, #58577)
• Use GetNodeAddressAndLiveness in raylet client pool (#58576)
• Ray Direct Transport improvements with NIXL integration (#57671, #58550, #58548, #56783, #58263)
• Fix symmetric-run (#58337)
• Make worker connection timeout parameters configurable (#58372)
• Define env for controlling UVloop (#58442)
• Allow 60 seconds for dashboard to start (#58341)
• Report driver stats (#58045)
• Fix idle node termination on object pulling (#57928)
• Check if temp_dir is subdir of virtualenv to prevent runtime virtualenv problems (#58084)

🔨 Fixes:

• Fixed use-after-free in RayletClient (#58747)
• Fixed deadlock when cancelling stale requests on in-order actors (#57746)
• Fixed "RayEventRecorder::StartExportingEvents() should be called only once" error (#57917)
• Fixed raylet shutdown races (#57198)
• Fixed incorrect usage of gRPC streaming API in ray syncer (#58307)
• Fixed log monitor seeking bug after log rotation (#56902)
• Fixed idempotency issues in RequestWorkerLease for scheduled leases (#58265)
• Fixed RAY_CHECK(inserted) inside reference counter (#58092)
• Fixed static type hints for ActorClass when setting options (#58439)
• Fixed exception type for accelerator ID visibility check (#58269)
• Fixed transport type handling in DAG node initialization (#57987)
• Fixed RAY_NODE_TYPE_NAME handling when autoscaler is in read-only mode (#58460)
• Ensure client_call_manager_ outlives metrics_agent_client_ in core worker (#58315)
• Fixed header validation in dashboard tests (#58648)
• Validation of Ray-on-Spark-on-YARN mode to enable it to run (#58335)

📖 Documentation:

• Fix pattern_async_actor demo typo (#58486)
• Add limitations of RDT documentation (#58063)
• Add actor+job+node event to ray event export documentation (#57930)
• Remove implementation details from get_runtime_context docstring (#58212)
• Improved monitoring section with links (#58472)

🏗 Architecture:

• Refactor ActorInfoAccessor in gcs_client to be mockable (#57241)
• Refactor reference_counter out of memory store and plasma store (#57590)
• Remove reference counter mock for real reference counter in testing (#57178)
• Split raylet cython file into multiple files (#56575)
• Move ray_syncer to top level directory (#58316)
• Move python_callbacks to common (#57909)
• Consolidate find_free_port to network_utils (#58304)
• Implement event merge logic at export time (#58070)
• Feature flag for enabling ray export event (#57999)
• Add comments explaining ray_syncer_ channels in Raylet (#58342)
• Integration tests for task event generation (#57636)

Dashboard

💫 Enhancements:

• Added percentage usage graphs for resources (#57549)
• Sub-tabs with full Grafana dashboard embeds on Metrics tab (#57561)
• Added queued blocks to operator panels (#57739)
• Improved operator metrics logging (#57702)
• Make do_reply accept status_code instead of success bool (#58384)
• Add denial of fetch headers (#58553)

🔨 Fixes:

• Fixed broken Ray Data per node metrics due to unsupported operator filter (#57970)
• Filtered out ANSI escape codes from logs (#53370)

📖 Documentation:

• Expose dashboard URL when deploying on Yarn using Skein (#57793)

Autoscaler + KubeRay

🎉 New Features:

• KubeRay autoscaling support with top-level Resources and Labels fields (#57260)
• Bundle label selector support in request_resources SDK (#54843)

💫 Enhancements:

• Azure VM launcher release test (#57921)
• Azure CLI added to base-extra image (#58012)

📖 Documentation:

• Label selector guide (#58157)
• Add minimum version requirement on kai-scheduler (#58161)
• Mention RayJob gang scheduling for Yunikorn (#58375)
• Add Volcano RayJob gang scheduling example (#58320)
• Add KAI scheduler integration documentation (#54857)
• Kuberay sidecar mode (#58273)
• Update RayJob documentation with new DeletionStrategy (#58306)
• Add guidance for RayService initialization timeout (#58238)
• Update version to 1.5.0 (#58452)
• Add output example of CLI commands (#58078)
• Fix invalid syntax in label_selector (#58352)

Thank You to all the Contributors! @marosset, @curiosity-hyf, @bveeramani, @Future-Outlier, @saihaj, @ZacAttack, @ArthurBook, @crypdick, @Aydin-ab, @elliot-barn, @Kunchd, @justinvyu, @jjyao, @gangsf, @sunsetxh, @Daraan, @justinyeh1995, @MatthewCWeston, @kyuds, @daiping8, @sauravvenkat, @omatthew98, @CowKeyMan, @morotti, @israbbani, @goutamvenkat-anyscale, @fscnick, @Zakelly, @xyuzh, @kouroshHakha, @owenowenisme, @Qiaolin-Yu, @czgdp1807, @shen-shanshan, @wph95, @iamjustinhsu, @MengjinYan, @jugalshah291, @Yicheng-Lu-llll, @ryanaoleary, @nadongjun, @xinyuangui2, @ideal, @my-vegetable-has-exploded, @lucaschadwicklam97, @tianyi-ge, @ahao-anyscale, @abrarsheikh, @Blaze-DSP, @rueian, @thomasdesr, @CaiZhanqi, @harshit-anyscale, @jeffreyjeffreywang, @TimothySeah, @codope, @sampan-s-nayak, @andrewsykim, @xingsuo-zbz, @aslonnie, @OneSizeFitsQuorum, @ryankert01, @Sparks0219, @soffer-anyscale, @akyang-anyscale, @alanwguo, @chrisfellowes-anyscale, @richo-anyscale, @alexeykudinkin, @JasonLi1909, @ruisearch42, @EkinKarabulut, @MarcoGorelli, @SolitaryThinker, @srinathk10, @dayshah, @richardliaw, @pseudo-rnd-thoughts, @win5923, @axreldable, @matthewdeng, @ArturNiederfahrenhorst, @can-anyscale, @khluu, @landscapepainter, @kevin85421, @seanlaii, @edoakes, @nrghosh, @eicherseiji, @Artimislyy, @cem-anyscale, @coqian, @chiayi, @liulehui

• Fix for CVE-2025-62593: reject Sec-Fetch-* other browser-specific headers in dashboard browser rejection logic

• Reuse previous metadata if transferring the same tensor list with nixl (https://github.com/ray-project/ray/pull/58309)

Release Highlights

Ray Train:

• Ray Train v2 is now enabled by default! Ray Train v2 provides usability and stability improvements, as well as new features. For more details, see the REP and Migration Guide. To disable Ray Train v2, set the environment variable RAY_TRAIN_V2_ENABLED=0.

Ray Serve:

• Application-level autoscaling: Introduces custom autoscaling policies that operate across all deployments in an application, enabling coordinated scaling decisions based on aggregate metrics. This is a significant advancement over per-deployment autoscaling, allowing for more intelligent resource management at the application level.
• Enhanced autoscaling capabilities with replica-level metrics: Wires up AutoscalingContext with total_running_requests, total_queued_requests, and total_num_requests, plus adds support for min, max, and time-weighted average aggregation functions. These improvements give users fine-grained control to implement sophisticated custom autoscaling policies based on real-time workload metrics.

Ray Libraries

Ray Data

🎉 New Features:

• Added enhanced support for Unity Catalog integration (#57954, #58049)
• New expression evaluator infrastructure for improved query optimization (#57778, #57855)
• Support for SaveMode in write operations (#57946)
• Added approximate quantile aggregator (#57598)
• MCAP datasource support for robotics data (#55716)
• Callback-based stat computation for preprocessors and ValueCounter (#56848)
• Support for multiple download URIs with improved error handling (#57775)

💫 Enhancements:

• Improved projection pushdown handling with renamed columns (#58033, #58037, #58040, #58071)
• Enhanced hash-shuffle performance with better retry policies (#57572)
• Streamlined concurrency parameter semantics (#57035)
• Improved execution progress rendering (#56992)
• Better handling of empty columns in pandas blocks (#57740)
• Enhanced support for complex data types and column operations (#57271)
• Reduced memory usage with improved streaming generator backpressure (#57688)
• Enhanced preemption testing and utilities (#57883)
• Improved Download operator display names (#57773)
• Better handling of variable-shaped tensors and tensor columns (#57240)
• Optimized aggregator execution with out-of-order processing by default (#57753)

🔨 Fixes:

• Fixed renamed columns to be appropriately dropped from output (#58040, #58071)
• Fixed handling of renames in projection pushdown (#58033, #58037)
• Fixed vLLMEngineStage field name inconsistency for images (#57980)
• Fixed driver hang during streaming generator block metadata retrieval (#56451)
• Fixed retry policy for hash-shuffle tasks (#57572)
• Fixed prefetch loop to avoid blocking on fetches (#57613)
• Fixed empty projection handling (#57740)
• Fixed errors with concatenation of mixed pyarrow native and extension types (#56811)

📖 Documentation:

• Updated document embedding benchmark to use canonical Ray Data API (#57977)
• Improved concurrency-related documentation (#57658)
• Updated preprocessing and data handling examples

Ray Train

🎉 New features

• Turn on Train v2 by default (#57857)
• Top-level ray.train aliases for public APIs (#57758)

💫 Enhancements

• Raise clear errors when mixing v1/v2 APIs (#57570)
• JAX backend: add jax.distributed.shutdown() for JaxBackend (#57802)
• Update TrainingFailedError module (#57865)
• Improve deprecation handling when ray.train methods are called from ray.tune (#57810)
• Enable deprecation warnings for legacy XGBoost/LightGBM trainers (#57280)

🔨 Fixes

• Fix ControllerError triggered by after_worker_group_poll_status errors (#57869)
• Fix iter_torch_batches use of ray.train.torch.get_device outside Train (#57816)
• Fix exception-queue race condition in ThreadRunner (#57249)

📖 Documentation

• Add validation and details to checkpoint docs (#57065)

🏗 Architecture / tests

• Enable Train v2 across test suites; migrate remaining tests and isolate/disable stragglers (#56868, #57256, #57534, #57722, #57764)
• Isolate circular-dependency tests and resolve circular imports (#57710, #56921)
• Replace Checkpoint Manager Pydantic v2 APIs with v1 (#57147)
• Bump test timeouts (test_util, torch_trainer) (#57939, #57873)

Ray Tune

💫 Enhancements:

• Updated release tests to import from tune (#57956)
• Better integration with Train V2 backend

Ray Serve

🎉 New Features:

• Application-level autoscaling. Introduces support for custom autoscaling policies that operate across all deployments in an application, enabling coordinated scaling decisions based on aggregate metrics. (#57535, #57548, #57637, #57756)
• Autoscaling metrics aggregation functions. Adds support for min, max, and time-weighted average aggregation over timeseries data, providing more flexible autoscaling control. (#56871)
• Enhanced autoscaling context with replica-level metrics. Wires up AutoscalingContext constructor arguments to expose total_running_requests, total_queued_requests, and total_num_requests for use in custom autoscaling policies. (#57202)
• Multiple task consumers in a single application. Ray Serve applications can now run multiple task consumer deployments concurrently. (#56618)

💫 Enhancements:

• Reconfigure invoked on replica rank changes. The reconfigure method now receives both user_config and rank parameters when ranks change, enabling replicas to adapt their configuration dynamically. (#57091)
• Celery adapter configuration improvements. Added default serializer and new configuration fields to enhance Celery integration flexibility. (#56707)
• AutoscalingContext promoted to public API. The autoscaling context is now officially part of the public API with comprehensive documentation. (#57600)
• Async inference telemetry. Added telemetry tracking to monitor the number of replicas using asynchronous inference. (#57665)
• Rank logging verbosity reduced. Changed seven rank-related INFO logs to DEBUG level, reducing log noise during normal operations. (#57831)
• Controller logging optimized. Removed expensive debug logs from the controller that were costly in large clusters. (#57813)

🔨 Fixes:

• Max constructor retry count test fixed for Windows. Adjusted test resource requirements to account for Windows process creation overhead compared to Linux forking. (#57541)
• Streaming test stability improvements. Added synchronization mechanisms to prevent chunk coalescing and rechunking, eliminating test flakiness. (#57592, #57728)
• Autoscaling test deflaking. Fixed race conditions in application-level autoscaling tests and removed flaky min aggregation test scenario. (#57784, #57967)
• State API usage test corrected. Fixed a unit test that was broken but not running in CI. (#56948)
• Controller recovery logging condition fixed. Updated test condition to properly verify debug and JSON logs after controller recovery. (#57568)

📖 Documentation:

• Custom autoscaling documentation. Added comprehensive guide for implementing custom autoscaling policies with examples and best practices. (#57600)
• Replica ranks documentation. Documented the replica rank feature, including how ranks are assigned and how to use them in reconfigure methods. (#57649)
• Application-level autoscaling guide. Added documentation explaining how to configure and use application-level autoscaling policies. (#57756)
• Autoscaling documentation improvements. Updated serve autoscaling docs with clearer explanations and examples. (#57652)
• Performance flags documentation. Documented performance-related configuration flags for Ray Serve. (#57845)
• Metrics documentation fix. Corrected ray_serve_deployment_queued_queries metric name discrepancy in documentation. (#57629)
• AutoscalingContext import added to examples. Fixed missing import statement in custom autoscaling policy example. (#57876)
• App builder guide typo corrected. Fixed command syntax error in typed application builder example. (#57634)
• Celery filesystem broker note. Added warning about using filesystem as a broker in Celery workers. (#57686)
• Async inference alpha stage warning. Added notice that async inference is in alpha stage. (#57268)

🏗 Architecture refactoring:

• Autoscaling control moved to application state. Migrated autoscaling control loop from deployment state to application state, preparing for application-level autoscaling. (#57548)
• Async capability enum removed. Cleaned up unused async capability enum from codebase. (#57666)

Ray Serve/Data LLM

🎉 New Features:

• Updated vLLM to 0.11.0 and Nixl to 0.6.0 (#57201)
• Video processor support for multimodal pipelines (#56785)
• Enhanced callback API for engine customization (#57257)
• Unified and extended builder configuration for LLM deployments (#57724)

💫 Enhancements:

• Protocol-based typing improvements and cleaner inheritance structure (#57743)
• Better engine metrics enabled by default (#57615)
• Simplified NIXL dependency management in ray-llm images (#57706)
• Per-stage map kwargs for LLM processor preprocessing/postprocessing (#57826)
• Improved architecture documentation (#57830)
• Better code structure alignment with architectural design (#57889)
• Enhanced multimodal support with Deepseek compatibility (#56906)

🔨 Fixes:

• Fixed NIXL limitations with proper exception handling (#58159)
• Improved runai_streamer for vLLM 0.10.2+ integration (#56906)

📖 Documentation:

• Added comprehensive architecture documentation for Ray Serve LLM (#57830)
• Reorganized LLM documentation with improved navigation (#57787)
• Added benchmark page for performance reference (#57960)
• Converted quick-start guide to MyST Markdown (#57782)
• Better organization of Ray Serve LLM documentation (#57181?)

RLlib

🎉 New Features:

• Prometheus metrics support for selected RLlib components (#57932)
• Enhanced support for complex observations in SingleAgentEpisode (#57017)

💫 Enhancements:

• LINT improvements with enabled ruff imports for rllib/utils (#56737)
• Better type hints for learner_connector (#57673)
• Improved throughput metrics to avoid biasing (#57215)

🔨 Fixes:

• Fixed segment_tree.py edge case (#57599)
• Fixed small bug in type hints (#57673)

Ray Core

🎉 New Features:

• Enhanced Ray Direct Transport (RDT) with improved NIXL integration and garbage collection (#57671, #57603, #58159)
• Cgroups support improvements with better system resource management (#57776, #57864, #57731, #58017, #58028, #58064)
• Fault-tolerant RPC improvements for better distributed reliability (#57786, #57861)
• Exponential backoff for retryable gRPCs (#56568)

💫 Enhancements:

• Migrated from STATS to metric interface in RPC components (#57926)
• Improved histogram metrics midpoint calculation (#57948)
• Made FreeObjects non-fatal for better error handling (#57550)
• Enhanced ReleaseUnusedBundles fault tolerance (#57786)
• Made DrainRaylet and ShutdownRaylet fault tolerant (#57861)
• Better error handling for metric and event exporter agent (#57925)
• Improved raylet shutdown process and file organization (#57817)
• Reporter agent can now get PID via RPC to raylet (#57004)
• Enhanced ray.get thread safety (#57911)
• Configurable proto naming during event JSON conversion (#57705)
• Better handling of detached actor restarts (#57931)
• Improved lease rescheduling in local lease manager during node draining (#57834)

🔨 Fixes:

• Fixed "RayEventRecorder::StartExportingEvents() should be called only once" error (#57917)
• Fixed deadlock when cancelling stale requests on in-order actors (#57746)
• Fixed raylet shutdown races (#57198)
• Fixed log monitor seeking bug after log rotation (#56902)
• Deflaked multiple test suites for better CI reliability
• Fixed various memory and resource management issues
• Better handling of actor and task failures

📖 Documentation:

• Added JaxTrainer API overview to Ray docs (#57182)
• Fixed various typos and documentation issues
• Updated autoscaling and system configuration guides
• Enhanced SLURM documentation with symmetric-run support (#56775)

🏗 Architecture refactoring:

• Dashboard API server subprocesses moved into system cgroup (#57864)
• Driver moved into workers cgroup for better isolation (#57776)
• Improved worker-raylet interface separation (#57804)
• Better plasma store provider architecture

Dashboard

💫 Enhancements:

• Added percentage usage graphs for resources (#57549)
• Introduced sub-tabs with full Grafana dashboard embeds on Metrics tab (#57561)
• Added queued blocks to operator panels (#57739)
• Improved operator metrics logging for better clarity (#57702)
• Better filtering and display in job lists

🔨 Fixes:

• Fixed filtering issue in job list (#56946)
• Fixed incomplete card content on overview page (#56947)
• Filtered out ANSI escape codes from logs (#53370)

Autoscaler

🎉 New Features:

• KubeRay autoscaling support with top-level Resources and Labels fields (#57260)
• Bundle label selector support in request_resources SDK (#54843)
• Application Gateway for Containers as ingress for Ray clusters on Azure

💫 Enhancements:

• Azure improvements: Cleaning up extra resources (MSI, VNET, NSG) during cluster teardown (#57610)
• Updated defaults for Azure cluster templates (#57716)
• Better availability zone support for Azure node pools (#55532)
• Hello world release tests for Azure and GCE (#57597, #57695)
• Improved cluster resource state handling to fix over-provisioning (#57130)

🔨 Fixes:

• Fixed autoscaler state synchronization issues (#57010)
• Better handling of node state information (#57130)
• Improved timeout handling for patch requests (#56605)

Thank you to everyone who contributed to this release! Special thanks to all the contributors who helped make Ray 2.51.0 possible through bug fixes, features, documentation improvements, and testing efforts.

Ray Core: Fix deadlock when cancelling stale requests on in-order actors (#57746)

Release Highlights

Ray Data: This release offers many updates to Ray Data, including:

• The default shuffle strategy is now changed from sort-based to hash-based. This will result in much lower peak memory usage and improved shuffle performance for aggregations.
• We’ve added a new expression API enables predicate-based filtering, UDF transformations with with_column, and column aliasing for more powerful data transformations
• Ray Data LLM has a number of new enhancements for multimodal data pipelines, including multi-node tensor and pipeline parallelism support per replica and ability to share vLLM engines across processors.

Ray Core:

Alpha release of Ray Direct Transport (formerly GPU objects) - simply enable it by adding the tensor_transport parameter to the existing native Ray Core API. This keeps GPU data in GPU memory until a transfer is needed, avoiding expensive serialization and copies to and from the Ray object store. It uses efficient data transports such as collective communication libraries (GLOO or NCCL) or point-to-point RDMA (via NVIDIA’s NIXL) to transfer data directly between devices, including both CPUs and GPUs.

Ray Train:

Local mode support for multi-process training with torchrun, enhanced checkpoint management with new upload modes and validation functions

Ray Serve:

• Async Inference alpha release - New Ray Serve APIs for supporting long-running asynchronous inference tasks, such as for video or large document processing. Includes capabilities for using different message brokers, adapters like celery and DLQ.
• Support for replica ranks - Replica level ranks are added for supporting large-model inference use-cases such as wide Data Parallel and Expert Parallel setups.
• FastAPI factory pattern support - Enables using FastAPI plugins that are not serializable via cloudpickle.
• Throughput optimizations - Enable these using the RAY_SERVE_THROUGHPUT_OPTIMIZED environment variable.

RLLib: Add StepFailedRecreateEnv exception for users with unsatisfiable environments

Ray Serve/Data LLM:

Improvements to multi node serving, loading models from remote storages, and sharing resources for efficiency (fractional gpus, sharing gpus on a data pipeline with shared stages)

Ray Libraries

Ray Data

🎉 New Features:

• Expression and Filtering API: New expression API enables predicate-based filtering, UDF transformations with with_column, and column aliasing for more powerful data transformations (#56716, #56313, #56550, #55915, #55788, #56193, #56596)
• Added support for projection pushdown into Parquet reads (#56500)
• New download expression enables efficient loading of data from columns containing URIs with improved performance and error handling (#55824, #56462, #56294, #56852, #57146)
• New explain() API provides insights into dataset execution plans (#55482)
• Added streaming_train_test_split to avoid materialization for train/test splits (#56803)
• Ray Data LLM:
  • Enabled multi-node tensor and pipeline parallelism for LLM processing (#56779)
  • Added chat_template_kwargs parameter for customizing chat templates (#56490)
  • Added support for OpenAI's nested image URL format in multimodal pipelines (#56584)
  • vLLM engines can now be shared across sequential processors for better resource utilization (#55179)
• Enhanced Dataset.stats() output with input/output row counts per operator (#56040)
• Added new metrics for task duration, inputs per task, and output blocks (#56958, #56379)
• Time to first batch metric for better iteration performance monitoring (#55758)
• Added type-specific aggregators for numerical, categorical, and vector columns (#56610)
• Added fine-grained concurrency controls with max_task_concurrency and resource allocation options (#56370, #56381)

💫 Enhancements:

• Join and shuffle improvements:
  • Default shuffle strategy changed from sort-based to hash-based for better performance (#55510)
  • Improved groupby performance with sort-shuffle pull-based approach (#57014)
  • Improved join operations with new abstractions (#57022, #56945, #55759)
• Tensor type handling improvements:
  • Improved compatibility between PyArrow native types, extension types, and pandas Arrow dtypes (#57566, #57176, #57057)
  • Joins now supported with list/tensor non-key columns (#5648)
  • Enhanced support for variable-shaped tensor arrays with different dimensions (#57240, #56918, #56457)
  • Added serialization/deserialization for PyArrow Extension Arrays (#51972)
• Removing Parquet metadata fetching in ParquetDatasource (#56105)
• Resource requirements (num_cpus/gpus, memory) are now top-level parameters in most APIs for easier configuration (#56419)
• zip() operator now supports combining multiple datasets, not just pairs (#56524)
• Concurrency parameter now accepts tuples for more flexible configuration (#55867)
• Write operations now use iterators instead of accumulating blocks in memory (#57108)
• Reduced memory usage for OneHotEncoder (#56565)
• Reduced memory usage for schema unification (#55880)
• Eliminated unnecessary block copying and double execution of arrow conversions (#56569, #56793)
• Improved Parquet encoding ratio estimation (#56268)
• Enabled per-block limiting for Limit operator (#55239)
• Optimized schema handling with deduplication and removed unnecessary unification (#55854, #55926)
• Improved issue detection with event emission instead of just logs (#55717)
• Better metric organization and external queue metric handling (#55495, #56604)
• New backpressure policy based on downstream processing capacity (#55463)

🔨 Fixes:

• Fixed streaming executor to properly drain output queues (#56941)
• Improved resource management and reservation for operators (#56319, #57123)
• Fixed retry logic for hash shuffle operations (#57575)
• Fix split_blocks produce empty blocks (#57085)
• Initialize datacontext after setting src_fn_name in actor worker (#57117)
• Fix mongo datasource collStats invocation (#57027)
• Fixing empty projection handling in ParquetDataSource (#56299)
• Fix UnboundLocalError when read_parquet with columns and no partitioning (#55820)
• Fix high memory usage with FileBasedDatasource & ParquetDatasource when using a large number of files (#55978)
• [llm] Fixed LLM processor deployment with Ray Serve (#57061)
• [llm] Fixed multimodal image extraction when system prompts are absent (#56435)
• Ignore metadata for pandas block (#56402)
• Remove metadata for hashing + truncate warning logs (#56093)

📖 Documentation:

• Error in ray.data.groupby example in docs. (#57036)
• Update on ray.data.Dataset.map() type hints. (#52455)
• Small typo fix. (#56560)
• Fix a typo. (#56587)
• Fix documentation for new execution options resource limits assignment. (#56051)
• Fix broken code snippets in user guides. (#55519)
• Add Autoscaling Config for Context docs. (#55712)
• Make object store tuning tips consistent with other pages. (#56705)
• New example of how to perform batch inference with embedding models (#56027)

Ray Train

🎉 New Features:

• Local mode support for Ray Train V2
  • Add local mode support to Ray Train v2 (num_workers=0). (#55487)
  • Add PyTorch local mode support for multi-process training with torchrun. (#56218)
• Async checkpoint and validation for Ray Train
  • Add checkpoint_upload_mode to ray.train.report. (#55637)
  • Add checkpoint_upload_function to ray.train.report. (#56208)
  • Add validate_function and validate_config to ray.train.report. (#56360)
  • Add ray.train.get_all_reported_checkpoints method. (#54555)

💫 Enhancements:

• Ray Train V2 Migration
  • Implement BaseWorkerGroup for V1/V2 compatibility. (#57151)
• Train Controller is always actor + fix tune integration to enable this. (#55556)
• Refactor AcceleratorSetupCallback to use before_init_train_context. (#56509)
• Move collective implementations to train_fn_utils. (#55689)
• Ray Train Framework support enhancements
  • Add hf trainer support for dictionary of datasets. (#56484)
  • Add usage tag key for JaxTrainer. (#55887)
• Add Torch process group shutdown timeout. (#56182)
• Ray Train disables blocking get inside async warning. (#56757)
• ThreadRunner captures exceptions from nested threads. (#55756)
• Abort reconciliation thread catches ray.util.state.get_actor exception. (#56600)
• Ray Data Integration
  • Minor rework of get_dataset_shard. (#55825)
  • Create a deepcopy of the data context on the split coordinator process. (#56211)
  • Enable debug logging; fix default actor_locality_enabled. (#56632)
• Refactor call_with_retry into shared library and use it to retry checkpoint upload. (#56608)
• Remove Placement Group on Train Run Abort. (#56011)

🔨 Fixes:

• Fix LightGBM v2 callbacks for Tune only usage. (#57042)
• Ignore tensorflow test for py312. (#56244)
• Revising test_jax_trainer flaky test. (#56854)
• Fix test_jax_trainer imports. (#55799)
• Fix test_jax_trainer::test_minimal_multihost Flaky Test. (#56548)
• Disable drop_last flag to fix division by zero in torch dataloader baselines. (#56395)
• Preload a subset of modules for torch dataloader forkserver multiprocessing. (#56343)

📖 Documentation:

• Add checkpoint_upload_mode to checkpoint docs. (#56860)
• Add get_all_reported_checkpoints and ReportedCheckpoint to API docs. (#56174)
• Fix typo for Instantiating in ray train doc. (#55826)

🏗 Architecture refactoring:

• Release tests for ray train local mode. (#56862)
• Migrate tune_rllib_connect_test & tune_cloud_long_running_cloud_storage to ray train v2. (#56844)
• Add v2 multinode persistence release test. (#56856)
• Attach a quick checkpoint when reporting metrics. (#56718)
• Upgrade tune_torch_benchmark to v2. (#56804)
• Move tune_with_frequent_pausing to Ray Train v2 and tune_tests folder. (#56799)
• Migrate xgboost/lgbm benchmarks to train V2. (#56792)

Ray Tune

🎉 New Features:

• Trigger Checkpointing via Trial / Tuner Callback. (#55527)

💫 Enhancements:

• Improve _PBTTrialState for dev/debugging usage. (#56890)
• Enable Train V2 in Tune unit tests and examples. (#56816)
• Enable Train v2 in doc examples. (#56820)
• Reintroduce keras tune callback. (#57121)

🔨 Fixes:

• Increase tune checkpoint test latency threshold. (#56251)
• Remove a bunch of low-signal/redundant train/air/tune tests. (#56477)
• Remove tune_air_oom test. (#57089)

Ray Serve

🎉 New Features:

• Add tests and DLQ business logic for async inference. (#55608)
• Foundation work for aggregating metrics on controller. (#55568)
• Include custom metrics method and report to controller. (#56005)
• Add post scaling api. (#56135)
• Introduce deployment rank manager. (#55729)
• Integrated deployment ranks with deployment state. (#55829)
• Add rank and world size in replica context. (#55827)
• Added ssl to ray serve. (#55228)
• Custom parameter for downscaling to zero. (#56573)
• Add optional APIType filter to /api/serve/applications/ endpoint. (#56458)
• Make deployment retry configurable. (#56530)

💫 Enhancements:

• Aggregate autoscaling metrics on controller. (#56306)
• Update metrics_utils for future global metrics aggregation in controller. (#55568)
• Use deployment method in access logs for replicas. (#56829)
• Cache router metrics. (#55897)
• Allow same event loop handle shutdown from sync context. (#55551)
• Additional deps to start with prometheus. (#57155)
• Require prefix RAY_SERVE_ for env vars + value verification. (#55864)
• Record queued metrics on timeseries. (#57024)
• Add throughput opt env var for serve. (#55804)
• Fix None pending Request. (#54775)
• Omit unnecessary newlines in the config generated by serve build app:app. (#56609)
• Expose actor name for target group api. (#56738)

🔨 Fixes:

• Fix proxy lua dependency in dockerfile. (#57221)
• Fix non thread safe asyncio task creation in router. (#56124)
• Fix throughput optimized benchmarks. (#56173)
• Move ingress validation for multiple fastapi deployment into client. (#56706)
• Explicitly close choose_replicas_with_backoff async generator. (#56357)
• Fix buffered logging reusing request context. (#56094)
• Use default gc frequency for proxy. (#56511)
• Fixing deployment scoped custom autoscaling. (#56192)

📖 Documentation:

• Stable links for Ray serve. (#56241)
• Add document for using fastapi factory pattern in serve. (#56607)
• Add documentation for async inference (#56453)

🏗 Architecture refactoring:

• Add microbenchmark for throughput optimized configuration. (#55900)
• Only checkpoint controller state when it is confirmed that target state has changed (#55848)
• Proxy Actor Interface. (#56288)
• Allow ProxyActor to return true/false for health check. (#56660)

Ray Serve/Data LLM

🎉 New Features:

• Score API Integration for Serve LLM. (#55914)
• Add start/stop_profile method to LLMServer. (#55920)
• Add prefix cache hit rate to Serve LLM dashboard. (#55675)
• Configure aggregation interval for dashboard. (#56591)

💫 Enhancements:

• Bump vLLM to 0.10.2. (#56535)
• Vllm bump -> 0.10.1.1. (#56099)
• Refactor: Improve Deployment Builder Ergonomics and Code Organization. (#57181)
• Fix build_llm_processor for ServeDeploymentProcessor. (#57061)
• Allow setting data_parallel_size=1 in engine_kwargs. (#55750)
• Allow tuple for concurrency arg. (#55867)
• Fix multimodal image extraction when no system prompt is present. (#56435)
• Support azure and abfss in LLM config. (#56441)
• Support custom s3 endpoint when downloading models from remote. (#55458)
• Skip safetensor file downloads for runai streamer mode. (#55662)
• Support colocating local DP ranks in DPRankAssigner. (#55720)
• Adjust LLM engine timing logic. (#55595)
• Fixed DP DSV3 issues. (#55802)
• Gracefully return timeouts as HTTPException. (#56264)
• Remove upstreamed workarounds 1/3. (#54512)

🔨 Fixes:

• Changed LMCache dependency to use 0.3.3 to avoid regressions in the release test. (#56104)
• Fix doc test for Working with LLMs guide. (#55917)
• Fix sglang byod on release. (#55885)

📖 Documentation:

• Add gpt oss deployment example. (#56400)
• Add serve llm example to index page + other minor fix. (#56788)
• Example serve llm deployment. (#55819)
• Fix serve llm examples. (#56382)
• Docs: serve llm deployment examples refinement. (#56287)
• Add example of serving a VLLM model on fractional gpu. (#57197)
• Add main pytest code snippet to those tests that were missing it. (#57167)

RLlib

🎉 New Features:

• Add StepFailedRecreateEnv exception. (#55146)

💫 Enhancements:

• Add tags to envrunner calls, count in flight requests in ActorManager. (#56930, #56953)
• Add spaces in case only offline data is used. (#56141)
• Add Footsies environment and tests. (#55041)

🔨 Fixes:

• Fix failing env step in MultiAgentEnvRunner. (#55567)
• Fix Metrics/Stats lifetime count and throughput measurement for async remote actors. (#56047)
• Fixes Implementation of Shared Encoder. (#54571)
• Fix MetricsLogger/Stats throughput bugs. (#55696)

📖 Documentation:

• [RLlib] [DOC] Fix documentation typos and grammatical issues in RLlib docs (#56130)
• Update rllib-env.rst - typo. (#56140)
• Fixing typo in the RLlib documentation. (#55752)
• Fix formatting of class references. (#55764)

🏗 Architecture refactoring:

• Remove checkpoint release tests. (#57105)
• Remove long_running_apex test. (#57097)
• LINT: Enable ruff imports for multiple directories in rllib. (#56736)
• Upgrade g3 to g4 machine for aws release test. (#56248)

Ray Core

🎉 New Features:

• Alpha release of Ray Direct Transport
  • Support ray.put() and ray.get() with nixl in gpu objects. (#56146)
  • Support using ray.get with nixl to retrieve data from GPU object refs created by remote tasks. (#56559)
  • Support tensor transfer from outside owners of actors. (#56485)
  • Automatically enable tensor transport for the actor if any method specifies one. (#55324)
  • Support cpu tensor transfer with NIXL in GPU Objects. (#55793)
  • Handle multiple transfers of the same object to an actor. (#55628)
  • Support NIXL as tensor transport backend. (#54459)
  • Add a user-facing call to wait for tensor to be freed. (#55076)
  • Always write to GPUObjectStore to avoid _get_tensor_meta() from hanging indefinitely. (#55433)
  • Add warning when GPU object refs passed back to the same actor. (#55639)
  • Avoid triggering a KeyError by the GPU object GC callback for intra-actor communication. (#54556)
• Enable autoscaler v2 on clusters launched by the cluster launcher. (#55865)
• Ray Symmetric Run Script and ray symmetric-run command. (#55111, #56497)

💫 Enhancements:

• Ray Event Export
  • GCS AddEvent support. (#55528)
  • Actor event: add proto schema. (#56221)
  • Node event: add proto schema and send node events to the aggregator. (#56031, #56426)
  • Job event: add schema for driver job event and send job events to the aggregator. (#55032, #55213)
  • Emit actor events to Event aggregator. (#56617)
  • Export node event by default. (#56810)
• Security
  • Bind ray internal servers to the specified node ip instead of 0.0.0.0. (#55178, #55210, #55298, #55484)
  • Bind dashboard agent http server to localhost in addition to the node ip. (#55910)
  • Bind dashboard agent grpc to specified ip instead of 0.0.0.0. (#55732)
  • Bind runtime env agent and dashboard agent http server to specified ip instead of 0.0.0.0. (#55431)
• RPC network fault tolerance
  • Making core worker pub sub RPCs fault tolerant. (#56436)
  • Make RequestWorkerLease RPC Fault Tolerant. (#56191)
  • Making ReturnWorkerLease Idempotent. (#56073)
  • Making CancelWorkerLease RPC Fault Tolerant. (#56195)
  • Make Free Objects RPC Fault Tolerant. (#56293)
  • Make PinObjectIDs RPC Fault Tolerant. (#56443)
  • Make Unsubscribe Idempotent. (#57546)
  • Core Worker GetObjStatus GRPC Fault Tolerance. (#54567)
• Not overriding accelerator id env vars when num_accelerators is 0 or not set. (#54928)
• Migrate metric collection from opencensus to opentelemetry. (#53098, #53740)
• Add per worker process group and deprecate process subreaper in favor of cleanup using process group. (#56476)
• Add node_id validation in NodeAffinitySchedulingStrategy. (#56708)
• Add io_context metrics to gcs and raylet. (#55762)
• Modify RedisDelKeyPrefixSync to use the Redis SCAN command instead of KEYS. (#56907)
• Add error_type to job failures. (#55578)
• Add PID to structured logs for tasks and actors. (#55176)
• Log actor name when warning about excess queueing. (#57124)
• Output the error log on the driver side if the failed task will still retry. (#56472)
• Prometheus http service discovery API. (#55656)
• Add node ip in runtime env error message to improve debug observability. (#56837)
• Fallback unserializable exceptions to their string representation. (#55476)
• Introduce new exception type for un-pickleable exceptions. (#55878)
• Improve docs for custom serialization for exceptions + add test. (#56156)
• Add a warning when returning an object w/ num_returns=0. (#56213)
• Adding ability to specify availability zones for ray cluster node pools on Azure. (#55532)
• Query for supported Microsoft.Network/virtualNetworks API versions instead of relying on resource_client.DEFAULT_API_VERSION. (#54874)
• Loosen Ray self-dependency check to allow matching versions. (#57019)
• Add support for pip_install_options for pip. (#53551)
• Proper typing for ObjectRef. (#55566)

🔨 Fixes:

• Use subscription id from azure profile if not provided in config during AzureNodeProvider init. (#56640)
• Always create standard public IP addresses (basic sku is deprecated). (#57131)
• Fix: bug with config key pairs when launching worker nodes. (#57107)
• If azure cluster launcher keypair doesnt exist create one automatically + doc typo fix. (#54596)
• Fix "objects_valid" for the case that multiple instances of the same task are storing returns. (#54904)
• Fix objects_valid check failure with except from BaseException. (#55602)
• Preserve err type in case of task cancellation due to actor death. (#57538)
• Fix checking for uv existence during ray_runtime setup. (#54141)
• Prevent sending SIGTERM after calling Worker::MarkDead. (#54377)
• Fixed the bug where the head was unable to submit tasks after redis is turned on. (#54267)
• Fix possible race by checking node cache status instead of just subscription. (#54745)
• Fix get actor timeout multiplier. (#54525)
• Use a temporary file to share default worker path in runtime env. (#53653)
• Fix check fail when task buffer periodical runner runs before RayEvent is initialized. (#55249)
• Patch grpc with RAY_num_grpc_threads to control grpc thread count. (#54988)
• Fix HandleRefRemoved thread safety. (#56445)
• Fix error handling for plasma put errors. (#56070)
• Fix batching logic in CoreWorkerPlasmaStoreProvider::Get. (#56041)
• Fix RAY_CHECK failure during shutdown due to plasma store race condition. (#55367)
• Fix autoscaler RAY_CHECK when GcsAutoscalerStateManager is out of sync with NodeManager. (#57010)
• Fix bug where inflight requests are not taken into account by retryable. (#57142)
• A timeout should be set when submitting patch requests for autoscaler. (#56605)
• Fix the bug in memray regarding the default configuration of -o {output_file_path}. (#56732)
• Fixed the issue of RemoveActorNameFromRegistry being called repeatedly. (#54955)
• Fixed an issue where the command executed when use_podman=true and run_env=None was not prefixed with podman exec. (#56619)
• Fix data race when using async gpu to gpu transfer. (#57112)
• Retry + Make FreeActorObject idempotent. (#56447)
• Fix check crash on gpu obj free if driver knows actor is dead. (#56404)
• Handle system errors with a background monitor thread. (#56513)
• Fix GPU metrics. (#56009)
• Don't disconnect worker client on OBOD unless the worker is dead. (#57185)
• Prevent stale GET request being registered if its lease was cleared. (#56766)
• Drop messages received after ClientConnection::Close. (#56240)
• Fix cancel race that leads to RAY_CHECK it->second.submitted_task_ref_count > 0. (#56123)
• Reorder asyncio actor shutdown to terminate asyncio thread first. (#56827)
• Fix actor import error message for async actors. (#55722)
• Allow task manager access with submitter mutex + unify retry. (#56216)
• Fix bug in restore_from_path such that connector states are also restored on remote EnvRunners. (#54672)
• Fix S3 access issue in AKS. (#56358)
• Add S3 public bucket fallback to handle NoCredentialsError. (#56334)
• Fix ABFSS (Azure Blob File System Secure) protocol support problems during E2E test. (#56188)
• Ray cluster commands (up, attach, status, etc) updates to work on Windows. (#54982)
• Update cluster scheduler to handle label selector hard node id constraint. (#56235)

📖 Documentation:

• Added guide on using type hints with Ray Core. (#55013)
• Lifecycle of a task. (#55496)
• Add OSS Document for Task Events. (#56203)
• Fix Missing Events Issue in Task Events. (#55916)
• Add docs for asyncio and object mutability. (#56790)
• Update getting started and set up document for ray on vsphere. (#56954)
• Docfix - rst annotation showing up in render. (#57104)
• Add threading requirement to NodeProvider interface. (#56349)
• Add guidance for matching Ray and Python versions with uv envs. (#56597)
• Fix documentation typos, grammar, and terminology inconsistencies. (#56066, #56067, #56068, #56069, #56128, #56129, #56130, #56131, #56132, #56272, #56273, #56274, #56275, #56277, #56278, #56279)
• Update SLURM docs with symmetric-run. (#56775)
• Update Kueue integration documentation to include RayService & RayCluster support. (#56781)
• Application Gateway for Containers as ingress to access Ray Cluster. (#56574)
• Update DLAMI Information in aws.md. (#55702)

Dashboard

💫 Enhancements:

• Use pynvml for GPU metrics. (#56000)
• Default dashboard usability improvements. (#55620)
• Make Ray Train Dashboard Panel Ids Static. (#55559)
• Small fixes to Metrics Tab for kube-ray clusters. (#57149)
• Add metadata to indicate full dashboard embedding is supported. (#56077)
• Use ray node id instead of ip for profilinglink. (#55439)
• Fix grafana dashboard generation bug. (#56346)
• Catch OSError when detecting the GPU. (#56158)

🔨 Fixes:

• Removed references to a deleted Data metrics panel. (#55478)
• Fix typo in memory_utils and adjust display formatting for clarity. (#56217)

Ray Images

🎉 New Features:

• Add support for building and publishing ray-extra images. (#56543)
• Add ray-llm and ray-ml extra images. (#56800)
• Build ray-extra images for aarch64. (#56818)
• Add slim image to the image build matrix. (#55723)

💫 Enhancements:

• Add haproxy binary, for ray serve use. (#56845)
• Add ~/.local/bin to PATH in slim image. (#56920)
• Remove slim's dependency on normal bases. (#56544)
• Add label for ray version and commit. (#56493)
• Refactor apt package installation. (#55701)
• Allow using explicit base type. (#56545)
• Add extra-test stage in image building. (#55725)
• Add test rules for image building files. (#56554)
• Add ray-llm image type check. (#56542)
• Unify label and tag conventions. (#56189)
• GKE GPU compat paths: PATH, LD_LIBRARY_PATH (temporarily). (#55569)
• Stop publishing ray-ml images. (#57070)
• Stop building and releasing x86 osx wheels. (#57077)

📖 Documentation:

• Update latest Docker dependencies for 2.49.0 release. (#55966)
• Update latest Docker dependencies for 2.49.2 release. (#56760)

Wheels and images

💫 Enhancements:

• Use bazel run to generate files required for the wheel and testing (#55957, #56527, #56969, #56004, #55928)
• Ban click 8.3.0. (#56789)
• Upgrade protobuf to v4. (#54496)
• Add adlfs[abfs] into image (#56084)
• Upgrade boto3 to 1.29.x. (#56363)
• Upgrading orjson to 3.9.15. (#55972)
• Update spdlog to 15.3. (#56711)

Thanks!

Thank you to everyone who contributed to this release!

@alexeykudinkin, @richardliaw, @nrghosh, @ljstrnadiii, @Daraan, @kouroshHakha, @Bye-legumes, @kamil-kaczmarek, @jugalshah291, @sampan-s-nayak, @jjyao, @Evelynn-V, @gangsf, @omatthew98, @TimothySeah, @kshanmol, @goutamvenkat-anyscale, @axreldable, @jiangwu300, @simonsays1980, @400Ping, @JasonLi1909, @chuang0221, @weiliango, @Myasuka, @win5923, @liulehui, @khluu, @ok-scale, @eicherseiji, @tianyi-ge, @MengjinYan, @kevin85421, @Yevet, @orangeQWJ, @vie-serendipity, @edoakes, @wyhong3103, @israbbani, @vickytsang, @HassamSheikh, @acrewdson, @czgdp1807, @daiping8, @carolynwang, @thc1006, @jeffreyjeffreywang, @Stack-Attack, @Catch-Bull, @elliot-barn, @Levi080513, @BestVIncent, @dragongu, @jmajety-dev, @jcarlson212, @tohtana, @abrarsheikh, @crypdick, @Yicheng-Lu-llll, @ZacAttack, @justinvyu, @lk-chen, @alanwguo, @mcoder6425, @my-vegetable-has-exploded, @yancanmao, @arcyleung, @rjpower, @codope, @harshit-anyscale, @dayshah, @stephanie-wang, @KaisennHu, @ryanaoleary, @saihaj, @mattip, @rueian, @Kunchd, @pavitrabhalla, @owenowenisme, @Aydin-ab, @gvspraveen, @minerharry, @JackGammack, @jpatra72, @coqian, @zcin, @dstrodtman, @aslonnie, @ahao-anyscale, @GuyStone, @iamjustinhsu, @seanlaii, @ruisearch42, @akyang-anyscale, @ArturNiederfahrenhorst, @bveeramani, @OneSizeFitsQuorum, @xinyuangui2, @sb-hakunamatata, @22quinn, @Sparks0219, @sven1977, @snehachhabria, @dioptre, @nadongjun, @eric-higgins-ai, @marosset, @MatthewCWeston, @pcmoritz, @can-anyscale, @pimdh, @roshankathawate, @matthewdeng, @martinbomio, @GokuMohandas, @alimaazamat, @ali-corpo, @landscapepainter, @Qiaolin-Yu, @vaishdho1, @avigyabb, @srinathk10, @tannerdwood

There is no difference between 2.49.2 and 2.49.1, though we needed a patch version for other out of band reasons. To fill the awkward blankness, here is a haiku about Ray:

Summit drawing near Ray advances, step by step Scaling without end

• Ray Dashboard: Fix issue where GPU metrics are missing (#56006)
• Ray Data: Fixed regression in handling very large schemas (#56058)

Release Highlights

Ray Data:

• We’ve implemented a variety of performance enhancements, including improved actor/node autoscaling with budget-aware decisions; faster/more accurate shuffle accounting; reduced Parquet metadata footprint; and out-of-order execution for higher throughput.
• We’ve also implemented anti/semi joins, stratified train_test_split, and added Snowflake connectors.

Ray Core:

• Performance/robustness cleanups around GCS publish path and raylet internals; simpler OpenTelemetry flagging; new user-facing API to wait for GPU tensor free; plus assorted test/infra tidy-ups

Ray Train:

• We’ve introduced a new JaxTrainer with SPMD support for TPUs.

Ray Serve:

• Custom Autoscaling per Deployment Serve now supports user-defined autoscaling policies via AutoscalingContext and AutoscalingPolicy, enabling fine-grained scaling logic at the deployment level. This is part of a large effort where we are adding support for autoscaling based on custom metrics in Serve, see this RFC for more details.
• Async Inference (Initial Support): Ray Serve introduces asynchronous inference execution, laying the foundation for better throughput and latency in async workloads. Please see this RFC for more details.
• Major Performance Gains: This version of ray serve brings double digit % performance improvements both in throughput and latency. See release notes for more details.

Ray Serve/Data LLM:

• We’ve refactored Ray Serve LLM to be fully compatible with the default vllm serve and also now supports vLLM=0.10.
• We’ve added a prefix cache-aware router with PrefixCacheAffinityRouter for optimized cache utilization; dynamic cache management via reset prefix cache remote methods; enhanced LMCacheConnectorV1 with kv_transfer_config support.

Ray Libraries

Ray Data

🎉 New Features:

• Wrapped batch indices in a BatchMetadata object to make per-batch metadata explicit. (#55643)
• Added support for Anti/Semi Join types. (#55272)
• Introduced an Issue Detection Framework. (#55155)
• Added an option to enable out-of-order execution for better performance. (#54504)
• Introduced a StreamingSplit logical operator for DAG rewrite. (#54994)
• Added a stratify parameter to train_test_split. (#54624)
• Added Snowflake connectors. (#51429)
• Updated Hudi integration to support incremental query. (#54301)
• Added an Actor location tracker. (#54590)
• Added BundleQueue.has_next. (#54710)
• Made DEFAULT_OBJECT_STORE_MEMORY_LIMIT_FRACTION configurable. (#54873)
• Added Expression support & a with_columns API. (#54322)
• Allocate GPU resources in ResourceManager. (#54445)

💫 Enhancements:

• Decoupled actor and node autoscaling; autoscaling now also considers budget. (#55673, #54902)
• Faster hash-shuffle resource usage calculation; more accurate shuffle progress totals. (#55503, #55543)
• Reduced Parquet metadata storage usage. (#54821)
• Export API improvements: refresh dataset/operator state, sanitize metadata, and truncate exported metadata. (#55355, #55379, #55216, #54623)
• Metrics & observability: task metric improvements, external-buffer block-count metric, row-based metrics, clearer operator names in logs, single debug log when aggregators are ready. (#55429, #55022, #54693, #52949, #54483)
• Dashboard: added “Max Bytes to Read” panel/budget, panels for blocks-per-task and bytes-per-block, and streaming executor duration. (#55024, #55020, #54614)
• Planner/execution & infra cleanups: ExecutionResources and StatsManager cleanup, planner interface refactor, node trackers init, removed ray.get in _MapWorker ctor, removed target_shuffle_max_block_size. (#54694, #55400, #55018, #54665, #54734, #55158)
• Behavior/interop tweaks: map_batches defaults to row_modification=False and avoids pushing past limit; limited operator pushdown; prefetch for PandasJSONDatasource; use cloudpickle for Arrow tensor extension ser/des; bumped Arrow to 21.0; schema warning tone change. (#54992, #54457, #54667, #54831, #55426, #54630)
• Removed randomize-blocks reorder rule for more stable behavior. (#55278)

🔨 Fixes:

• AutoscalingActorPool now properly downscales after execution. (#55565)
• StatsManager handles StatsActor loss on disconnect. (#55163)
• Handle missing chunks key when Databricks UC query returns zero rows. (#54526)
• Handle empty fragments in sampling when num_row_groups=0. (#54822)
• Restored handling of PyExtensionType to keep compatibility with previously written datasets. (#55498)
• Prevent negative resource budget when concurrency exceeds the global limit; fixed resource-manager log calculation. (#54986, #54878)
• Default write_parquet warning removed; handled unhashable types in OneHotEncoding. (#54864, #54863)
• Overwrite mode now maps to the correct Arrow behavior for parallel writes. (#55118)
• Added back from_daft Arrow-version checks. (#54907)
• Pandas chained in-place assignment warning resolved. (#54486)
• Test stability/infra: fixed flaky tests, adjusted bounds and sizes, added additional release tests/chaos variants for image workloads, increased join test size, adjusted sorting release test to produce 1 GB blocks. (#55485, #55489, #54806, #55120, #54716, #55402, #54971)

📖 Documentation:

• Added a user guide for aggregations. (#53568)
• Added a code snippet in docs for partitioned writes. (#55002)
• Updated links to Lance documentation. (#54836)

Ray Train

🎉 New Features:

• Introduced JaxTrainer with SPMD support on TPUs (#55207)

💫 Enhancements:

• ray.train.get_dataset_shard now lazily configures dataset sharding for better startup behavior (#55230)
• Clearer worker error logging (#55222)
• Fail fast when placement group requirements can never be satisfied (#54402)
• New ControllerError surfaced and handled via failure policy for improved resiliency (#54801, #54833)
• TrainStateActor periodically checks controller health and aborts when necessary (#53818)

🔨 Fixes:

• Resolve circular import in ray.train.v2.lightning.lightning_utils (#55668)
• Fix XGBoost v2 callback behavior (#54787)
• Suppress a spurious type error (#50994)
• Reduce test flakiness: remove randomness and bump a data-integration test size (#55315, #55633)

📖 Documentation:

• New LightGBMTrainer user guide (#54492)
• Fix code-snippet syntax highlighting (#54909)
• Minor correction in experiment-tracking guide comment (#54605)

🏗 Architecture refactoring:

• Public Train APIs routed through TrainFnUtils for consistency (#55226)
• LoggingManager utility for Train logging (#55121)
• Convert DEFAULT variables from strings to bools (#55581)

Ray Tune

🎉 New Features:

• Add video FPS support to WandbLoggerCallback (#53638)

💫 Enhancements:

• Typing: reset_config now explicitly returns bool (#54581)
• CheckpointManager supports recording scoring metric only (#54642)

🔨 Fixes:

• Fix XGBoost v2 callback integration (#54787)
• Correct type for RunConfig.progress_reporter (#48439)

📖 Documentation:

• Minor fixes (#55125, #54942)

Ray Serve

🎉 New Features:

• Async inference support in Ray Serve (initial phase). Provides basic asynchronous inference execution, with follow-up work planned for failed/unprocessed queues and additional tests. #54824
• Per-deployment custom autoscaling controls. Introduces AutoscalingContext and AutoscalingPolicy classes, enabling user-defined autoscaling strategies at the deployment level. #55253
• Same event loop router. Adds option to run the Serve router in the same event loop as the proxy, yielding ~17% throughput improvement. #55030

💫 Enhancements:

• Async get_current_servable_instance(). Converts the FastAPI dependency to async def, removing threadpool overhead and boosting performance: 35% higher RPS and reduced latency. #55457
• Access log optimization. Cached contexts in request path logging improved request throughput by ~16% with lower average latency. #55166
• Batching improvements. Default batch wait timeout increased from 0.0s to 0.01s (10ms) to enable meaningful batching. #55126
• HTTP receive refactor. Cleaned up handling of replica-side HTTP receive tasks. #54543 / #54565
• Configurable replica router backoff. Added knobs for retry/backoff control when routing to replicas. #54723
• Autoscaling ergonomics. Marked per-deployment autoscaling metrics push interval config as deprecated for consistency. #55102
• Health check & env var safety. Introduced warnings for invalid/zero/negative environment variable values, with migration path planned for Ray 2.50.0. #55464, #54944
• Improved CLI UX. serve config now prints No configuration was found. instead of an empty string. #54767

🔨 Fixes:

• Removed brittle ray._private dependency usage. #55659
• HTTP route test fixes. Migrated to get_application_url() to avoid hardcoded URLs, reducing flakiness on Windows. #55623, #54974, #54924, #54911, #54704, #54903, #54882, #54877, #54631, #53933
• Semaphore bug fix. Corrected race where more workers than allowed could acquire the semaphore. #55147
• LongPollClient cancellation. Prevented spurious cancellation of listen_for_change. #54832
• Backpressure error code. gRPC now returns RESOURCE_EXHAUSTED instead of UNAVAILABLE on overload. #54537
• Logging improvements. Added request IDs to proxy access logs; avoided duplicate shutdown log lines. #54657, #54534
• Test stability. Various waits, deflakes, and sync fixes across Serve tests. #54794, #54522, #54585

📖 Documentation:

• Unexpected queuing behavior. Documented quirks in handle request queuing. #54542

🏗 Architecture refactoring:

• Router/handle internals refactored for clarity and future feature expansion. #55635
• Model composition benchmarks. Added benchmarking to track performance of common composition patterns. #55549
• Constants refactor. Utility functions moved out of constants.py for better readability and stricter env var validation. #54944, #55464
• Ray internals migration. Moved usage, ray_option_utils, and selected constants from _private to _common. #54915, #54578

Ray Serve/Data LLM

🎉 New Features:

• Prefix cache-aware router with PrefixCacheAffinityRouter for optimized cache utilization. (#55218, #55588)
• Reset prefix cache remote method for dynamic cache management. (#55658)
• LMCacheConnectorV1 support for kv_transfer_config to enhance key-value transfer configurations. (#54579)
• LLMServer and LLMEngine major refactor for 100% vLLM serve frontend compatibility. (#54554)

💫 Enhancements:

• vLLM engine upgrade to version 0.10.0 with improved performance and compatibility. (#55067)
• Enhanced error handling for invalid model_id parameters with clearer error messages. (#55589)
• Improved telemetry handling with better race condition management for push operations. (#55558)
• Optimized deployment defaults with better configuration values to prevent bottlenecks. (#54696)
• LoRA workflow improvements with refactored downloading and utility functions. (#54946)
• LLMServer refactor to synchronous initialization for better reliability. (#54835)
• Mistral tokenizer support for tekken tokenizer compatibility. (#54666)
• Smart batching logic that skips batching when batch_interval_ms == 0. (#54751)
• Dashboard enhancements with improved LLM metrics and monitoring capabilities. (#54797)

🔨 Fixes:

• Pyright linting corrections for Ray Serve LLM examples. (#55284)
• Test stability improvements for DeepSeek model and vLLM engine processor tests. (#55401, #55120)
• Serialization fixes for ChatCompletionRequest tool_calls ValidatorIterator objects. (#55538)

📖 Documentation:

• Prefix cache router documentation with comprehensive usage examples. (#55218)
• Multi-LoRA documentation improvements with clearer setup instructions. (#54788)
• STRICT_PACK strategy FAQ documentation explaining data.llm packing behavior. (#55505)

🏗 Architecture refactoring:

• Docker image optimizations with UCX and NCCL updates, plus GKE GPU operator compatibility paths. (#54598, #55206)

RLlib

🎉 New Features:

• Implemented Implicit Q-Learning (IQL). (#55304, #55422)
• DreamerV3 is now available in PyTorch. (#45463, #55140)
• Discrete actions support for SAC. (#53982)

💫 Enhancements:

• Upgraded RLlink protocol for external env/simulator training. (#53550)
• Performance improvements in Offline RL API through switching to iter_torch_batches. (#54277)
• Added an example for curriculum learning in Atari Pong. (#55304)

🔨 Fixes:

• Corrected TensorType handling. (#55694)
• Fixed a bug with multi-learner setups in Offline RL API. (#55693)
• Addressed ImportError in Atari examples. (#54967)
• Fixed some bugs in the docs for IQL and CQL. (#55614)
• Increased default timesteps on two experiments. (#54185)
• Fixed TorchMultiCategorical.to_deterministic when having different number of categories and logits with time dimension. (#54414)
• Added missing documentation for SACConfig's training(). (#53918)
• Fixed bug in restore_from_path such that connector states are also restored on remote EnvRunners. (#54672)
• Fixed missing support for config.count_steps_by = "agent_steps". (#54885)
• Added missing colon to CUBLAS_WORKSPACE_CONFIG. (#53913)
• Removed rllib_contrib completely from RLlib. (#55182)

🏗 Architecture refactoring:

• Deprecated TensorFlow support from new API stack. (#55042)
• Deprecated input/output specs from RLModule. (#55141)
• Deprecated --enable-new-api-stack flag from all scripts. (#54853, #54702)

Ray Core

🎉 New Features:

• [Core] Add AcceleratorManager implementation for Rebellions NPU (#53985)

💫 Enhancements:

• [core][gpu-objects] Garbage collection (#53911)
• [core] Support pip_install_options for pip (#53551)
• [core][gpu-objects] Move data transfers to a background thread (#54256)
• [core][gpu-objects] Pass tensor_transport to store_task_errors even if the actor task throws an exception (#55427)
• [core][gpu-objects] Exception handling for application errors (#55442)
• [core][gpu-object] Add a user-facing call to wait for tensor to be freed (#55076)
• [Core] Bind ray internal servers to the specified node ip instead of 0.0.0.0 which improves the security (#55178, #55210, #55298)
• [core] Fallback unserializable exceptions to their string representation (#55476)

🔨 Fixes:

• [core] Fix objects_valid check failure with except from BaseException (#55602)
• [core][gpu-objects] Avoid triggering a KeyError by the GPU object GC callback for intra-actor communication (#54556)
• [core] fix checking for uv existence during ray_runtime setup (#54141)
• [core][autoscaler][v1] add heartbeat timeout logic to determine node activity status (#54030)
• [core] prevent sending SIGTERM after calling Worker::MarkDead (#54377)
• [Core] Fixed the bug where the head was unable to submit tasks after redis is turned on. (#54267)
• [Core] [Azure] query for supported Microsoft.Network/virtualNetworks API versions instead of relying on resource_client.DEFAULT_API_VERSION (#54874)
• [core] Fix possible race by checking node cache status instead of just subscription (#54745)
• [core] Fix get actor timeout multiplier (#54525)
• [core]: Use a temporary file to share default worker path in runtime env (#53653)
• [core] Fix check fail when task buffer periodical runner runs before RayEvent is initialized (#55249)
• [core] Patch grpc with RAY_num_grpc_threads to control grpc thread count (#54988)
• [core][gpu-objects] Always write to GPUObjectStore to avoid _get_tensor_meta() from hanging indefinitely. (#55433)
• [Core] Core Worker GetObjStatus GRPC Fault Tolerance (#54567)

📖 Documentation:

• Added guide on using type hints with Ray Core. (#55013)

🏗 Architecture refactoring:

• Migrate metric collection from opencensus to opentelemetry (#53098, #53740)

Dashboard

💫 Enhancements:

• Grafana: new Operator filter for Data; Prometheus adds RayNodeType label on for nodes. (#55493, #55192)

🔨 Fixes:

• Removed references to a deleted Data metrics panel. (#55478)

Ray Images

🎉 New Features:

• Added cuda 12.6 based images (#55425, #55444)

💫 Enhancements:

• Upgraded protobuf to v4 (#54496)

Docs

💫 Enhancements:

• KubeRay docs: added InteractiveMode quick-start details; expanded Core type-hints guidance; Serve LLM example coverage; Data LLM batching FAQ (#55570, #55284)

🔨 Fixes:

• Various formatting/mis-highlighting and lints across Train/Tune/Serve LLM docs. (#55284, #54763)

Thanks!

Thank you to everyone who contributed to this release! @pavitrabhalla, @Daraan, @Sparks0219, @daiping8, @abrarsheikh, @sven1977, @Toshaksha, @bveeramani, @MengjinYan, @GokuMohandas, @codope, @nadongjun, @SolitaryThinker, @matthewdeng, @elliot-barn, @isimluk, @avibasnet31, @OneSizeFitsQuorum, @Future-Outlier, @marosset, @jackfrancis, @kshanmol, @eicherseiji, @dayshah, @iamjustinhsu, @Qiaolin-Yu, @goutamvenkat-anyscale, @Yicheng-Lu-llll, @yantarou, @rclough, @zcin, @NeilGirdhar, @VarunBhandary, @400Ping, @akshay-anyscale, @vickytsang, @xushiyan, @JasonLi1909, @n-elia, @simonsays1980, @dragongu, @Kishanthan, @ruisearch42, @jectpro7, @TimothySeah, @liulehui, @rueian, @HollowMan6, @akyang-anyscale, @axreldable, @czgdp1807, @alanwguo, @justinvyu, @ok-scale, @my-vegetable-has-exploded, @landscapepainter, @fscnick, @machichima, @mpashkovskii, @ZacAttack, @gvspraveen, @sword865, @lmsh7, @Ziy1-Tan, @rebel-scottlee, @sampan-s-nayak, @coqian, @can-anyscale, @Bye-legumes, @win5923, @MortalHappiness, @angelinalg, @khluu, @aslonnie, @krishnakalyan3, @minosvasilias, @x-tong, @xinyuangui2, @raulchen, @Yangruipis, @edoakes, @kevin85421, @wingkitlee0, @Fokko, @cristianjd, @srinathk10, @owenowenisme, @JoshKarpel, @MengqingCao, @leopardracer, @westonpace, @LeslieWongCV, @VassilisVassiliadis, @crypdick, @alexeykudinkin, @mjacar, @kunling-anyscale, @saihaj, @kouroshHakha, @ema-pe, @markjm, @avigyabb, @dshepelev15, @mauvilsa, @omatthew98, @nrghosh, @ryanaoleary, @Aydin-ab, @lk-chen, @stephanie-wang, @harshit-anyscale, @jjyao, @bullgom, @Yevet, @israbbani

Release Highlights

• Ray Data: This release features a new Delta Lake and Unity Catalog integration and performance improvements to various reading/writing operators.
• Ray Core: Enhanced GPU object support with intra-process communication and improved Autoscaler v2 functionality
• Ray Train: Improved hardware metrics integration with Grafana and enhanced collective operations support
• Ray Serve LLM: This release features early proof of concept for prefill-decode disaggregation deployment and LLM-aware request routing such as prefix-cache aware routing.
• Ray Data LLM: Improved throughput and CPU memory utilization for ray data workers.

Ray Libraries

Ray Data

🎉 New Features:

• Add reading from Delta Lake tables and Unity Catalog integration (#53701)
• Enhanced pin_memory support in iter_torch_batches (#53792)
• Add pin_memory to iter_torch_batches (#53792)

💫 Enhancements:

• Re-enabled sorting in Ray Data tests with performance improvements (#54475)
• Enhanced handling of mismatched columns and pandas.NA values (#53861, #53859)
• Improved read_text trailing newline semantics (#53860)
• Optimized backpressure handling with policy-based resource management (#54376)
• Enhanced write_parquet with support for both partition_by and row limits (#53930)
• Prevent filename collisions on write operations (#53890)
• Improved execution performance for One Hot encoding in preprocessors (#54022)

🔨 Fixes:

• Fixing map_groups issues (#54462)
• Prevented Op fusion for streaming repartition to avoid performance degradation (#54469)
• Fixed ActorPool autoscaler scaling up logic (#53983)
• Resolved empty dataset repartitioning issues (#54107)
• Fixed PyArrow overflow handling in data processing (#53971, #54390)
• Fixed IcebergDatasink to properly generate individual file uuids (#52956)
• Avoid OOMs with read_json(..., lines=True) (#54436)
• Handle HuggingFace parquet dataset resolve URLs (#54146)
• Fixed BlockMetadata derivation for Read operator (#53908)

📖 Documentation:

• Updated AggregateFnV2 documentation to clarify finalize method (#53835)
• Improved preprocessor and vectorizer API documentation

Ray Train

🎉 New Features:

• Added broadcast_from_rank_zero and barrier collective operations (#54066)
• Enhanced hardware metrics integration with Grafana dashboards (#53218)
• Added support for dynamically loading callbacks via environment variables (#54233)

💫 Enhancements:

• Improved checkpoint population from before_init_train_context (#54453)
• Enhanced controller state logging and metrics (#52805)
• Added structured logging environment variable support (#52952)
• Improved handling of Noop scaling decisions for smoother scaling logic (#53180)
• Logging of controller state transitions to aid in debugging and analysis (#53344)

🔨 Fixes:

• Fixed GPU tensor reporting in ray.train.report (#53725)
• Enhanced move_tensors_to_device utility for complex tensor structures (#53109)
• Improved worker health check error handling with trace information (#53626)
• Fixed GPU transfer support for non-contiguous tensors (#52548)
• Force abort on SIGINT spam and do not abort finished runs (#54188)

📖 Documentation:

• Updated beginner PyTorch example (#54124)
• Added documentation for ray.train.collective APIs (#54340)
• Added a note about PyTorch DataLoader's multiprocessing and forkserver usage (#52924)
• Fixed various docstring format and indentation issues (#52855, #52878)
• Added note that ray.train.report API docs should mention optional checkpoint_dir_name (#54391)

🏗 Architecture refactoring:

• Removed subclass relationship between RunConfig and RunConfigV1 (#54293)
• Enhanced error handling for finished training runs (#54188)
• Deduplicated ML doctest runners in CI for efficiency (#53157)
• Converted isort configuration to Ruff for consistency (#52869)

Ray Tune

💫 Enhancements:

• Updated test_train_v2_integration to use the correct RunConfig (#52882)

🔨 Fixes:

• Fixed RayTaskError serialization logic (#54396)
• Improved experiment restore timeout handling (#53387)

📖 Documentation:

• Replaced session.report with tune.report and corrected import paths (#52801)
• Removed outdated graphics cards reference in docs (#52922)
• Fixed various docstring format issues (#52879)

Ray Serve

🎉 New Features:

• Added RouterConfig field to DeploymentConfig for custom RequestRouter configuration (#53870)
• Added support for implementing custom request routing algorithms (#53251)

💫 Enhancements:

• Enhanced FastAPI ingress deployment validation for multiple deployments (#53647)
• Optimized get_live_deployments performance (#54454)
• Progress towards making ray.serve.llm compatible with vLLM serve frontend (#54481, #54443, #54440)

🔨 Fixes:

• Fixed deployment scheduler issues with component scheduling (#54479)
• Fixed runtime_env validation for py_modules (#53186)
• Added descriptive error message when deployment name is not found (#45181)

📖 Documentation:

• Added troubleshooting guide for DeepSeek/multi-node GPU deployment on KubeRay (#54229)
• Updated the guide on serving models with Triton Server in Ray Serve
• Added documentation for custom request routing algorithms
• Added custom request router docs (#53511)

🏗 Architecture refactoring:

• Remove indirection layers of node initialization (#54481)
• Incremental refactor of LLMEngine (#54443)
• Remove random v0 logic from serve endpoints (#54440)
• Remove usage of internal_api.memory_summary() (#54417)
• Remove usage of ray._private.state (#54140)

Ray Serve/Data LLM

🎉 New Features

• Support separate deployment config for PDProxy in PrefixAwareReplicaSet (#53935)
• Support for prefix-aware request router (#52725)

💫 Enhancements

• Log engine stats after each batch task is done. (#54360)
• Decouple max_tasks_in_flight from max_concurrent_batches (#54362)
• Make llm serve endpoints compatible with vLLM serve frontend, including streaming, tool_code, and health check support (#54440)
• Remove botocore dependency in Ray Serve LLM (#54156)
• Update vLLM version to 0.9.2 (#54407)

🔨 Fixes

• Fix health check in prefill disagg (#53937)
• Fix doc to only support int concurrency (#54196)
• Fix vLLM batch test by changing to Pixtral (#53744)
• Fix pickle error with remote code models in vLLM Ray workloads (#53868)
• Adaption of the change of vllm.PoolingOutput (#54467)

📖 Documentation

• Ray serve/lora doc fix (#53553)
• Add Ray serve/LLM doc (#52832)
• Add a doc snippet to inform users about existing diffs between vLLM and Ray Serve LLM behavior in some APIs like streaming, tool_code, and health check (#54123)
• Troubleshooting DeepSeek/multi-node GPU deployment on KubeRay (#54229)

🏗 Architecture refactoring

• Make llm serve endpoints compatible with vLLM serve frontend, including streaming, tool_code, and health check support (#54490)
• Prefix-aware scheduler [2/N] Configure PrefixAwareReplicaSet to correctly handle the number of available GPUs for each worker and to ensure efficient GPU utilization in vLLM (#53192)
• Organize spread out utils.py (#53722)
• Remove ImageRetriever class and related tests from the LLM serving codebase. (#54018)
• Return a batch of rows in the udf instead of row by row (#54329)

RLlib

🎉 New Features:

• Implemented Offline Policy Evaluation (OPE) via Importance Sampling (#53702)
• Enhanced ConnectorV2 ObservationPreprocessor APIs with multi-agent support (#54209)
• Add GPU inference to offline evaluation (#52718)

💫 Enhancements:

• Enhanced MetricsLogger to handle tensors in state management (#53514)
• Improved env seeding in EnvRunners with deterministic training example rewrite (#54039)
• Cleanup of meta learning classes and examples (#52680)

🔨 Fixes:

• Fixed EnvRunner restoration when no local EnvRunner is available (#54091)
• Fixed shapes in explained_variance for recurrent policies (#54005)
• Resolved device check issues in Learner implementation (#53706)
• Enhanced numerical stability in MeanStdFilter (#53484)
• Fixed weight synching in offline evaluation (#52757)
• Fixed bug in split_and_zero_pad utility function (#52818)

📖 Documentation:

• Do-over of examples for connector pipelines (#52604)
• Remove "new API stack" banner from all RLlib docs pages as it's now the default (#54282)

Ray Core

🎉 New Features:

• Enhanced GPU object support with intra-process communication (#53798)
• Integrated single-controller collective APIs with GPU objects (#53720)
• Added support for ray.get on driver process for GPU objects (#53902)
• Supporting allreduce on list of input nodes in compiled graphs (#51047)
• Add single-controller API for ray.util.collective and torch gloo backend (#53319)

💫 Enhancements:

• Improved autoscaler v2 functionality with cloud instance ID reusing (#54397)
• Enhanced cluster task manager with better resource management (#54413)
• Upgraded OpenTelemetry SDK for better observability (#53745)
• Improved actor scheduling to prevent deadlocks in ordered actors (#54034)
• Enhanced get_max_resources_from_cluster_config functionality (#54455)
• Use std::move in cluster task manager constructor (#54413)
• Improve status messages and add comments about stale seq_no handling (#54470)
• uv run integration is now enabled by default (#53060)

🔨 Fixes:

• Fixed race conditions in object eviction and repinning for recovery (#53934)
• Resolved GCS crash issues on duplicate MarkJobFinished RPCs (#53951)
• Enhanced actor restart handling on node failures (#54088)
• Improved reference counting during worker graceful shutdown (#53002)
• Fix race condition when canceling task that hasn't started yet (#52703)
• Fix the issue where a valid RestartActor rpc is ignored (#53330)
• Fixed "Check failed: it->second.num_retries_left == -1" error (#54116)
• Fix detached actor being unexpectedly killed (#53562)

📖 Documentation:

• Enhanced troubleshooting guides and API documentation
• Updated reStructuredText formatting on Resources page (#53882)
• Fix working code snippets (#52748)
• Add doc for running KubeRay dashboard (#53830)
• Add antipattern for nested ray.get (#43184)

🏗 Architecture refactoring:

• Delete old skipped tests and unused code (#54427)
• Consolidate TaskManager interface (#54317)
• Move dependencies of NodeManager to main.cc for better testability (#53782)
• Use smart pointer in logging.cc (#54351)
• Delete event_label and unused environment variables (#54378, #54095)
• Remove actor task path in normal task submitter (#53996)
• Rename GcsFunctionManager and use fake in test (#53973)

Dashboard

🎉 New Features:

• Add dynolog for on-demand GPU profiling for Torch training (#53191)

💫 Enhancements:

• Added TPU usage metrics to reporter agent (#53678)
• Enhanced GPU profiling manager IP address retrieval (#53807)
• Improved configurability of Grafana dashboard parameters (#53236)
• Add configurability of 'orgId' param for requesting Grafana dashboards (#53236)

🔨 Fixes:

• Fixed Grafana dashboard dropdowns for data and train dashboards (#52752)
• Resolved daylight savings time issues in dashboard (#52755)
• Fix retrieving IP address from the GPUProfilingManager on the dashboard agent (#53807)

Docs 🎉 New Features:

• New end-to-end examples:
  • Multi-modal AI pipeline (#52342)
  • Xgboost tutorial (#52383)
  • Audio transcription and LLM as judge curation (#53189)
  • LLM training and inference (#53415)
  • Scalable video processing (#50965)

💫 Enhancements:

• Add pydoclint to pre-commit (#52974)
• Add vale to pre-commit (#53564)

Breaking Changes

• Removed deprecated ray.workflow package (#53612)
• Removed deprecated storage parameter from ray.init (#53669)
• Removed deprecated ray start CLI options (#53675)
• Removed experimental "array" library (#54105)
• Remove dask from byod 3.9 deps (#54521)

Dependencies & Build

• Added uv binary v0.7.19 for improved package management (#54437)
• Upgraded datasets in release tests (#54425)
• Enhanced wheel building process with single bazel call optimization (#54476)
• Fixed uv run parser for handling extra arguments (#54488)
• Upgrade h11, requests, starlette, jinja2, pyopenssl and cryptography
• Generate multi-arch image indexes (#52816)

Thanks!

Thank you to everyone who contributed to this release! @kouroshHakha, @davidwagnerkc, @MengjinYan, @minerharry, @simonsays1980, @Myasuka, @noemotiovon, @goutamvenkat-anyscale, @harshit-anyscale, @jugalshah291, @tianyi-ge, @sven1977, @crypdick, @JohnsonKuan, @lk-chen, @richardsliu, @alexeykudinkin, @EagleLo, @soffer-anyscale, @zcin, @AdrienVannson, @nilsmelchert, @raulchen, @jujipotle, @DrehanM, @vigneshka, @Ziy1-Tan, @Blaze-DSP, @ArthurBook, @GokuMohandas, @walkoss, @bveeramani, @edoakes, @omatthew98, @SeanQuant, @CheyuWu, @cszhu, @win5923, @kevin85421, @angelinalg, @iamjustinhsu, @eicherseiji, @kunling-anyscale, @vickytsang, @MortalHappiness, @aslonnie, @psr-ai, @sbhat98, @anyadontfly, @marwan116, @cristianjd, @2niuhe, @codope, @fscnick, @ryanaoleary, @srinathk10, @TimothySeah, @han-steve, @Future-Outlier, @Syulin7, @Qiaolin-Yu, @elliot-barn, @JoshKarpel, @dayshah, @can-anyscale, @ok-scale, @mattip, @SolitaryThinker, @owenowenisme, @nehiljain, @GeneDer, @rnkrtt, @israbbani, @DriverSong, @sinalallsite, @pcmoritz, @akyang-anyscale, @xinyuangui2, @nrghosh, @davidxia, @rueian, @stephanie-wang, @jjyao, @chris-ray-zhang, @czgdp1807, @justinvyu, @Daraan, @landscapepainter, @troychiu, @khluu, @hipudding, @ruisearch42, @robertnishihara, @ArturNiederfahrenhorst, @abrarsheikh, @alanwguo, @HollowMan6, @ran1995data, @matthewdeng

Ray 2.47.1 fixed an issue where Ray failed to start on Mac (https://github.com/ray-project/ray/pull/53807)

Release Highlights

• Prefill disaggregation is now supported in initial support in Ray Serve LLM (#53092). This is critical for production LLM serving use cases.
• Ray Data features a variety of performance improvements (locality-based scheduling, non-blocking execution) as well as improvements to observability, preprocessors, and other stability fixes.
• Ray Serve now features custom request routing algorithms, which is critical for high throughput traffic for large model use cases.

Ray Libraries

Ray Data

🎉 New Features:

• Add save modes support to file data sinks (#52900)
• Added flattening capability to the Concatenator preprocessor to support output vectorization use cases (#53378)

💫 Enhancements:

• Re-enable Actor locality-based scheduling. This PR also improves algorithms for ranking the locations for the bundle. (#52861)
• Disable blocking pipeline by default until Actor Pool fully scales up to min actors (#52754)
• Progress bar and dashboard improvements to show name of partial functions properly(#52280)

🔨 Fixes:

• Make Ray Data from_torch respect Dataset len (#52804)
• Fixing flaky aggregation test (#53383)
• Fix race condition bug in fault tolerance by disabling on_exit hook (#53249)
• Fix move_tensors_to_device utility for the list/tuple[tensor] case (#53109)
• Fix ActorPool scaling to avoid scaling down when the input queue is empty (#53009)
• Fix internal queues accounting for all Operators w/ an internal queue (#52806)
• Fix backpressure for FileBasedDatasource. This fixes potential OOMs for workloads using FileBasedDatasources (#52852)

📖 Documentation:

• Fix working code snippets (#52748)
• Improve AggregateFnV2 docstrings and examples (#52911)
• Improved documentation for vectorizers and API visibility in Data (#52456)

Ray Train

🎉 New Features:

• Added support for configuring Ray Train worker actor runtime environments. (#52421)
• Included Grafana panel data in Ray Train export for improved monitoring. (#53072)
• Introduced a structured logging environment variable to standardize log formats. (#52952)
• Added metrics for TrainControllerState to enhance observability. (#52805)

💫 Enhancements:

• Logging of controller state transitions to aid in debugging and analysis. (#53344)
• Improved handling of Noop scaling decisions for smoother scaling logic. (#53180)

🔨 Fixes:

• Improved move_tensors_to_device utility to correctly handle list / tuple of tensors. (#53109)
• Fixed GPU transfer support for non-contiguous tensors. (#52548)
• Increased timeout in test_torch_device_manager to reduce flakiness. (#52917)

📖 Documentation:

• Added a note about PyTorch DataLoader’s multiprocessing and forkserver usage. (#52924)
• Fixed various docstring format and indentation issues. (#52855, #52878)
• Removed unused "configuration-overview" documentation page. (#52912)
• General typo corrections. (#53048)

🏗 Architecture refactoring:

• Deduplicated ML doctest runners in CI for efficiency. (#53157)
• Converted isort configuration to Ruff for consistency. (#52869)
• Removed unused PARALLEL_CI blocks and combined imports. (#53087, #52742)

Ray Tune

💫 Enhancements:

• Updated test_train_v2_integration to use the correct RunConfig. (#52882)

📖 Documentation:

• Replaced session.report with tune.report and corrected import paths. (#52801)
• Removed outdated graphics cards reference in docs. (#52922)
• Fixed various docstring format issues. (#52879)

Ray Serve

🎉 New Features:

• Added support for implementing custom request routing algorithms. (#53251)
• Introduced an environment variable to prioritize custom resources during deployment scheduling. (#51978)

💫 Enhancements:

• The ingress API now accepts a builder function in addition to an ASGI app object. (#52892)

🔨 Fixes:

• Fixed runtime_env validation for py_modules. (#53186)
• Disallowed special characters in Serve deployment and application names. (#52702)
• Added a descriptive error message when a deployment name is not found. (#45181)

📖 Documentation:

• Updated the guide on serving models with Triton Server in Ray Serve.
• Added documentation for custom request routing algorithms.

Ray Serve/Data LLM

🎉 New Features:

• Added initial support for prefill decode disaggregation (#53092)
• Expose vLLM Metrics to serve.llm API (#52719)
• Embedding API (#52229)

💫 Enhancements:

• Allow setting name_prefix in build_llm_deployment (#53316)
• Minor bug fix for 53144: stop tokens cannot be null (#53288)
• Add missing repetition_penalty vLLM sampling parameter (#53222)
• Mitigate the serve.llm streaming overhead by properly batching stream chunks (#52766)
• Fix test_batch_vllm leaking resources by using larger wait_for_min_actors_s

🔨 Fixes:

• LLMRouter.check_health() should check LLMServer.check_health() (#53358)
• Fix runtime passthrough and auto-executor class selection (#53253)
• Update check_health return type (#53114)
• Bug fix for duplication of <bos> token (#52853)
• In stream batching, first part of the stream was always consumed and not streamed back from the router (#52848)

RLlib

🎉 New Features:

• Add GPU inference to offline evaluation. (#52718)

💫 Enhancements:

• Do-over of examples for connector pipelines. (#52604)
• Cleanup of meta learning classes and examples. (#52680)

🔨 Fixes:

• Fixed weight synching in offline evaluation. (#52757)
• Fixed bug in split_and_zero_pad utility function (related to complex structures vs simple values or np.arrays). (#52818)

Ray Core

💫 Enhancements:

• uv run integration is now enabled by default, so you don't need to set the RAY_RUNTIME_ENV_HOOK any more (#53060). If you rely on the previous behavior where uv run only runs the Ray driver but not the workers in the uv environment, you can switch back to the old behavior by setting the RAY_ENABLE_UV_RUN_RUNTIME_ENV=0 environment variable.
• Record gcs process metrics (#53171)

🔨 Fixes:

• Improvements for using RuntimeEnv in the Job Submission API. (#52704)
• Close unused pipe file descriptor of child processes of Raylet (#52700)
• Fix race condition when canceling task that hasn't started yet (#52703)
• Implement a thread pool and call the CPython API on all threads within the same concurrency group (#52575)
• cgraph: Fix execution schedules with collective operations (#53007)
• cgraph: Fix scalar tensor serialization edge case with serialize_to_numpy_or_scalar (#53160)
• Fix the issue where a valid RestartActor rpc is ignored (#53330)
• Fix reference counter crashes during worker graceful shutdown (#53002)

Dashboard

🎉 New Features:

• train: Add dynolog for on-demand GPU profiling for Torch training (#53191)

💫 Enhancements:

• Add configurability of 'orgId' param for requesting Grafana dashboards (#53236)

🔨 Fixes:

• Fix Grafana dashboards dropdowns for data and train dashboard (#52752)
• Fix dashboard for daylight savings (#52755)

Ray Container Images

💫 Enhancements:

• Upgrade h11 (#53361), requests, starlette, jinja2 (#52951), pyopenssl and cryptography (#52941)
• Generate multi-arch image indexes (#52816)

Docs

🎉 New Features:

• End-to-end example: Entity recognition with LLMs (#52342) - new end-to-end example
• End-to-end example: xgboost tutorial (#52383)
• End-to-end tutorial for audio transcription and LLM as judge curation (#53189)

💫 Enhancements:

• Adds pydoclint to pre-commit (#52974)

Thanks!

Thank you to everyone who contributed to this release!

@NeilGirdhar, @ok-scale, @JiangJiaWei1103, @brandonscript, @eicherseiji, @ktyxx, @MichalPitr, @GeneDer, @rueian, @khluu, @bveeramani, @ArturNiederfahrenhorst, @c8ef, @lk-chen, @alanwguo, @simonsays1980, @codope, @ArthurBook, @kouroshHakha, @Yicheng-Lu-llll, @jujipotle, @aslonnie, @justinvyu, @machichima, @pcmoritz, @saihaj, @wingkitlee0, @omatthew98, @can-anyscale, @nadongjun, @chris-ray-zhang, @dizer-ti, @matthewdeng, @ryanaoleary, @janimo, @crypdick, @srinathk10, @cszhu, @TimothySeah, @iamjustinhsu, @mimiliaogo, @angelinalg, @gvspraveen, @kevin85421, @jjyao, @elliot-barn, @xingyu-long, @LeoLiao123, @thomasdesr, @ishaan-mehta, @noemotiovon, @hipudding, @davidxia, @omahs, @MengjinYan, @dengwxn, @MortalHappiness, @alhparsa, @emmanuel-ferdman, @alexeykudinkin, @KunWuLuan, @dev-goyal, @sven1977, @akyang-anyscale, @GokuMohandas, @raulchen, @abrarsheikh, @edoakes, @JoshKarpel, @bhmiller, @seanlaii, @ruisearch42, @dayshah, @Bye-legumes, @petern48, @richardliaw, @rclough, @israbbani, @jiwq

Release Highlights

The 2.46 Ray release comes with a couple core highlights:

1. Ray Data now supports hash shuffling for repartition and aggregations, along with support for joins. This enables many new data processing workloads to be run on Ray Data. Please give it a try and let us know if you have any feedback!
2. Ray Serve LLM now supports vLLM v1 to be forward-compatible with upcoming vLLM releases. This also opens up significant performance improvements that come with vLLM's v1 refactor.
3. There is a new Train Grafana dashboard which provides in-depth metrics on Grafana for better metrics on training workloads.

Ray Libraries

Ray Data

🎉 New Features:

• Adding support for hash-shuffle based repartitioning and aggregations (#52664)
• Added support for Joins (using hash-shuffle) (#52728)
• [LLM] vLLM support upgrades to 0.8.5 (#52344)

💫 Enhancements:

• Add memory attribute to ExecutionResources (#51127)
• Support ray_remote_args for read_tfrecords #52450
• [data.dashboard] Skip reporting internal metrics (#52666)
• Add PhysicalOperator.min_max_resource_usage_bounds (#52502)
• Speed up printing the schema (#52612)
• [data.dashboard] Dataset logger for worker (#52706)
• Support new pyiceberg version (#51744)
• Support num_cpus, memory, concurrency, batch_size for preprocess (#52574)

🔨 Fixes:

• Handle Arrow Array null types in to_numpy (#52572)
• Fix S3 serialization wrapper compatibility with RetryingPyFileSystem (#52568)
• Fixing Optimizer to apply rules until plan stabilize; (#52663)
• Fixing FuseOperators rule to properly handle the case of transformations drastically changing size of the dataset (#52570)

📖 Documentation:

• [LLM] Improve concurrency settings, improve prompt to achieve better throughput (#52634)

Ray Train

🎉 New Features:

• Add initial Train Grafana dashboard (#52709)

💫 Enhancements:

• Lazily import torch FSDP for ray.train.torch module to improve performance and reduce unnecessary dependencies (#52707)
• Deserialize the user-defined training function directly on workers, improving efficiency (#52684)

🔨 Fixes:

• Fixed error when no arguments are passed into TorchTrainer (#52693)

📖 Documentation:

• Added new XGBoostTrainer user guide (#52355)

🏗 Architecture refactoring:

• Re-enabled isort for python/ray/train to maintain code formatting consistency (#52717)

Ray Tune

📖 Documentation:

• Fixed typo in Ray Tune PyTorch Lightning docs (#52756)

Ray Serve

💫 Enhancements:

• [LLM] Refactor LLMServer and LLMEngine to not diverge too much from vllm chat formatting logic (#52597)
• Bump vllm from 0.8.2 to 0.8.5 in /python (#52344)
• [LLM] Add router replicas and batch size to llm config (#52655)

🔨 Fixes:

• Request cancellation not propagating correctly across deployments (#52591)
• BackpressureError not properly propagated in FastAPI ingress deployments (#52397)
• Hanging issue when awaiting deployment responses (#52561)
• [Serve.llm] made Ray Serve LLM compatible with vLLM v1 (#52668)

📖 Documentation:

• [Serve][LLM] Add doc for deploying DeepSeek (#52592)

RLLib

🎉 New Features:

• Offline Evaluation with loss function for Offline RL pipeline. Introduces three new callbacks, on_offline_evaluate_start, on_offline_evaluate_end, on_offline_eval_runners_recreated (#52308)

💫 Enhancements:

• New custom_data attribute for SingleAgentEpisode and MultiAgentEpisode to store custom metrics. Deprecates add|get_temporary_timestep_data() (#52603)

Ray Core

💫 Enhancements:

• Only get serialization context once for all .remote args (#52690)
• Add grpc server success and fail count metric (#52711)

🔨 Fixes:

• Fix open leak for plasma store memory (shm/fallback) by workers (#52622)
• Assure closing of unused pipe for dashboard subprocesses (#52678)
• Expand protection against dead processes in reporter agent (#52657)
• [cgraph] Separate metadata and data in cross-node shared memory transport (#52619)
• Fix JobID check for detached actor tasks (#52405)
• Fix potential log loss of tail_job_logs (#44709)

🏗 Architecture refactoring:

• Cancel tasks when an owner dies instead of checking if an owner is dead during scheduling (#52516)
• Unify GcsAioClient and GcsClient (#52735)
• Remove worker context dependency from the task receiver (#52740)

Dashboard

🎉 New Features:

• Ray Train Grafana Dashboard added with a few built-in metrics. More to come.

Thanks!

Thank you to everyone who contributed to this release! @kevin85421, @edoakes, @wingkitlee0, @alexeykudinkin, @chris-ray-zhang, @sophie0730, @zcin, @raulchen, @matthewdeng, @abrarsheikh, @popojk, @Jay-ju, @ruisearch42, @eicherseiji, @lk-chen, @justinvyu, @dayshah, @kouroshHakha, @NeilGirdhar, @omatthew98, @ishaan-mehta, @davidxia, @ArthurBook, @GeneDer, @srinathk10, @dependabot[bot], @JoshKarpel, @aslonnie, @khluu, @can-anyscale, @israbbani, @saihaj, @MortalHappiness, @alanwguo, @bveeramani, @iamjustinhsu, @Ziy1-Tan, @xingyu-long, @simonsays1980, @fscnick, @chuang0221, @sven1977, @jjyao

Ray Core

💫 Enhancements

• Make Object Store Fallback Directory configurable (#51189).
• [cgraph] Support with_tensor_transport(transport='shm') (#51872).
• [cgraph] Support reduce scatter and all gather collective for GPU communicator in compiled graph (#50624).

🔨 Fixes

• Make sure KillActor RPC with force_kill=True can actually kill the threaded actor (#51414).
• [Autoscaler] Do not remove idle nodes for upcoming placement groups (#51122).
• Threaded actors get stuck forever if they receive two exit signals (#51582).
• [cgraph] Fix illegal memory access of cgraph when used in PP (#51734).
• Avoid resubmitted actor tasks from hanging indefinitely (#51904).
• Fix interleaved placement group creation process due to node failure (#52202).
• Flush task events in CoreWorker::Shutdown instead of CoreWorker::Disconnect (#52374).

🏗 Architecture refactoring

• Split dashboard single process into multiple processes to improve stability and avoid interference between different heads (#51282, #51489, #51555, #51507, #51587, #51553, #51676, #51733, #51809, #51877, #51876, #51980, #52114).

Ray Libraries

Ray Data

🎉 New Features

• New ClickHouse sink via Dataset.write_clickhouse() (#50377)
• Support ray_remote_args_fn in Dataset.groupby().map_groups() to set per-group runtime env and resource hints (#51236)
• Expose Dataset.name / set_name as a public API for easier lineage tracking (#51076)
• Allow async callable classes in Dataset.flat_map() (#51180)
• Introduce Ruleset abstraction for rule-based query optimisation (#51558)
• Add seamless conversion from Daft DataFrame to Ray Dataset (#51531)
• Improved support for line-delimited JSONL reading in read_json() (#52083)
• Provide Dataset.export_metadata() for schema & stats snapshots (#52227)

💫 Enhancements

• Improved performance of sorting and sort-shuffle based operations (by more than 5x in benchmarks) (#51943)
• Metrics: number of map-actor workers alive / pending / restarting (#51082)
• Continuous memory-usage polling per map task (#51324)
• Auto-tune map-task memory based on output size (#51536)
• More informative back-pressure progress bar (#51697)
• Faster RefBundle.get_cached_location() lookup (#52097)
• Speed-up for PandasBlock.size_bytes() (#52510)
• Expanded BlockColumnAccessor utilities and ops (#51326, #51571)

🔨 Fixes

• Correct MapTransformFn.__eq__ equality check (#51434)
• Persist unresolved wildcard paths in FileBasedDataSource (#51424)
• Repair Hugging Face dynamic-module loading on workers (#51488)
• Prevent HTTP URLs from being expanded by _expand_paths (#50178)
• Fix Databricks host-URL parsing in Delta datasource (#49926)
• Restore reproducibility of Dataset.random_sample() (#51401)
• Correct RandomAccessDataset.multiget() return values (#51421)
• Ensure executor shutdown after schema fetch to avoid leaked actors (#52379)
• Repair streaming shutdown regression (#52509)
• Honour minimum resource reservation in ResourceManager (#52226)

📖 Documentation

• Clarified shuffle-section wording (#51289)
• Documented concurrency semantics in API reference (#51963)
• Updated Ray Data guides for the 2.45 release (#52082)

Ray Train

🎉 New Features

• Fold v2.LightGBMTrainer API into the public trainer class as an alternate constructor (#51265).

💫 Enhancements

• Use the user-defined function name as the training thread name (#52514).
• Upgrade LightGBM to version 4.6.0 (#52410).
• Adjust test size further for better results (#52283).
• Log errors raised by workers during training (#52223).
• Add worker group setup finished log to track progress (#52120).
• Change test_telemetry to medium size (#52178).
• Improve dataset name observability for better tracking (#52059).
• Differentiate between train v1 and v2 export data for clarity (#51728).
• Include scheduling status detail to improve debugging (#51480).
• Move train library usage check to Trainer initialization (#50966).

🔨 Fixes

• Separate OutputSplitter._locality_hints from actor_locality_enabled and locality_with_output (#52005).
• Fix print redirection to handle new lines correctly (#51542).
• Mark RunAttempt workers as dead after completion to avoid stale states (#51540).
• Fix setup_wandb rank_zero_only logic (#52381).

📖 Documentation

• Add links to the Train v2 migration guide in the Train API pages (#51924).

🏗 Architecture refactoring

• Replace AMD device environment variable with HIP_VISIBLE_DEVICES (#51104).
• Remove unnecessary string literal splits (#47360).

Ray Tune

📖 Documentation

• Improve Tune documentation structure (#51684).
• Fix syntax errors in Ray Tune example pbt_ppo_example.ipynb (#51626).

Ray Serve

🎉 New Features

• Add request timeout sec for gRPC (#52276).
• [Serve.llm] ray.llm support custom accelerators (#51359).

💫 Enhancements

• Improve Serve deploy ignore behavior (#49336).
• [Serve.llm] Telemetry GPU type fallback to cluster hardware when unspecified (#52003).

🔨 Fixes

• Fix multiplex fallback logic during burst requests (#51389).
• Don't stop retrying replicas when a deployment is scaling back up from zero (#51600).
• Remove RAY_SERVE_ENABLE_QUEUE_LENGTH_CACHE flag (#51649).
• Remove RAY_SERVE_EAGERLY_START_REPLACEMENT_REPLICAS flag (#51722).
• Unify request cancellation errors (#51768).
• Catch timeout error when checking if proxy is dead (#52002).
• Suppress cancelled errors in proxy (#52423).
• [Serve.llm] Fix loading model from remote storage and add docs (#51617).
• [Serve.llm] Fix ServeReplica deployment failure for DeepSeek (#51989).
• [Serve.llm] Check GPUType enum value rather than enum itself (#52037).
• [Serve.llm] Fix inconsistent v0/v1 config passed to vLLM (#52185).
• [Serve.llm] Fix V0/V1 auto-resolution more broadly (#52357).
• [Serve.llm] Fix getting prompt limit for input-too-long error (#52315).

📖 Documentation

• Update deployment status docs (#51610).
• Update engine kwargs in LoRA instructions (#52203).
• Remove outdated docs about request retries (#52438).
• [Serve.llm] Add gen config related doc (#51572).
• [Serve.llm] Fix loading model from remote storage and add docs (#51617).
• [Serve.llm] Update Ray Serve LLM docs (#51867).
• [Serve.llm] Update model_source doc (#52054).
• [Serve.llm] Add doc example on using tokenizer_pool_size (#52356).
• [Serve.llm] Add API links (#52384).

RLlib

🎉 New Features

• Meta-learning, differentiable learner (#51859, #52494).

💫 Enhancements

• Offline RL:
  • Add configuration parameter ignore_final_observation and automatic episode ID generation (#52107).
  • BC performance improvements and adjustments to new Learner.update logic (#51425).
• Better handling of tensors in MetricsLogger and Stats (#52334).
• Make min/max env steps per evaluation sample call configurable for duration="auto" (#51637).
• Access to all episode chunks in EnvRunner callback on_episode_end (#52252).

🔨 Fixes

• Various smaller fixes (#52503, #51371, #51816, #51621, #44700, #51522, #51406).

🏗 Architecture refactoring

• Remove local EnvRunner (optional) (#52091).

Dashboard

💫 Enhancements

• [Core/Dashboard] Add CPU and memory metrics for subprocesses (#52208).

🔨 Fixes

• [Observability] Fix metrics in the core Grafana dashboard to sum by labels (#51894).
• [Dashboard] Remove millisecond padding in DurationText component (#52395).
• [Core][Dashboard-Agent] Surface the root cause of job submission failure when the dashboard agent fails to launch HTTP server (#51869).

---

Many thanks to all those who contributed to this release! @samhallam-reverb, @vickytsang, @anyadontfly, @zhaoch23, @bryant1410, @khluu, @akyang-anyscale, @angelinalg, @RocketRider, @wingkitlee0, @robertnishihara, @liuxsh9, @KepingYan, @SumanthRH, @emmanuel-ferdman, @ashwinsnambiar, @ArturNiederfahrenhorst, @KPCOFGS, @Bye-legumes, @400Ping, @dayshah, @aslonnie, @justinvyu, @rugggg, @zhiqiwangebay, @comaniac, @thusoy, @JDarDagran, @chuang0221, @davidxia, @tnixon, @israbbani, @win5923, @leibovitzgil, @simonsays1980, @machichima, @VamshikShetty, @zcin, @lk-chen, @abrarsheikh, @edoakes, @alexeykudinkin, @ruisearch42, @tespent, @jecsand838, @sijieamoy, @can-anyscale, @JonDum, @jyakaranda, @nadongjun, @d-miketa, @MortalHappiness, @kevin85421, @Ziy1-Tan, @matthewdeng, @crypdick, @hongpeng-guo, @richardliaw, @Qiaolin-Yu, @bhmiller, @soffer-anyscale, @kenchung285, @nishi-t, @Drice1999, @ryanaoleary, @chris-ray-zhang, @MengjinYan, @saihaj, @jjyao, @jaganmolleti7, @iamjustinhsu, @fscnick, @pcmoritz, @Jay-ju, @westonpace, @han-steve, @GeneDer, @denadai2, @thomasdesr, @jaychia, @raulchen, @omatthew98, @srinathk10, @alanwguo, @rueian, @akshay-anyscale, @bveeramani, @dentiny, @dhakshin32, @kouroshHakha, @sven1977

There is no difference between 2.44.1 and 2.44.0, though we needed a patch version for other out of band reasons. To fill the awkward blankness, here is a haiku about Ray:

Under screen-lit skies A ray of bliss in each patch Joy at any scale

Release Highlights

• This release features Ray Compiled Graph (beta). Ray Compiled Graph gives you a classic Ray Core-like API, but with (1) less than 50us system overhead for workloads that repeatedly execute the same task graph; and (2) native support for GPU-GPU communication via NCCL. Ray Compiled Graph APIs simplify high-performance multi-GPU workloads such as LLM inference and training. The beta release refines the API, enhances stability, and adds or improves features like visualization, profiling and experimental GPU compute/computation overlap. For more information, refer to Ray documentation: https://docs.ray.io/en/latest/ray-core/compiled-graph/ray-compiled-graph.html
• The experimental Ray Workflows library has been deprecated and will be removed in a future version of Ray. Ray Workflows has been marked experimental since its inception and hasn’t been maintained due to the Ray team focusing on other priorities. If you are using Ray Workflows, we recommend pinning your Ray version to 2.44.

Ray Libraries

Ray Data

🎉 New Features:

• Add Iceberg write support through pyiceberg (#50590)
• [LLM] Various feature enhancements to Ray Data LLM, including LoRA support #50804 and structured outputs #50901

💫 Enhancements:

• Add dataset/operator state, progress, total metrics (#50770)
• Make chunk combination threshold configurable (#51200)
• Store average memory use per task in OpRuntimeMetrics (#51126)
• Avoid unnecessary conversion to Numpy when creating Arrow/Pandas blocks (#51238)
• Append-mode API for preprocessors -- #50848, #50847, #50642, #50856, #50584. Note that vectorizers and hashers now output a single column instead 1 column per feature. In the near future, we will be graduating preprocessors to beta.

🔨 Fixes:

• Fixing Map Operators to avoid unconditionally overriding generator's back-pressure configuration (#50900)
• Fix filter expr equating negative numbers (#50932)
• Fix error message for override_num_blocks when reading from a HuggingFace Dataset (#50998)
• Make num_blocks in repartition optional (#50997)
• Always pin the seed when doing file-based random shuffle (#50924)
• Fix StandardScaler to handle NaN stats (#51281)

Ray Train

🎉 New Features:

• Implement state export API (#50622, #51085, #51177)

💫 Enhancements:

• Folded v2.XGBoostTrainer API into the public trainer class as an alternate constructor (#50045)
• Created a default ScalingConfig if one is not provided to the trainer (#51093)
• Improved TrainingFailedError message (#51199)
• Utilize FailurePolicy factory (#51067)

🔨 Fixes:

• Fixed trainer import deserialization when captured within a Ray task (#50862)
• Fixed serialize import test for Python 3.12 (#50963)
• Fixed RunConfig deprecation message in Tune being emitted in trainer.fit usage (#51198)

📖 Documentation:

• [Train V2] Updated API references (#51222)
• [Train V2] Updated persistent storage guide (#51202)
• [Train V2] Updated user guides for metrics, checkpoints, results, and experiment tracking (#51204)
• [Train V2] Added updated Train + Tune user guide (#51048)
• [Train V2] Added updated fault tolerance user guide (#51083)
• Improved HF Transformers example (#50896)
• Improved Train DeepSpeed example (#50906)
• Use correct mean and standard deviation norm values in image tutorials (#50240)

🏗 Architecture refactoring:

• Deprecated Torch AMP wrapper utilities (#51066)
• Hid private functions of train context to avoid abuse (#50874)
• Removed ray storage dependency and deprecated RAY_STORAGE env var configuration option (#50872)
• Moved library usage tests out of core (#51161)

Ray Tune

📖 Documentation:

• Various improvements to Tune Pytorch CIFAR tutorial (#50316)
• Various improvements to the Ray Tune XGBoost tutorial (#50455)
• Various enhancements to Tune Keras example (#50581)
• Minor improvements to Hyperopt tutorial (#50697)
• Various improvements to LightGBM tutorial (#50704)
• Fixed non-runnable Optuna tutorial (#50404)
• Added documentation for Asynchronous HyperBand Example in Tune (#50708)
• Replaced reuse actors example with a fuller demonstration (#51234)
• Fixed broken PB2/RLlib example (#51219)
• Fixed typo and standardized equations across the two APIs (#51114)
• Improved PBT example (#50870)
• Removed broken links in documentation (#50995, #50996)

🏗 Architecture refactoring:

• Removed ray storage dependency and deprecated RAY_STORAGE env var configuration option (#50872)
• Moved library usage tests out of core (#51161)

Ray Serve

🎉 New Features:

• Faster bulk imperative Serve Application deploys (#49168)
• [LLM] Add gen-config (#51235)

💫 Enhancements:

• Clean up shutdown behavior of serve (#51009)
• Add additional_log_standard_attrs to serve logging config (#51144)
• [LLM] remove asyncache and cachetools from dependencies (#50806)
• [LLM] remove backoff dependency (#50822)
• [LLM] Remove asyncio_timeout from ray[llm] deps on python<3.11 (#50815)
• [LLM] Made JSON validator a singleton and jsonref packages lazy imported (#50821)
• [LLM] Reuse AutoscalingConfig and DeploymentConfig from Serve (#50871)
• [LLM] Use pyarrow FS for cloud remote storage interaction (#50820)
• [LLM] Add usage telemetry for serve.llm (#51221)

🔨 Fixes:

• Exclude redirects from request error count (#51130)
• [LLM] Fix the wrong device_capability issue in vllm on quantized models (#51007)
• [LLM] add gen-config related data file to the package (#51347)

📖 Documentation:

• [LLM] Fix quickstart serve LLM docs (#50910)
• [LLM] update build_openai_app to include yaml example (#51283)
• [LLM] remove old vllm+serve doc (#51311)

RLlib

💫 Enhancements:

• APPO/IMPALA accelerate:
  • LearnerGroup should not pickle remote functions on each update-call; Refactor LearnerGroup and Learner APIs. (#50665)
  • EnvRunner sync enhancements. (#50918)
  • Various other speedups: #51302, #50923, #50919, #50791
• Unify namings for actor managers' outstanding in-flight requests metrics. (#51159)
• Add timers to env step, forward pass, and complete connector pipelines runs. (#51160)

🔨 Fixes:

• Multi-agent env vectorization:
  • Fix MultiAgentEnvRunner env check bug. (#50891)
  • Add single_action_space and single_observation_space to VectorMultiAgentEnv. (#51096)
• Other fixes: #51255, #50920, #51369

📖 Documentation:

• Smaller fixes: #51015, #51219

Ray Core and Ray Clusters

Ray Core

🎉 New Features:

• Enhanced uv support (#51233)

💫 Enhancements:

• Made infeasible task errors much more obvious (#45909)
• Log rotation for workers, runtime env agent, and dashboard agent (#50759, #50877, #50909)
• Support customizing gloo timeout (#50223)
• Support torch profiling in Compiled Graph (#51022)
• Change default tensor deserialization in Compiled Graph (#50778)
• Use current node id if no node is specified on ray drain-node (#51134)

🔨 Fixes:

• Fixed an issue where the raylet continued to have high CPU overhead after a job was terminated (#49999).
• Fixed compiled graph buffer release issues (#50434)
• Improved logic for ray.wait on object store objects (#50680)
• Ray metrics performing validation the same validation as Prometheus for invalid names (#40586)
• Make executor a long-running Python thread (#51016)
• Fix plasma client memory leak (#51051)
• Fix using ray.actor.exit_actor() from within an async background thread (#49451)
• Fix UV hook to support Ray Job submission (#51150)
• Fix resource leakage after ray job is finished (#49999)
• Use the correct way to check whether an actor task is running (#51158)
• Controllably destroy CUDA events in GPUFuture’s (Compiled Graph) (#51090)
• Avoid creating a thread pool with 0 threads (#50837)
• Fix the logic to calculate the number of workers based on the TPU version (#51227)

📖 Documentation:

• Updated error message and anti-pattern when forking new processes in worker processes (#50705)
• Compiled Graph API Documentation (#50754)
• Doc for nsight and torch profile for Compiled Graph (#51037)
• Compiled Graph Troubleshooting Doc (#51030)
• Completion of of Compiled Graph Docs (#51206)
• Updated jemalloc profiling doc (#51031)
• Add information about standard Python logger attributes (#51038)
• Add description for named placement groups to require a namespace (#51285)
• Deprecation warnings for Ray Workflows and cluster-wide storage (#51309)

Ray Clusters

🎉 New Features:

• Add cuda 12.8 images (#51210)

💫 Enhancements:

• Add Pod names to the output of ray status -v (#51192)

🔨 Fixes:

• Fix autoscaler v1 crash from infeasible strict spread placement groups (#39691)

🏗 Architecture refactoring:

• Refactor autoscaler v2 log formatting (#49350)
• Update yaml example for CoordinatorSenderNodeProvider (#51292)

Dashboard

🎉 New Features:

• Discover TPU logs on the Ray Dashboard (#47737)

🔨 Fixes:

• Return the correct error message when trying to kill non-existent actors (#51341)

---

Many thanks to all those who contributed to this release! @crypdick, @rueian, @justinvyu, @MortalHappiness, @CheyuWu, @GeneDer, @dayshah, @lk-chen, @matthewdeng, @co63oc, @win5923, @sven1977, @akshay-anyscale, @ShaochenYu-YW, @gvspraveen, @bveeramani, @jakac, @VamshikShetty, @raulchen, @PaulFenton, @elimelt, @comaniac, @qinyiyan, @ruisearch42, @nadongjun, @AndyUB, @israbbani, @hongpeng-guo, @laysfire, @alexeykudinkin, @Drice1999, @harborn, @scottsun94, @abrarsheikh, @martinbomio, @MengjinYan, @HollowMan6, @orcahmlee, @kenchung285, @csy1204, @noemotiovon, @jujipotle, @davidxia, @kevin85421, @hcc429, @edoakes, @kouroshHakha, @omatthew98, @alanwguo, @farridav, @aslonnie, @simonsays1980, @pcmoritz, @terraflops1048576, @JoshKarpel, @SumanthRH, @sijieamoy, @zcin, @can-anyscale, @akyang-anyscale, @angelinalg, @saihaj, @jjyao, @anmscale, @ryanaoleary, @dentiny, @jimmyxie-figma, @stephanie-wang, @khluu, @maofagui

Highlights

• This release features new modules in Ray Serve and Ray Data for integration with large language models, marking the first step of addressing #50639. Existing Ray Data and Ray Serve have limited support for LLM deployments, where users have to manually configure and manage the underlying LLM engine. In this release, we offer APIs for both batch inference and serving of LLMs within Ray in ray.data.llm and ray.serve.llm. See the below notes for more details. These APIs are marked as alpha -- meaning they may change in future releases without a deprecation period.
• Ray Train V2 is available to try starting in Ray 2.43! Run your next Ray Train job with the RAY_TRAIN_V2_ENABLED=1 environment variable. See the migration guide for more information.
• A new integration with uv run that allows easily specifying Python dependencies for both driver and workers in a consistent way and enables quick iterations for development of Ray applications (#50160, 50462), check out our blog post

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New Features:

• Ray Data LLM: We are introducing a new module in Ray Data for batch inference with LLMs (currently marked as alpha). It offers a new Processor abstraction that interoperates with existing Ray Data pipelines. This abstraction can be configured two ways:
  • Using the vLLMEngineProcessorConfig, which configures vLLM to load model replicas for high throughput model inference
  • Using the HttpRequestProcessorConfig, which sends HTTP requests to an OpenAI-compatible endpoint for inference.
  • Documentation for these features can be found here.
• Implement accurate memory accounting for UnionOperator (#50436)
• Implement accurate memory accounting for all-to-all operations (#50290)

💫 Enhancements:

• Support class constructor args for filter() (#50245)
• Persist ParquetDatasource metadata. (#50332)
• Rebasing ShufflingBatcher onto try_combine_chunked_columns (#50296)
• Improve warning message if required dependency isn't installed (#50464)
• Move data-related test logic out of core tests directory (#50482)
• Pass executor as an argument to ExecutionCallback (#50165)
• Add operator id info to task+actor (#50323)
• Abstracting common methods, removing duplication in ArrowBlockAccessor, PandasBlockAccessor (#50498)
• Warn if map UDF is too large (#50611)
• Replace AggregateFn with AggregateFnV2, cleaning up Aggregation infrastructure (#50585)
• Simplify Operator.repr (#50620)
• Adding in TaskDurationStats and on_execution_step callback (#50766)
• Print Resource Manager stats in release tests (#50801)

🔨 Fixes:

• Fix invalid escape sequences in grouped_data.py docstrings (#50392)
• Deflake test_map_batches_async_generator (#50459)
• Avoid memory leak with pyarrow.infer_type on datetime arrays (#50403)
• Fix parquet partition cols to support tensors types (#50591)
• Fixing aggregation protocol to be appropriately associative (#50757)

📖 Documentation:

• Remove "Stable Diffusion Batch Prediction with Ray Data" example (#50460)

Ray Train<a id="ray-train"></a>

🎉 New Features:

• Ray Train V2 is available to try starting in Ray 2.43! Run your next Ray Train job with the RAY_TRAIN_V2_ENABLED=1 environment variable. See the migration guide for more information.

💫 Enhancements:

• Add a training ingest benchmark release test (#50019, #50299) with a fault tolerance variant (#50399)
• Add telemetry for Trainer usage in V2 (#50321)
• Add pydantic as a ray[train] extra install (#46682)
• Add state tracking to train v2 to make run status, run attempts, and training worker metadata observable (#50515)

🔨 Fixes:

• Increase doc test parallelism (#50326)
• Disable TF test for py312 (#50382)
• Increase test timeout to deflake (#50796)

📖 Documentation:

• Add missing xgboost pip install in example (#50232)

🏗 Architecture refactoring:

• Add deprecation warnings pointing to a migration guide for Ray Train V2 (#49455, #50101, #50322)
• Refactor internal Train controller state management (#50113, #50181, #50388)

Ray Tune<a id="ray-tune"></a>

🔨 Fixes:

• Fix worker node failure test (#50109)

📖 Documentation:

• Update all doc examples off of ray.train imports (#50458)
• Update all ray/tune/examples off of ray.train imports (#50435)
• Fix typos in persistent storage guide (#50127)
• Remove Binder notebook links in Ray Tune docs (#50621)

🏗 Architecture refactoring:

• Update RLlib to use ray.tune imports instead of ray.air and ray.train (#49895)

Ray Serve<a id="ray-serve"></a>

🎉 New Features:

• Ray Serve LLM: We are introducing a new module in Ray Serve to easily integrate open source LLMs in your Ray Serve deployment, currently marked as alpha. This opens up a powerful capability of composing complex applications with multiple LLMs, which is a use case in emerging applications like agentic workflows. Ray Serve LLM offers a couple core components, including:
  • VLLMService: A prebuilt deployment that offers a full-featured vLLM engine integration, with support for features such as LoRA multiplexing and multimodal language models.
  • LLMRouter: An out-of-the-box OpenAI compatible model router that can route across multiple LLM deployments.
  • Documentation can be found at https://docs.ray.io/en/releases-2.43.0/serve/llm/overview.html

💫 Enhancements:

• Add required_resources to REST API (#50058)

🔨 Fixes:

• Fix batched requests hanging after cancellation (#50054)
• Properly propagate backpressure error (#50311)

RLlib<a id="rllib"></a>

🎉 New Features:

• Added env vectorization support for multi-agent (new API stack). (#50437)

💫 Enhancements:

• APPO/IMPALA various acceleration efforts. Reached 100k ts/sec on Atari benchmark with 400 EnvRunners and 16 (multi-node) GPU Learners: #50760, #50162, #50249, #50353, #50368, #50379, #50440, #50477, #50527, #50528, #50600, #50309
• Offline RL:
  • Remove all weight synching to eval_env_runner_group from the training steps. (#50057)
  • Enable single-learner/multi-learner GPU training. (#50034)
  • Remove reference to MARWILOfflinePreLearner in OfflinePreLearner docstring. (#50107)
  • Add metrics to multi-agent replay buffers. (#49959)

🔨 Fixes:

• Fix SPOT preemption tolerance for large AlgorithmConfig: Pass by reference to RolloutWorker (#50688)
• on_workers/env_runners_recreated callback would be called twice. (#50172)
• default_resource_request: aggregator actors missing in placement group for local Learner. (#50219, #50475)

📖 Documentation:

• Docs re-do (new API stack):
  • Rewrite/enhance "getting started" rst page. (#49950)
  • Remove rllib-models.rst and fix broken html links. (#49966, #50126)

Ray Core and Ray Clusters

Ray Core<a id="ray-core"></a>

💫 Enhancements:

• [Core] Enable users to configure python standard log attributes for structured logging (#49871)
• [Core] Prestart worker with runtime env (#49994)
• [compiled graphs] Support experimental_compile(_default_communicator=comm) (#50023)
• [Core] ray.util.Queue Empty and Full exceptions extend queue.Empty and Full (#50261)
• [Core] Initial port of Ray to Python 3.13 (#47984)

🔨 Fixes:

• [Core] Ignore stale ReportWorkerBacklogRequest (#50280)
• [Core] Fix check failure due to negative available resource (#50517)

Ray Clusters <a id="ray-clusters"></a>

📖 Documentation:

• Update the KubeRay docs to v1.3.0.

Ray Dashboard <a id="ray-dashboard"></a>

🎉 New Features:

• Additional filters for job list page (#50283)

Thanks

Thank you to everyone who contributed to this release! 🥳 @liuxsh9, @justinrmiller, @CheyuWu, @400Ping, @scottsun94, @bveeramani, @bhmiller, @tylerfreckmann, @hefeiyun, @pcmoritz, @matthewdeng, @dentiny, @erictang000, @gvspraveen, @simonsays1980, @aslonnie, @shorbaji, @LeoLiao123, @justinvyu, @israbbani, @zcin, @ruisearch42, @khluu, @kouroshHakha, @sijieamoy, @SergeCroise, @raulchen, @anson627, @bluenote10, @allenyin55, @martinbomio, @rueian, @rynewang, @owenowenisme, @Betula-L, @alexeykudinkin, @crypdick, @jujipotle, @saihaj, @EricWiener, @kevin85421, @MengjinYan, @chris-ray-zhang, @SumanthRH, @chiayi, @comaniac, @angelinalg, @kenchung285, @tanmaychimurkar, @andrewsykim, @MortalHappiness, @sven1977, @richardliaw, @omatthew98, @fscnick, @akyang-anyscale, @cristianjd, @Jay-ju, @spencer-p, @win5923, @wxsms, @stfp, @letaoj, @JDarDagran, @jjyao, @srinathk10, @edoakes, @vincent0426, @dayshah, @davidxia, @DmitriGekhtman, @GeneDer, @HYLcool, @gameofby, @can-anyscale, @ryanaoleary, @eddyxu

Ray Data<a id="ray-data"></a>

🔨 Fixes:

• Fixes incorrect assertion (#50210)

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New Features:

• Added read_audio and read_video (#50016)

💫 Enhancements:

• Optimized multi-column groupbys (#45667)
• Included Ray user-agent in BigQuery client construction (#49922)

🔨 Fixes:

• Fixed bug that made read tasks non-deterministic (#49897)

🗑️ Deprecations:

• Deprecated num_rows_per_file in favor of min_rows_per_file (#49978)

Ray Train<a id="ray-train"></a>

💫 Enhancements:

• Add Train v2 user-facing callback interface (#49819)
• Add TuneReportCallback for propagating intermediate Train results to Tune (#49927)

Ray Tune<a id="ray-tune"></a>

📖 Documentation:

• Fix BayesOptSearch docs (#49848)

Ray Serve<a id="ray-serve"></a>

💫 Enhancements:

• Cache metrics in replica and report on an interval (#49971)
• Cache expensive calls to inspect.signature (#49975)
• Remove extra pickle serialization for gRPCRequest (#49943)
• Shared LongPollClient for Routers (#48807)
• DeploymentHandle API is now stable (#49840)

🔨 Fixes:

• Fix batched requests hanging after request cancellation bug (#50054)

RLlib<a id="rllib"></a>

💫 Enhancements:

• Add metrics to replay buffers. (#49822)
• Enhance node-failure tolerance (new API stack). (#50007)
• MetricsLogger cleanup throughput logic. (#49981)
• Split AddStates... connectors into 2 connector pieces (AddTimeDimToBatchAndZeroPad and AddStatesFromEpisodesToBatch) (#49835)

🔨 Fixes:

• Old API stack IMPALA/APPO: Re-introduce mixin-replay-buffer pass, even if replay-ratio=0 (fixes a memory leak). (#49964)
• Fix MetricsLogger race conditions. (#49888)
• APPO/IMPALA: Bug fix for > 1 Learner actor. (#49849)

📖 Documentation:

• New MetricsLogger API rst page. (#49538)
• Move "new API stack" info box right below page titles for better visibility. (#49921)
• Add example script for how to log custom metrics in training_step(). (#49976)
• Enhance/redo autoregressive action distribution example. (#49967)
• Make the "tiny CNN" example RLModule run with APPO (by implementing TargetNetAPI) (#49825)

Ray Core and Ray Clusters

Ray Core<a id="ray-core"></a>

💫 Enhancements:

• Only get single node info rather then all when needed (#49727)
• Introduce with_tensor_transport API (#49753)

🔨 Fixes:

• Fix tqdm manager thread safe #50040

Ray Clusters <a id="ray-clusters"></a>

🔨 Fixes:

• Fix token expiration for ray autoscaler (#48481)

Thanks

Thank you to everyone who contributed to this release! 🥳 @wingkitlee0, @saihaj, @win5923, @justinvyu, @kevin85421, @edoakes, @cristianjd, @rynewang, @richardliaw, @LeoLiao123, @alexeykudinkin, @simonsays1980, @aslonnie, @ruisearch42, @pcmoritz, @fscnick, @bveeramani, @mattip, @till-m, @tswast, @ujjawal-khare, @wadhah101, @nikitavemuri, @akshay-anyscale, @srinathk10, @zcin, @dayshah, @dentiny, @LydiaXwQ, @matthewdeng, @JoshKarpel, @MortalHappiness, @sven1977, @omatthew98

Highlights

• Major update of RLlib docs and example scripts for the new API stack.

Ray Libraries

Ray Data

🎉 New Features:

• Expression support for filters (#49016)
• Support partition_cols in write_parquet (#49411)
• Feature: implement multi-directional sort over Ray Data datasets (#49281)

💫 Enhancements:

• Use dask 2022.10.2 (#48898)
• Clarify schema validation error (#48882)
• Raise ValueError when the data sort key is None (#48969)
• Provide more messages when webdataset format is error (#48643)
• Upgrade Arrow version from 17 to 18 (#48448)
• Update hudi version to 0.2.0 (#48875)
• webdataset: expand JSON objects into individual samples (#48673)
• Support passing kwargs to map tasks. (#49208)
• Add ExecutionCallback interface (#49205)
• Add seed for read files (#49129)
• Make select_columns and rename_columns use Project operator (#49393)

🔨 Fixes:

• Fix partial function name parsing in map_groups (#48907)
• Always launch one task for read_sql (#48923)
• Reimplement of fix memory pandas (#48970)
• webdataset: flatten return args (#48674)
• Handle numpy > 2.0.0 behaviour in _create_possibly_ragged_ndarray (#48064)
• Fix DataContext sealing for multiple datasets. (#49096)
• Fix to_tf for List types (#49139)
• Fix type mismatch error while mapping nullable column (#49405)
• Datasink: support passing write results to on_write_completes (#49251)
• Fix groupby hang when value contains np.nan (#49420)
• Fix bug where file_extensions doesn't work with compound extensions (#49244)
• Fix map operator fusion when concurrency is set (#49573)

Ray Train

🎉 New Features:

• Output JSON structured log files for system and application logs (#49414)
• Add support for AMD ROCR_VISIBLE_DEVICES (#49346)

💫 Enhancements:

• Implement Train Tune API Revamp REP (#49376, #49467, #49317, #49522)

🏗 Architecture refactoring:

• LightGBM: Rewrite get_network_params implementation (#49019)

Ray Tune

🎉 New Features:

• Update optuna_search to allow users to configure optuna storage (#48547)

🏗 Architecture refactoring:

• Make changes to support Train Tune API Revamp REP (#49308, #49317, #49519)

Ray Serve

💫 Enhancements:

• Improved request_id generation to reduce proxy CPU overhead (#49537)
• Tune GC threshold by default in proxy (#49720)
• Use pickle.dumps for faster serialization from proxy to replica (#49539)

🔨 Fixes:

• Handle nested ‘=’ in serve run arguments (#49719)
• Fix bug when ray.init() is called multiple times with different runtime_envs (#49074)

🗑️ Deprecations:

• Adds a warning that the default behavior for sync methods will change in a future release. They will be run in a threadpool by default. You can opt into this behavior early by setting RAY_SERVE_RUN_SYNC_IN_THREADPOOL=1. (#48897)

RLlib

🎉 New Features:

• Add support for external Envs to new API stack: New example script and custom tcp-capable EnvRunner. (#49033)

💫 Enhancements:

• Offline RL:
  • Add sequence sampling to EpisodeReplayBuffer. (#48116)
  • Allow incomplete SampleBatch data and fully compressed observations. (#48699)
  • Add option to customize OfflineData. (#49015)
  • Enable offline training without specifying an environment. (#49041)
  • Various fixes: #48309, #49194, #49195
• APPO/IMPALA acceleration (new API stack):
  • Add support for AggregatorActors per Learner. (#49284)
  • Auto-sleep time AND thread-safety for MetricsLogger. (#48868)
  • Activate APPO cont. actions release- and CI tests (HalfCheetah-v1 and Pendulum-v1 new in tuned_examples). (#49068)
  • Add "burn-in" period setting to the training of stateful RLModules. (#49680)
• Callbacks API: Add support for individual lambda-style callbacks. (#49511)
• Other enhancements: #49687, #49714, #49693, #49497, #49800, #49098

📖 Documentation:

• New example scripts:
  • How to write a custom algorithm (VPG) from scratch. (#49536)
  • How to customize an offline data pipeline. (#49046)
  • GPUs on EnvRunners. (#49166)
  • Hierarchical training. (#49127)
  • Async gym vector env. (#49527)
  • Other fixes and enhancements: #48988, #49071
• New/rewritten html pages:
  • Rewrite checkpointing page. (#49504)
  • New scaling guide. (#49528)
  • New callbacks page. (#49513)
  • Rewrite RLModule page. (#49387)
  • New AlgorithmConfig page and redo package_ref page for algo configs. (#49464)
  • Rewrite offline RL page. (#48818)
  • Rewrite “key concepts" rst page. (#49398)
  • Rewrite RL environments pages. (#49165, #48542)
  • Fixes and enhancements: #49465, #49037, #49304, #49428, #49474, #49399, #49713, #49518

🔨 Fixes:

• Add on_episode_created callback to SingleAgentEnvRunner. (#49487)
• Fix train_batch_size_per_learner problems. (#49715)
• Various other fixes: #48540, #49363, #49418, #49191

🏗 Architecture refactoring:

• RLModule: Introduce Default[algo]RLModule classes (#49366, #49368)
• Remove RLlib dependencies from setup.py; add ormsgpack (#49489)

🗑️ Deprecations:

• #49488, #49144

Ray Core and Ray Clusters

Ray Core

💫 Enhancements:

• Add task_name, task_function_name and actor_name in Structured Logging (#48703)
• Support redis/valkey authentication with username (#48225)
• Add v6e TPU Head Resource Autoscaling Support (#48201)
• compiled graphs: Support all driver and actor read combinations (#48963)
• compiled graphs: Add ascii based CG visualization (#48315)
• compiled graphs: Add ray[cg] pip install option (#49220)
• Allow uv cache at installation (#49176)
• Support != Filter in GCS for Task State API (#48983)
• compiled graphs: Add CPU-based NCCL communicator for development (#48440)
• Support gcs and raylet log rotation (#48952)
• compiled graphs: Support nsight.nvtx profiling (#49392)

🔨 Fixes:

• autoscaler: Health check logs are not visible in the autoscaler container's stdout (#48905)
• Only publish WORKER_OBJECT_EVICTION when the object is out of scope or manually freed (#47990)
• autoscaler: Autoscaler doesn't scale up correctly when the KubeRay RayCluster is not in the goal state (#48909)
• autoscaler: Fix incorrectly terminating nodes misclassified as idle in autoscaler v1 (#48519)
• compiled graphs: Fix the missing dependencies when num_returns is used (#49118)
• autoscaler: Fuse scaling requests together to avoid overloading the Kubernetes API server (#49150)
• Fix bug to support S3 pre-signed url for .whl file (#48560)
• Fix data race on gRPC client context (#49475)
• Make sure draining node is not selected for scheduling (#49517)

Ray Clusters

💫 Enhancements:

• Azure: Enable accelerated networking as a flag in azure vms (#47988)

📖 Documentation:

• Kuberay: Logging: Add Fluent Bit DaemonSet and Grafana Loki to "Persist KubeRay Operator Logs" (#48725)
• Kuberay: Logging: Specify the Helm chart version in "Persist KubeRay Operator Logs" (#48937)

Dashboard

💫 Enhancements:

• Add instance variable to many default dashboard graphs (#49174)
• Display duration in milliseconds if under 1 second. (#49126)
• Add RAY_PROMETHEUS_HEADERS env for carrying additional headers to Prometheus (#49353)
• Document about the RAY_PROMETHEUS_HEADERS env for carrying additional headers to Prometheus (#49700)

🏗 Architecture refactoring:

• Move memray dependency from default to observability (#47763)
• Move StateHead's methods into free functions. (#49388)

Thanks

@raulchen, @alanwguo, @omatthew98, @xingyu-long, @tlinkin, @yantzu, @alexeykudinkin, @andrewsykim, @win5923, @csy1204, @dayshah, @richardliaw, @stephanie-wang, @gueraf, @rueian, @davidxia, @fscnick, @wingkitlee0, @KPostOffice, @GeneDer, @MengjinYan, @simonsays1980, @pcmoritz, @petern48, @kashiwachen, @pfldy2850, @zcin, @scottjlee, @Akhil-CM, @Jay-ju, @JoshKarpel, @edoakes, @ruisearch42, @gorloffslava, @jimmyxie-figma, @bthananjeyan, @sven1977, @bnorick, @jeffreyjeffreywang, @ravi-dalal, @matthewdeng, @angelinalg, @ivanthewebber, @rkooo567, @srinathk10, @maresb, @gvspraveen, @akyang-anyscale, @mimiliaogo, @bveeramani, @ryanaoleary, @kevin85421, @richardsliu, @hartikainen, @coltwood93, @mattip, @Superskyyy, @justinvyu, @hongpeng-guo, @ArturNiederfahrenhorst, @jecsand838, @Bye-legumes, @hcc429, @WeichenXu123, @martinbomio, @HollowMan6, @MortalHappiness, @dentiny, @zhe-thoughts, @anyadontfly, @smanolloff, @richo-anyscale, @khluu, @xushiyan, @rynewang, @japneet-anyscale, @jjyao, @sumanthratna, @saihaj, @aslonnie

Many thanks to all those who contributed to this release!

Ray Libraries

Ray Data

🎉 New Features:

• Added read_hudi (https://github.com/ray-project/ray/pull/46273)

💫 Enhancements:

• Improved performance of DelegatingBlockBuilder (https://github.com/ray-project/ray/pull/48509)
• Improved memory accounting of pandas blocks (https://github.com/ray-project/ray/pull/46939)

🔨 Fixes:

• Fixed bug where you can’t specify a schema with write_parquet (https://github.com/ray-project/ray/issues/48630)
• Fixed bug where to_pandas errors if your dataset contains Arrow and pandas blocks (https://github.com/ray-project/ray/pull/48583)
• Fixed bug where map_groups doesn’t work with pandas data (https://github.com/ray-project/ray/pull/48287)
• Fixed bug where write_parquet errors if your data contains nullable fields (https://github.com/ray-project/ray/pull/48478)
• Fixed bug where “Iteration Blocked Time” charts looks incorrect (https://github.com/ray-project/ray/pull/48618)
• Fixed bug where unique fails with null values (https://github.com/ray-project/ray/pull/48750)
• Fixed bug where “Rows Outputted” is 0 in the Data dashboard (https://github.com/ray-project/ray/pull/48745)
• Fixed bug where methods like drop_columns cause spilling (https://github.com/ray-project/ray/pull/48140)
• Fixed bug where async map tasks hang (https://github.com/ray-project/ray/pull/48861)

🗑️ Deprecations:

• Deprecated read_parquet_bulk https://github.com/ray-project/ray/pull/48691
• Deprecated iter_tf_batches https://github.com/ray-project/ray/pull/48693
• Deprecated meta_provider parameter of read functions (https://github.com/ray-project/ray/pull/48690)
• Deprecated to_torch (https://github.com/ray-project/ray/pull/48692)

Ray Train

🔨 Fixes:

• Fix StartTracebackWithWorkerRank serialization (#48548)

📖 Documentation:

• Add example for fine-tuning Llama3.1 with AWS Trainium (#48768)

Ray Tune

🔨 Fixes:

• Remove the clear_checkpoint function during Trial restoration error handling. (#48532)

Ray Serve

🎉 New Features:

• Initial version of local_testing_mode (#48477)

💫 Enhancements:

• Handle multiple changed objects per LongPollHost.listen_for_change RPC (#48803)
• Add more nuanced checks for http proxy status errors (#47896)
• Improve replica access log messages to include HTTP status info and better resemble standard log format (#48819)
• Propagate replica constructor error to deployment status message and print num retries left (#48531)

🔨 Fixes:

• Pending requests that are cancelled before they were assigned to a replica now also return a serve.RequestCancelledError (#48496)

RLlib

💫 Enhancements:

• Release test enhancements. (#45803, #48681)
• Make opencv-python-headless default over opencv-python (#48776)
• Reverse learner queue behavior of IMPALA/APPO (consume oldest batches first, instead of newest, BUT drop oldest batches if queue full). (#48702)

🔨 Fixes:

• Fix torch scheduler stepping and reporting. (#48125)
• Fix accumulation of results over n training_step calls within same iteration (new API stack). (#48136)
• Various other fixes: #48563, #48314, #48698, #48869.

📖 Documentation:

• Upgrade examples script overview page (new API stack). (#48526)
• Enable RLlib + Serve example in CI and translate to new API stack. (#48687)

🏗 Architecture refactoring:

• Switch new API stack on by default, APPO, IMPALA, BC, MARWIL, and CQL. (#48516, #48599)
• Various APPO enhancements (new API stack): Circular buffer (#48798), minor loss math fixes (#48800), target network update logic (#48802), smaller cleanups (#48844).
• Remove rllib_contrib from repo. (#48565)

Ray Core and Ray Clusters

Ray Core

🎉 New Features:

• [Core] uv runtime env support (#48479, #48486, #48611, #48619, #48632, #48634, #48637, #48670, #48731)
• [Core] GCS FT with redis sentinel (#47335)

💫 Enhancements:

• [CompiledGraphs] Refine schedule visualization (#48594)

🔨 Fixes:

• [CompiledGraphs] Don't persist input_nodes in _CollectiveOperation to avoid wrong understanding about DAGs (#48463)
• [Core] Fix Ascend NPU discovery to support 8+ cards per node (#48543)
• [Core] Make Placement Group Wildcard and Indexed Resource Assignments Consistent (#48088)
• [Core] Stop the GRPC server before Shut down the Object Store (#48572)

Ray Clusters

🔨 Fixes:

• [KubeRay]: Fix ConnectionError on Autoscaler CR lookups in K8s clusters with custom DNS for Kubernetes API. (#48541)

Dashboard

💫 Enhancements:

• Add global UTC timezone button in navbar with local storage (#48510)
• Add memory graphs optimized for OOM debugging (#48530)
• Improve tasks/actors metric naming and add graph for running tasks (#48528) add actor pid to dashboard (#48791)

🔨 Fixes:

• Fix Placement Group Table table cells overflow (#47323)
• Fix Rows Outputted being zero on Ray Data Dashboard (#48745)
• fix confusing dataset operator name (#48805)

Thanks

Thanks to all those who contributed to this release! @rynewang, @rickyyx, @bveeramani, @marwan116, @simonsays1980, @dayshah, @dentiny, @KepingYan, @mimiliaogo, @kevin85421, @SeaOfOcean, @stephanie-wang, @mohitjain2504, @azayz, @xushiyan, @richardliaw, @can-anyscale, @xingyu-long, @kanwang, @aslonnie, @MortalHappiness, @jjyao, @SumanthRH, @matthewdeng, @alexeykudinkin, @sven1977, @raulchen, @andrewsykim, @zcin, @nadongjun, @hongpeng-guo, @miguelteixeiraa, @saihaj, @khluu, @ArturNiederfahrenhorst, @ryanaoleary, @ltbringer, @pcmoritz, @JoshKarpel, @akyang-anyscale, @frances720, @BeingGod, @edoakes, @Bye-legumes, @Superskyyy, @liuxsh9, @MengjinYan, @ruisearch42, @scottjlee, @angelinalg

Ray Libraries

Ray Data

🔨 Fixes:

• Fixed InvalidObjectError edge case with Dataset.split() (https://github.com/ray-project/ray/pull/48130)
• Made Concatenator preserve order of concatenated columns (https://github.com/ray-project/ray/pull/47997)

📖 Documentation:

• Improved documentation around Parquet column and predicate pushdown (https://github.com/ray-project/ray/pull/48095)
• Marked num_rows_per_file parameter of write APIs as experimental (https://github.com/ray-project/ray/pull/48208)
• One hot encoder now returns an encoded vector (https://github.com/ray-project/ray/pull/48173)
• transform_batch no longer fails on missing columns (https://github.com/ray-project/ray/pull/48137)

🏗 Architecture refactoring:

• Dataset.count() now uses a Count logical operator (https://github.com/ray-project/ray/pull/48126)

🗑 Deprecations:

• Removed long-deprecated set_progress_bars (https://github.com/ray-project/ray/pull/48203)

Ray Train

🔨 Fixes:

• Safely check if the storage filesystem is pyarrow.fs.S3FileSystem (#48216)

Ray Tune

🔨 Fixes:

• Safely check if the storage filesystem is pyarrow.fs.S3FileSystem (#48216)

Ray Serve

💫 Enhancements:

• Cancelled requests now return a serve.RequestCancelledError (https://github.com/ray-project/ray/pull/48444)
• Exposed application source in app details model (https://github.com/ray-project/ray/pull/45522)

🔨 Fixes:

• Basic HTTP deployments will now return “Internal Server Error” instead of a traceback to match FastAPI behavior (https://github.com/ray-project/ray/pull/48491)
• Fixed an issue where high values of max_ongoing_requests couldn’t be reached due to an interaction with core’s max_concurrency (https://github.com/ray-project/ray/pull/48274)
• Fixed an edge case where pending requests were not canceled properly (https://github.com/ray-project/ray/pull/47873)
• Removed deprecated API to set route_prefix per-deployment (https://github.com/ray-project/ray/pull/48223)

📖 Documentation:

• Added ProxyStatus model to reference docs (https://github.com/ray-project/ray/pull/48299)
• Added ApplicationStatus model to reference docs (https://github.com/ray-project/ray/pull/48220)

RLlib

💫 Enhancements:

• Upgrade to gymnasium==1.0.0 (support new API for vector env resets). (#48443, #45328)
• Add off-policy'ness metric to new API stack. (#48227)
• Validate episodes before adding them to the buffer. (#48083)

📖 Documentation:

• New example script for custom metrics on EnvRunners (using MetricsLogger API on the new stack). (#47969)
• Do-over: New RLlib index page. (#48285, #48442)
• Do-over: Example script for AutoregressiveActionsRLM. (#47972)

🏗 Architecture refactoring:

• New API stack on by default for PPO. (#48284)
• Change config.fault_tolerance default behavior (from recreate_failed_env_runners=False to True). (#48286)

🔨 Fixes:

• Various bug and CI fixes: #47993, #48450, #48213
• Cleanup evaluation folder (#48493)

Ray Core

🎉 New Features:

• [CompiledGraphs] Support all reduce collective in aDAG (#47621)
• [CompiledGraphs] Add visualization of compiled graphs (#47958)

💫 Enhancements:

• [Distributed Debugger] The distributed debugger can now be used without having to set RAY_DEBUG=1, see https://github.com/ray-project/ray/pull/48301 and https://docs.ray.io/en/latest/ray-observability/ray-distributed-debugger.html. If you want to restore the previous behavior and use the CLI based debugger, you need to set RAY_DEBUG=legacy.
• [Core] Add more infos to each breakpoint for ray debug CLI (#48202)
• [Core] Add demands info to GCS debug state (#48115)
• [Core] Add PENDING_ACTOR_TASK_ARGS_FETCH and PENDING_ACTOR_TASK_ORDERING_OR_CONCURRENCY TaskStatus (#48242)
• [Core] Add metrics ray_io_context_event_loop_lag_ms. (#47989)
• [Core] Better log format when show the disk size (#46869)
• [CompiledGraphs] Support asyncio.gather on multiple CompiledDAGFutures (#47860)
• [CompiledGraphs] Raise an exception if a leaf node is found during compilation (#47757)

🔨 Fixes:

• [Core] Posts CoreWorkerMemoryStore callbacks onto io_context to fix deadlock (#47833)

Dashboard

🔨 Fixes:

• [Dashboard] Reworking dashboard_max_actors_to_cache to RAY_maximum_gcs_destroyed_actor_cached_count (#48229)

Thanks

Many thanks to all those who contributed to this release!

@akyang-anyscale, @rkooo567, @bveeramani, @dayshah, @martinbomio, @khluu, @justinvyu, @slfan1989, @alexeykudinkin, @simonsays1980, @vigneshka, @ruisearch42, @rynewang, @scottjlee, @jjyao, @JoshKarpel, @win5923, @MengjinYan, @MortalHappiness, @ujjawal-khare-27, @zcin, @ccoulombe, @Bye-legumes, @dentiny, @stephanie-wang, @LeoLiao123, @dengwxn, @richo-anyscale, @pcmoritz, @sven1977, @omatthew98, @GeneDer, @srinathk10, @can-anyscale, @edoakes, @kevin85421, @aslonnie, @jeffreyjeffreywang, @ArturNiederfahrenhorst

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New Features:

• Add Dataset.rename_columns (#47906)
• Basic structured logging (#47210)

💫 Enhancements:

• Add partitioning parameter to read_parquet (#47553)
• Add SERVICE_UNAVAILABLE to list of retried transient errors (#47673)
• Re-phrase the streaming executor current usage string (#47515)
• Remove ray.kill in ActorPoolMapOperator (#47752)
• Simplify and consolidate progress bar outputs (#47692)
• Refactor OpRuntimeMetrics to support properties (#47800)
• Refactor plan_write_op and Datasinks (#47942)
• Link PhysicalOperator to its LogicalOperator (#47986)
• Allow specifying both num_cpus and num_gpus for map APIs (#47995)
• Allow specifying insertion index when registering custom plan optimization Rules (#48039)
• Adding in better framework for substituting logging handlers (#48056)

🔨 Fixes:

• Fix bug where Ray Data incorrectly emits progress bar warning (#47680)
• Yield remaining results from async map_batches (#47696)
• Fix event loop mismatch with async map (#47907)
• Make sure num_gpus provide to Ray Data is appropriately passed to ray.remote call (#47768)
• Fix unequal partitions when grouping by multiple keys (#47924)
• Fix reading multiple parquet files with ragged ndarrays (#47961)
• Removing unneeded test case (#48031)
• Adding in better json checking in test logging (#48036)
• Fix bug with inserting custom optimization rule at index 0 (#48051)
• Fix logging output from write_xxx APIs (#48096)

📖 Documentation:

• Add docs section for Ray Data progress bars (#47804)
• Add reference to parquet predicate pushdown (#47881)
• Add tip about how to understand map_batches format (#47394)

Ray Train<a id="ray-train"></a>

🏗 Architecture refactoring:

• Remove deprecated mosaic and sklearn trainer code (#47901)

Ray Tune<a id="ray-tune"></a>

🔨 Fixes:

• Fix WandbLoggerCallback to reuse actors upon restore (#47985)

Ray Serve<a id="ray-serve"></a>

🔨 Fixes:

• Stop scheduling task early when requests have been canceled (#47847)

RLlib<a id="rllib"></a>

🎉 New Features:

• Enable cloud checkpointing. (#47682)

💫 Enhancements:

• PPO on new API stack now shuffles batches properly before each epoch. (#47458)
• Other enhancements: #47705, #47501, #47731, #47451, #47830, #47970, #47157

🔨 Fixes:

• Fix spot node preemption problem (RLlib now run stably with EnvRunner workers on spot nodes) (#47940)
• Fix action masking example. (#47817)
• Various other fixes: #47973, #46721, #47914, #47880, #47304, #47686

🏗 Architecture refactoring:

• Switch on new API stack by default for SAC and DQN. (#47217)
• Remove Tf support on new API stack for PPO/IMPALA/APPO (only DreamerV3 on new API stack remains with tf now). (#47892)
• Discontinue support for "hybrid" API stack (using RLModule + Learner, but still on RolloutWorker and Policy) (#46085)
• RLModule (new API stack) refinements: #47884, #47885, #47889, #47908, #47915, #47965, #47775

📖 Documentation:

• Add new API stack migration guide. (#47779)
• New API stack example script: BC pre training, then PPO finetuning using same RLModule class. (#47838)
• New API stack: Autoregressive actions example. (#47829)
• Remove old API stack connector docs entirely. (#47778)

Ray Core and Ray Clusters

Ray Core <a id="ray-core"></a>

🎉 New Features:

• CompiledGraphs: support multi readers in multi node when DAG is created from an actor (#47601)

💫 Enhancements:

• Add a flag to raise exception for out of band serialization of ObjectRef (#47544)
• Store each GCS table in its own Redis Hash (#46861)
• Decouple create worker vs pop worker request. (#47694)
• Add metrics for GCS jobs (#47793)

🔨 Fixes:

• Fix broken dashboard cluster page when there are dead nodes (#47701)
• Fix the ray_tasks{State="PENDING_ARGS_FETCH"} metric counting (#47770)
• Separate the attempt_number with the task_status in memory summary and object list (#47818)
• Fix object reconstruction hang on arguments pending creation (#47645)
• Fix check failure: sync_reactors_.find(reactor->GetRemoteNodeID()) == sync_reactors_.end() (#47861)
• Fix check failure RAY_CHECK(it != current_tasks_.end()); (#47659)

📖 Documentation:

• KubeRay docs: Add docs for YuniKorn Gang scheduling #47850

Dashboard<a id="dashboard"></a>

💫 Enhancements:

• Performance improvements for large scale clusters (#47617)

🔨 Fixes:

• Placement group and required resources not showing correctly in dashboard (#47754)

Thanks

Many thanks to all those who contributed to this release! @GeneDer, @rkooo567, @dayshah, @saihaj, @nikitavemuri, @bill-oconnor-anyscale, @WeichenXu123, @can-anyscale, @jjyao, @edoakes, @kekulai-fredchang, @bveeramani, @alexeykudinkin, @raulchen, @khluu, @sven1977, @ruisearch42, @dentiny, @MengjinYan, @Mark2000, @simonsays1980, @rynewang, @PatricYan, @zcin, @sofianhnaide, @matthewdeng, @dlwh, @scottjlee, @MortalHappiness, @kevin85421, @win5923, @aslonnie, @prithvi081099, @richardsliu, @milesvant, @omatthew98, @Superskyyy, @pcmoritz

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

💫 Enhancements:

• Simplify custom metadata provider API (#47575)
• Change counts of metrics to rates of metrics (#47236)
• Throw exception for non-streaming HF datasets with "override_num_blocks" argument (#47559)
• Refactor custom optimizer rules (#47605)

🔨 Fixes:

• Remove ineffective retry code in plan_read_op (#47456)
• Fix incorrect pending task size if outputs are empty (#47604)

Ray Train<a id="ray-train"></a>

💫 Enhancements:

• Update run status and add stack trace to TrainRunInfo (#46875)

Ray Serve<a id="ray-serve"></a>

💫 Enhancements:

• Allow control of some serve configuration via env vars (#47533)
• [serve] Faster detection of dead replicas (#47237)

🔨 Fixes:

• [Serve] fix component id logging field (#47609)

RLlib<a id="rllib"></a>

💫 Enhancements:

• New API stack:
  • Add restart-failed-env option to EnvRunners. (#47608)
  • Offline RL: Store episodes in state form. (#47294)
  • Offline RL: Replace GAE in MARWILOfflinePreLearner with GeneralAdvantageEstimation connector in learner pipeline. (#47532)
  • Off-policy algos: Add episode sampling to EpisodeReplayBuffer. (#47500)
  • RLModule APIs: Add SelfSupervisedLossAPI for RLModules that bring their own loss and InferenceOnlyAPI. (#47581, #47572)

Ray Core<a id="ray-core"></a>

💫 Enhancements:

• [aDAG] Allow custom NCCL group for aDAG (#47141)
• [aDAG] support buffered input (#47272)
• [aDAG] Support multi node multi reader (#47480)
• [Core] Make is_gpu, is_actor, root_detached_id fields late bind to workers. (#47212)
• [Core] Reconstruct actor to run lineage reconstruction triggered actor task (#47396)
• [Core] Optimize GetAllJobInfo API for performance (#47530)

🔨 Fixes:

• [aDAG] Fix ranks ordering for custom NCCL group (#47594)

Ray Clusters<a id="ray-clusters"></a>

📖 Documentation:

• [KubeRay] add a guide for deploying vLLM with RayService (#47038)

Thanks

Many thanks to all those who contributed to this release! @ruisearch42, @andrewsykim, @timkpaine, @rkooo567, @WeichenXu123, @GeneDer, @sword865, @simonsays1980, @angelinalg, @sven1977, @jjyao, @woshiyyya, @aslonnie, @zcin, @omatthew98, @rueian, @khluu, @justinvyu, @bveeramani, @nikitavemuri, @chris-ray-zhang, @liuxsh9, @xingyu-long, @peytondmurray, @rynewang

Ray Core<a id="ray-core"></a>

🔨 Fixes:

• Fix broken dashboard cluster page when there are dead nodes (#47701)
• Fix broken dashboard worker page (#47714)

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

💫 Enhancements:

• Remove limit on number of tasks launched per scheduling step (#47393)
• Allow user-defined Exception to be caught. (#47339)

🔨 Fixes:

• Display pending actors separately in the progress bar and not count them towards running resources (#46384)
• Fix bug where arrow_parquet_args aren't used (#47161)
• Skip empty JSON files in read_json() (#47378)
• Remove remote call for initializing Datasource in read_datasource() (#47467)
• Remove dead from_*_operator modules (#47457)
• Release test fixes
• Add AWS ACCESS_DENIED as retryable exception for multi-node Data+Train benchmarks (#47232)
• Get AWS credentials with boto (#47352)
• Use worker node instead of head node for read_images_comparison_microbenchmark_single_node release test (#47228)

📖 Documentation:

• Add docstring to explain Dataset.deserialize_lineage (#47203)
• Add a comment explaining the bundling behavior for map_batches with default batch_size (#47433)

Ray Train<a id="ray-train"></a>

💫 Enhancements:

• Decouple device-related modules and add Huawei NPU support to Ray Train (#44086)

🔨 Fixes:

• Update TORCH_NCCL_ASYNC_ERROR_HANDLING env var (#47292)

📖 Documentation:

• Add missing Train public API reference (#47134)

Ray Tune<a id="ray-tune"></a>

📖 Documentation:

• Add missing Tune public API references (#47138)

Ray Serve<a id="ray-serve"></a>

💫 Enhancements:

• Mark proxy as unready when its routers are aware of zero replicas (#47002)
• Setup default serve logger (#47229)

🔨 Fixes:

• Allow get_serve_logs_dir to run outside of Ray's context (#47224)
• Use serve logger name for logs in serve (#47205)

📖 Documentation:

• [HPU] [Serve] [experimental] Add vllm HPU support in vllm example (#45893)

🏗 Architecture refactoring:

• Remove support for nested DeploymentResponses (#47209)

RLlib<a id="rllib"></a>

🎉 New Features:

• New API stack: Add CQL algorithm. (#47000, #47402)
• New API stack: Enable GPU and multi-GPU support for DQN/SAC/CQL. (#47179)

💫 Enhancements:

• New API stack: Offline RL enhancements: #47195, #47359
• Enhance new API stack stability: #46324, #47196, #47245, #47279
• Fix large batch size for synchronous algos (e.g. PPO) after EnvRunner failures. (#47356)
• Add torch.compile config options to old API stack. (#47340)
• Add kwargs to torch.nn.parallel.DistributedDataParallel (#47276)
• Enhanced CI stability: #47197, #47249

📖 Documentation:

• New API stack example scripts:
  • Float16 training example script. (#47362)
  • Mixed precision training example script (#47116)
  • ModelV2 -> RLModule wrapper for migrating to new API stack. (#47425)
• Remove "new API stack experimental" hint from docs. (#47301)

🏗 Architecture refactoring:

• Remove 2nd Learner ConnectorV2 pass from PPO (#47401)
• Add separate learning rates for policy and alpha to SAC. (#47078)

🔨 Fixes:

• Various bug fixes: #47401, #47194, #47259, #47271, #47277, #47382

Ray Core<a id="ray-core"></a>

💫 Enhancements:

• [ADAG] Raise proper error message for nccl within the same actor (#47250)
• [ADAG] Support multi-read of the same shm channel (#47311)
• Log why core worker is not idle during HandleExit (#47300)
• Add PREPARED state for placement groups in GCS for better fault tolerance. (#46858)

🔨 Fixes:

• Fix ray_unintentional_worker_failures_total to only count unintentional worker failures (#47368)
• Fix runtime env race condition when uploading the same package concurrently (#47482)

Dashboard<a id="dashboard"></a>

🔨 Fixes:

• Performance optimizations for dashboard backend logic (#47392) (#47367) (#47160) (#47213)
• Refactor to simplify dashboard backend logic (#47324)

Docs<a id="dashboard"></a>

💫 Enhancements:

• Add sphinx-autobuild and documentation for make local (#47275): Speed up of local docs builds with make local.
• Add Algolia search to docs (#46477)
• Update PyTorch Mnist Training doc for KubeRay 1.2.0 (#47321)
• Life-cycle of documentation policy of Ray APIs

Thanks

Many thanks to all those who contributed to this release! @GeneDer, @Bye-legumes, @nikitavemuri, @kevin85421, @MortalHappiness, @LeoLiao123, @saihaj, @rmcsqrd, @bveeramani, @zcin, @matthewdeng, @raulchen, @mattip, @jjyao, @ruisearch42, @scottjlee, @can-anyscale, @khluu, @aslonnie, @rynewang, @edoakes, @zhanluxianshen, @venkatram-dev, @c21, @allenyin55, @alexeykudinkin, @snehakottapalli, @BitPhinix, @hongchaodeng, @dengwxn, @liuxsh9, @simonsays1980, @peytondmurray, @KepingYan, @bryant1410, @woshiyyya, @sven1977

Notice: Starting from this release, pip install ray[all] will not include ray[cpp], and will not install the respective ray-cpp package. To install everything that includes ray-cpp, one can use pip install ray[cpp-all] instead.

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New Features:

• Upgrade supported Arrow version from 16 to 17 (#47034)
• Add support for reading from Iceberg (#46889)

💫 Enhancements:

• Various Progress Bar UX improvements (#46816, #46801, #46826, #46692, #46699, #46974, #46928, #47029, #46924, #47120, #47095, #47106)
• Try get size_bytes from metadata and consolidate metadata methods (#46862)
• Improve warning message when read task is large (#46942)
• Extend API to enable passing sample weights via ray.dataset.to_tf (#45701)
• Add a parameter to allow overriding LanceDB scanner options (#46975)
• Add failure retry logic for read_lance (#46976)
• Clarify warning for reading old Parquet data (#47049)
• Move datasource implementations to _internal subpackage (#46825)
• Handle logs from tensor extensions (#46943)

🔨 Fixes:

• Change type of DataContext.retried_io_errors from tuple to list (#46884)
• Make Parquet tests more robust and expose Parquet logic (#46944)
• Change pickling log level from warning to debug (#47032)
• Add validation for shuffle arg (#47055)
• Fix validation bug when size=0 in ActorPoolStrategy (#47072)
• Fix exception in async map (#47110)
• Fix wrong metrics group for Object Store Memory metrics on Ray Data Dashboard (#47170)
• Handle errors in SplitCoordinator when generating a new epoch (#47176)

📖 Documentation:

• Auto-gen GroupedData api (#46925)
• Fix signature of Rule.plan (#47094)

Ray Train<a id="ray-train"></a>

💫 Enhancements:

• [train] Updates to support xgboost==2.1.0 (#46667)
• [train] Add hardware stats (#46719)

Ray Tune<a id="ray-tune"></a>

🔨 Fixes:

• [RLlib; Tune] Fix WandB metric overlap after restore from checkpoint. (#46897)

Ray Serve<a id="ray-serve"></a>

💫 Enhancements:

• Improved handling of replica death and replica unavailability in deployment handle routers before controller restarts replica (#47008)
• Eagerly create routers in proxy for better GCS fault tolerance (#47031)
• Immediately send ping in router when receiving new replica set (#47053)

🏗 Architecture refactoring:

• Deprecate passing arguments that contain DeploymentResponses in nested objects to downstream deployment handle calls (#46806)

RLlib<a id="rllib"></a>

🎉 New Features:

• Offline RL on the new API stack:
  • Record offline data (#46818, #47046, #47133, #47155) and support to directly read from episodes. (#46865)
  • RLUnplugged example. (#46792)
  • Progress on BC/MARWIL migration: #44970, #47154, #46799
  • Progress on CQL migration: #46969, #47105

💫 Enhancements:

• Add ObservationPreprocessor (ConnectorV2). (#47077)

🔨 Fixes:

• New API stack: Fix IMPALA/APPO + LSTM for single- and multi-GPU. (#47132, #47158)
• Various bug fixes: #46898, #47047, #46963, #47021, #46897
• Add more control to Algorithm.add_module/policy methods. (#46932, #46836)

📖 Documentation:

• Example scripts for new API stack:
  • Curiosity (inverse dynamics model-based) RLModule example. (#46841)
  • Add example script for Env with protobuf observation space. (#47071)
• New API stack documentation:
  • Cleanup old API stack docs (rllib-dev.rst). (#47172)
  • Episodes (SingleAgentEpisode). (#46985)
  • Redo rllib-algorithms.rst page. (#46916)

🏗 Architecture refactoring:

• Rename MultiAgent...RLModule... into MultiRL...Module for more generality. (#46840)
• Add learner_only flag to RLModuleConfig/Spec and simplify creation of RLModule specs from algo-config. (#46900)

Ray Core<a id="ray-core"></a>

💫 Enhancements:

• Emit total lineage bytes metrics (#46725)
• Adding accelerator type H100 (#46823)
• More structured logging in core worker (#46906)
• Change all callbacks to move to save copies. (#46971)
• Add ray[adag] option to pip install (#47009)

🔨 Fixes:

• Fix dashboard process reporting on windows (#45578)
• Fix Ray-on-Spark cluster crashing bug when user cancels cell execution (#46899)
• Fix PinExistingReturnObject segfault by passing owner_address (#46973)
• Fix raylet CHECK failure from runtime env creation failure. (#46991)
• Fix typo in memray command (#47006)
• [ADAG] Fix for asyncio outputs (#46845)

📖 Documentation:

• Clarify behavior of placement_group_capture_child_tasks in docs (#46885)
• Update ray.available_resources() docstring (#47018)

🏗 Architecture refactoring:

• Async APIs for the New GcsClient. (#46788)
• Replace GCS stubs in the dashboard to use NewGcsAioClient. (#46846)

Dashboard<a id="dashboard"></a>

💫 Enhancements:

• Polish and minor improvements to the Serve page (#46811)

🔨 Fixes:

• Fix CPU/GPU/RAM not being reported correctly on Windows (#44578)

Docs<a id="dashboard"></a>

💫 Enhancements:

• Add more information about developer tooling for docs contributions (#46636), including esbonio section

🔨 Fixes:

• Use PyData Sphinx theme version switcher (#46936)

Thanks

Many thanks to all those who contributed to this release! @simonsays1980, @bveeramani, @tungh2, @zcin, @xingyu-long, @WeichenXu123, @aslonnie, @MaxVanDijck, @can-anyscale, @galenhwang, @omatthew98, @matthewdeng, @raulchen, @sven1977, @shrekris-anyscale, @deepyaman, @alexeykudinkin, @stephanie-wang, @kevin85421, @ruisearch42, @hongchaodeng, @khluu, @alanwguo, @hongpeng-guo, @saihaj, @Superskyyy, @tespent, @slfan1989, @justinvyu, @rynewang, @nikitavemuri, @amogkam, @mattip, @dev-goyal, @ryanaoleary, @peytondmurray, @edoakes, @venkatajagannath, @jjyao, @cristianjd, @scottjlee, @Bye-legumes

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

💫 Enhancements:

• Add better support for UDF returns from list of datetime objects (#46762)

🔨 Fixes:

• Remove read task warning if size bytes not set in metadata (#46765)

📖 Documentation:

• Fix read_tfrecords() docstring to display tfx-bsl tip (#46717)
• Update Dataset.zip() docs (#46757)

Ray Train<a id="ray-train"></a>

🔨 Fixes:

• Sort workers by node ID rather than by node IP (#46163)

🏗 Architecture refactoring:

• Remove dead RayDatasetSpec (#46764)

RLlib<a id="rllib"></a>

🎉 New Features:

• Offline RL support on new API stack:
  • Initial design for Ray-Data based offline RL Algos (on new API stack). (#44969)
  • Add user-defined schemas for data loading. (#46738)
  • Make data pipeline better configurable and tuneable for users. (#46777)

💫 Enhancements:

• Move DQN into the TargetNetworkAPI (and deprecate RLModuleWithTargetNetworksInterface). (#46752)

🔨 Fixes:

• Numpy version fix: Rename all np.product usage to np.prod (#46317)

📖 Documentation:

• Examples for new API stack: Add 2 (count-based) curiosity examples. (#46737)
• Remove RLlib CLI from docs (soon to be deprecated and replaced by python API). (#46724)

🏗 Architecture refactoring:

• Cleanup, rename, clarify: Algorithm.workers/evaluation_workers, local_worker(), etc.. (#46726)

Ray Core<a id="ray-core"></a>

🏗 Architecture refactoring:

• New python GcsClient binding (#46186)

Many thanks to all those who contributed to this release! @KyleKoon, @ruisearch42, @rynewang, @sven1977, @saihaj, @aslonnie, @bveeramani, @akshay-anyscale, @kevin85421, @omatthew98, @anyscalesam, @MaxVanDijck, @justinvyu, @simonsays1980, @can-anyscale, @peytondmurray, @scottjlee

Ray Libraries<a id="ray-libraries"></a>

Ray Core<a id="ray-core"></a>

💫 Enhancements:

• Add "last exception" to error message when GCS connection fails in ray.init() (#46516)

🔨 Fixes:

• Add object back to memory store when object recovery is skipped (#46460)
• Task status should start with PENDING_ARGS_AVAIL when retry (#46494)
• Fix ObjectFetchTimedOutError (#46562)
• Make working_dir support files created before 1980 (#46634)
• Allow full path in conda runtime env. (#45550)
• Fix worker launch time formatting in state api (#43516)

Ray Data<a id="ray-data"></a>

🎉 New Features:

• Deprecate Dataset.get_internal_block_refs() (#46455)
• Add read API for reading Databricks table with Delta Sharing (#46072)
• Add support for objects to Arrow blocks (#45272)

💫 Enhancements:

• Change offsets to int64 and change to LargeList for ArrowTensorArray (#45352)
• Prevent from_pandas from combining input blocks (#46363)
• Update Dataset.count() to avoid unnecessarily keeping BlockRefs in-memory (#46369)
• Use Set to fix inefficient iteration over Arrow table columns (#46541)
• Add AWS Error UNKNOWN to list of retried write errors (#46646)
• Always print traceback for internal exceptions (#46647)
• Allow unknown estimate of operator output bundles and ProgressBar totals (#46601)
• Improve filesystem retry coverage (#46685)

🔨 Fixes:

• Replace lambda mutable default arguments (#46493)

📖 Documentation:

• Auto-generate Dataset API documentation (#46557)
• Update outdated ExecutionPlan docstring (#46638)

Ray Train<a id="ray-train"></a>

💫 Enhancements:

• Update run status and actor status for train runs. (#46395)

🔨 Fixes:

• Replace lambda default arguments (#46576)

📖 Documentation:

• Add MNIST training using KubeRay doc page (#46123)
• Add example of pre-training Llama model on Intel Gaudi (#45459)
• Fix tensorflow example by using ScalingConfig (#46565)

Ray Tune<a id="ray-tune"></a>

🔨 Fixes:

• Replace lambda default arguments (#46596)

Ray Serve<a id="ray-serve"></a>

🎉 New Features:

• Fully deprecate target_num_ongoing_requests_per_replica and max_concurrent_queries, respectively replaced by max_ongoing_requests and target_ongoing_requests (#46392 and #46427)
• Configure the task launched by the controller to build an application with Serve’s logging config (#46347)

RLlib<a id="rllib"></a>

💫 Enhancements:

• Moving sampling coordination for batch_mode=complete_episodes to synchronous_parallel_sample. (#46321)
• Enable complex action spaces with stateful modules. (#46468)

🏗 Architecture refactoring:

• Enable multi-learner setup for hybrid stack BC. (#46436)
• Introduce Checkpointable API for RLlib components and subcomponents. (#46376)

🔨 Fixes:

• Replace Mapping typehint with Dict: #46474

📖 Documentation:

• More example scripts for new API stack: Two separate optimizers (w/ different learning rates). (#46540) and custom loss function. (#46445)

Dashboard<a id="dashboard"></a>

🔨 Fixes:

• Task end time showing the incorrect time (#46439)
• Events Table rows having really bad spacing (#46701)
• UI bugs in the serve dashboard page (#46599)

Thanks

Many thanks to all those who contributed to this release!

@alanwguo, @hongchaodeng, @anyscalesam, @brucebismarck, @bt2513, @woshiyyya, @terraflops1048576, @lorenzoritter, @omrishiv, @davidxia, @cchen777, @nono-Sang, @jackhumphries, @aslonnie, @JoshKarpel, @zjregee, @bveeramani, @khluu, @Superskyyy, @liuxsh9, @jjyao, @ruisearch42, @sven1977, @harborn, @saihaj, @zcin, @can-anyscale, @veekaybee, @chungen04, @WeichenXu123, @GeneDer, @sergey-serebryakov, @Bye-legumes, @scottjlee, @rynewang, @kevin85421, @cristianjd, @peytondmurray, @MortalHappiness, @MaxVanDijck, @simonsays1980, @mjovanovic9999

Highlight: aDAG Developer Preview <a id="highlilght"></a>

This is a new Ray Core specific feature called Ray accelerated DAGs (aDAGs).

• aDAGs give you a Ray Core-like API but with extensibility to pre-compile execution paths across pre-allocated resources on a Ray Cluster to possible benefits for optimization on throughput and latency. Some practical examples include:
  • Up to 10x lower task execution time on single-node.
  • Native support for GPU-GPU communication, via NCCL.
• This is still very early, but please reach out on #ray-core on Ray Slack to learn more!

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

💫 Enhancements:

• Support async callable classes in map_batches() (#46129)

🔨 Fixes:

• Ensure InputDataBuffer doesn't free block references (#46191)
• MapOperator.num_active_tasks should exclude pending actors (#46364)
• Fix progress bars being displayed as partially completed in Jupyter notebooks (#46289)

📖 Documentation:

• Fix docs: read_api.py docstring (#45690)
• Correct API annotation for tfrecords_datasource (#46171)
• Fix broken links in README and in ray.data.Dataset (#45345)

Ray Train<a id="ray-train"></a>

📖 Documentation:

• Update PyTorch Data Ingestion User Guide (#45421)

Ray Serve<a id="ray-serve"></a>

💫 Enhancements:

• Optimize ServeController.get_app_config() (#45878)
• Change default for max and target ongoing requests (#45943)
• Integrate with Ray structured logging (#46215)
• Allow configuring handle cache size and controller max concurrency (#46278)
• Optimize DeploymentDetails.deployment_route_prefix_not_set() (#46305)

RLlib<a id="rllib"></a>

🎉 New Features:

• APPO on new API stack (w/ EnvRunners). (#46216)

💫 Enhancements:

• Stability: APPO, SAC, and DQN activate multi-agent learning tests (#45542, #46299)
• Make Tune trial ID available in EnvRunners (and callbacks). (#46294)
• Add env- and agent_steps to custom evaluation function. (#45652)
• Remove default-metrics from Algorithm (tune does NOT error anymore if any stop-metric is missing). (#46200)

🔨 Fixes:

• Various bug fixes: #45542

📖 Documentation:

• Example for new API stack: Offline RL (BC) training on single-agent, while evaluating w/ multi-agent setup. (#46251)
• Example for new API stack: Custom RLModule with an LSTM. (#46276)

Ray Core<a id="ray-core"></a>

🎉 New Features:

• aDAG Developer Preview.

💫 Enhancements:

• Allow env setup logger encoding (#46242)
• ray list tasks filter state and name on GCS side (#46270)
• Log ray version and ray commit during GCS start (#46341)

🔨 Fixes:

• Decrement lineage ref count of an actor when the actor task return object reference is deleted (#46230)
• Fix negative ALIVE actors metric and introduce IDLE state (#45718)
• psutil process attr num_fds is not available on Windows (#46329)

Dashboard<a id="dashboard"></a>

🎉 New Features:

• Added customizable refresh frequency for metrics on Ray Dashboard (#44037)

💫 Enhancements:

• Upgraded to MUIv5 and React 18 (#45789)

🔨 Fixes:

• Fix for multi-line log items breaking log viewer rendering (#46391)
• Fix for UI inconsistency when a job submission creates more than one Ray job. (#46267)
• Fix filtering by job id for tasks API not filtering correctly. (#45017)

Docs<a id="docs"></a>

🔨 Fixes:

• Re-enabled automatic cross-reference link checking for Ray documentation, with Sphinx nitpicky mode (#46279)
• Enforced naming conventions for public and private APIs to maintain accuracy, starting with Ray Data API documentation (#46261)

📖 Documentation:

• Upgrade Python 3.12 support to alpha, marking the release of the Ray wheel to PyPI and conducting a sanity check of the most critical tests.

Thanks

Many thanks to all those who contributed to this release!

@stephanie-wang, @MortalHappiness, @aslonnie, @ryanaoleary, @jjyao, @jackhumphries, @nikitavemuri, @woshiyyya, @JoshKarpel, @ruisearch42, @sven1977, @alanwguo, @GeneDer, @saihaj, @raulchen, @liuxsh9, @khluu, @cristianjd, @scottjlee, @bveeramani, @zcin, @simonsays1980, @SumanthRH, @davidxia, @can-anyscale, @peytondmurray, @kevin85421

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🔨 Fixes:

• Fixed bug where preserve_order doesn’t work with file reads (#46135)

📖 Documentation:

• Added documentation for dataset.Schema (#46170)

Ray Train<a id="ray-train"></a>

💫 Enhancements:

• Add API for Ray Train run stats (#45711)

Ray Tune<a id="ray-tune"></a>

💫 Enhancements:

• Missing stopping criterion should not error (just warn). (#45613)

📖 Documentation:

• Fix broken references in Ray Tune documentation (#45233)

Ray Serve<a id="ray-serve"></a>

WARNING: the following default values will change in Ray 2.32:

• Default for max_ongoing_requests will change from 100 to 5.
• Default for target_ongoing_requests will change from 1 to 2.

💫 Enhancements:

• Optimize DeploymentStateManager.get_deployment_statuses (#45872)

🔨 Fixes:

• Fix logging error on passing traceback object into exc_info (#46105)
• Run del even if constructor is still in-progress (#45882)
• Spread replicas with custom resources in torch tune serve release test (#46093)
• [1k release test] don't run replicas on head node (#46130)

📖 Documentation:

• Remove todo since issue is fixed (#45941)

RLlib<a id="rllib"></a>

🎉 New Features:

• IMPALA runs on the new API stack (with EnvRunners and ConnectorV2s). (#42085)
• SAC/DQN: Prioritized multi-agent episode replay buffer. (#45576)

💫 Enhancements:

• New API stack stability: Add systematic CI learning tests for all possible combinations of: [PPO|IMPALA] + [1CPU|2CPU|1GPU|2GPU] + [single-agent|multi-agent]. (#46162, #46161)

📖 Documentation:

• New API stack: Example script for action masking (#46146)
• New API stack: PyFlight example script cleanup (#45956)
• Old API stack: Enhanced ONNX example (+LSTM). (#43592)

Ray Core and Ray Clusters<a id="ray-core-and-ray-clusters"></a>

Ray Core<a id="ray-core"></a>

💫 Enhancements:

• [runtime-env] automatically infer worker path when starting worker in container (#42304)

🔨 Fixes:

• On GCS restart, destroy not forget the unused workers. Fixing PG leaks. (#45854)
• Cancel lease requests before returning a PG bundle (#45919)
• Fix boost fiber stack overflow (#46133)

Thanks

Many thanks to all those who contributed to this release!

@jjyao, @kevin85421, @vincent-pli, @khluu, @simonsays1980, @sven1977, @rynewang, @can-anyscale, @richardsliu, @jackhumphries, @alexeykudinkin, @bveeramani, @ruisearch42, @shrekris-anyscale, @stephanie-wang, @matthewdeng, @zcin, @hongchaodeng, @ryanaoleary, @liuxsh9, @GeneDer, @aslonnie, @peytondmurray, @Bye-legumes, @woshiyyya, @scottjlee, @JoshKarpel

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

💫 Enhancements:

• Improve fractional CPU/GPU formatting (#45673)
• Use sampled fragments to estimate Parquet reader batch size (#45749)
• Refactoring ParquetDatasource and metadata fetching logic (#45728, #45727, #45733, #45734, #45767)
• Refactor planner.py (#45706)

Ray Tune<a id="ray-tune"></a>

💫 Enhancements:

• Change the behavior of a missing stopping criterion metric to warn instead of raising an error. This enables the use case of reporting different sets of metrics on different iterations (ex: a separate set of training and validation metrics). (#45613)

Ray Serve<a id="ray-serve"></a>

💫 Enhancements:

• Create internal request id to track request objects (#45761)

RLLib<a id="rllib"></a>

💫 Enhancements:

• Stability: DreamerV3 weekly release test (#45654); Add "official" benchmark script for Atari PPO benchmarks. (#45697)
• Enhance env-rendering callback (#45682)

🔨 Fixes:

• Bug fix in new MetricsLogger API: EMA stats w/o window would lead to infinite list mem-leak. (#45752)
• Various other bug fixes: (#45819, #45820, #45683, #45651, #45753)

📖 Documentation:

• Re-do examples overview page (new API stack): #45382
  • PyFlyt QuadX WayPoints example #44758, #45956
  • RLModule inference on new API stack (#45831, #45845)
  • How to resume a tune.Tuner.fit() experiment from checkpoint. (#45681)
  • Custom RLModule (tiny CNN): #45774
  • Connector examples docstrings (#45864)
• Old API stack examples: #43592, #45829

Ray Core<a id="ray-core"></a>

🎉 New Features:

• Alpha release of job level logging configuration: users can now config the user logging to be logfmt format with logging context attached. (#45344)

💫 Enhancements:

• Integrate amdsmi in AMDAcceleratorManager (#44572)

🔨 Fixes:

• Fix the C++ GcsClient Del not respecting del_by_prefix (#45604)
• Fix exit handling of FiberState threads (#45834)

Dashboard<a id="ray-dashboard"></a>

💫 Enhancements:

• Parse out json logs (#45853)

Many thanks to all those who contributed to this release: @liuxsh9, @peytondmurray, @pcmoritz, @GeneDer, @saihaj, @khluu, @aslonnie, @yucai, @vickytsang, @can-anyscale, @bthananjeyan, @raulchen, @hongchaodeng, @x13n, @simonsays1980, @peterghaddad, @kevin85421, @rynewang, @angelinalg, @jjyao, @BenWilson2, @jackhumphries, @zcin, @chris-ray-zhang, @c21, @shrekris-anyscale, @alanwguo, @stephanie-wang, @Bye-legumes, @sven1977, @WeichenXu123, @bveeramani, @nikitavemuri

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New Features:

• Allow user to configure timeout for actor pool (#45508)
• Add override_num_blocks to from_pandas and perform auto-partition (#44937)
• Upgrade Arrow version to 16 in CI (#45565)

💫 Enhancements:

• Clarify that num_rows_per_file isn't strict (#45529)
• Record more telemetry for newly added datasources (#45647)
• Avoid pickling LanceFragment when creating read tasks for Lance (#45392)

Ray Train<a id="ray-train"></a>

📖 Documentation:

• [HPU] Add example of Stable Diffusion fine-tuning and serving on Intel Gaudi (#45217)
• [HPU] Add example of Llama-2 fine-tuning on Intel Gaudi (#44667)

Ray Tune<a id="ray-tune"></a>

🏗 Architecture refactoring:

• Improve excessive syncing warning and deprecate TUNE_RESULT_DIR, RAY_AIR_LOCAL_CACHE_DIR, local_dir (#45210)

Ray Serve<a id="ray-serve"></a>

💫 Enhancements:

• Clean up Serve proxy files (#45486)

📖 Documentation:

• vllm example to serve llm models (#45430)

RLLib<a id="rllib"></a>

💫 Enhancements:

• DreamerV3 on tf: Bug fix, so it can run again with tf==2.11.1 (2.11.0 is not available anymore) (#45419); Added weekly release test for DreamerV3.
• Added support for multi-agent off-policy algorithms (DQN and SAC) in the new (#45182)
• Config option for APPO/IMPALA to change number of GPU-loader threads (#45467)

🔨 Fixes:

• Various MetricsLogger bug fixes (#45543, #45585, #45575)
• Other fixes: #45588, #45617, #45517, #45465

📖 Documentation:

• Example script for new API stack: How-to restore 1 of n agents from a checkpoint. (#45462)
• Example script for new API stack: Autoregressive action module. #45525

Ray Core<a id="ray-core"></a>

💫 Enhancements:

• Improve node death observability (#45320, #45357, #45533, #45644, #45497)
• Ray c++ backend structured logging (#44468)

🔨 Fixes:

• Fix worker crash when getting actor name from runtime context (#45194)
• log dedup should not dedup number only lines (#45385)

📖 Documentation:

• Improve doc for --object-store-memory to describe how the default value is set (#45301)

Dashboard<a id="ray-dashboard"></a>

🔨 Fixes:

• Move Job package uploading to another thread to unblock the event loop. (#45282)

Many thanks to all those who contributed to this release: @maxliuofficial, @simonsays1980, @GeneDer, @dudeperf3ct, @khluu, @justinvyu, @andrewsykim, @Catch-Bull, @zcin, @bveeramani, @rynewang, @angelinalg, @matthewdeng, @jjyao, @kira-lin, @harborn, @hongchaodeng, @peytondmurray, @aslonnie, @timkpaine, @982945902, @maxpumperla, @stephanie-wang, @ruisearch42, @alanwguo, @can-anyscale, @c21, @Atry, @KamenShah, @sven1977, @raulchen

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New Features:

• Add support for using GPUs with map_groups (#45305)
• Add support for using actors with map_groups (#45310)

💫 Enhancements:

• Refine exception handling from arrow data conversion (#45294)

🔨 Fixes:

• Fix Ray databricks UC reader with dynamic Databricks notebook scope token (#45153)
• Fix bug where you can't return objects and array from UDF (#45287 )
• Fix bug where map_groups triggers execution during input validation (#45314)

Ray Tune<a id="ray-tune"></a>

🔨 Fixes:

• [tune] Fix PB2 scheduler error resulting from trying to sort by Trial objects (#45161)

Ray Serve<a id="ray-serve"></a>

🔨 Fixes:

• Log application unhealthy errors at error level instead of warning level (#45211)

RLLib<a id="rllib"></a>

💫 Enhancements:

• Examples and tuned_examples learning test for new API stack are now “self-executable” (don’t require a third-party script anymore to run them). + WandB support. (#45023)

🔨 Fixes:

• Fix result dict “spam” (duplicate, deprecated keys, e.g. “sampler_results” dumped into top level). (#45330)

📖 Documentation:

• Add example for training with fractional GPUs on new API stack. (#45379)
• Cleanup examples folder and remove deprecated sub directories. (#45327)

Ray Core<a id="ray-core"></a>

💫 Enhancements:

• [Logs] Add runtime env started logs to job driver (#45255)
• ray.util.collective support torch.bfloat16 (#39845)
• [Core] Better propagate node death information (#45128)

🔨 Fixes:

• [Core] Fix worker process leaks after job finishes (#44214)

Many thanks to all those who contributed to this release: @hongchaodeng, @khluu, @antoni-jamiolkowski, @ameroyer, @bveeramani, @can-anyscale, @WeichenXu123, @peytondmurray, @jackhumphries, @kevin85421, @jjyao, @robcaulk, @rynewang, @scottsun94, @swang, @GeneDer, @zcin, @ruisearch42, @aslonnie, @angelinalg, @raulchen, @ArthurBook, @sven1977, @wuxibin89

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New Features:

• Add function to dynamically generate ray_remote_args for Map APIs (#45143)
• Allow manually setting resource limits for training jobs (#45188)

💫 Enhancements:

• Introduce abstract interface for data autoscaling (#45002)
• Add debugging info for SplitCoordinator (#45226)

🔨 Fixes:

• Don’t show AllToAllOperator progress bar if the disable flag is set (#45136)
• Don't load Arrow PyExtensionType by default (#45084)
• Don't raise batch size error if num_gpus=0 (#45202)

Ray Train<a id="ray-train"></a>

💫 Enhancements:

• [XGBoost][LightGBM] Update RayTrainReportCallback to only save checkpoints on rank 0 (#45083)

Ray Core<a id="ray-core"></a>

🔨 Fixes:

• Fix the cpu percentage metrics for dashboard process (#45124)

Dashboard<a id="dashboard"></a>

💫 Enhancements:

• Improvements to log viewer so line numbers do not get selected when copying text.
• Improvements to the log viewer to avoid unnecessary re-rendering which causes text selection to clear.

Many thanks to all those who contributed to this release: @justinvyu, @simonsays1980, @chris-ray-zhang, @kevin85421, @angelinalg, @rynewang, @brycehuang30, @alanwguo, @jjyao, @shaikhismail, @khluu, @can-anyscale, @bveeramani, @jrosti, @WeichenXu123, @MortalHappiness, @raulchen, @scottjlee, @ruisearch42, @aslonnie, @alexeykudinkin

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New features:

• Add read_lance API to read Lance Dataset (#45106)

🔨 Fixes:

• Retry RaySystemError application errors (#45079)

📖 Documentation:

• Fix broken references in data documentation (#44956)

Ray Train<a id="ray-train"></a>

📖 Documentation:

• Fix broken links in Train documentation (#44953)

Ray Tune<a id="ray-tune"></a>

📖 Documentation:

• Update Hugging Face example to add reference (#42771)

🏗 Architecture refactoring:

• Remove deprecated ray.air.callbacks modules (#45104)

Ray Serve<a id="ray-serve"></a>

💫 Enhancements:

• Allow methods to pass type @serve.batch type hint (#45004)
• Allow configuring Serve control loop interval (#45063)

🔨 Fixes:

• Fix bug with controller failing to recover for autoscaling deployments (#45118)
• Fix control+c after serve run doesn't shutdown serve components (#45087)
• Fix lightweight update max ongoing requests (#45006)

RLlib<a id="rllib"></a>

🎉 New Features:

• New MetricsLogger API now fully functional on the new API stack (working now also inside Learner classes, i.e. loss functions). (#44995, #45109)

💫 Enhancements:

• Renamings and cleanups (toward new API stack and more consistent naming schemata): WorkerSet -> EnvRunnerGroup, DEFAULT_POLICY_ID -> DEFAULT_MODULE_ID, config.rollouts() -> config.env_runners(), etc.. (#45022, #44920)
• Changed behavior of EnvRunnerGroup.foreach_worker… methods to new defaults: mark_healthy=True (used to be False) and healthy_only=True (used to be False). (#44993)
• Fix get_state()/from_state() methods in SingleAgent- and MultiAgentEpisodes. (#45012)

🔨 Fixes:

• Bug fix for (torch) global_norm clipping overflow problem: (#45055)
• Various bug- and test case fixes: #45030, #45031, #45070, #45053, #45110, #45111

📖 Documentation:

• Example scripts using the MetricsLogger for env rendering and recording w/ WandB: #45073, #45107

Ray Core<a id="ray-core"></a>

🔨 Fixes:

• Fix ray.init(logging_format) argument is ignored (#45037)
• Handle unserializable user exception (#44878)
• Fix dashboard process event loop blocking issues (#45048, #45047)

Dashboard<a id="dashboard"></a>

🔨 Fixes:

• Fix Nodes page sorting not working correctly.
• Add back “actors per page” UI control in the actors page.

Many thanks to all those who contributed to this release: @rynewang, @can-anyscale, @scottsun94, @bveeramani, @ceddy4395, @GeneDer, @zcin, @JoshKarpel, @nikitavemuri, @stephanie-wang, @jackhumphries, @matthewdeng, @yash97, @simonsays1980, @peytondmurray, @evalaiyc98, @c21, @alanwguo, @shrekris-anyscale, @kevin85421, @hongchaodeng, @sven1977, @st--, @khluu

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

💫 Enhancements:

• Dedupe repeated schema during ParquetDatasource metadata prefetching (#44750)
• Update map_groups implementation to better handle large outputs (#44862)
• Deprecate prefetch_batches arg of iter_rows and change default value (#44982)
• Adding in default behavior to false for creating dirs on s3 writes (#44972)
• Make internal UDF names more descriptive (#44985)
• Make name a required argument for AggregateFn (#44880)

📖 Documentation:

• Add key concepts to and revise "Data Internals" page (#44751)

Ray Train<a id="ray-train"></a>

💫 Enhancements:

• Setup XGBoost CommunicatorContext automatically (#44883)
• Track Train Run Info with TrainStateActor (#44585)

📖 Documentation:

• Add documentation for accelerator_type (#44882)
• Update Ray Train example titles (#44369)

Ray Tune<a id="ray-tune"></a>

💫 Enhancements:

• Remove trial table when running Ray Train in a Jupyter notebook (#44858)
• Clean up temporary checkpoint directories for class Trainables (ex: RLlib) (#44366)

📖 Documentation:

• Fix minor doc format issues (#44865)
• Remove outdated ScalingConfig references (#44918)

Ray Serve<a id="ray-serve"></a>

💫 Enhancements:

• Handle push metric interval is now configurable with environment variable RAY_SERVE_HANDLE_METRIC_PUSH_INTERVAL_S (#32920)
• Improve performance of developer API serve.get_app_handle (#44812)

🔨 Fixes:

• Fix memory leak in handles for autoscaling deployments (the leak happens when
• RAY_SERVE_COLLECT_AUTOSCALING_METRICS_ON_HANDLE=1) (#44877)

RLlib<a id="rllib"></a>

🎉 New Features:

• Introduce MetricsLogger, a unified API for users of RLlib to log custom metrics and stats in all of RLlib’s components (Algorithm, EnvRunners, and Learners). Rolled out for new API stack for Algorithm (training_step) and EnvRunners (custom callbacks). Learner (custom loss functions) support in progress. #44888, #44442
• Introduce “inference-only” (slim) mode for RLModules that run inside an EnvRunner (and thus don’t require value-functions or target networks): #44797

💫 Enhancements:

• MultiAgentEpisodeReplayBuffer for new API stack (preparation for multi-agent support of SAC and DQN): #44450
• AlgorithmConfig cleanup and renaming of properties and methods for better consistency/transparency: #44896

🔨 Fixes:

• Various minor bug fixes: #44989, #44988, #44891, #44898, #44868, #44867, #44845

Ray Core and Ray Clusters<a id="ray-core"></a>

💫 Enhancements:

• Report GCS internal pubsub buffer metrics and cap message size (#44749)

🔨 Fixes:

• Fix task submission never return when network partition happens (#44692)
• Fix incorrect use of ssh port forward option. (#44973)
• Make sure dashboard will exit if grpc server fails (#44928)
• Make sure dashboard agent will exit if grpc server fails (#44899)

Thanks @can-anyscale, @hongchaodeng, @zcin, @marwan116, @khluu, @bewestphal, @scottjlee, @andrewsykim, @anyscalesam, @MortalHappiness, @justinvyu, @JoshKarpel, @woshiyyya, @rynewang, @Abirdcfly, @omatthew98, @sven1977, @marcelocarmona, @rueian, @mattip, @angelinalg, @aslonnie, @matthewdeng, @abizjakpro, @simonsays1980, @jjyao, @terraflops1048576, @hongpeng-guo, @stephanie-wang, @bw-matthew, @bveeramani, @ruisearch42, @kevin85421, @Tongruizhe

Many thanks to all those who contributed to this release!

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New Features:

• Store Ray Data logs in special subdirectory (#44743)

💫 Enhancements:

• Add in local_read option to from_torch (#44752)

🔨 Fixes:

• Fix the config to disable progress bar (#44342)

📖 Documentation:

• Clarify deprecated Datasource docstrings (#44790)

Ray Train<a id="ray-train"></a>

🔨 Fixes:

• Disable gathering the full state dict in RayFSDPStrategy for lightning>2.1 (#44569)

Ray Tune<a id="ray-tune"></a>

💫 Enhancements:

• Remove spammy log for "new output engine" (#44824)
• Enable isort (#44693)

Ray Serve<a id="ray-serve"></a>

🔨 Fixes:

• [Serve] fix getting attributes on stdout during Serve logging redirect (#44787)

RLlib<a id="rllib"></a>

🎉 New Features:

• Support of images and video logging in WandB (env rendering example script for the new API stack coming up). (#43356)

💫 Enhancements:

• Better support and separation-of-concerns for model_config_dict in new API stack. (#44263)
• Added example script to pre-train an RLModule in single-agent fashion, then bring checkpoint into multi-agent setup and continue training. (#44674)
• More examples scripts got translated from the old- to the new API stack: Curriculum learning, custom-gym-env, etc..: (#44706, #44707, #44735, #44841)

Ray Core and Ray Clusters<a id="ray-core"></a>

🔨 Fixes:

• Fix GetAllJobInfo is_running_tasks is not returning the correct value when driver starts ray (#44459)

Thanks

Many thanks to all those who contributed to this release! @can-anyscale, @hongpeng-guo, @sven1977, @zcin, @shrekris-anyscale, @liuxsh9, @jackhumphries, @GeneDer, @woshiyyya, @simonsays1980, @omatthew98, @andrewsykim, @n30111, @architkulkarni, @bveeramani, @aslonnie, @alexeykudinkin, @WeichenXu123, @rynewang, @matthewdeng, @angelinalg, @c21

Release Highlights<a id="release-highlights"></a>

• [data] Support reading Avro files with ray.data.read_avro
• [train] Added experimental support for AWS Trainium (Neuron) and Intel HPU.

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New Features:

• Support reading Avro files with ray.data.read_avro (#43663)

💫 Enhancements:

• Pin ipywidgets==7.7.2 to enable Data progress bars in VSCode Web (#44398)
• Change log level for ignored exceptions (#44408)

🔨 Fixes:

• Change Parquet encoding ratio lower bound from 2 to 1 (#44470)
• Fix throughput time calculations for metrics (#44138)
• Fix nested ragged numpy.ndarray (#44236)
• Fix Ray debugger incompatibility caused by trimmed error stack trace (#44496)

📖 Documentation:

• Update "Data Loading and Preprocessing" doc (#44165)
• Move imports into TFPRedictor in batch inference example (#44434)

Ray Train<a id="ray-train"></a>

🎉 New Features:

• Add experimental support for AWS Trainium (Neuron) (#39130)
• Add experimental support for Intel HPU (#43343)

💫 Enhancements:

• Log a deprecation warning for local_dir and related environment variables (#44029)
• Enforce xgboost>=1.7 for XGBoostTrainer usage (#44269)

🔨 Fixes:

• Fix ScalingConfig(accelerator_type) to request an appropriate resource amount (#44225)
• Fix maximum recursion issue when serializing exceptions (#43952)
• Remove base config deepcopy when initializing the trainer actor (#44611)

🏗 Architecture refactoring:

• Remove deprecated BatchPredictor (#43934)

Ray Tune<a id="ray-tune"></a>

💫 Enhancements:

• Add support for new style lightning import (#44339)
• Log a deprecation warning for local_dir and related environment variables (#44029)

🏗 Architecture refactoring:

• Remove scikit-optimize search algorithm (#43969)

Ray Serve<a id="ray-serve"></a>

🔨 Fixes:

• Dynamically-created applications will no longer be deleted when a config is PUT via the REST API (#44476).
• Fix _to_object_ref memory leak (#43763)
• Log warning to reconfigure max_ongoing_requests if max_batch_size is less than max_ongoing_requests (#43840)
• Deployment fails to start with ModuleNotFoundError in Ray 3.10 (#44329)
  • This was fixed by reverting the original core changes on the sys.path behavior. Revert "[core] If there's working_dir, don't set _py_driver_sys_path." (#44435)
• The batch_queue_cls parameter is removed from the @serve.batch decorator (#43935)

RLlib<a id="rllib"></a>

🎉 New Features:

• New API stack: DQN Rainbow is now available for single-agent (#43196, #43198, #43199)
• PrioritizedEpisodeReplayBuffer is available for off-policy learning using the EnvRunner API (SingleAgentEnvRunner) and supports random n-step sampling (#42832, #43258, #43458, #43496, #44262)

💫 Enhancements:

• Restructured examples/ folder; started moving example scripts to the new API stack (#44559, #44067, #44603)
• Evaluation do-over: Deprecate enable_async_evaluation option (in favor of existing evaluation_parallel_to_training setting). (#43787)
• Add: module_for API to MultiAgentEpisode (analogous to policy_for API of the old Episode classes). (#44241)
• All rllib_contrib old stack algorithms have been removed from rllib/algorithms (#43656)

🔨 Fixes:

• New API stack: Multi-GPU + multi-agent has been fixed. This completes support for any combinations of the following on the new API stack: [single-agent, multi-agent] vs [0 GPUs, 1 GPU, >1GPUs] vs [any number of EnvRunners] (#44420, #44664, #44594, #44677, #44082, #44669, #44622)
• Various other bug fixes: #43906, #43871, #44000, #44340, #44491, #43959, #44043, #44446, #44040

📖 Documentation:

• Re-announced new API stack in alpha stage (#44090).

Ray Core and Ray Clusters<a id="ray-core"></a>

🎉 New Features:

• Added Ray check-open-ports CLI for checking potential open ports to the public (#44488)

💫 Enhancements:

• Support nodes sharing the same spilling directory without conflicts. (#44487)
• Create two subclasses of RayActorError to distinguish between actor died (ActorDiedError) and actor temporarily unavailable (ActorUnavailableError) cases.

🔨 Fixes:

• Fixed the ModuleNotFound issued introduced in 2.10 (#44435)
• Fixed an issue where agent process is using too much CPU (#44348)
• Fixed race condition in multi-threaded actor creation (#44232)
• Fixed several streaming generator bugs (#44079, #44257, #44197)
• Fixed an issue where user exception raised from tasks cannot be subclassed (#44379)

Dashboard <a id="dashboard"></a>

💫 Enhancements:

• Add serve controller metrics to serve system dashboard page (#43797)
• Add Serve Application rows to Serve top-level deployments details page (#43506)
• [Actor table page enhancements] Include "NodeId", "CPU", "Memory", "GPU", "GRAM" columns in the actor table page. Add sort functionality to resource utilization columns. Enable searching table by "Class" and "Repr". (#42588) (#42633) (#42788)

🔨 Fixes:

• Fix default sorting of nodes in Cluster table page to first be by "Alive" nodes, then head nodes, then alphabetical by node ID. (#42929)
• Fix bug where the Serve Deployment detail page fails to load if the deployment is in "Starting" state (#43279)

Docs <a id="docs"></a>

💫 Enhancements:

• Landing page refreshes its look and feel. (#44251)

Thanks

Many thanks to all those who contributed to this release!

@aslonnie, @brycehuang30, @MortalHappiness, @astron8t-voyagerx, @edoakes, @sven1977, @anyscalesam, @scottjlee, @hongchaodeng, @slfan1989, @hebiao064, @fishbone, @zcin, @GeneDer, @shrekris-anyscale, @kira-lin, @chappidim, @raulchen, @c21, @WeichenXu123, @marian-code, @bveeramani, @can-anyscale, @mjd3, @justinvyu, @jackhumphries, @Bye-legumes, @ashione, @alanwguo, @Dreamsorcerer, @KamenShah, @jjyao, @omatthew98, @autolisis, @Superskyyy, @stephanie-wang, @simonsays1980, @davidxia, @angelinalg, @architkulkarni, @chris-ray-zhang, @kevin85421, @rynewang, @peytondmurray, @zhangyilun, @khluu, @matthewdeng, @ruisearch42, @pcmoritz, @mattip, @jerome-habana, @alexeykudinkin

Release Highlights<a id="release-highlights"></a>

Ray 2.10 release brings important stability improvements and enhancements to Ray Data, with Ray Data becoming generally available (GA).

• [Data] Ray Data becomes generally available with stability improvements in streaming execution, reading and writing data, better tasks concurrency control, and debuggability improvement with dashboard, logging and metrics visualization.
• [RLlib] “New API Stack” officially announced as alpha for PPO and SAC.
• [Serve] Added a default autoscaling policy set via num_replicas=”auto” (#42613).
• [Serve] Added support for active load shedding via max_queued_requests (#42950).
• [Serve] Added replica queue length caching to the DeploymentHandle scheduler (#42943).
  • This should improve overhead in the Serve proxy and handles.
  • max_ongoing_requests (max_concurrent_queries) is also now strictly enforced (#42947).
  • If you see any issues, please report them on GitHub and you can disable this behavior by setting: RAY_SERVE_ENABLE_QUEUE_LENGTH_CACHE=0.
• [Serve] Renamed the following parameters. Each of the old names will be supported for another release before removal.
  • max_concurrent_queries -> max_ongoing_requests
  • target_num_ongoing_requests_per_replica -> target_ongoing_requests
  • downscale_smoothing_factor -> downscaling_factor
  • upscale_smoothing_factor -> upscaling_factor
• [Core] Autoscaler v2 is in alpha and can be tried out with Kuberay. It has improved observability and stability compared to v1.
• [Train] Added support for accelerator types via ScalingConfig(accelerator_type).
• [Train] Revamped the XGBoostTrainer and LightGBMTrainer to no longer depend on xgboost_ray and lightgbm_ray. A new, more flexible API will be released in a future release.
• [Train/Tune] Refactored local staging directory to remove the need for local_dir and RAY_AIR_LOCAL_CACHE_DIR.

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New Features:

• Streaming execution stability improvement to avoid memory issue, including per-operator resource reservation, streaming generator output buffer management, and better runtime resource estimation (#43026, #43171, #43298, #43299, #42930, #42504)
• Metadata read stability improvement to avoid AWS transient error, including retry on application-level exception, spread tasks across multiple nodes, and configure retry interval (#42044, #43216, #42922, #42759).
• Allow tasks concurrency control for read, map, and write APIs (#42849, #43113, #43177, #42637)
• Data dashboard and statistics improvement with more runtime metrics for each components (#43790, #43628, #43241, #43477, #43110, #43112)
• Allow to specify application-level error to retry for actor task (#42492)
• Add num_rows_per_file parameter to file-based writes (#42694)
• Add DataIterator.materialize (#43210)
• Skip schema call in DataIterator.to_tf if tf.TypeSpec is provided (#42917)
• Add option to append for Dataset.write_bigquery (#42584)
• Deprecate legacy components and classes (#43575, #43178, #43347, #43349, #43342, #43341, #42936, #43144, #43022, #43023)

💫 Enhancements:

• Restructure stdout logging for better readability (#43360)
• Add a more performant way to read large TFRecord datasets (#42277)
• Modify ImageDatasource to use Image.BILINEAR as the default image resampling filter (#43484)
• Reduce internal stack trace output by default (#43251)
• Perform incremental writes to Parquet files (#43563)
• Warn on excessive driver memory usage during shuffle ops (#42574)
• Distributed reads for ray.data.from_huggingface (#42599)
• Remove Stage class and related usages (#42685)
• Improve stability of reading JSON files to avoid PyArrow errors (#42558, #42357)

🔨 Fixes:

• Turn off actor locality by default (#44124)
• Normalize block types before internal multi-block operations (#43764)
• Fix memory metrics for OutputSplitter (#43740)
• Fix race condition issue in OpBufferQueue (#43015)
• Fix early stop for multiple Limit operators. (#42958)
• Fix deadlocks caused by Dataset.streaming_split for job hanging (#42601)

📖 Documentation:

• Revamp Ray Data documentation for GA (#44006, #44007, #44008, #44098, #44168, #44093, #44105)

Ray Train<a id="ray-train"></a>

🎉 New Features:

• Add support for accelerator types via ScalingConfig(accelerator_type) for improved worker scheduling (#43090)

💫 Enhancements:

• Add a backend-specific context manager for train_func for setup/teardown logic (#43209)
• Remove DEFAULT_NCCL_SOCKET_IFNAME to simplify network configuration (#42808)
• Colocate Trainer with rank 0 Worker for to improve scheduling behavior (#43115)

🔨 Fixes:

• Enable scheduling workers with memory resource requirements (#42999)
• Make path behavior OS-agnostic by using Path.as_posix over os.path.join (#42037)
• [Lightning] Fix resuming from checkpoint when using RayFSDPStrategy (#43594)
• [Lightning] Fix deadlock in RayTrainReportCallback (#42751)
• [Transformers] Fix checkpoint reporting behavior when get_latest_checkpoint returns None (#42953)

📖 Documentation:

• Enhance docstring and user guides for train_loop_config (#43691)
• Clarify in ray.train.report docstring that it is not a barrier (#42422)
• Improve documentation for prepare_data_loader shuffle behavior and set_epoch (#41807)

🏗 Architecture refactoring:

• Simplify XGBoost and LightGBM Trainer integrations. Implemented XGBoostTrainer and LightGBMTrainer as DataParallelTrainer. Removed dependency on xgboost_ray and lightgbm_ray. (#42111, #42767, #43244, #43424)
• Refactor local staging directory to remove the need for local_dir and RAY_AIR_LOCAL_CACHE_DIR. Add isolation between driver and distributed worker artifacts so that large files written by workers are not uploaded implicitly. Results are now only written to storage_path, rather than having another copy in the user’s home directory (~/ray_results). (#43369, #43403, #43689)
• Split overloaded ray.train.torch.get_device into another get_devices API for multi-GPU worker setup (#42314)
• Refactor restoration configuration to be centered around storage_path (#42853, #43179)
• Deprecations related to SyncConfig (#42909)
• Remove deprecated preprocessor argument from Trainers (#43146, #43234)
• Hard-deprecate MosaicTrainer and remove SklearnTrainer (#42814)

Ray Tune<a id="ray-tune"></a>

💫 Enhancements:

• Increase the minimum number of allowed pending trials for faster auto-scaleup (#43455)
• Add support to TBXLogger for logging images (#37822)
• Improve validation of Experiment(config) to handle RLlib AlgorithmConfig (#42816, #42116)

🔨 Fixes:

• Fix reuse_actors error on actor cleanup for function trainables (#42951)
• Make path behavior OS-agnostic by using Path.as_posix over os.path.join (#42037)

📖 Documentation:

• Minor documentation fixes (#42118, #41982)

🏗 Architecture refactoring:

• Refactor local staging directory to remove the need for local_dir and RAY_AIR_LOCAL_CACHE_DIR. Add isolation between driver and distributed worker artifacts so that large files written by workers are not uploaded implicitly. Results are now only written to storage_path, rather than having another copy in the user’s home directory (~/ray_results). (#43369, #43403, #43689)
• Deprecations related to SyncConfig and chdir_to_trial_dir (#42909)
• Refactor restoration configuration to be centered around storage_path (#42853, #43179)
• Add back NevergradSearch (#42305)
• Clean up invalid checkpoint_dir and reporter deprecation notices (#42698)

Ray Serve<a id="ray-serve"></a>

🎉 New Features:

• Added support for active load shedding via max_queued_requests (#42950).
• Added a default autoscaling policy set via num_replicas=”auto” (#42613).

🏗 API Changes:

• Renamed the following parameters. Each of the old names will be supported for another release before removal.
  • max_concurrent_queries to max_ongoing_requests
  • target_num_ongoing_requests_per_replica to target_ongoing_requests
  • downscale_smoothing_factor to downscaling_factor
  • upscale_smoothing_factor to upscaling_factor
• WARNING: the following default values will change in Ray 2.11:
  • Default for max_ongoing_requests will change from 100 to 5.
  • Default for target_ongoing_requests will change from 1 to 2.

💫 Enhancements:

• Add RAY_SERVE_LOG_ENCODING env to set the global logging behavior for Serve (#42781).
• Config Serve's gRPC proxy to allow large payload (#43114).
• Add blocking flag to serve.run() (#43227).
• Add actor id and worker id to Serve structured logs (#43725).
• Added replica queue length caching to the DeploymentHandle scheduler (#42943).
  • This should improve overhead in the Serve proxy and handles.
  • max_ongoing_requests (max_concurrent_queries) is also now strictly enforced (#42947).
  • If you see any issues, please report them on GitHub and you can disable this behavior by setting: RAY_SERVE_ENABLE_QUEUE_LENGTH_CACHE=0.
• Autoscaling metrics (tracking ongoing and queued metrics) are now collected at deployment handles by default instead of at the Serve replicas (#42578).
  • This means you can now set max_ongoing_requests=1 for autoscaling deployments and still upscale properly, because requests queued at handles are properly taken into account for autoscaling.
  • You should expect deployments to upscale more aggressively during bursty traffic, because requests will likely queue up at handles during bursts of traffic.
  • If you see any issues, please report them on GitHub and you can switch back to the old method of collecting metrics by setting the environment variable RAY_SERVE_COLLECT_AUTOSCALING_METRICS_ON_HANDLE=0
• Improved the downscaling behavior of smoothing_factor for low numbers of replicas (#42612).
• Various logging improvements (#43707, #43708, #43629, #43557).
• During in-place upgrades or when replicas become unhealthy, Serve will no longer wait for old replicas to gracefully terminate before starting new ones (#43187). New replicas will be eagerly started to satisfy the target number of healthy replicas.
  • This new behavior is on by default and can be turned off by setting RAY_SERVE_EAGERLY_START_REPLACEMENT_REPLICAS=0

🔨 Fixes:

• Fix deployment route prefix override by default route prefix from serve run cli (#43805).
• Fixed a bug causing batch methods to hang upon cancellation (#42593).
• Unpinned FastAPI dependency version (#42711).
• Delay proxy marking itself as healthy until it has routes from the controller (#43076).
• Fixed an issue where multiplexed deployments could go into infinite backoff (#43965).
• Silence noisy KeyError on disconnects (#43713).
• Fixed the prometheus counter metrics emitted as gauge bug (#43795, #43901).
  • All the serve counter metrics are emitted as counters with _total suffix. The old gauge metrics are still emitted for compatibility.

📖 Documentation:

• Update serve logging config docs (#43483).
• Added documentation for max_replicas_per_node (#42743).

RLlib<a id="rllib"></a>

🎉 New Features:

• The “new API stack” is now in alpha stage and available for PPO single- (#42272) and multi-agent and for SAC single-agent (#42571, #42570, #42568)
  • ConnectorV2 API (#43669, #43680, #43040, #41074, #41212)
  • Episode APIs (SingleAgentEpisode and MultiAgentEpisode) (#42009, #43275, #42296, #43818, #41631)
  • EnvRunner APIs (SingleAgentEnvRunner and MultiAgentEnvRunner) (#41558, #41825, #42296, #43779)
• In preparation of DQN on the new API stack: PrioritizedEpisodeReplayBuffer (#43258, #42832)

💫 Enhancements:

• Old API Stack cleanups:
  • Move SampleBatch column names (e.g. SampleBatch.OBS) into new class (Columns). (#43665)
  • Remove old exec_plan API code. (#41585)
  • Introduce OldAPIStack decorator (#43657)
  • RLModule API: Add functionality to define kernel and bias initializers via config. (#42137)
• Learner/LearnerGroup APIs:
  • Replace Learner/LearnerGroup specific config classes (e.g. LearnerHyperparameters) with AlgorithmConfig. (#41296)
  • Learner/LearnerGroup: Allow updating from Episodes. (#41235)
• In preparation of DQN on the new API stack: (#43199, #43196)

🔨 Fixes:

• New API Stack bug fixes: Fix policy_to_train logic (#41529), fix multi-APU for PPO on the new API stack. (#44001), Issue 40347: (#42090)
• Other fixes: MultiAgentEnv would NOT call env.close() on a failed sub-env (#43664), Issue 42152 (#43317), issue 42396: (#43316), issue 41518 (#42011), issue 42385 (#43313)

📖 Documentation:

• New API Stack examples: Self-play and league-based self-play (#43276), MeanStdFilter (for both single-agent and multi-agent) (#43274), Prev-actions/prev-rewards for multi-agent (#43491)
• Other docs fixes and enhancements: (#43438, #41472, #42117, #43458)

Ray Core and Ray Clusters<a id="ray-core-and-ray-clusters"></a>

Ray Core<a id="ray-core"></a>

🎉 New Features:

• Autoscaler v2 is in alpha and can be tried out with Kuberay.
• Introduced subreaper to prevent leaks of sub-processes created by user code. (#42992)

💫 Enhancements:

• Ray state api get_task() now accepts ObjectRef (#43507)
• Add an option to disable task tracing for task/actor (#42431)
• Improved object transfer throughput. (#43434)
• Ray client now compares the Ray and Python version for compatibility with the remote Ray cluster. (#42760)

🔨 Fixes:

• Fixed several bugs for streaming generator (#43775, #43772, #43413)
• Fixed Ray counter metrics emitted as gauge bug (#43795)
• Fixed a bug where empty resource task doesn’t work with placement group (#43448)
• Fixed a bug where CPU resource is not released for a blocked worker inside placement group (#43270)
• Fixed GCS crashes when PG commit phase failed due to node failure (#43405)
• Fixed a bug where Ray memory monitor prematurely kill tasks (#43071)
• Fixed placement group resource leak (#42942)
• Upgraded cloudpickle to 3.0 which fixes the incompatibility with dataclasses (#42730)

📖 Documentation:

• Updated the doc for Ray accelerators support (#41849)

Ray Clusters<a id="ray-clusters"></a>

💫 Enhancements:

• [spark] Add heap_memory param for setup_ray_cluster API, and change default value of per ray worker node config, and change default value of ray head node config for global Ray cluster (#42604)
• [spark] Add global mode for ray on spark cluster (#41153)

🔨 Fixes:

• [VSphere] Only deploy ovf to first host of cluster (#42258)

Thanks

Many thanks to all those who contributed to this release!

@ronyw7, @xsqian, @justinvyu, @matthewdeng, @sven1977, @thomasdesr, @veryhannibal, @klebster2, @can-anyscale, @simran-2797, @stephanie-wang, @simonsays1980, @kouroshHakha, @Zandew, @akshay-anyscale, @matschaffer-roblox, @WeichenXu123, @matthew29tang, @vitsai, @Hank0626, @anmyachev, @kira-lin, @ericl, @zcin, @sihanwang41, @peytondmurray, @raulchen, @aslonnie, @ruisearch42, @vszal, @pcmoritz, @rickyyx, @chrislevn, @brycehuang30, @alexeykudinkin, @vonsago, @shrekris-anyscale, @andrewsykim, @c21, @mattip, @hongchaodeng, @dabauxi, @fishbone, @scottjlee, @justina777, @surenyufuz, @robertnishihara, @nikitavemuri, @Yard1, @huchen2021, @shomilj, @architkulkarni, @liuxsh9, @Jocn2020, @liuyang-my, @rkooo567, @alanwguo, @KPostOffice, @woshiyyya, @n30111, @edoakes, @y-abe, @martinbomio, @jiwq, @arunppsg, @ArturNiederfahrenhorst, @kevin85421, @khluu, @JingChen23, @masariello, @angelinalg, @jjyao, @omatthew98, @jonathan-anyscale, @sjoshi6, @gaborgsomogyi, @rynewang, @ratnopamc, @chris-ray-zhang, @ijrsvt, @scottsun94, @raychen911, @franklsf95, @GeneDer, @madhuri-rai07, @scv119, @bveeramani, @anyscalesam, @zen-xu, @npuichigo

This patch release contains fixes for Ray Core, Ray Data, and Ray Serve.

Ray Core

🔨 Fixes:

• Fix protobuf breaking change by adding a compat layer. (#43172)
• Bump up task failure logs to warnings to make sure failures could be troubleshooted (#43147)
• Fix placement group leaks (#42942)

Ray Data

🔨 Fixes:

• Skip schema call in to_tf if tf.TypeSpec is provided (#42917)
• Skip recording memory spilled stats when get_memory_info_reply is failed (#42824)

Ray Serve

🔨 Fixes:

• Fixing DeploymentStateManager qualifying replicas as running prematurely (#43075)

Thanks

Many thanks to all those who contributed to this release!

@rynewang, @GeneDer, @alexeykudinkin, @edoakes, @c21, @rkooo567

This patch release contains fixes for Ray Core, Ray Data, and Ray Serve.

Ray Core

🔨 Fixes:

• Fix out of disk test on release branch (https://github.com/ray-project/ray/pull/42724)

Ray Data

🔨 Fixes:

• Fix failing huggingface test (https://github.com/ray-project/ray/pull/42727)
• Fix deadlocks caused by streaming_split (https://github.com/ray-project/ray/pull/42601) (https://github.com/ray-project/ray/pull/42755)
• Fix locality config not being respected in DataConfig (https://github.com/ray-project/ray/pull/42204 https://github.com/ray-project/ray/pull/42204) (https://github.com/ray-project/ray/pull/42722)
• Stability & accuracy improvements for Data+Train benchmark (https://github.com/ray-project/ray/pull/42027)
• Add retry for _sample_fragment during ParquetDatasource._estimate_files_encoding_ratio() (https://github.com/ray-project/ray/pull/42759) (https://github.com/ray-project/ray/pull/42774)
• Skip recording memory spilled stats when get_memory_info_reply is failed (https://github.com/ray-project/ray/pull/42824) (https://github.com/ray-project/ray/pull/42834)

Ray Serve

🔨 Fixes:

• Pin the fastapi & starlette version to avoid breaking proxy (https://github.com/ray-project/ray/pull/42740 https://github.com/ray-project/ray/pull/42740)
• Fix IS_PYDANTIC_2 logic for pydantic<1.9.0 (https://github.com/ray-project/ray/pull/42704) (https://github.com/ray-project/ray/pull/42708)
• fix missing message body for json log formats (https://github.com/ray-project/ray/pull/42729) (https://github.com/ray-project/ray/pull/42874)

Thanks

Many thanks to all those who contributed to this release!

@c21, @raulchen, @can-anyscale, @edoakes, @peytondmurray, @scottjlee, @aslonnie, @architkulkarni, @GeneDer, @Zandew, @sihanwang41

This patch release contains fixes for Ray Core, Ray Data, and Ray Serve.

Ray Core

🔨 Fixes:

• Adding debupgy as the ray debugger (#42311)
• Fix task events profile events per task leak (#42248)
• Make sure redis sync context and async context connect to the same redis instance (#42040)

Ray Data

🔨 Fixes:

• [Data] Retry write if error during file clean up (#42326)

Ray Serve

🔨 Fixes:

• Improve handling the websocket server disconnect scenario (#42130)
• Fix pydantic config documentation (#42216)
• Address issues under high network delays:
  • Enable setting queue length response deadline via environment variable (#42001)
  • Add exponential backoff for queue_len_response_deadline_s (#42041)