NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI · #4694 most downloaded on PyPI
LLM Evaluations
Last release 13 days ago
21 Sep 2026
Ships fairly regularly
a new release about every 2 weeks
Some releases are documented
notes for 33 of the last 60 stable releases
1 version withdrawn
withdrawn after publishing
3 years old
92 releases · first in 2024
evals: deprecate document relevance evaluators
One column per month.
evals: deprecate document relevance evaluators
The Phoenix evals module is graduating out of experimental! You can now install Phoenix evals as a standalone package with pip install arize-phoenix-evals or you can include the new version of phoenix.evals along with the Phoenix install with pip install -U arize-phoenix[evals]. Swapping to the new evals module includes a few small breaking changes which might require some migration work. Details can be found in MIGRATION.md.
phoenix.experimental.evals is being deprecated and will remain in Phoenix for about a month before being removed.
phoenix.evals into phoenix (#2420) (dd3e7b4)evals: deprecate document relevance evaluators
evals: add session-level pii_detection evaluator
3.5.1 (2026-08-25) Bug Fixes upgrade to anthropic SDK v1
evals: add retrieval relevance evaluator
phoenix.evals (#2421) (fbd4961)BedrockModel (#2425) (81a720c)evals: add hallucination evaluator
pxi: enable phoenix-gql mutations by default with approval in manual mode
server: add POST /traces/transfer
evals: compute macro/weighted F-score per class to match sklearn semantics
agents: Move PXI prompt assembly server-side and cache prompts
Nothing published for this version
evals: deprecate evals 1.0 and remove legacy experiments module
add Python 3.14 support (except Windows)
evals: deprecate evals 1.0 docs
evals: add conciseness classification evaluator
This PR was generated with Release Please. See documentation.
<!-- CURSOR_SUMMARY -->
add tool response handling evaluator template
This PR was generated with Release Please. See documentation.
<!-- CURSOR_SUMMARY -->
add FaithfulnessEvaluator and deprecate HallucinationEvaluator
Enable sync- and async- client kwargs for LLM constructor
## 2.7.1 (2025-12-12) ### Bug Fixes * evals: rate limiter fixes
evals: support prompt/template messages
handle None values for top_p and temperature in AnthropicModel
Add anthropic and google-genai adapters
This PR was generated with Release Please. See documentation.
<!-- CURSOR_SUMMARY -->
d67ea3f : Deprecate createDocumentRelevanceEvaluator in favor of createRetrievalRelevanceEvaluator . Rename documentText to context and the unrelated…
createCompletenessEvaluator to judge whether every active user request in a conversation was actually completed.createDocumentRelevanceEvaluator in favor ofcreateRetrievalRelevanceEvaluator. Rename documentText to context and theunrelated label to irrelevant. The deprecated factory and its types will beCoroutine Fn support for phoenix.evals.create_evaluator
<img width="1858" height="1045" alt="coroutine_evaluator" src="https://github.com/user-attachments/assets/331d425f-bc03-434b-bc18-34ad1ac9317f" />
llm_classify (#2068) (3233d56)Allow dot delimited fstring keys in evals Templates
evals: add document relevance evaluator
This PR was generated with Release Please. See documentation.
<!-- CURSOR_SUMMARY -->
## 2.1.0 (2025-09-24) ### Features * Add Azure provider support (#9605) (edf448a) ### Bug Fixes * Remove openai sdk retries (#9587) (08d78f9) * Use OI
v1/traces HTTP endpoint to handle ExportTraceServiceRequest (3c94dea)v1/traces HTTP endpoint to handle ExportTraceServiceRequest (#1968) (3c94dea)## 2.0.1 (2025-09-17) ### Documentation * fix docs preview import paths
evals: add rate limiting to llm methods
llm_classify and llm_generate interfaces (#1974)llm_generate (#1965) (5999133)llm_classify and llm_generate interfaces (#1974) (9fd35a1)evals: add evaluate_dataframe function
evals: make evaluator input schema more explicit and discoverable
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Your coding agent can read these notes before it upgrades. Set up the MCP server →