NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI · #3927 most downloaded on PyPI
Open-source evaluators for LLM applications
Last release 6 months ago
07 Apr 2026
Ships unpredictably
gaps range from 8 days to 5 months
Nearly every release is documented
notes for 24 of 26 stable releases
Nothing withdrawn
no release was ever pulled
2 years old
60 releases · first in 2025
One column per month.
Fix all Dependabot security vulnerabilities by @jkennedyvz in https://github.com/langchain-ai/openevals/pull/169
Full Changelog: https://github.com/langchain-ai/openevals/compare/openevals-js==0.1.5...openevals==0.2.0
ci(js): Fix JS trusted publishing by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/135
Full Changelog: https://github.com/langchain-ai/openevals/compare/openevals==0.1.3...openevals==0.1.4
release(js): 0.1.2 by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/127
Full Changelog: https://github.com/langchain-ai/openevals/compare/openevals==0.1.2...openevals==0.1.3
feat(py,js): Improve source run linking for JSON match evaluators by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/125
Full Changelog: https://github.com/langchain-ai/openevals/compare/openevals==0.1.1...openevals==0.1.2
chore(ci): Run CI against beta by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/117
Full Changelog: https://github.com/langchain-ai/openevals/compare/js==0.1.1...openevals==0.1.1
Nothing published for this version
feat(ci): Add release workflows by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/81
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.20...py==0.1.0
Nothing published for this version
release(js): 0.0.14 by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/92
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.19...py==0.0.20
release(js): 0.0.13 by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/86
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.18...py==0.0.19
Nothing published for this version
release(js): 0.0.12 by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/77
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.17...py==0.0.18
release(js): 0.0.11 by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/74
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.16...py==0.0.17
release(js): 0.0.10 by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/66
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.15...py==0.0.16
Nothing published for this version
docs: Unnest markdown headers by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/61
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.13...py==0.0.15
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
fix(py): Move py.typed to the proper place by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/59
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.12...py==0.0.13
Nothing published for this version
feat(python): Add py.typed file, fix mypy issues by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/57
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.11...py==0.0.12
release(js): 0.0.9 by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/54
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.10...py==0.0.11
Nothing published for this version
feat(python,js): Make input and output optional for LLM as judge by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/52
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.9...py==0.0.10
Nothing published for this version
feat(python,js): Allow evaluators to return metadata, update code evaluators by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/48
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.8...py==0.0.9
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
feat(js): Export prompts from their own entrypoint by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/42
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.7...py==0.0.8
Nothing published for this version
release(js): 0.0.5 by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/36
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.6...py==0.0.7
feat(python,js): Adds code evals by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/32
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.5...py==0.0.6
fix(python): Make things actually work in Python 3.9 by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/30
Full Changelog: https://github.com/langchain-ai/openevals/compare/py==0.0.4...py==0.0.5
docs: Adds video quickstart by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/25
Full Changelog: https://github.com/langchain-ai/openevals/compare/0.0.3...py==0.0.4
feat(python,js): Export base run evaluator, use explicit kwargs by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/23
Full Changelog: https://github.com/langchain-ai/openevals/compare/0.0.2...0.0.3
Nothing published for this version
feat: Adds metadata to traced runs by @jacoblee93 in https://github.com/langchain-ai/openevals/pull/21
Full Changelog: https://github.com/langchain-ai/openevals/compare/0.0.1...0.0.2
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Your coding agent can read these notes before it upgrades. Set up the MCP server →