What's Changed

• feat: migrate SQLSemanticEquivalence to collections API by @anistark in #2496
• feat: migrate DataCompyScore to collections API by @anistark in #2499
• fix: migrate CHRF Score to new metrics collections by @anistark in #2500
• Feat/improve rag quickstart by @anistark in #2501
• fix: handle classification array length mismatch in TopicAdherence metric by @anistark in #2503
• feat: migrate quoted spans metric to collections api by @anistark in #2504
• fix: handle instructor modes for json and tools by @anistark in #2505
• docs: remove obsolete 'write your own metric' guides by @sanjeed5 in #2488
• fix: BasePrompt.adapt() structured output + language adaptation guide by @sanjeed5 in #2487
• Add AG-UI Protocol Integration for Agent Evaluation by @contextablemark in #2395
• docs: fix modifying-prompts-metrics guide with correct API by @sanjeed5 in #2486
• fix: handle nested dicts/lists in ToolCallF1 args to prevent unhashable type error by @anistark in #2507
• feat: add Claude docs auto-update workflow by @sanjeed5 in #2508
• fix: allow unannotated parameters to accept any type by @dhyaneesh in #2513
• feat: migrate MultiModalFaithfulness to collections API by @anistark in #2515
• feat: migrate MultiModalRelevance to collections API by @anistark in #2518
• feat: add support for new google-genai SDK with backwards compatibility for deprecated google-generativeai by @anistark in #2517
• fix: use instructor Mode.JSON for litellm and generic providers to fix Dict type validation errors by @anistark in #2514
• [AG-UI] Replacing "OpenAI" with "AsyncOpenAI". by @contextablemark in #2520
• Feat: document PDF export workflow by @cgaswin in #2522
• refactor: deprecate legacy metrics by @anistark in #2519
• fix: claude workflows to use github token by @anistark in #2527
• feat: add HuggingFace tokenizer support in knowledge graph operations by @anistark in #2524
• feat: add generate_with_chunks for pre-chunked documents by @MinseongS in #2526
• feat(docs): add offline mermaid support to PDF export by @cgaswin in #2530
• fix: claude workflows to use pat token instead of github token to support forked PRs by @anistark in #2531
• docs: change query execution to use asyncio.run by @yangzq50 in #2532
• fix: increase max-turns and update prompt by @anistark in #2534
• chore: remove survey link by @anistark in #2535
• feat: add caching support for metrics collections by @anistark in #2533
• feat: add caching support for embeddings by @anistark in #2536

New Contributors

• @contextablemark made their first contribution in #2395
• @cgaswin made their first contribution in #2522
• @MinseongS made their first contribution in #2526
• @yangzq50 made their first contribution in #2532

Full Changelog: v0.4.1...v0.4.2

What's Changed

• feat: add save/load methods to BasePrompt by @anistark in #2465
• docs: update run_config guide to use collections API by @sanjeed5 in #2468
• fix: add anthropic and gemini clients for custom clients by @anistark in #2472
• feat: migrate ToolCallAccuracy to collections API by @anistark in #2476
• chore: add survey link to readme and docs banner by @anistark in #2478
• Fix: apply_transforms(kg, transforms, run_config=run_config or RunCon… by @narabi in #2480
• feat: migrate ToolCallF1 to collections API with set-based F1 scoring by @anistark in #2483
• New input parameter for the TestsetGenerator Class : LLM_CONTEXT by @narabi in #2474
• feat: migrate TopicAdherence and AgentGoalAccuracy to collections API by @anistark in #2485
• docs: organize integrations nav into collapsible groups by @sanjeed5 in #2492
• aembed_text() replace embed_text() in AnswerRelevancy by @anexplore in #2495
• docs: remove broken reference to train_your_own_metric.md by @sanjeed5 in #2489
• feat: migrate Rubrics Metrics to collections API by @anistark in #2494

New Contributors

• @narabi made their first contribution in #2480
• @anexplore made their first contribution in #2495

Full Changelog: v0.4.0...v0.4.1

What's Changed

• docs: complete collections API documentation for remaining metrics by @sanjeed5 in #2420
• feat: support GPT-5 and o-series models with automatic temperature and top_p constraint handling by @anistark in #2418
• update: add llm options as tabs to quickstart by @anistark in #2421
• feat: migrate to instructor.from_provider for universal provider support by @anistark in #2424
• docs: fix typos in some files by @Edge-Seven in #2429
• feat: implement prompt class for context precision by @anistark in #2433
• fix: docs for quickstart by @anistark in #2434
• Fix 2432: Update import statements for langchain modules by @rvernica in #2436
• Fix 2440 Adjust LLM parameters in evals.py by @rvernica in #2441
• feat: migrate context recall, answer relevancy, and context entity recall metrics to modular BasePrompt architecture by @anistark in #2435
• feat: migrate 6 metrics (ContextRelevance, Response Groundedness, AnswerAccuracy, Faithfulness, AnswerCorrectness, SummaryScore) to BasePrompt by @anistark in #2443
• feat: migrate final metrics (FactualCorrectness, NoiseSensitivity) to modular BasePrompt architecture and update docs by @anistark in #2444
• chore: add COC by @anistark in #2437
• docs: clarify MLflow is required, not optional in RAG evaluation guide by @anistark in #2447
• chore: cleanup old patterns and update links by @anistark in #2449
• chore: rebranding efforts by @jjmachan in #2445
• feat: dual adapter support (Instructor + LiteLLM) by @anistark in #2446
• fix: resolve InstructorLLM detection bug and add EvaluationDataset backend support for experiments by @anistark in #2451
• fix: retrieved_contexts string filtering in LangChain integration by @dhyaneesh in #2452
• fix: correct MultiTurnSample user_input validation logic by @harshil-sanghvi in #2426
• fix: automatic embedding provider matching for LLMs by @anistark in #2454
• fix: make GitPython an optional dependency by @anistark in #2453
• docs: Update customizations how-to guides to use collections API and LLM factory by @sanjeed5 in #2425
• fix: detect async clients in closures for instructor-wrapped litellm routers by @anistark in #2458
• fix: quickstart by @anistark in #2463
• chore: update calendar email by @anistark in #2462
• fix: make GoogleEmbeddings handle GenerativeModel clients by auto-extracting genai module by @anistark in #2466
• docs: add migration guide for v0.4 by @anistark in #2461

New Contributors

• @Edge-Seven made their first contribution in #2429
• @rvernica made their first contribution in #2436
• @dhyaneesh made their first contribution in #2452

Full Changelog: v0.3.9...v0.4.0

What's Changed

• fix(docs): add missing line break so the step title and description a… by @nkch1k in #2391
• Migrate SummaryScore by @rhlbhatnagar in #2376
• feat: add metadata fields for synthetic data traceability by @dev-jonathan in #2389
• Migrate noise sensitivity by @rhlbhatnagar in #2379
• docs: quickstart guide with interactive LLM and project structure by @anistark in #2380
• Migrate Faithfullness by @rhlbhatnagar in #2384
• fix: docs for discrete, numeric and ranking using instructor by @anistark in #2397
• Migrate Answer Accuracy + Context Relevance by @rhlbhatnagar in #2390
• refactor: remove aspect critic and simple criteria metrics with discrete metric examples by @anistark in #2399
• Migrate Context Pricision with + without ref by @rhlbhatnagar in #2398
• docs: fix recall formula label in SQL metrics by @tysoncung in #2405
• chore: remove deprecated ground_truths by @anistark in #2402
• docs: Add documentation for metrics.collections API by @sanjeed5 in #2407
• Migrate factual correctness by @rhlbhatnagar in #2401
• refactor: remove redundant AnswerSimilarity from collections API in favor of SemanticSimilarity by @anistark in #2410
• docs: Update documentation structure to reflect experiments-first paradigm by @sanjeed5 in #2394
• Response Groundedness by @rhlbhatnagar in #2403
• fix: office hours link update by @anistark in #2415
• Refactor/removing deprecated by @anistark in #2412
• fix: handle max_completeion_tokens error for newer openai models by @anistark in #2413
• refactor: make embeddings optional in AnswerCorrectness when using pure factuality mode by @anistark in #2414
• Feat/migrate context recall by @jjmachan in #2372
• chore: update quickstart llm config by @anistark in #2417

New Contributors

• @nkch1k made their first contribution in #2391
• @dev-jonathan made their first contribution in #2389
• @tysoncung made their first contribution in #2405

Full Changelog: v0.3.8...v0.3.9

What's Changed

• feat: semantic similarity migrated to collections by @anistark in #2361
• feat: Add reusable testing infrastructure for metrics migration by @jjmachan in #2370
• add: console scripts for ragas_examples by @anistark in #2367
• feat: add quickstart cmd with templates to run by @anistark in #2374
• fix: detect uvloop and skip nest_asyncio to prevent patching errors by @anistark in #2369
• Remove import not used by @ChenyangLi4288 in #2364
• Migrate answer_correctness by @rhlbhatnagar in #2365
• Migrate context_entity_recall by @rhlbhatnagar in #2366
• feat: aspect critic metric for coherence, harmfulness, maliciousness, correctness by @anistark in #2375
• Fixed: NameError during evalutation of llamaindex query engine by @Prigoistic in #2331
• Remove error suppressor in async_utils.py and engine.py by @ChenyangLi4288 in #2362
• docs: clarify Context Relevance implementation differs from paper design by @anistark in #2378
• fix: add missing metrics (ToolCallF1, ChrfScore) to sidebar and document deprecated ContextUtilization by @anistark in #2381
• refactor: instructor_llm_factory merge with llm_factory by @anistark in #2382
• fix: handle tuple-formatted entities in SingleHopSpecificQuerySynthesizer by @anistark in #2377
• feat: simple criteria migrated to collections by @anistark in #2386
• chore: remove deprecation warnings for LangchainLLMWrapper, LlamaIndexLLMWrapper, and embedding wrappers by @anistark in #2387

New Contributors

• @ChenyangLi4288 made their first contribution in #2364

Full Changelog: v0.3.7...v0.3.8

What's Changed

• refactor: improve metrics code quality by @anistark in #2337
• chore: remove old analtyics by @jjmachan in #2338
• Fix/query distribution robustness by @yatoyun in #2340
• Simplify earlier how to guides in docs by @sanjeed5 in #2319
• docs: reorganize prompt evaluation guides in navigation by @sanjeed5 in #2346
• Metrics migration, migrate rouge + answer relevance by @rhlbhatnagar in #2335
• fix: streamline theme extraction from overlaps in MultiHopSpecificQue… by @kenzoyan in #2347
• Test/metric new compare by @anistark in #2349
• feat: bleu score migrated to collections by @anistark in #2352
• fix: Add List[List[str]] formats for overlapped items in theme extration (Continuation in #2347) by @kenzoyan in #2355
• feat: string metrics migrated to collections by @anistark in #2356
• feat: answer similarity migrated to collections by @anistark in #2358
• fix: add missing props token_usage_parser for test generation methods #2359 by @bhkj9999 in #2360
• feat: add bypass_n option to LangchainLLMWrapper for n-completion control by @SimFG in #2354
• docs: Add how-to guide for aligning LLM-as-Judge by @sanjeed5 in #2348

New Contributors

• @yatoyun made their first contribution in #2340
• @kenzoyan made their first contribution in #2347
• @bhkj9999 made their first contribution in #2360
• @SimFG made their first contribution in #2354

Full Changelog: v0.3.6...v0.3.7

What's Changed

• Feature/chrf score by @kauabh in #2221
• Fix/asyncio by @anistark in #2294
• Fix: update simple RAG init to use embed_text(s) (docs) by @s3pi in #2292
• Update _bleu_score.py by @kauabh in #2297
• Refactor/update gemini to genai sdk by @sahusiddharth in #2240
• Feature/metrics input flexibility by @anistark in #2298
• Ensure old_temperature is set correctly. Fixes #1937 and #2110 by @claudepi in #2295
• Enhance EmbeddingExtractor to support both async and sync methods for… by @telesoho in #2286
• Tokens counting by @anistark in #2299
• Fix/tool call accuracy by @anistark in #2300
• fix: coroutine warning for bleu by @anistark in #2301
• Add base_url parameter to embedding_factor by @anistark in #2303
• fix: add disallowed_special on tiktoken encode by @anistark in #2304
• Feat/tool call f1 1893 by @anistark in #2305
• Feature/azure token usage extraction by @anistark in #2306
• fix: improve metric decorators with better validation and error handling by @jjmachan in #2302
• Metric/parallel tool call by @anistark in #2307
• Fix: avoid ambiguous truth value for empty numpy array in HuggingfaceEmbeddings (fixes #2080) by @Rahul2512Chauhan in #2308
• Devpod cn/main by @anistark in #2309
• Feat/quoted spans metric by @anistark in #2311
• Fix noise sensitivity compute by @anistark in #2312
• Corrected numerous typos in Markdown files. by @ker2xu in #1994
• Deprecation warnings for LLMs and Prompts by @rhlbhatnagar in #2253
• Docs/eval_rag_agent - how to evaluate and improve rag app by @sanjeed5 in #2293
• Add llamaindex agentic evals gemini by @anistark in #2317
• fix: type str in tests by @anistark in #2318
• Fix generate_multiple caching issue (#1980) by @Rahul2512Chauhan in #2314
• fix: metric inheritance patterns: separate factory-created metrics from class-instantiated metrics by @jjmachan in #2316
• fix: concurrent ResponseRelevancy by @anistark in #2328
• fix: answer_relevancy scoring logic to prevent false zero by @anistark in #2327
• feat: Add OCI Gen AI Integration for Direct LLM Support by @harshil-sanghvi in #2321
• feat: Add save/load functionality and improved repr for LLM-based metrics by @jjmachan in #2320
• Fix: Fixed the Numpy 3.13 issue by @Prigoistic in #2282
• refactor: docs and warnings for metric base new structure by @anistark in #2333
• fix: typing by @anistark in #2334

New Contributors

• @kauabh made their first contribution in #2221
• @s3pi made their first contribution in #2292
• @claudepi made their first contribution in #2295
• @telesoho made their first contribution in #2286
• @ker2xu made their first contribution in #1994
• @harshil-sanghvi made their first contribution in #2321
• @Prigoistic made their first contribution in #2282

Full Changelog: v0.3.5...v0.3.6

What's Changed

• Docs/howto-texttosqlagent by @sanjeed5 in #2264
• fix: preview logo was too small. by @anistark in #2277
• modified the documentation to be in sync with current output format by @kotalaraghava in #2281
• removed some meta properties to test by @jjmachan in #2278
• feature: improve async / executor functionality by @ahgraber in #2070
• modification of the translate instruction by @anistark in #2284
• Remove experimental from docs and fix examples in docs by @sanjeed5 in #2270
• fix: resolve TypeError in TopicAdherenceScore bitwise operations by @anistark in #2258
• Knowledge graph/optimize for large corpus by @anistark in #2267
• Update _nv_metrics.py by @titericz in #2053
• Add telemetry by @rhlbhatnagar in #2260
• OpenAI model cost by @anistark in #2287
• docs: agent metrics code examples improvement by @yesidc in #1983
• Prompt Optimization Tutorial by @sahusiddharth in #1993
• Feature/metric type checking by @anistark in #2288
• improved the release script for ragas-examples by @jjmachan in #2289
• fix: removed the need for regex patterns by @jjmachan in #2290

New Contributors

• @kotalaraghava made their first contribution in #2281
• @yesidc made their first contribution in #1983

Full Changelog: v0.3.4...v0.3.5

v0.3.5rc2 Pre-release

Pre-release

Compare

Choose a tag to compare

What's Changed

• Docs/howto-texttosqlagent by @sanjeed5 in https://github.com/explodinggradients/ragas/pull/2264
• fix: preview logo was too small. by @anistark in https://github.com/explodinggradients/ragas/pull/2277
• modified the documentation to be in sync with current output format by @kotalaraghava in https://github.com/explodinggradients/ragas/pull/2281
• removed some meta properties to test by @jjmachan in https://github.com/explodinggradients/ragas/pull/2278
• feature: improve async / executor functionality by @ahgraber in https://github.com/explodinggradients/ragas/pull/2070
• modification of the translate instruction by @anistark in https://github.com/explodinggradients/ragas/pull/2284
• Remove experimental from docs and fix examples in docs by @sanjeed5 in https://github.com/explodinggradients/ragas/pull/2270
• fix: resolve TypeError in TopicAdherenceScore bitwise operations by @anistark in https://github.com/explodinggradients/ragas/pull/2258
• Knowledge graph/optimize for large corpus by @anistark in https://github.com/explodinggradients/ragas/pull/2267
• Update _nv_metrics.py by @titericz in https://github.com/explodinggradients/ragas/pull/2053
• Add telemetry by @rhlbhatnagar in https://github.com/explodinggradients/ragas/pull/2260
• OpenAI model cost by @anistark in https://github.com/explodinggradients/ragas/pull/2287
• docs: agent metrics code examples improvement by @yesidc in https://github.com/explodinggradients/ragas/pull/1983
• Prompt Optimization Tutorial by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1993
• Feature/metric type checking by @anistark in https://github.com/explodinggradients/ragas/pull/2288
• improved the release script for ragas-examples by @jjmachan in https://github.com/explodinggradients/ragas/pull/2289

New Contributors

• @kotalaraghava made their first contribution in https://github.com/explodinggradients/ragas/pull/2281
• @yesidc made their first contribution in https://github.com/explodinggradients/ragas/pull/1983

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.3.4...v0.3.5rc1

What's Changed

• Update context_precision.md by @anupamck in https://github.com/explodinggradients/ragas/pull/2262
• Reduce find_indirect_clusters() runtime through neighborhood detection and sampling by @ahgraber in https://github.com/explodinggradients/ragas/pull/2144
• fix: handle langchain multiple batching by @anistark in https://github.com/explodinggradients/ragas/pull/2257
• chore: fix the dead space in the header for docsite by @jjmachan in https://github.com/explodinggradients/ragas/pull/2265
• total bin coverage for default_transform() in Knowledge Graph transformations by @tolgaerdonmez in https://github.com/explodinggradients/ragas/pull/1950
• Improved context precision documentation by @anupamck in https://github.com/explodinggradients/ragas/pull/2266
• Fix docs: correct inheritance class for Non-LLM metrics by @AlanPonnachan in https://github.com/explodinggradients/ragas/pull/2272
• added rb2b analytics by @jjmachan in https://github.com/explodinggradients/ragas/pull/2273

New Contributors

• @anupamck made their first contribution in https://github.com/explodinggradients/ragas/pull/2262
• @tolgaerdonmez made their first contribution in https://github.com/explodinggradients/ragas/pull/1950
• @AlanPonnachan made their first contribution in https://github.com/explodinggradients/ragas/pull/2272

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.3.3...v0.3.4

What's Changed

• feat: refactor Ragas embeddings to maintain backward compatibility by @NirantK in https://github.com/explodinggradients/ragas/pull/2184
• feat: move tracing from experimental to ragas/integrations by @NirantK in https://github.com/explodinggradients/ragas/pull/2188
• feat: Refactor LLM structure and add InstructorLLM by @NirantK in https://github.com/explodinggradients/ragas/pull/2191
• fix: setup and local ci by @anistark in https://github.com/explodinggradients/ragas/pull/2199
• docs: correct small typos in Testset Generation page and related md components by @deibid in https://github.com/explodinggradients/ragas/pull/2123
• Docs/how-to-benchmark-new-llm-guide by @sanjeed5 in https://github.com/explodinggradients/ragas/pull/2158
• fix: access model fields via class by @emmanuel-ferdman in https://github.com/explodinggradients/ragas/pull/2146
• add: PR template for better review and overview of changes by @anistark in https://github.com/explodinggradients/ragas/pull/2202
• Move prompts to ragas by @NirantK in https://github.com/explodinggradients/ragas/pull/2197
• Fix: make the answer be the same as the split sentences by @wlbksy in https://github.com/explodinggradients/ragas/pull/2128
• Merge workspace.toml into pyproject.toml by @NirantK in https://github.com/explodinggradients/ragas/pull/2189
• Cleanup/move-benchmark-llm-to-examples by @sanjeed5 in https://github.com/explodinggradients/ragas/pull/2203
• fix: shields badge for downloads replace with pepy. by @anistark in https://github.com/explodinggradients/ragas/pull/2204
• refactor: removing some section to streamline pr template by @anistark in https://github.com/explodinggradients/ragas/pull/2206
• Move dataset module to main package and update imports by @NirantK in https://github.com/explodinggradients/ragas/pull/2205
• fix: type check for pyright by @anistark in https://github.com/explodinggradients/ragas/pull/2211
• refactor: move ragas out to project root. by @anistark in https://github.com/explodinggradients/ragas/pull/2209
• chore: fix the OIDC error in claude code by @jjmachan in https://github.com/explodinggradients/ragas/pull/2212
• fix: _version.py is to be ignored in git by @anistark in https://github.com/explodinggradients/ragas/pull/2213
• chore: only review for /claude-review command by @jjmachan in https://github.com/explodinggradients/ragas/pull/2214
• Fix a couple of typos in docs/getstarted/evals.md by @sadra-barikbin in https://github.com/explodinggradients/ragas/pull/2081
• Fix repeating template bug in _nv_metrics.py by @dev-nid in https://github.com/explodinggradients/ragas/pull/2216
• fix: ragas dir structure path fixes by @anistark in https://github.com/explodinggradients/ragas/pull/2215
• fix: minor typo in context_precision.md by @AbhishekRP2002 in https://github.com/explodinggradients/ragas/pull/2050
• fix: update docstrings for single-hop use case by @terry07 in https://github.com/explodinggradients/ragas/pull/1947
• Fix issue with cache hits by @Nicba1010 in https://github.com/explodinggradients/ragas/pull/1977
• Use generate_multiple instead of generate in a loop in ResponseRelevancy by @aabanic in https://github.com/explodinggradients/ragas/pull/1978
• fix: update required columns in RefusalRate (docs) by @DmitrySirakov in https://github.com/explodinggradients/ragas/pull/2008
• Add support for make install-minimal to speed up CI + dev by @rhlbhatnagar in https://github.com/explodinggradients/ragas/pull/2218
• Move experimental metrics to main package by @NirantK in https://github.com/explodinggradients/ragas/pull/2220
• Fix e2e test failures with datasets>=4.0.0 and missing dependencies by @anistark in https://github.com/explodinggradients/ragas/pull/2222
• Move Utils / Exceptions module to main package and update imports by @rhlbhatnagar in https://github.com/explodinggradients/ragas/pull/2219
• update compare_embeddings doc links by @resitaydin in https://github.com/explodinggradients/ragas/pull/2037
• chore: add py.typed marker file to support type checking by @sisp in https://github.com/explodinggradients/ragas/pull/2029
• Retire ragas.experimental namespace by @NirantK in https://github.com/explodinggradients/ragas/pull/2223
• chore: cleanup docs for ragas directory by @anistark in https://github.com/explodinggradients/ragas/pull/2224
• fix: update make commands to use uv for docs by @anistark in https://github.com/explodinggradients/ragas/pull/2217
• Remove experimental folder by @NirantK in https://github.com/explodinggradients/ragas/pull/2225
• Docs/how-to-iterate-prompt by @sanjeed5 in https://github.com/explodinggradients/ragas/pull/2226
• security: stop supporting v0.2 security releases by @anistark in https://github.com/explodinggradients/ragas/pull/2230
• Optimize factual correctness metric runtime by 50% by @vignesh14052002 in https://github.com/explodinggradients/ragas/pull/2153
• Fixed the mermaid which was breaking on navigation by @ganeshrvel in https://github.com/explodinggradients/ragas/pull/2121
• Fix typos and clean style by @kwinkunks in https://github.com/explodinggradients/ragas/pull/2042
• Feature/examples-as-installable-package by @sanjeed5 in https://github.com/explodinggradients/ragas/pull/2210
• chore: added reo.js analytics by @jjmachan in https://github.com/explodinggradients/ragas/pull/2232
• Move tutorials by @rhlbhatnagar in https://github.com/explodinggradients/ragas/pull/2235
• Move howtos by @rhlbhatnagar in https://github.com/explodinggradients/ragas/pull/2236
• Move references to docs/ by @rhlbhatnagar in https://github.com/explodinggradients/ragas/pull/2234
• Fix BadRequestError while using OpenAI O1 series models | Add option to bypass temperature by @vignesh14052002 in https://github.com/explodinggradients/ragas/pull/2151
• feature!: knowledge graph flexibility by @ahgraber in https://github.com/explodinggradients/ragas/pull/2030
• return False when pred_sequence length and ref_requence length is dif… by @licux in https://github.com/explodinggradients/ragas/pull/1894
• Feature: improve relationship builders for better async and reduced memory utilization by @ahgraber in https://github.com/explodinggradients/ragas/pull/2077
• Mirgate all leftover docks from experimental by @rhlbhatnagar in https://github.com/explodinggradients/ragas/pull/2243
• Add deprecation warnings for embeddings by @rhlbhatnagar in https://github.com/explodinggradients/ragas/pull/2244
• set temperature to 0.01 for better llm compatibility by @0Falli0 in https://github.com/explodinggradients/ragas/pull/939
• docs: fix broken link in FAQ (metrics tracing) by @Rahul2512Chauhan in https://github.com/explodinggradients/ragas/pull/2238
• fix: temperature change to 0.01 bugs by @anistark in https://github.com/explodinggradients/ragas/pull/2247
• Scrubbed the references to app.ragas.io by @ganeshrvel in https://github.com/explodinggradients/ragas/pull/2207
• feat: cancellable task by @anistark in https://github.com/explodinggradients/ragas/pull/2242
• update: add og images and metadata to docs by @anistark in https://github.com/explodinggradients/ragas/pull/2248
• fix: add support for both langchain llm and ragas llm by @anistark in https://github.com/explodinggradients/ragas/pull/2229
• chore: analytics support for commonroom and scarf by @jjmachan in https://github.com/explodinggradients/ragas/pull/2250

New Contributors

• @deibid made their first contribution in https://github.com/explodinggradients/ragas/pull/2123
• @wlbksy made their first contribution in https://github.com/explodinggradients/ragas/pull/2128
• @sadra-barikbin made their first contribution in https://github.com/explodinggradients/ragas/pull/2081
• @dev-nid made their first contribution in https://github.com/explodinggradients/ragas/pull/2216
• @AbhishekRP2002 made their first contribution in https://github.com/explodinggradients/ragas/pull/2050
• @terry07 made their first contribution in https://github.com/explodinggradients/ragas/pull/1947
• @Nicba1010 made their first contribution in https://github.com/explodinggradients/ragas/pull/1977
• @aabanic made their first contribution in https://github.com/explodinggradients/ragas/pull/1978
• @rhlbhatnagar made their first contribution in https://github.com/explodinggradients/ragas/pull/2218
• @resitaydin made their first contribution in https://github.com/explodinggradients/ragas/pull/2037
• @sisp made their first contribution in https://github.com/explodinggradients/ragas/pull/2029
• @vignesh14052002 made their first contribution in https://github.com/explodinggradients/ragas/pull/2153
• @kwinkunks made their first contribution in https://github.com/explodinggradients/ragas/pull/2042
• @0Falli0 made their first contribution in https://github.com/explodinggradients/ragas/pull/939
• @Rahul2512Chauhan made their first contribution in https://github.com/explodinggradients/ragas/pull/2238

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.3.2...v0.3.3

What's Changed

• feat: refactor Ragas embeddings to maintain backward compatibility by @NirantK in https://github.com/explodinggradients/ragas/pull/2184
• feat: move tracing from experimental to ragas/integrations by @NirantK in https://github.com/explodinggradients/ragas/pull/2188
• feat: Refactor LLM structure and add InstructorLLM by @NirantK in https://github.com/explodinggradients/ragas/pull/2191
• fix: setup and local ci by @anistark in https://github.com/explodinggradients/ragas/pull/2199
• docs: correct small typos in Testset Generation page and related md components by @deibid in https://github.com/explodinggradients/ragas/pull/2123
• Docs/how-to-benchmark-new-llm-guide by @sanjeed5 in https://github.com/explodinggradients/ragas/pull/2158
• fix: access model fields via class by @emmanuel-ferdman in https://github.com/explodinggradients/ragas/pull/2146
• add: PR template for better review and overview of changes by @anistark in https://github.com/explodinggradients/ragas/pull/2202
• Move prompts to ragas by @NirantK in https://github.com/explodinggradients/ragas/pull/2197
• Fix: make the answer be the same as the split sentences by @wlbksy in https://github.com/explodinggradients/ragas/pull/2128
• Merge workspace.toml into pyproject.toml by @NirantK in https://github.com/explodinggradients/ragas/pull/2189
• Cleanup/move-benchmark-llm-to-examples by @sanjeed5 in https://github.com/explodinggradients/ragas/pull/2203
• fix: shields badge for downloads replace with pepy. by @anistark in https://github.com/explodinggradients/ragas/pull/2204
• refactor: removing some section to streamline pr template by @anistark in https://github.com/explodinggradients/ragas/pull/2206
• Move dataset module to main package and update imports by @NirantK in https://github.com/explodinggradients/ragas/pull/2205
• fix: type check for pyright by @anistark in https://github.com/explodinggradients/ragas/pull/2211
• refactor: move ragas out to project root. by @anistark in https://github.com/explodinggradients/ragas/pull/2209
• chore: fix the OIDC error in claude code by @jjmachan in https://github.com/explodinggradients/ragas/pull/2212
• fix: _version.py is to be ignored in git by @anistark in https://github.com/explodinggradients/ragas/pull/2213
• chore: only review for /claude-review command by @jjmachan in https://github.com/explodinggradients/ragas/pull/2214
• Fix a couple of typos in docs/getstarted/evals.md by @sadra-barikbin in https://github.com/explodinggradients/ragas/pull/2081
• Fix repeating template bug in _nv_metrics.py by @dev-nid in https://github.com/explodinggradients/ragas/pull/2216
• fix: ragas dir structure path fixes by @anistark in https://github.com/explodinggradients/ragas/pull/2215
• fix: minor typo in context_precision.md by @AbhishekRP2002 in https://github.com/explodinggradients/ragas/pull/2050
• fix: update docstrings for single-hop use case by @terry07 in https://github.com/explodinggradients/ragas/pull/1947
• Fix issue with cache hits by @Nicba1010 in https://github.com/explodinggradients/ragas/pull/1977
• Use generate_multiple instead of generate in a loop in ResponseRelevancy by @aabanic in https://github.com/explodinggradients/ragas/pull/1978
• fix: update required columns in RefusalRate (docs) by @DmitrySirakov in https://github.com/explodinggradients/ragas/pull/2008
• Add support for make install-minimal to speed up CI + dev by @rhlbhatnagar in https://github.com/explodinggradients/ragas/pull/2218
• Move experimental metrics to main package by @NirantK in https://github.com/explodinggradients/ragas/pull/2220
• Fix e2e test failures with datasets>=4.0.0 and missing dependencies by @anistark in https://github.com/explodinggradients/ragas/pull/2222
• Move Utils / Exceptions module to main package and update imports by @rhlbhatnagar in https://github.com/explodinggradients/ragas/pull/2219
• update compare_embeddings doc links by @resitaydin in https://github.com/explodinggradients/ragas/pull/2037
• chore: add py.typed marker file to support type checking by @sisp in https://github.com/explodinggradients/ragas/pull/2029
• Retire ragas.experimental namespace by @NirantK in https://github.com/explodinggradients/ragas/pull/2223
• chore: cleanup docs for ragas directory by @anistark in https://github.com/explodinggradients/ragas/pull/2224
• fix: update make commands to use uv for docs by @anistark in https://github.com/explodinggradients/ragas/pull/2217
• Remove experimental folder by @NirantK in https://github.com/explodinggradients/ragas/pull/2225
• Docs/how-to-iterate-prompt by @sanjeed5 in https://github.com/explodinggradients/ragas/pull/2226
• security: stop supporting v0.2 security releases by @anistark in https://github.com/explodinggradients/ragas/pull/2230
• Optimize factual correctness metric runtime by 50% by @vignesh14052002 in https://github.com/explodinggradients/ragas/pull/2153
• Fixed the mermaid which was breaking on navigation by @ganeshrvel in https://github.com/explodinggradients/ragas/pull/2121
• Fix typos and clean style by @kwinkunks in https://github.com/explodinggradients/ragas/pull/2042
• Feature/examples-as-installable-package by @sanjeed5 in https://github.com/explodinggradients/ragas/pull/2210
• chore: added reo.js analytics by @jjmachan in https://github.com/explodinggradients/ragas/pull/2232
• Move tutorials by @rhlbhatnagar in https://github.com/explodinggradients/ragas/pull/2235
• Move howtos by @rhlbhatnagar in https://github.com/explodinggradients/ragas/pull/2236
• Move references to docs/ by @rhlbhatnagar in https://github.com/explodinggradients/ragas/pull/2234
• Fix BadRequestError while using OpenAI O1 series models | Add option to bypass temperature by @vignesh14052002 in https://github.com/explodinggradients/ragas/pull/2151
• feature!: knowledge graph flexibility by @ahgraber in https://github.com/explodinggradients/ragas/pull/2030
• return False when pred_sequence length and ref_requence length is dif… by @licux in https://github.com/explodinggradients/ragas/pull/1894
• Feature: improve relationship builders for better async and reduced memory utilization by @ahgraber in https://github.com/explodinggradients/ragas/pull/2077
• Mirgate all leftover docks from experimental by @rhlbhatnagar in https://github.com/explodinggradients/ragas/pull/2243
• Add deprecation warnings for embeddings by @rhlbhatnagar in https://github.com/explodinggradients/ragas/pull/2244
• set temperature to 0.01 for better llm compatibility by @0Falli0 in https://github.com/explodinggradients/ragas/pull/939
• docs: fix broken link in FAQ (metrics tracing) by @Rahul2512Chauhan in https://github.com/explodinggradients/ragas/pull/2238
• fix: temperature change to 0.01 bugs by @anistark in https://github.com/explodinggradients/ragas/pull/2247
• Scrubbed the references to app.ragas.io by @ganeshrvel in https://github.com/explodinggradients/ragas/pull/2207
• feat: cancellable task by @anistark in https://github.com/explodinggradients/ragas/pull/2242
• update: add og images and metadata to docs by @anistark in https://github.com/explodinggradients/ragas/pull/2248
• fix: add support for both langchain llm and ragas llm by @anistark in https://github.com/explodinggradients/ragas/pull/2229
• chore: analytics support for commonroom and scarf by @jjmachan in https://github.com/explodinggradients/ragas/pull/2250

New Contributors

• @deibid made their first contribution in https://github.com/explodinggradients/ragas/pull/2123
• @wlbksy made their first contribution in https://github.com/explodinggradients/ragas/pull/2128
• @sadra-barikbin made their first contribution in https://github.com/explodinggradients/ragas/pull/2081
• @dev-nid made their first contribution in https://github.com/explodinggradients/ragas/pull/2216
• @AbhishekRP2002 made their first contribution in https://github.com/explodinggradients/ragas/pull/2050
• @terry07 made their first contribution in https://github.com/explodinggradients/ragas/pull/1947
• @Nicba1010 made their first contribution in https://github.com/explodinggradients/ragas/pull/1977
• @aabanic made their first contribution in https://github.com/explodinggradients/ragas/pull/1978
• @rhlbhatnagar made their first contribution in https://github.com/explodinggradients/ragas/pull/2218
• @resitaydin made their first contribution in https://github.com/explodinggradients/ragas/pull/2037
• @sisp made their first contribution in https://github.com/explodinggradients/ragas/pull/2029
• @vignesh14052002 made their first contribution in https://github.com/explodinggradients/ragas/pull/2153
• @kwinkunks made their first contribution in https://github.com/explodinggradients/ragas/pull/2042
• @0Falli0 made their first contribution in https://github.com/explodinggradients/ragas/pull/939
• @Rahul2512Chauhan made their first contribution in https://github.com/explodinggradients/ragas/pull/2238

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.3.2...v0.3.3rc1

What's Changed

• feat: save and load prompt object by @jjmachan in https://github.com/explodinggradients/ragas/pull/2152
• refactor: use basemetric as base for metric by @anistark in https://github.com/explodinggradients/ragas/pull/2163
• add: pre-commit hook to run ci on every commit by @anistark in https://github.com/explodinggradients/ragas/pull/2164
• Make AnswerSimilarity a dataclass by @hutec in https://github.com/explodinggradients/ragas/pull/2159
• docs: Add discord link to Community page in docs by @sanjeed5 in https://github.com/explodinggradients/ragas/pull/2157
• Fixed a small typo in context_precision.md by @ImadSaddik in https://github.com/explodinggradients/ragas/pull/2166
• Move Experimental to inside Ragas by @NirantK in https://github.com/explodinggradients/ragas/pull/2167
• refactor: experiment function to use model as additional optional arg by @anistark in https://github.com/explodinggradients/ragas/pull/2165
• Add python 3.13 target, refactor on pre-commit and benchmarking tests by @anistark in https://github.com/explodinggradients/ragas/pull/2169
• fix: enum values instead of names in testset synthesizers by @lovets18 in https://github.com/explodinggradients/ragas/pull/2137
• Cleaner, simpler pyproject.toml by @NirantK in https://github.com/explodinggradients/ragas/pull/2173
• Move Examples to root by @NirantK in https://github.com/explodinggradients/ragas/pull/2176
• Add backend to ragas main by @NirantK in https://github.com/explodinggradients/ragas/pull/2174
• Remove txt requirements by @NirantK in https://github.com/explodinggradients/ragas/pull/2180
• Remove black by @NirantK in https://github.com/explodinggradients/ragas/pull/2178
• feat: move experiment from experimental to ragas main by @NirantK in https://github.com/explodinggradients/ragas/pull/2175
• feat: migrate CLI from experimental to main package by @NirantK in https://github.com/explodinggradients/ragas/pull/2183
• Reformat examples by @NirantK in https://github.com/explodinggradients/ragas/pull/2186
• Remove simulation by @NirantK in https://github.com/explodinggradients/ragas/pull/2187
• refactor: avoid notebook in test. run all in test-all by @anistark in https://github.com/explodinggradients/ragas/pull/2171
• Add Claude Code GitHub Workflow by @jjmachan in https://github.com/explodinggradients/ragas/pull/2192
• hotfix: Include LICENSE and README.md in PyPI package by @NirantK in https://github.com/explodinggradients/ragas/pull/2195
• fix: pypi requirements and image absolute path by @anistark in https://github.com/explodinggradients/ragas/pull/2196
• chore: fix release script by @jjmachan in https://github.com/explodinggradients/ragas/pull/2198

New Contributors

• @anistark made their first contribution in https://github.com/explodinggradients/ragas/pull/2163
• @hutec made their first contribution in https://github.com/explodinggradients/ragas/pull/2159
• @ImadSaddik made their first contribution in https://github.com/explodinggradients/ragas/pull/2166
• @lovets18 made their first contribution in https://github.com/explodinggradients/ragas/pull/2137

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.3.1...v0.3.2

What's Changed

• fix: pypi requirements and image absolute path by @anistark in https://github.com/explodinggradients/ragas/pull/2196

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.3.2-rc1...v0.3.2-rc2

What's Changed

• feat: save and load prompt object by @jjmachan in https://github.com/explodinggradients/ragas/pull/2152
• refactor: use basemetric as base for metric by @anistark in https://github.com/explodinggradients/ragas/pull/2163
• add: pre-commit hook to run ci on every commit by @anistark in https://github.com/explodinggradients/ragas/pull/2164
• Make AnswerSimilarity a dataclass by @hutec in https://github.com/explodinggradients/ragas/pull/2159
• docs: Add discord link to Community page in docs by @sanjeed5 in https://github.com/explodinggradients/ragas/pull/2157
• Fixed a small typo in context_precision.md by @ImadSaddik in https://github.com/explodinggradients/ragas/pull/2166
• Move Experimental to inside Ragas by @NirantK in https://github.com/explodinggradients/ragas/pull/2167
• refactor: experiment function to use model as additional optional arg by @anistark in https://github.com/explodinggradients/ragas/pull/2165
• Add python 3.13 target, refactor on pre-commit and benchmarking tests by @anistark in https://github.com/explodinggradients/ragas/pull/2169
• fix: enum values instead of names in testset synthesizers by @lovets18 in https://github.com/explodinggradients/ragas/pull/2137
• Cleaner, simpler pyproject.toml by @NirantK in https://github.com/explodinggradients/ragas/pull/2173
• Move Examples to root by @NirantK in https://github.com/explodinggradients/ragas/pull/2176
• Add backend to ragas main by @NirantK in https://github.com/explodinggradients/ragas/pull/2174
• Remove txt requirements by @NirantK in https://github.com/explodinggradients/ragas/pull/2180
• Remove black by @NirantK in https://github.com/explodinggradients/ragas/pull/2178
• feat: move experiment from experimental to ragas main by @NirantK in https://github.com/explodinggradients/ragas/pull/2175
• feat: migrate CLI from experimental to main package by @NirantK in https://github.com/explodinggradients/ragas/pull/2183
• Reformat examples by @NirantK in https://github.com/explodinggradients/ragas/pull/2186
• Remove simulation by @NirantK in https://github.com/explodinggradients/ragas/pull/2187
• refactor: avoid notebook in test. run all in test-all by @anistark in https://github.com/explodinggradients/ragas/pull/2171
• Add Claude Code GitHub Workflow by @jjmachan in https://github.com/explodinggradients/ragas/pull/2192
• hotfix: Include LICENSE and README.md in PyPI package by @NirantK in https://github.com/explodinggradients/ragas/pull/2195

New Contributors

• @anistark made their first contribution in https://github.com/explodinggradients/ragas/pull/2163
• @hutec made their first contribution in https://github.com/explodinggradients/ragas/pull/2159
• @ImadSaddik made their first contribution in https://github.com/explodinggradients/ragas/pull/2166
• @lovets18 made their first contribution in https://github.com/explodinggradients/ragas/pull/2137

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.3.1...v0.3.2-rc1

What's Changed

• docs: improve hello world by @shahules786 in https://github.com/explodinggradients/ragas/pull/2127
• examples: fix api changes by @shahules786 in https://github.com/explodinggradients/ragas/pull/2129
• docs: add explanation by @shahules786 in https://github.com/explodinggradients/ragas/pull/2139
• adds Google Drive backend for storing datasets as sheets by @dmikey in https://github.com/explodinggradients/ragas/pull/2138
• Fix experimental docs navigation, fix broken tutorials, improve examples for better user understanding by @sanjeed5 in https://github.com/explodinggradients/ragas/pull/2156
• fix: correct mkdocstrings path configuration for local development by @sanjeed5 in https://github.com/explodinggradients/ragas/pull/2155

New Contributors

• @dmikey made their first contribution in https://github.com/explodinggradients/ragas/pull/2138
• @sanjeed5 made their first contribution in https://github.com/explodinggradients/ragas/pull/2156

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.3.0...v0.3.1

What's Changed

• llamaindex agentic integration by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/2026
• CVE-2025-45691 Fix: Arbitrary File Read Vulnerability + Feat: Security settings by @adithyan-ak in https://github.com/explodinggradients/ragas/pull/1991
• feat: add devin's deepwiki by @jjmachan in https://github.com/explodinggradients/ragas/pull/2039
• feat: added ragas_experimental back into main by @jjmachan in https://github.com/explodinggradients/ragas/pull/2031
• fix: dataset slicing by @shahules786 in https://github.com/explodinggradients/ragas/pull/2057
• feat: user simulator by @shahules786 in https://github.com/explodinggradients/ragas/pull/2062
• chore: make CI pass by @jjmachan in https://github.com/explodinggradients/ragas/pull/2063
• Update analytics source by @emmanuel-ferdman in https://github.com/explodinggradients/ragas/pull/2059
• fix: update embedding extraction to use appropriate async method by @ahgraber in https://github.com/explodinggradients/ragas/pull/2068
• feat: check embedding shapes in CosineSimilarityBuilders by @ahgraber in https://github.com/explodinggradients/ragas/pull/2069
• feat: llm as prompt as optional by @shahules786 in https://github.com/explodinggradients/ragas/pull/2084
• chore: added claude code workflow by @jjmachan in https://github.com/explodinggradients/ragas/pull/2089
• chore: refactor the experimental code off nbdev by @jjmachan in https://github.com/explodinggradients/ragas/pull/2087
• feat: align and validate alignment rate by @shahules786 in https://github.com/explodinggradients/ragas/pull/2094
• Fix/docs upgrade shade 2 by @ganeshrvel in https://github.com/explodinggradients/ragas/pull/2097
• feat: ragas evals CLI by @shahules786 in https://github.com/explodinggradients/ragas/pull/2086
• docs: hello world for ragas experimental by @shahules786 in https://github.com/explodinggradients/ragas/pull/2100
• feat: refactored out Project by @jjmachan in https://github.com/explodinggradients/ragas/pull/2106
• Updated the header color and moved away from yellow shade to more dar… by @ganeshrvel in https://github.com/explodinggradients/ragas/pull/2104
• examples : example evals by @shahules786 in https://github.com/explodinggradients/ragas/pull/2109
• Fix/theme black primary color by @ganeshrvel in https://github.com/explodinggradients/ragas/pull/2113
• fix: cleaned up some metrics by @jjmachan in https://github.com/explodinggradients/ragas/pull/2111
• feat: add llm_factory and embedding_factory by @jjmachan in https://github.com/explodinggradients/ragas/pull/2112
• chore: release scripts and cleanup for monorepo by @jjmachan in https://github.com/explodinggradients/ragas/pull/2117
• fix: release script by @jjmachan in https://github.com/explodinggradients/ragas/pull/2118
• docs: hello world & explanation by @shahules786 in https://github.com/explodinggradients/ragas/pull/2114
• chore: added ragas.experimental and cleaned dependencies by @jjmachan in https://github.com/explodinggradients/ragas/pull/2119
• fix: metric alignment with inmemory backend by @jjmachan in https://github.com/explodinggradients/ragas/pull/2116
• docs: added structure to sidebar by @jjmachan in https://github.com/explodinggradients/ragas/pull/2120

New Contributors

• @emmanuel-ferdman made their first contribution in https://github.com/explodinggradients/ragas/pull/2059

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.15...v0.3.0

What's Changed

• md of vertexai notebooks by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1957
• Single-hop query testset generation tutorial by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1959
• AWS Bedrock Integration by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1974
• Benchmarking Tutorial by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1984
• Create SECURITY.md by @adithyan-ak in https://github.com/explodinggradients/ragas/pull/1987
• Evaluation function documentation improvements by @gbrlbd in https://github.com/explodinggradients/ragas/pull/1965
• directory name change in docs by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1966
• Detailed error message for noise sensitivity metric by @KalyanKS-NLP in https://github.com/explodinggradients/ragas/pull/1986
• Docs: Fixed broken links in nav bar for NVIDIA metrics by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1971
• Refactor: Update AWS Bedrock Naming to Amazon Bedrock Across Docs & Repo Structure by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1982
• LlamaStack integration by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/2011
• Adding the link to download the utils and minor changes by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/2003
• Enhance MultiTurnSample Validator to Support Multiple Tool Calls by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/2017
• Griptape integration by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/2021
• Fix shared examples bug in FactualCorrectness metric by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/2016
• docs: removed SUPPORTED_LANGUAGE by @DmitrySirakov in https://github.com/explodinggradients/ragas/pull/2020
• docs: add CTA for Ragas app by @shahules786 in https://github.com/explodinggradients/ragas/pull/2023

New Contributors

• @adithyan-ak made their first contribution in https://github.com/explodinggradients/ragas/pull/1987
• @gbrlbd made their first contribution in https://github.com/explodinggradients/ragas/pull/1965
• @KalyanKS-NLP made their first contribution in https://github.com/explodinggradients/ragas/pull/1986
• @DmitrySirakov made their first contribution in https://github.com/explodinggradients/ragas/pull/2020

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.14...v0.2.15

What's Changed

• feat: add http request-response logging with env flag control by @ganeshrvel in https://github.com/explodinggradients/ragas/pull/1903
• Metric with different parameter gives different score by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1908
• Updated the instruction of StatementGeneratorPrompt in faithfulness by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1906
• Fixed simple criteria metric by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1909
• Evaluating Multi-Turn Conversations by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1919
• fix SingleHopQuerySynthesizer::prepare_combinations() by @lryan599 in https://github.com/explodinggradients/ragas/pull/1921
• docs: fix semantic similarity description (cross-encoder -> bi-encoder) by @Ayaka-mogumogu in https://github.com/explodinggradients/ragas/pull/1910
• docs: list googles model provider by @jjmachan in https://github.com/explodinggradients/ragas/pull/1920
• nvidia end-2-end accuracy, relevance and groundedness metrics by @titericz in https://github.com/explodinggradients/ragas/pull/1913
• Enable Runtime Checks for ModeMetric Protocol by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1929
• r2r integration by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1918
• Haystack llm and embedding wrapper by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1901
• Easy fix, Groundedness metric, 5 retries early break. by @titericz in https://github.com/explodinggradients/ragas/pull/1935
• nvidia docs by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1940
• unnecessary Noise Sensitivity name update by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1943
• When saving kg only save the node id in the relationships by @lryan599 in https://github.com/explodinggradients/ragas/pull/1926
• fix: return valid JSON output schema by @alasdairsmith in https://github.com/explodinggradients/ragas/pull/1933
• Update default.py - Missing cosine_similarity transform for docs with Token Count 101–500 by @rgrizzo-linksmt in https://github.com/explodinggradients/ragas/pull/1934
• Ndarray dtype fix, improve error wording by @zybinmikhail in https://github.com/explodinggradients/ragas/pull/1924

New Contributors

• @lryan599 made their first contribution in https://github.com/explodinggradients/ragas/pull/1921
• @Ayaka-mogumogu made their first contribution in https://github.com/explodinggradients/ragas/pull/1910
• @titericz made their first contribution in https://github.com/explodinggradients/ragas/pull/1913
• @alasdairsmith made their first contribution in https://github.com/explodinggradients/ragas/pull/1933
• @rgrizzo-linksmt made their first contribution in https://github.com/explodinggradients/ragas/pull/1934
• @zybinmikhail made their first contribution in https://github.com/explodinggradients/ragas/pull/1924

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.13...v0.2.14

What's Changed

• Improved the documentation in the migration docs by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1873
• docs: match default timeout in RunConfig by @neo in https://github.com/explodinggradients/ragas/pull/1872
• fix(prompt/mixin): Add name property and add it to saving/loading path. by @tim-hilde in https://github.com/explodinggradients/ragas/pull/1853
• Adding missing evaluate import to evals guide by @carlosgsouza in https://github.com/explodinggradients/ragas/pull/1876
• Improve error message for missing embeddings in Answer Relevancy Metric by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1877
• Change the validate_samples functionality by @Vidit-Ostwal in https://github.com/explodinggradients/ragas/pull/1879
• chore: added openhands for solving some github issues by @jjmachan in https://github.com/explodinggradients/ragas/pull/1884
• Fix Broken Test Case Due to Changed Error Message by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1881
• Faithfulness prompt update to avoid having single quotes in response by @michaelromagne in https://github.com/explodinggradients/ragas/pull/1874
• Internal logging and tracing improvements. by @ganeshrvel in https://github.com/explodinggradients/ragas/pull/1886
• docs: added office hour links by @jjmachan in https://github.com/explodinggradients/ragas/pull/1888
• added the end of sequence token for the WatsonX family of models by @malikaltakrori in https://github.com/explodinggradients/ragas/pull/1883
• Enhance langgraph integration to preserve metadata by @i-w-a in https://github.com/explodinggradients/ragas/pull/1878
• Adding tutorial for Haystack Integration by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1887
• Remove llm argument from ToolCallAccuracy initialization by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1891
• Updated Application Page by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1899
• fix: UnicodeEncode error when save/load knowledge graph by @jjmachan in https://github.com/explodinggradients/ragas/pull/1900

New Contributors

• @tim-hilde made their first contribution in https://github.com/explodinggradients/ragas/pull/1853
• @carlosgsouza made their first contribution in https://github.com/explodinggradients/ragas/pull/1876
• @Vidit-Ostwal made their first contribution in https://github.com/explodinggradients/ragas/pull/1879
• @malikaltakrori made their first contribution in https://github.com/explodinggradients/ragas/pull/1883

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.12...v0.2.13

What's Changed

• Add token parser for Bedrock & fix anthropic typo by @michaelromagne in https://github.com/explodinggradients/ragas/pull/1851
• fix tp&fp calc error by @oslijunw in https://github.com/explodinggradients/ragas/pull/1848
• docs: added canonical_url by @jjmachan in https://github.com/explodinggradients/ragas/pull/1855
• docs: be consistent with the variable naming of evaluator_embeddings by @neo in https://github.com/explodinggradients/ragas/pull/1858
• Add optional use_effective_order parameter to BLEU score by @JonasElburgUVA in https://github.com/explodinggradients/ragas/pull/1862
• docs: updated broken links in ReadMe and evaluation documentation by @iamarunbrahma in https://github.com/explodinggradients/ragas/pull/1847
• docs: fix incorrect reference to LLM in NonLLMContextPrecisionWithReference metric by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1863
• fix: output parser bug by @jjmachan in https://github.com/explodinggradients/ragas/pull/1864

New Contributors

• @michaelromagne made their first contribution in https://github.com/explodinggradients/ragas/pull/1851
• @neo made their first contribution in https://github.com/explodinggradients/ragas/pull/1858
• @JonasElburgUVA made their first contribution in https://github.com/explodinggradients/ragas/pull/1862
• @iamarunbrahma made their first contribution in https://github.com/explodinggradients/ragas/pull/1847

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.11...v0.2.12

What's Changed

• fix: add min requirement for optimisation by @shahules786 in https://github.com/explodinggradients/ragas/pull/1820
• fix: rubrics based metrics by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1821
• Enhance ToolCall message to support all argument types by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1823
• Added from_url method to fetch evaluation annotation from the api. by @ganeshrvel in https://github.com/explodinggradients/ragas/pull/1795
• docs: fix incorrect imports by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1824
• docs: update some misc doc errors by @jjmachan in https://github.com/explodinggradients/ragas/pull/1825
• Feat: Fix for Instance Based Metrics and Updated docs by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1827
• Docs/updating the LangSmith docs by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1828
• fix: remove pysbd and sentence segmenting by @shahules786 in https://github.com/explodinggradients/ragas/pull/1826
• docs: Fixed most of the broken links by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1830
• docs: updated the formula for some metrics by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1834
• docs: update _language_adaptation.md by @eltociear in https://github.com/explodinggradients/ragas/pull/1839
• feat: added option to provide experiment_name to evaluate() by @jjmachan in https://github.com/explodinggradients/ragas/pull/1843
• Change save and load encode to utf-8 by @joaorura in https://github.com/explodinggradients/ragas/pull/1836
• fix numpy invert error in metrics by @oslijunw in https://github.com/explodinggradients/ragas/pull/1835
• Adding swarm integration and tutorial by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1841
• Adding Swarm integration to the list of integrations by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1845
• Adjustment when calculating hash | Adjustment of the hash calculation… by @joaorura in https://github.com/explodinggradients/ragas/pull/1837
• docs: removed SUPPORTED_LANGUAGE by @jjmachan in https://github.com/explodinggradients/ragas/pull/1846

New Contributors

• @ganeshrvel made their first contribution in https://github.com/explodinggradients/ragas/pull/1795
• @oslijunw made their first contribution in https://github.com/explodinggradients/ragas/pull/1835

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.10...v0.2.11

What's Changed

• docs: new get started guide by @shahules786 in https://github.com/explodinggradients/ragas/pull/1786
• Update README.md by @shahules786 in https://github.com/explodinggradients/ragas/pull/1794
• docs: fix typo by @shahules786 in https://github.com/explodinggradients/ragas/pull/1801
• fix: llamaindex testset generator by @jjmachan in https://github.com/explodinggradients/ragas/pull/1804
• Docs improvements by @EricGustin in https://github.com/explodinggradients/ragas/pull/1805
• Docs improvements by @hundredeuk2 in https://github.com/explodinggradients/ragas/pull/1808
• docs: add new rag eval tutorial by @shahules786 in https://github.com/explodinggradients/ragas/pull/1815
• Fix typo in index.md by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1811
• docs: clean get-started by @shahules786 in https://github.com/explodinggradients/ragas/pull/1816
• Improve question generation efficiency In Response Relevancy metrics by @hundredeuk2 in https://github.com/explodinggradients/ragas/pull/1810
• docs: langchain v3 integration tutorial by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1818

New Contributors

• @EricGustin made their first contribution in https://github.com/explodinggradients/ragas/pull/1805
• @hundredeuk2 made their first contribution in https://github.com/explodinggradients/ragas/pull/1808

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.9...v0.2.10

What's Changed

• feat: replaced NLTK's implementation of BLEU with sacrebleu's implementation by @trent-sp in https://github.com/explodinggradients/ragas/pull/1744
• doc: add note by @shahules786 in https://github.com/explodinggradients/ragas/pull/1753
• docs: add blog to readme by @shahules786 in https://github.com/explodinggradients/ragas/pull/1755
• fixes: some minor fixes by @shahules786 in https://github.com/explodinggradients/ragas/pull/1756
• docs: added ragas-app to getting started by @jjmachan in https://github.com/explodinggradients/ragas/pull/1754
• feat: add exact match caching by @ayulockin in https://github.com/explodinggradients/ragas/pull/1717
• fix: temperature needs to be added handled effectively by @jjmachan in https://github.com/explodinggradients/ragas/pull/1759
• Fix/hhem: divide by zero error by @jjmachan in https://github.com/explodinggradients/ragas/pull/1765
• fix: add reference to simple scoring by @shahules786 in https://github.com/explodinggradients/ragas/pull/1758
• fix: import train config and add tests by @shahules786 in https://github.com/explodinggradients/ragas/pull/1776
• chore: make analytics into threading by @jjmachan in https://github.com/explodinggradients/ragas/pull/1777
• fix: change to schema by @shahules786 in https://github.com/explodinggradients/ragas/pull/1780
• Fix link to langchain and add pip install by @pamelafox in https://github.com/explodinggradients/ragas/pull/1762
• fix: distance measure in non llm CR by @shahules786 in https://github.com/explodinggradients/ragas/pull/1787
• docs: caching in ragas by @jjmachan in https://github.com/explodinggradients/ragas/pull/1779
• fix: remove reproducibility arg by @shahules786 in https://github.com/explodinggradients/ragas/pull/1790

New Contributors

• @trent-sp made their first contribution in https://github.com/explodinggradients/ragas/pull/1744
• @pamelafox made their first contribution in https://github.com/explodinggradients/ragas/pull/1762

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.8...v0.2.9

What's Changed

• fix: Update README.md by @avanco in https://github.com/explodinggradients/ragas/pull/1738
• feat: genetic algo based optimizer by @shahules786 in https://github.com/explodinggradients/ragas/pull/1724
• Added native support for Gemini models generation completion signals in LangchainLLMWrapper class / is_finished method by @cnemri in https://github.com/explodinggradients/ragas/pull/1727
• Added custom is_finished_parser logic to Google Vertex AI customizati… by @cnemri in https://github.com/explodinggradients/ragas/pull/1728
• feat: added basics for upload by @jjmachan in https://github.com/explodinggradients/ragas/pull/1741
• feat: few shot example optimzier by @jjmachan in https://github.com/explodinggradients/ragas/pull/1739
• fix: correct url for upload() by @jjmachan in https://github.com/explodinggradients/ragas/pull/1742
• fix: prompt naming related issues by @shahules786 in https://github.com/explodinggradients/ragas/pull/1743
• feat: load from annotated testset by @jjmachan in https://github.com/explodinggradients/ragas/pull/1747
• docs: add demo example for metric training by @shahules786 in https://github.com/explodinggradients/ragas/pull/1746
• docs: demo related fixes by @shahules786 in https://github.com/explodinggradients/ragas/pull/1750

New Contributors

• @avanco made their first contribution in https://github.com/explodinggradients/ragas/pull/1738
• @cnemri made their first contribution in https://github.com/explodinggradients/ragas/pull/1727

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.7...v0.2.8

What's Changed

• feat: removed critics and rubrics from examples by @jjmachan in https://github.com/explodinggradients/ragas/pull/1691
• feat: test-generation improvements by @shahules786 in https://github.com/explodinggradients/ragas/pull/1698
• chore: updated tracking for single scores by @jjmachan in https://github.com/explodinggradients/ragas/pull/1716
• feat: remove examples by @jjmachan in https://github.com/explodinggradients/ragas/pull/1708
• Bug in documentation rag_testset_generation.md: API changes required arguments by @bmerkle in https://github.com/explodinggradients/ragas/pull/1714
• feat: add output type to metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1722
• feat: dataloader for annotated json by @shahules786 in https://github.com/explodinggradients/ragas/pull/1723
• feat: train your own evaluators by @shahules786 in https://github.com/explodinggradients/ragas/pull/1701
• fix: attempting a simple fix for #1718 by @ayulockin in https://github.com/explodinggradients/ragas/pull/1721
• fix: add missing query qualifier by @shahules786 in https://github.com/explodinggradients/ragas/pull/1733
• feat: Test generation for non-english corpus by @shahules786 in https://github.com/explodinggradients/ragas/pull/1734

New Contributors

• @bmerkle made their first contribution in https://github.com/explodinggradients/ragas/pull/1714
• @ayulockin made their first contribution in https://github.com/explodinggradients/ragas/pull/1721

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.6...v0.2.7

What's Changed

• Avoid unnecessary decompose_claims call when mode='precision' in _factual_correctness.py by @dfhssilva in https://github.com/explodinggradients/ragas/pull/1669
• fixes: instruction format by @shahules786 in https://github.com/explodinggradients/ragas/pull/1672
• docs: updated readme by @jjmachan in https://github.com/explodinggradients/ragas/pull/1674
• docs: update readme by @jjmachan in https://github.com/explodinggradients/ragas/pull/1675
• fixes: add extraction limit to Extractors by @shahules786 in https://github.com/explodinggradients/ragas/pull/1673
• fix: upload for testset by @jjmachan in https://github.com/explodinggradients/ragas/pull/1676
• removed MetricWithLLM from SemanticSimilarity by @Luka958Pixion in https://github.com/explodinggradients/ragas/pull/1678
• feat: improvements in default test generation by @shahules786 in https://github.com/explodinggradients/ragas/pull/1661
• Update rag.md by @TimWue in https://github.com/explodinggradients/ragas/pull/1686
• fix: missing embeddings argument in testset and some E2E tests by @jjmachan in https://github.com/explodinggradients/ragas/pull/1690
• fixes: handle long context extraction by @shahules786 in https://github.com/explodinggradients/ragas/pull/1680
• feat: make general purpose metrics more general by @jjmachan in https://github.com/explodinggradients/ragas/pull/1666
• Fixes ToolCallAccuracy raises ZeroDivisionError when called without any arguments. by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1685

New Contributors

• @dfhssilva made their first contribution in https://github.com/explodinggradients/ragas/pull/1669
• @Luka958Pixion made their first contribution in https://github.com/explodinggradients/ragas/pull/1678
• @TimWue made their first contribution in https://github.com/explodinggradients/ragas/pull/1686

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.5...v0.2.6

What's Changed

• feat: Unify the use of sentence_segmenter by @jasinliu in https://github.com/explodinggradients/ragas/pull/1629
• feat: aspect critic with reference by @shahules786 in https://github.com/explodinggradients/ragas/pull/1638
• docs: change setting llm by @shahules786 in https://github.com/explodinggradients/ragas/pull/1639
• fixed wrong variable names in starter guide using azure openai by @MichaelPerger in https://github.com/explodinggradients/ragas/pull/1641
• feat: improvements in test gen by @shahules786 in https://github.com/explodinggradients/ragas/pull/1645
• Add tutorial for evaluating LangGraph agents by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1636
• docs: add cost tracker back by @shahules786 in https://github.com/explodinggradients/ragas/pull/1653
• docs: tracing and debugging metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1652
• feat: upload testset generation upload by @jjmachan in https://github.com/explodinggradients/ragas/pull/1647
• docs: reorganise files by @shahules786 in https://github.com/explodinggradients/ragas/pull/1662
• docs: fix _arize.md by @suekou in https://github.com/explodinggradients/ragas/pull/1643
• docs: persona generator by @jjmachan in https://github.com/explodinggradients/ragas/pull/1664
• Bug fixing recall in _factual_correctness.py by @GitMarco27 in https://github.com/explodinggradients/ragas/pull/1650

New Contributors

• @MichaelPerger made their first contribution in https://github.com/explodinggradients/ragas/pull/1641
• @GitMarco27 made their first contribution in https://github.com/explodinggradients/ragas/pull/1650

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.4...v0.2.5

What's Changed

• Fix : Filename in the path by @satodayo in https://github.com/explodinggradients/ragas/pull/1605
• fix: remove abstract class in Metric by @jjmachan in https://github.com/explodinggradients/ragas/pull/1603
• Added pandas as a method for reading datasets by @joaorura in https://github.com/explodinggradients/ragas/pull/1595
• fix: treat anthropic-direct API the same as bedrock in llamaindex wrapper by @ahgraber in https://github.com/explodinggradients/ragas/pull/1584
• feat: add support for llamaindex by @jjmachan in https://github.com/explodinggradients/ragas/pull/1609
• docs: Add embedding_model attribute when creating TestsetGenerator object by @Youngrok123 in https://github.com/explodinggradients/ragas/pull/1606
• Fix: Add instance check before casting ChatGeneration object by @Youngrok123 in https://github.com/explodinggradients/ragas/pull/1607
• docs: add comment for claude-3-5-sonnet-20241022 by @SokiKawashima in https://github.com/explodinggradients/ragas/pull/1611
• docs: add community docs by @Yunnglin in https://github.com/explodinggradients/ragas/pull/1612
• Handle embeddings for empty strings in AnswerSimilarity class by @baptiste-pasquier in https://github.com/explodinggradients/ragas/pull/994
• Feature: batched execution by @ahgraber in https://github.com/explodinggradients/ragas/pull/1589
• docs: how to add custom metrics by @jjmachan in https://github.com/explodinggradients/ragas/pull/1616
• docs: added azure openai to default docs by @jjmachan in https://github.com/explodinggradients/ragas/pull/1610
• Introduce epsilon in denominators to avoid division by zero by @jltham in https://github.com/explodinggradients/ragas/pull/1622
• feat: Add support for llamaIndex in evaluation by @suekou in https://github.com/explodinggradients/ragas/pull/1619
• feat: added upload for EvaluationResult by @jjmachan in https://github.com/explodinggradients/ragas/pull/1625
• feat: automatic persona generation by @shahules786 in https://github.com/explodinggradients/ragas/pull/1618
• docs: Refactor Code for Syntax Highlighting and URL Updates by @suekou in https://github.com/explodinggradients/ragas/pull/1634
• fix: remove . as a split condition in noise_sensitivity.py by @jasinliu in https://github.com/explodinggradients/ragas/pull/1627
• feat: improvements in test synthesization by @shahules786 in https://github.com/explodinggradients/ragas/pull/1621

New Contributors

• @satodayo made their first contribution in https://github.com/explodinggradients/ragas/pull/1605
• @joaorura made their first contribution in https://github.com/explodinggradients/ragas/pull/1595
• @SokiKawashima made their first contribution in https://github.com/explodinggradients/ragas/pull/1611
• @jltham made their first contribution in https://github.com/explodinggradients/ragas/pull/1622
• @jasinliu made their first contribution in https://github.com/explodinggradients/ragas/pull/1627

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.3...v0.2.4

What's Changed

• feat: measure cost of testset generator by @jjmachan in https://github.com/explodinggradients/ragas/pull/1560
• docs: added luka's blogs by @jjmachan in https://github.com/explodinggradients/ragas/pull/1554
• Fix: add LLMContextPrecisionWithReference to init.py by @licux in https://github.com/explodinggradients/ragas/pull/1561
• chore: add verbose option (V=1) to make targets by @trevorbowen in https://github.com/explodinggradients/ragas/pull/1556
• add embeddings to TestsetGenerator by @hunter-walden2113 in https://github.com/explodinggradients/ragas/pull/1562
• fixed verb agreement @ available_metrics by @gabrielhomsi in https://github.com/explodinggradients/ragas/pull/1574
• Fix: Limit number of retries for parse failures by @rskew in https://github.com/explodinggradients/ragas/pull/1569
• Fix: reference key error in LLMContextPrecisionWithoutReference by @xizhou-vw in https://github.com/explodinggradients/ragas/pull/1570
• Updated return type description for evaluate function by @taihim in https://github.com/explodinggradients/ragas/pull/1579
• Feat: add multimodal eval support by @Yunnglin in https://github.com/explodinggradients/ragas/pull/1559
• fix: add reference_topics as default required columns in TopicAdherenceScore #1564 by @luqmansen in https://github.com/explodinggradients/ragas/pull/1566
• fix: add reference tool call to required cols by @shahules786 in https://github.com/explodinggradients/ragas/pull/1580
• Improve efficiency in factual correctness for precision mode by @Jeff-67 in https://github.com/explodinggradients/ragas/pull/1578
• This commit implements the F-beta score metric by @Yuri-Albuquerque in https://github.com/explodinggradients/ragas/pull/1543
• fix: agent goal accuracy by @shahules786 in https://github.com/explodinggradients/ragas/pull/1583
• chores: fix pypi rendering by @shahules786 in https://github.com/explodinggradients/ragas/pull/1581
• fix: typo: ROUGE is a metric, ROGUE is a scoundrel by @ahgraber in https://github.com/explodinggradients/ragas/pull/1585

New Contributors

• @trevorbowen made their first contribution in https://github.com/explodinggradients/ragas/pull/1556
• @hunter-walden2113 made their first contribution in https://github.com/explodinggradients/ragas/pull/1562
• @gabrielhomsi made their first contribution in https://github.com/explodinggradients/ragas/pull/1574
• @rskew made their first contribution in https://github.com/explodinggradients/ragas/pull/1569
• @xizhou-vw made their first contribution in https://github.com/explodinggradients/ragas/pull/1570
• @taihim made their first contribution in https://github.com/explodinggradients/ragas/pull/1579
• @luqmansen made their first contribution in https://github.com/explodinggradients/ragas/pull/1566
• @Yuri-Albuquerque made their first contribution in https://github.com/explodinggradients/ragas/pull/1543

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.2...v0.2.3

What's Changed

• docs: update summarization calculation by @jCobbSK in https://github.com/explodinggradients/ragas/pull/1510
• fix: use get_temperature in base LLM generate by @sapphire-arches in https://github.com/explodinggradients/ragas/pull/1520
• doc: Update modifying prompts metrics doc by @FrankyCTY in https://github.com/explodinggradients/ragas/pull/1525
• fixbug: Expand sentence-ending punctuation check in _create_statements method by @Jeff-67 in https://github.com/explodinggradients/ragas/pull/1523
• fix: zero division error by @shahules786 in https://github.com/explodinggradients/ragas/pull/1530
• feat: add traces to EvaluationResult by @jjmachan in https://github.com/explodinggradients/ragas/pull/1531
• feat: translate instruction when adapting prompt by @Yunnglin in https://github.com/explodinggradients/ragas/pull/1529
• chore: remove old prompt object by @jjmachan in https://github.com/explodinggradients/ragas/pull/1535
• feat: improved the testset generation to_pandas and docs by @jjmachan in https://github.com/explodinggradients/ragas/pull/1536
• fixed typo in "single_turn_ascore" by @minorun365 in https://github.com/explodinggradients/ragas/pull/1542
• update AspectCritic docs examples by @licux in https://github.com/explodinggradients/ragas/pull/1547
• Fix: Resolve Agentic failure loop with improved json parsing by @ahgraber in https://github.com/explodinggradients/ragas/pull/1541
• feat: throw error when max_token limit is reached by @jjmachan in https://github.com/explodinggradients/ragas/pull/1549
• docs: fix Amazon bedrock info by @jjmachan in https://github.com/explodinggradients/ragas/pull/1551
• Added community blog posts in Japanese by @minorun365 in https://github.com/explodinggradients/ragas/pull/1550
• docs: make llms and embeddings explicit by @jjmachan in https://github.com/explodinggradients/ragas/pull/1553

New Contributors

• @jCobbSK made their first contribution in https://github.com/explodinggradients/ragas/pull/1510
• @sapphire-arches made their first contribution in https://github.com/explodinggradients/ragas/pull/1520
• @FrankyCTY made their first contribution in https://github.com/explodinggradients/ragas/pull/1525
• @Jeff-67 made their first contribution in https://github.com/explodinggradients/ragas/pull/1523
• @Yunnglin made their first contribution in https://github.com/explodinggradients/ragas/pull/1529
• @licux made their first contribution in https://github.com/explodinggradients/ragas/pull/1547
• @ahgraber made their first contribution in https://github.com/explodinggradients/ragas/pull/1541

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.1...v0.2.2

What's Changed

• Fix Discord link by @NirantK in https://github.com/explodinggradients/ragas/pull/1498
• Fix Quickstart link by @NirantK in https://github.com/explodinggradients/ragas/pull/1499
• Rewrite Open Analytics by @NirantK in https://github.com/explodinggradients/ragas/pull/1500
• Suggestions for Introduction and Key Features by @NirantK in https://github.com/explodinggradients/ragas/pull/1501
• docs: fix broken links in testset generation custimisation by @jjmachan in https://github.com/explodinggradients/ragas/pull/1502
• Update Bedrock modules for LangChain v0.3.x by @minorun365 in https://github.com/explodinggradients/ragas/pull/1487
• Fixes: Quickstart Ragas Metrics by @NirantK in https://github.com/explodinggradients/ragas/pull/1504
• Add Contributors by @NirantK in https://github.com/explodinggradients/ragas/pull/1505
• docs: Change test_size parameter name to testset_size by @Youngrok123 in https://github.com/explodinggradients/ragas/pull/1507
• docs: readme add llm by @shahules786 in https://github.com/explodinggradients/ragas/pull/1511
• docs: add faq by @shahules786 in https://github.com/explodinggradients/ragas/pull/1503
• fix: propagate callbacks in RagasOutputParser by @dlyz in https://github.com/explodinggradients/ragas/pull/1513
• docs: fix usage of eval dataset by @shahules786 in https://github.com/explodinggradients/ragas/pull/1514
• feat: use ensure_ascii=False for KnowledgeGraph files by @dlyz in https://github.com/explodinggradients/ragas/pull/1518
• fix: import error for TestsetGeneration and small fixes by @jjmachan in https://github.com/explodinggradients/ragas/pull/1516

New Contributors

• @NirantK made their first contribution in https://github.com/explodinggradients/ragas/pull/1498
• @minorun365 made their first contribution in https://github.com/explodinggradients/ragas/pull/1487
• @dlyz made their first contribution in https://github.com/explodinggradients/ragas/pull/1513

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.2.0...v0.2.1

What's Changed

• feat: topic adherence metric by @shahules786 in https://github.com/explodinggradients/ragas/pull/1327
• feat: tool call accuracy by @shahules786 in https://github.com/explodinggradients/ragas/pull/1311
• feat: sql sementic equivalence by @shahules786 in https://github.com/explodinggradients/ragas/pull/1323
• feat: Factual correctnes metric by @shahules786 in https://github.com/explodinggradients/ragas/pull/1334
• docs: migrating over to mkdocs by @jjmachan in https://github.com/explodinggradients/ragas/pull/1301
• feat: Adding transforms for testset generation by @jjmachan in https://github.com/explodinggradients/ragas/pull/1344
• docs: traditional metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1346
• feat: transforms for Knowledge Graphs by @jjmachan in https://github.com/explodinggradients/ragas/pull/1345
• docs: core concepts in metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1347
• chores: cleanup metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1348
• docs: refactor core concepts by @shahules786 in https://github.com/explodinggradients/ragas/pull/1351
• wrap as_complete with async by @liupgd in https://github.com/explodinggradients/ragas/pull/1350
• chores: add datasets to init by @shahules786 in https://github.com/explodinggradients/ragas/pull/1356
• docs: new getting started by @shahules786 in https://github.com/explodinggradients/ragas/pull/1355
• docs: metrics concepts by @shahules786 in https://github.com/explodinggradients/ragas/pull/1361
• docs: add mindmap by @shahules786 in https://github.com/explodinggradients/ragas/pull/1362
• Fixed Nav, search, and some styling issues for the docs by @joelk9895 in https://github.com/explodinggradients/ragas/pull/1363
• feat: transform engine to execute transforms by @jjmachan in https://github.com/explodinggradients/ragas/pull/1352
• chores: remove fuzzywuzzy by @shahules786 in https://github.com/explodinggradients/ragas/pull/1369
• docs: component guides by @shahules786 in https://github.com/explodinggradients/ragas/pull/1365
• chores: fix typo by @shahules786 in https://github.com/explodinggradients/ragas/pull/1370
• fix:ensure correct sequence in tool calls by @shahules786 in https://github.com/explodinggradients/ragas/pull/1371
• [howtos] update Langfuse cookbook by @jannikmaierhoefer in https://github.com/explodinggradients/ragas/pull/1368
• feat: adding an implementation of abstractQA by @jjmachan in https://github.com/explodinggradients/ragas/pull/1359
• feat: implement TestsetGenerator by @jjmachan in https://github.com/explodinggradients/ragas/pull/1375
• docs: testset generation by @shahules786 in https://github.com/explodinggradients/ragas/pull/1373
• feat: added more simulators by @jjmachan in https://github.com/explodinggradients/ragas/pull/1376
• Update models in Ragas documentation: change gpt-3.5-turbo-16k to gpt-4o-mini and gpt-4 to gpt-4o by @suekou in https://github.com/explodinggradients/ragas/pull/1380
• fix: aspect_critique documentation by @jjmachan in https://github.com/explodinggradients/ragas/pull/1382
• docs: added google analytics to mkdocs by @jjmachan in https://github.com/explodinggradients/ragas/pull/1384
• docs: fix index for metric/index.md by @jjmachan in https://github.com/explodinggradients/ragas/pull/1388
• fix: fixed verdict comparison in evolutions.py by @MarkHmnv in https://github.com/explodinggradients/ragas/pull/1366
• feat: output new Testset object for generation by @jjmachan in https://github.com/explodinggradients/ragas/pull/1386
• Update answer_relevance.md: Minor spell correct by @soumendrak in https://github.com/explodinggradients/ragas/pull/1390
• feat: support for callbacks and traces in testset generation by @jjmachan in https://github.com/explodinggradients/ragas/pull/1392
• non-asii characters support in faithfulness by @gargewp in https://github.com/explodinggradients/ragas/pull/1396
• feat: get and set prompts mixin by @shahules786 in https://github.com/explodinggradients/ragas/pull/1391
• fixes: rename naming in testset generation by @shahules786 in https://github.com/explodinggradients/ragas/pull/1393
• chore!: moving to pydantic2 by @jjmachan in https://github.com/explodinggradients/ragas/pull/1394
• docs: improve getting started by @shahules786 in https://github.com/explodinggradients/ragas/pull/1398
• Fix typo in metric error message by @suekou in https://github.com/explodinggradients/ragas/pull/1404
• feat!: moving experimental.testset to replace old testset by @jjmachan in https://github.com/explodinggradients/ragas/pull/1401
• feat: renamed sample operations and cleaned up test warnings by @jjmachan in https://github.com/explodinggradients/ragas/pull/1407
• added better styling to reder jupyter notebooks nicely. by @zodwick in https://github.com/explodinggradients/ragas/pull/1389
• docs: add core concepts for test generation by @shahules786 in https://github.com/explodinggradients/ragas/pull/1402
• docs:Auto Gen of Reference Page by @zodwick in https://github.com/explodinggradients/ragas/pull/1408
• fix key error for FaithfulnesswithHHEM by @linzzzzzz in https://github.com/explodinggradients/ragas/pull/1410
• fix: remove usage of contexts in response relevancy by @shahules786 in https://github.com/explodinggradients/ragas/pull/1409
• fix: score for jupyter by @shahules786 in https://github.com/explodinggradients/ragas/pull/1411
• feat: NER extractor and Jaccard by @shahules786 in https://github.com/explodinggradients/ragas/pull/1406
• feat(styles): refactor CSS to use custom variables by @zodwick in https://github.com/explodinggradients/ragas/pull/1413
• docs : add customisation by @shahules786 in https://github.com/explodinggradients/ragas/pull/1412
• feat(docs): add support for reusable snippets by @zodwick in https://github.com/explodinggradients/ragas/pull/1414
• Feature/css variables by @zodwick in https://github.com/explodinggradients/ragas/pull/1415
• docs: fix nav bar by @shahules786 in https://github.com/explodinggradients/ragas/pull/1416
• feat: more features to prompt object by @jjmachan in https://github.com/explodinggradients/ragas/pull/1418
• chore: add tracking for new testset generation by @jjmachan in https://github.com/explodinggradients/ragas/pull/1419
• feat: adapt prompt to different languages by @jjmachan in https://github.com/explodinggradients/ragas/pull/1429
• chores: add reference to gitignore by @shahules786 in https://github.com/explodinggradients/ragas/pull/1428
• docs: redo intro by @shahules786 in https://github.com/explodinggradients/ragas/pull/1427
• Add Custom Ragas Light and Dark Themes (v1) by @zodwick in https://github.com/explodinggradients/ragas/pull/1431
• fix attribute error for faithfulness metric by @linzzzzzz in https://github.com/explodinggradients/ragas/pull/1436
• docs: language adaptation by @shahules786 in https://github.com/explodinggradients/ragas/pull/1446
• fix: data export by @shahules786 in https://github.com/explodinggradients/ragas/pull/1461
• feat: save and load Prompts by @jjmachan in https://github.com/explodinggradients/ragas/pull/1458
• refactor: convert faithfulness, answer_correctness and noise_sensitivity to new prompt object by @jjmachan in https://github.com/explodinggradients/ragas/pull/1462
• Minor typo in _answer_relevance.py by @pjanowski in https://github.com/explodinggradients/ragas/pull/1452
• feat: added repr for prompt objects by @jjmachan in https://github.com/explodinggradients/ragas/pull/1459
• feat: change context recall and precision prompts to new by @jjmachan in https://github.com/explodinggradients/ragas/pull/1467
• docs: fix invalid links and imports by @shahules786 in https://github.com/explodinggradients/ragas/pull/1473
• feat: changed summary to new prompt by @jjmachan in https://github.com/explodinggradients/ragas/pull/1469
• docs: redo readme by @shahules786 in https://github.com/explodinggradients/ragas/pull/1480
• docs: new build process with rtd and API documentation by @jjmachan in https://github.com/explodinggradients/ragas/pull/1474
• chore: added install for mkdocs insiders by @jjmachan in https://github.com/explodinggradients/ragas/pull/1481
• chores: update gitignore by @shahules786 in https://github.com/explodinggradients/ragas/pull/1483
• docs: improve testgen namings by @shahules786 in https://github.com/explodinggradients/ragas/pull/1489
• Fixed an issue where non-ASCII characters were changed to Unicode characters within the prompt by @Youngrok123 in https://github.com/explodinggradients/ragas/pull/1490
• docs: fix search by @shahules786 in https://github.com/explodinggradients/ragas/pull/1491
• feat: added save and load to RagasDataset by @jjmachan in https://github.com/explodinggradients/ragas/pull/1492
• docs: moved from ipnb files to md files by @jjmachan in https://github.com/explodinggradients/ragas/pull/1482
• feat: set ensure_ascii to false by @jjmachan in https://github.com/explodinggradients/ragas/pull/1494
• feat: small refactors and cleanups by @jjmachan in https://github.com/explodinggradients/ragas/pull/1493

New Contributors

• @liupgd made their first contribution in https://github.com/explodinggradients/ragas/pull/1350
• @joelk9895 made their first contribution in https://github.com/explodinggradients/ragas/pull/1363
• @jannikmaierhoefer made their first contribution in https://github.com/explodinggradients/ragas/pull/1368
• @suekou made their first contribution in https://github.com/explodinggradients/ragas/pull/1380
• @MarkHmnv made their first contribution in https://github.com/explodinggradients/ragas/pull/1366
• @soumendrak made their first contribution in https://github.com/explodinggradients/ragas/pull/1390
• @gargewp made their first contribution in https://github.com/explodinggradients/ragas/pull/1396
• @zodwick made their first contribution in https://github.com/explodinggradients/ragas/pull/1389
• @linzzzzzz made their first contribution in https://github.com/explodinggradients/ragas/pull/1410
• @pjanowski made their first contribution in https://github.com/explodinggradients/ragas/pull/1452
• @Youngrok123 made their first contribution in https://github.com/explodinggradients/ragas/pull/1490

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.20...v0.2.0

What's Changed

• feat: topic adherence metric by @shahules786 in https://github.com/explodinggradients/ragas/pull/1327
• feat: tool call accuracy by @shahules786 in https://github.com/explodinggradients/ragas/pull/1311
• feat: sql sementic equivalence by @shahules786 in https://github.com/explodinggradients/ragas/pull/1323
• feat: Factual correctnes metric by @shahules786 in https://github.com/explodinggradients/ragas/pull/1334
• docs: migrating over to mkdocs by @jjmachan in https://github.com/explodinggradients/ragas/pull/1301
• feat: Adding transforms for testset generation by @jjmachan in https://github.com/explodinggradients/ragas/pull/1344
• docs: traditional metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1346
• feat: transforms for Knowledge Graphs by @jjmachan in https://github.com/explodinggradients/ragas/pull/1345
• docs: core concepts in metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1347
• chores: cleanup metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1348
• docs: refactor core concepts by @shahules786 in https://github.com/explodinggradients/ragas/pull/1351
• wrap as_complete with async by @liupgd in https://github.com/explodinggradients/ragas/pull/1350
• chores: add datasets to init by @shahules786 in https://github.com/explodinggradients/ragas/pull/1356
• docs: new getting started by @shahules786 in https://github.com/explodinggradients/ragas/pull/1355
• docs: metrics concepts by @shahules786 in https://github.com/explodinggradients/ragas/pull/1361
• docs: add mindmap by @shahules786 in https://github.com/explodinggradients/ragas/pull/1362
• Fixed Nav, search, and some styling issues for the docs by @joelk9895 in https://github.com/explodinggradients/ragas/pull/1363
• feat: transform engine to execute transforms by @jjmachan in https://github.com/explodinggradients/ragas/pull/1352
• chores: remove fuzzywuzzy by @shahules786 in https://github.com/explodinggradients/ragas/pull/1369
• docs: component guides by @shahules786 in https://github.com/explodinggradients/ragas/pull/1365
• chores: fix typo by @shahules786 in https://github.com/explodinggradients/ragas/pull/1370
• fix:ensure correct sequence in tool calls by @shahules786 in https://github.com/explodinggradients/ragas/pull/1371
• [howtos] update Langfuse cookbook by @jannikmaierhoefer in https://github.com/explodinggradients/ragas/pull/1368
• feat: adding an implementation of abstractQA by @jjmachan in https://github.com/explodinggradients/ragas/pull/1359
• feat: implement TestsetGenerator by @jjmachan in https://github.com/explodinggradients/ragas/pull/1375
• docs: testset generation by @shahules786 in https://github.com/explodinggradients/ragas/pull/1373
• feat: added more simulators by @jjmachan in https://github.com/explodinggradients/ragas/pull/1376
• Update models in Ragas documentation: change gpt-3.5-turbo-16k to gpt-4o-mini and gpt-4 to gpt-4o by @suekou in https://github.com/explodinggradients/ragas/pull/1380
• fix: aspect_critique documentation by @jjmachan in https://github.com/explodinggradients/ragas/pull/1382
• docs: added google analytics to mkdocs by @jjmachan in https://github.com/explodinggradients/ragas/pull/1384
• docs: fix index for metric/index.md by @jjmachan in https://github.com/explodinggradients/ragas/pull/1388
• fix: fixed verdict comparison in evolutions.py by @MarkHmnv in https://github.com/explodinggradients/ragas/pull/1366
• feat: output new Testset object for generation by @jjmachan in https://github.com/explodinggradients/ragas/pull/1386
• Update answer_relevance.md: Minor spell correct by @soumendrak in https://github.com/explodinggradients/ragas/pull/1390
• feat: support for callbacks and traces in testset generation by @jjmachan in https://github.com/explodinggradients/ragas/pull/1392
• non-asii characters support in faithfulness by @gargewp in https://github.com/explodinggradients/ragas/pull/1396
• feat: get and set prompts mixin by @shahules786 in https://github.com/explodinggradients/ragas/pull/1391
• fixes: rename naming in testset generation by @shahules786 in https://github.com/explodinggradients/ragas/pull/1393
• chore!: moving to pydantic2 by @jjmachan in https://github.com/explodinggradients/ragas/pull/1394
• docs: improve getting started by @shahules786 in https://github.com/explodinggradients/ragas/pull/1398
• Fix typo in metric error message by @suekou in https://github.com/explodinggradients/ragas/pull/1404
• feat!: moving experimental.testset to replace old testset by @jjmachan in https://github.com/explodinggradients/ragas/pull/1401
• feat: renamed sample operations and cleaned up test warnings by @jjmachan in https://github.com/explodinggradients/ragas/pull/1407
• added better styling to reder jupyter notebooks nicely. by @zodwick in https://github.com/explodinggradients/ragas/pull/1389
• docs: add core concepts for test generation by @shahules786 in https://github.com/explodinggradients/ragas/pull/1402
• docs:Auto Gen of Reference Page by @zodwick in https://github.com/explodinggradients/ragas/pull/1408
• fix key error for FaithfulnesswithHHEM by @linzzzzzz in https://github.com/explodinggradients/ragas/pull/1410
• fix: remove usage of contexts in response relevancy by @shahules786 in https://github.com/explodinggradients/ragas/pull/1409
• fix: score for jupyter by @shahules786 in https://github.com/explodinggradients/ragas/pull/1411
• feat: NER extractor and Jaccard by @shahules786 in https://github.com/explodinggradients/ragas/pull/1406
• feat(styles): refactor CSS to use custom variables by @zodwick in https://github.com/explodinggradients/ragas/pull/1413
• docs : add customisation by @shahules786 in https://github.com/explodinggradients/ragas/pull/1412
• feat(docs): add support for reusable snippets by @zodwick in https://github.com/explodinggradients/ragas/pull/1414
• Feature/css variables by @zodwick in https://github.com/explodinggradients/ragas/pull/1415
• docs: fix nav bar by @shahules786 in https://github.com/explodinggradients/ragas/pull/1416
• feat: more features to prompt object by @jjmachan in https://github.com/explodinggradients/ragas/pull/1418
• chore: add tracking for new testset generation by @jjmachan in https://github.com/explodinggradients/ragas/pull/1419

New Contributors

• @liupgd made their first contribution in https://github.com/explodinggradients/ragas/pull/1350
• @joelk9895 made their first contribution in https://github.com/explodinggradients/ragas/pull/1363
• @jannikmaierhoefer made their first contribution in https://github.com/explodinggradients/ragas/pull/1368
• @suekou made their first contribution in https://github.com/explodinggradients/ragas/pull/1380
• @MarkHmnv made their first contribution in https://github.com/explodinggradients/ragas/pull/1366
• @soumendrak made their first contribution in https://github.com/explodinggradients/ragas/pull/1390
• @gargewp made their first contribution in https://github.com/explodinggradients/ragas/pull/1396
• @zodwick made their first contribution in https://github.com/explodinggradients/ragas/pull/1389
• @linzzzzzz made their first contribution in https://github.com/explodinggradients/ragas/pull/1410

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.20...v0.2.0-beta

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.16...v0.1.21

• Pins to langchain<0.3
• based on v0.1.16 which was the last working version for 0.1
• hopefully last release before v0.2

What's Changed

• fix: pin langchain_core to <3 by @jjmachan in https://github.com/explodinggradients/ragas/pull/1329

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.19...v0.1.20

What's Changed

• feat: add non llm based context recall by @shahules786 in https://github.com/explodinggradients/ragas/pull/1266
• feat: Non LLM based context precision by @shahules786 in https://github.com/explodinggradients/ragas/pull/1264
• feat: added n and temperature passthrough to prompt by @jjmachan in https://github.com/explodinggradients/ragas/pull/1281
• Add option to disable progress bar in evaluate by @freddiev4 in https://github.com/explodinggradients/ragas/pull/1279
• feat: new aspect critic metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1286
• feat: add simple scorer metric by @shahules786 in https://github.com/explodinggradients/ragas/pull/1291
• docs: add docs for new metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1292
• Fix wrong input key name by @ruanhao in https://github.com/explodinggradients/ragas/pull/1300
• chore: move rubrics based metric into single file by @shahules786 in https://github.com/explodinggradients/ragas/pull/1287
• feat: Instance specifc rubrics metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1304
• feat: agent goal accuracy metric by @shahules786 in https://github.com/explodinggradients/ragas/pull/1303
• docs: agent metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1314
• Update Opik notebook example and integration by @jverre in https://github.com/explodinggradients/ragas/pull/1295
• fix: column rename by @shahules786 in https://github.com/explodinggradients/ragas/pull/1324

New Contributors

• @freddiev4 made their first contribution in https://github.com/explodinggradients/ragas/pull/1279
• @ruanhao made their first contribution in https://github.com/explodinggradients/ragas/pull/1300

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.18...v0.1.19

What's Changed

• fix: inverse distance by @shahules786 in https://github.com/explodinggradients/ragas/pull/1267
• fix: non llm based metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1268
• fix: v1 to v2 dataset by @shahules786 in https://github.com/explodinggradients/ragas/pull/1275
• fix: make score nested if loop_is_running by @jjmachan in https://github.com/explodinggradients/ragas/pull/1276

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.17...v0.1.18

What's Changed

• update documentation for faithfulness with HHEM by @Miaoranmmm in https://github.com/explodinggradients/ragas/pull/1242
• New data representation by @shahules786 in https://github.com/explodinggradients/ragas/pull/1222
• Feat: Opik Integration by @jverre in https://github.com/explodinggradients/ragas/pull/1256
• Add support to MultiTurn Metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1249
• fix: add single_turn_ascore by @shahules786 in https://github.com/explodinggradients/ragas/pull/1261
• fix: handle v0.1 dataset by @shahules786 in https://github.com/explodinggradients/ragas/pull/1262
• Non LLM based metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1260
• feat: set and get prompts for metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1259

New Contributors

• @jverre made their first contribution in https://github.com/explodinggradients/ragas/pull/1256

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.16...v0.1.17

What's Changed

• load HHEM on specified device by @Miaoranmmm in https://github.com/explodinggradients/ragas/pull/1235
• feat: new Prompt object and faithfulness metric by @jjmachan in https://github.com/explodinggradients/ragas/pull/1232
• feat (experimental) added new prompt and metric into ragas.experimental by @jjmachan in https://github.com/explodinggradients/ragas/pull/1240

New Contributors

• @Miaoranmmm made their first contribution in https://github.com/explodinggradients/ragas/pull/1235

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.15...v0.1.16

What's Changed

• fix: typo in model name by @shahules786 in https://github.com/explodinggradients/ragas/pull/1198
• metrics: add domain specific rubrics based scoring by @vaishakhRaveendran in https://github.com/explodinggradients/ragas/pull/1189
• docs: rubrics based metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1200
• docs: add context utilisation to the index by @shahules786 in https://github.com/explodinggradients/ragas/pull/1202
• Docs: fix typo in metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/1203
• Implementation of Noise sensitivity metrics from RAGChecker by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1190
• Tiny typo fix: it's -> its by @peterkaminski in https://github.com/explodinggradients/ragas/pull/1219
• fix : solves #1215 - Inconsistency in the Value of EVALMODE_TO_COLUMNS[EvaluationMode.ca] by @jucgonzalezes in https://github.com/explodinggradients/ragas/pull/1216
• chore: removed extra of requirements by @jjmachan in https://github.com/explodinggradients/ragas/pull/1225
• Bugfix of evaluate function in llama_index.py by @vcim in https://github.com/explodinggradients/ragas/pull/1223
• [FIX] - Fix for summarization edge case by @sky-2002 in https://github.com/explodinggradients/ragas/pull/1201

New Contributors

• @peterkaminski made their first contribution in https://github.com/explodinggradients/ragas/pull/1219
• @jucgonzalezes made their first contribution in https://github.com/explodinggradients/ragas/pull/1216
• @vcim made their first contribution in https://github.com/explodinggradients/ragas/pull/1223

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.14...v0.1.15

What's Changed

• Add question to context recall doc by @shahules786 in https://github.com/explodinggradients/ragas/pull/1173
• Add context utilization by @shahules786 in https://github.com/explodinggradients/ragas/pull/1174
• doc: fix output for testset generation and missing files by @jjmachan in https://github.com/explodinggradients/ragas/pull/1176
• Helicone Integration by @chitalian in https://github.com/explodinggradients/ragas/pull/1178
• docs: minor fixes by @shahules786 in https://github.com/explodinggradients/ragas/pull/1183
• solved issue that adaptation crashes when havig multiple nested promp… by @ssoima in https://github.com/explodinggradients/ragas/pull/1181
• Bugfix Summarization Score by @emreds in https://github.com/explodinggradients/ragas/pull/1164
• [DOCS] - Fix summarization score expression by @sky-2002 in https://github.com/explodinggradients/ragas/pull/1184
• metrics: add faithfulness with HHEM model by @shahules786 in https://github.com/explodinggradients/ragas/pull/1191
• Fix test command in development.md to match Makefile configuration by @sahusiddharth in https://github.com/explodinggradients/ragas/pull/1195
• feat: deeper helicon integration by @jjmachan in https://github.com/explodinggradients/ragas/pull/1196
• fix: added a test for prompts by @jjmachan in https://github.com/explodinggradients/ragas/pull/1197

New Contributors

• @chitalian made their first contribution in https://github.com/explodinggradients/ragas/pull/1178
• @ssoima made their first contribution in https://github.com/explodinggradients/ragas/pull/1181
• @emreds made their first contribution in https://github.com/explodinggradients/ragas/pull/1164
• @sahusiddharth made their first contribution in https://github.com/explodinggradients/ragas/pull/1195

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.13...v0.1.14

What's Changed

• Fix "global flags not at the start" raised in extract_text function by @vaishakhRaveendran in https://github.com/explodinggradients/ragas/pull/1113
• Random seed fixed. by @Gwenn-LR in https://github.com/explodinggradients/ragas/pull/1142
• feat: Usage tracking for evaluations by @jjmachan in https://github.com/explodinggradients/ragas/pull/1129
• Fixing Demo Code - Compare Embeddings for retriever Docs by @matheusft in https://github.com/explodinggradients/ragas/pull/1153
• fix: run_config post_init issue by @jjmachan in https://github.com/explodinggradients/ragas/pull/1158
• Add save function for statement_prompt by @miyaketomoya in https://github.com/explodinggradients/ragas/pull/1157
• Fix: adapt function to translate cached prompts in place. by @HerrIvan in https://github.com/explodinggradients/ragas/pull/889
• fix: RunConfig documentation and improvements by @jjmachan in https://github.com/explodinggradients/ragas/pull/1159
• Docs/run config by @jjmachan in https://github.com/explodinggradients/ragas/pull/1160
• change segmenter language when language adaptaion by @miyaketomoya in https://github.com/explodinggradients/ragas/pull/1163
• change default model to gpt4o-mini by @shahules786 in https://github.com/explodinggradients/ragas/pull/1166
• Add tests to ragas.run_config.RunConfig rng attribute. by @Gwenn-LR in https://github.com/explodinggradients/ragas/pull/1169
• feat: added track_completed decorator to track completed functions by @jjmachan in https://github.com/explodinggradients/ragas/pull/1171
• docs: added development.md file by @jjmachan in https://github.com/explodinggradients/ragas/pull/1115
• Allow Metric.score to work within an existing asyncio loop by @abramowskia in https://github.com/explodinggradients/ragas/pull/1161

New Contributors

• @Gwenn-LR made their first contribution in https://github.com/explodinggradients/ragas/pull/1142
• @matheusft made their first contribution in https://github.com/explodinggradients/ragas/pull/1153
• @miyaketomoya made their first contribution in https://github.com/explodinggradients/ragas/pull/1157
• @abramowskia made their first contribution in https://github.com/explodinggradients/ragas/pull/1161

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.12...v0.1.13

What's Changed

• [Fix bug] The adaptation for rewrite_invalid_question_prompt was not saved by @psydok in https://github.com/explodinggradients/ragas/pull/1071
• [Fix bug] utils.py by @jrredondo in https://github.com/explodinggradients/ragas/pull/1083
• Llama Index integration: optional RunConfig for evaluate() by @antono in https://github.com/explodinggradients/ragas/pull/1085
• Create a custom directory loader for RAGAS by @vaishakhRaveendran in https://github.com/explodinggradients/ragas/pull/1122
• Fix "sentence" -> "claim" the context recall metric docs by @artmatsak in https://github.com/explodinggradients/ragas/pull/1126
• fix: temperature parameter in generate_text not ignored. by @HerrIvan in https://github.com/explodinggradients/ragas/pull/887

New Contributors

• @psydok made their first contribution in https://github.com/explodinggradients/ragas/pull/1071
• @jrredondo made their first contribution in https://github.com/explodinggradients/ragas/pull/1083
• @antono made their first contribution in https://github.com/explodinggradients/ragas/pull/1085
• @vaishakhRaveendran made their first contribution in https://github.com/explodinggradients/ragas/pull/1122
• @artmatsak made their first contribution in https://github.com/explodinggradients/ragas/pull/1126
• @HerrIvan made their first contribution in https://github.com/explodinggradients/ragas/pull/887

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.11...v0.1.12

What's Changed

• Testgeneration v3 by @shahules786 in https://github.com/explodinggradients/ragas/pull/1024
• fix: import errors and make experimental runnable by @jjmachan in https://github.com/explodinggradients/ragas/pull/1078
• chore: added analytics to count the num of testsets generated with experimental by @jjmachan in https://github.com/explodinggradients/ragas/pull/1081
• Fix mutable dataclass defaults by @yasyf in https://github.com/explodinggradients/ragas/pull/1082
• fix(executor): simplify executor by removing threading by @jjmachan in https://github.com/explodinggradients/ragas/pull/1093
• chore: added ragas as a dependency by @jjmachan in https://github.com/explodinggradients/ragas/pull/1095
• test: added more tests for executor by @jjmachan in https://github.com/explodinggradients/ragas/pull/1110
• chore: remove context_relevancy from library by @jjmachan in https://github.com/explodinggradients/ragas/pull/1111
• Update index.md by @IdaJess in https://github.com/explodinggradients/ragas/pull/1117
• Improve Clarity and Structure of Faithfulness Explanation in Documentation by @aybruhm in https://github.com/explodinggradients/ragas/pull/1114
• fix: remove is_async argument since everything is async now by @jjmachan in https://github.com/explodinggradients/ragas/pull/1116

New Contributors

• @yasyf made their first contribution in https://github.com/explodinggradients/ragas/pull/1082
• @IdaJess made their first contribution in https://github.com/explodinggradients/ragas/pull/1117
• @aybruhm made their first contribution in https://github.com/explodinggradients/ragas/pull/1114

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.10...v0.1.11

What's Changed

• Update testset_generation.md - fix typo by @GTimothee in https://github.com/explodinggradients/ragas/pull/1008
• bugfix: evaluation thread timeout by @NIL-zhuang in https://github.com/explodinggradients/ragas/pull/858
• Update index.md by @shahules786 in https://github.com/explodinggradients/ragas/pull/1023
• Add initial version of summarization metric by @sky-2002 in https://github.com/explodinggradients/ragas/pull/990
• feat: added experimental package in ragas by @jjmachan in https://github.com/explodinggradients/ragas/pull/1056
• Docs for summarization metric by @sky-2002 in https://github.com/explodinggradients/ragas/pull/1026
• adding get_available_metrics method by @immortal3 in https://github.com/explodinggradients/ragas/pull/1025
• chore: Fix typo in index.md by @anfindsen in https://github.com/explodinggradients/ragas/pull/1058
• fix(testset): for generating testset with new docs by @Manav916 in https://github.com/explodinggradients/ragas/pull/1001
• Fix counter in Runner and Executor in case of any exception by @baptiste-pasquier in https://github.com/explodinggradients/ragas/pull/995
• Chore: Fix RuntimeError: ... got Future <..> attached to a different loop by @abetatos in https://github.com/explodinggradients/ragas/pull/979
• fix: langchain EvaluatorChain OpenAI validation error by @danielfornarini in https://github.com/explodinggradients/ragas/pull/1051
• chore: removed async time tests from benchmarks by @jjmachan in https://github.com/explodinggradients/ragas/pull/1061
• Feature/updating prompt by @louky123 in https://github.com/explodinggradients/ragas/pull/1050

New Contributors

• @GTimothee made their first contribution in https://github.com/explodinggradients/ragas/pull/1008
• @NIL-zhuang made their first contribution in https://github.com/explodinggradients/ragas/pull/858
• @immortal3 made their first contribution in https://github.com/explodinggradients/ragas/pull/1025
• @anfindsen made their first contribution in https://github.com/explodinggradients/ragas/pull/1058
• @Manav916 made their first contribution in https://github.com/explodinggradients/ragas/pull/1001
• @baptiste-pasquier made their first contribution in https://github.com/explodinggradients/ragas/pull/995
• @abetatos made their first contribution in https://github.com/explodinggradients/ragas/pull/979
• @danielfornarini made their first contribution in https://github.com/explodinggradients/ragas/pull/1051
• @louky123 made their first contribution in https://github.com/explodinggradients/ragas/pull/1050

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.9...v0.1.10

What's Changed

• Devanshu's Blog Added in Doc's community folder. by @DevanshuBrahmbhatt in https://github.com/explodinggradients/ragas/pull/869
• Update data_preparation doc link in evaluation.md by @ragul-kachiappan in https://github.com/explodinggradients/ragas/pull/874
• docs: langchain.document_loaders is deprecated by @bioerrorlog in https://github.com/explodinggradients/ragas/pull/881
• chores: change cal link by @shahules786 in https://github.com/explodinggradients/ragas/pull/892
• chore: added issue manager by @jjmachan in https://github.com/explodinggradients/ragas/pull/907
• fix: Feat/issue closer by @jjmachan in https://github.com/explodinggradients/ragas/pull/909
• fix(docs): update document regarding langfuse usage by @jjmachan in https://github.com/explodinggradients/ragas/pull/904
• docs: clean up integrations by @jjmachan in https://github.com/explodinggradients/ragas/pull/919
• doc: Updated vertexai python notebook to use langchain wrapper approach for vertex ai by @shawnmittal in https://github.com/explodinggradients/ragas/pull/915
• Add new RAG evaluation articles by @Yongtae723 in https://github.com/explodinggradients/ragas/pull/932
• Fix/ci: fix the broken CI by @jjmachan in https://github.com/explodinggradients/ragas/pull/933
• reproducibility PR1 : introduce ensembler by @shahules786 in https://github.com/explodinggradients/ragas/pull/924
• Cleaned installation step in README by @andreped in https://github.com/explodinggradients/ragas/pull/921
• reproducibility PR2: add new statement creation #930 by @shahules786 in https://github.com/explodinggradients/ragas/pull/941
• Update generator.py - Redundant code by @omkar-334 in https://github.com/explodinggradients/ragas/pull/942
• docs: improve vertex ai docs by @jjmachan in https://github.com/explodinggradients/ragas/pull/951
• docs: small fixes in vertex ai docs by @jjmachan in https://github.com/explodinggradients/ragas/pull/952
• Update discord invite link by @shahules786 in https://github.com/explodinggradients/ragas/pull/953
• fix: sentence segmenter for multi languages by @shahules786 in https://github.com/explodinggradients/ragas/pull/946
• answer-correctness : fix edge cases by @shahules786 in https://github.com/explodinggradients/ragas/pull/970
• Fix ensembler : handle edge cases by @shahules786 in https://github.com/explodinggradients/ragas/pull/972
• feat: adding new defaults to openai Model and embeddings by @jjmachan in https://github.com/explodinggradients/ragas/pull/973
• Improve grammar and punctuation in critique prompts: addresses #971 by @ruankie in https://github.com/explodinggradients/ragas/pull/974
• Fix testset generator issue on context selection by @Jovillios in https://github.com/explodinggradients/ragas/pull/937
• feat: Ragas CI/CD by @jjmachan in https://github.com/explodinggradients/ragas/pull/976
• fix: patch type issue in evolution parsing by @shahules786 in https://github.com/explodinggradients/ragas/pull/980

New Contributors

• @ragul-kachiappan made their first contribution in https://github.com/explodinggradients/ragas/pull/874
• @bioerrorlog made their first contribution in https://github.com/explodinggradients/ragas/pull/881
• @shawnmittal made their first contribution in https://github.com/explodinggradients/ragas/pull/915
• @andreped made their first contribution in https://github.com/explodinggradients/ragas/pull/921
• @omkar-334 made their first contribution in https://github.com/explodinggradients/ragas/pull/942
• @ruankie made their first contribution in https://github.com/explodinggradients/ragas/pull/974
• @Jovillios made their first contribution in https://github.com/explodinggradients/ragas/pull/937

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.7...v0.1.8

What's Changed

• minor: fix llm expansion in docs by @glenacota in https://github.com/explodinggradients/ragas/pull/847
• Update llama_index imports by @sestinj in https://github.com/explodinggradients/ragas/pull/849
• feat: Change parsers and add custom critic model by @shahules786 in https://github.com/explodinggradients/ragas/pull/838
• fix: change documentation to reflect accurate method name by @dbtreasure in https://github.com/explodinggradients/ragas/pull/837
• Update answer_relevance.md by @carllapierre in https://github.com/explodinggradients/ragas/pull/832
• fix ci: some linting issues by @jjmachan in https://github.com/explodinggradients/ragas/pull/852

New Contributors

• @glenacota made their first contribution in https://github.com/explodinggradients/ragas/pull/847
• @sestinj made their first contribution in https://github.com/explodinggradients/ragas/pull/849
• @dbtreasure made their first contribution in https://github.com/explodinggradients/ragas/pull/837
• @carllapierre made their first contribution in https://github.com/explodinggradients/ragas/pull/832

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.6...v0.1.7

What's Changed

• Fix: json format inconsistency in context recall (prompt for statemen… by @jokokojote in https://github.com/explodinggradients/ragas/pull/776
• fix: Explanaitions typo on index.md by @dcamejo1 in https://github.com/explodinggradients/ragas/pull/801
• feat(integrations): add support for langchain and langsmith by @jjmachan in https://github.com/explodinggradients/ragas/pull/795
• Docs: fix confusion regarding getting started by @shahules786 in https://github.com/explodinggradients/ragas/pull/815
• docs: add openlayer integration by @shahules786 in https://github.com/explodinggradients/ragas/pull/827
• fix: Eventloop unclosed by @yunfwe in https://github.com/explodinggradients/ragas/pull/796
• More robust JSON prompting and parsing by @mrtj in https://github.com/explodinggradients/ragas/pull/807
• fix(integration): langchain integration and CI fixes by @jjmachan in https://github.com/explodinggradients/ragas/pull/830
• Added more examples to the translation prompts by @baswenneker in https://github.com/explodinggradients/ragas/pull/826
• fix: use aparse in all metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/831

New Contributors

• @dcamejo1 made their first contribution in https://github.com/explodinggradients/ragas/pull/801
• @yunfwe made their first contribution in https://github.com/explodinggradients/ragas/pull/796
• @mrtj made their first contribution in https://github.com/explodinggradients/ragas/pull/807
• @baswenneker made their first contribution in https://github.com/explodinggradients/ragas/pull/826

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.5...v0.1.6

What's Changed

• fix: handle string score in node filter by @shahules786 in https://github.com/explodinggradients/ragas/pull/755
• Add ipython notebook demonstrating the ragas-haystack integration. by @annthurium in https://github.com/explodinggradients/ragas/pull/754
• Update Bring Your Own LLM Docs Link by @pmbaumgartner in https://github.com/explodinggradients/ragas/pull/765
• fix: edge case in answer_correctness by @shahules786 in https://github.com/explodinggradients/ragas/pull/778
• Added metadata field to datarows by @rnbokade in https://github.com/explodinggradients/ragas/pull/782
• fix: is_async not passed for llm generation in context recall by @jokokojote in https://github.com/explodinggradients/ragas/pull/777
• fix: Nan in metrics/test set gen by @shahules786 in https://github.com/explodinggradients/ragas/pull/786
• Fix adapt issue for keyphrase extractor by @smodlich in https://github.com/explodinggradients/ragas/pull/766
• updated context_scoring_prompt and provide extra examples. by @nofrish in https://github.com/explodinggradients/ragas/pull/725
• fix: faithfulness by @shahules786 in https://github.com/explodinggradients/ragas/pull/787

New Contributors

• @annthurium made their first contribution in https://github.com/explodinggradients/ragas/pull/754
• @rnbokade made their first contribution in https://github.com/explodinggradients/ragas/pull/782
• @smodlich made their first contribution in https://github.com/explodinggradients/ragas/pull/766
• @nofrish made their first contribution in https://github.com/explodinggradients/ragas/pull/725

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.4...v0.1.5

What's Changed

• docs: Add docs for new entity based metric by @sky-2002 in https://github.com/explodinggradients/ragas/pull/674
• Fix typos that cause fatal assertion errors in prompts by @joy13975 in https://github.com/explodinggradients/ragas/pull/676
• fix: edge case in faithfulness by @shahules786 in https://github.com/explodinggradients/ragas/pull/682
• docs: Fix the precision calculation in context-precision metric doc by @amit-timalsina in https://github.com/explodinggradients/ragas/pull/685
• fix: improving reproducability of evals when running everything together by @jjmachan in https://github.com/explodinggradients/ragas/pull/671
• added octlane docs by @jjmachan in https://github.com/explodinggradients/ragas/pull/695
• Docs: Fix content precision description by @jokokojote in https://github.com/explodinggradients/ragas/pull/693
• Docs: Created own section for Calculation (instead of Dropdown) by @GeroVanMi in https://github.com/explodinggradients/ragas/pull/696
• Fix: Enable Critique Prompt Customization in AspectCritique by @AaronKim218 in https://github.com/explodinggradients/ragas/pull/700
• chore: improve bug/question template by @shahules786 in https://github.com/explodinggradients/ragas/pull/702
• docs: clean and smooth by @shahules786 in https://github.com/explodinggradients/ragas/pull/703
• Docs: Fix and supplement answer relevancy description by @jokokojote in https://github.com/explodinggradients/ragas/pull/705
• fix: remove unwanted quotes from questions by @shahules786 in https://github.com/explodinggradients/ragas/pull/708
• Readme Formatting by @gkamradt in https://github.com/explodinggradients/ragas/pull/710
• docs: customising embeddings and llms by @shahules786 in https://github.com/explodinggradients/ragas/pull/712
• chore: add copyright by @shahules786 in https://github.com/explodinggradients/ragas/pull/716
• feat: test set generation improvements by @shahules786 in https://github.com/explodinggradients/ragas/pull/719
• Adding Tonic Validate integration notebook by @akamor in https://github.com/explodinggradients/ragas/pull/707
• fix: edge case in CR by @shahules786 in https://github.com/explodinggradients/ragas/pull/728
• fix(ragas): remove mutable defaults by @mspronesti in https://github.com/explodinggradients/ragas/pull/684
• fix: remove experimental filters by @shahules786 in https://github.com/explodinggradients/ragas/pull/743
• Correct Faithfulness Verdict Map by @pmbaumgartner in https://github.com/explodinggradients/ragas/pull/739
• feat(generator): extend construction to any langchain LLM and Embeddings by @mspronesti in https://github.com/explodinggradients/ragas/pull/670
• docs: minor corrections by @shahules786 in https://github.com/explodinggradients/ragas/pull/747

New Contributors

• @amit-timalsina made their first contribution in https://github.com/explodinggradients/ragas/pull/685
• @jokokojote made their first contribution in https://github.com/explodinggradients/ragas/pull/693
• @GeroVanMi made their first contribution in https://github.com/explodinggradients/ragas/pull/696
• @AaronKim218 made their first contribution in https://github.com/explodinggradients/ragas/pull/700
• @gkamradt made their first contribution in https://github.com/explodinggradients/ragas/pull/710
• @akamor made their first contribution in https://github.com/explodinggradients/ragas/pull/707
• @mspronesti made their first contribution in https://github.com/explodinggradients/ragas/pull/684
• @pmbaumgartner made their first contribution in https://github.com/explodinggradients/ragas/pull/739

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.3...v0.1.4

What's Changed

• feat: improved node selection and question fixing by @shahules786 in https://github.com/explodinggradients/ragas/pull/644
• docs: add explanation to metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/658
• fix: edge cases in metrics by @shahules786 in https://github.com/explodinggradients/ragas/pull/661
• Update evolution by @cuent in https://github.com/explodinggradients/ragas/pull/666
• feat: Add new "entity-based context recall" metric by @sky-2002 in https://github.com/explodinggradients/ragas/pull/667
• fix: filter and context selection by @shahules786 in https://github.com/explodinggradients/ragas/pull/672
• Add support for optional max concurrency by @joy13975 in https://github.com/explodinggradients/ragas/pull/643

New Contributors

• @cuent made their first contribution in https://github.com/explodinggradients/ragas/pull/666
• @sky-2002 made their first contribution in https://github.com/explodinggradients/ragas/pull/667

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.2...v0.1.3

What's Changed

• docs: arize/ ragas notebook by @axiomofjoy in https://github.com/explodinggradients/ragas/pull/594
• fix: prompt adaptation type checks by @shahules786 in https://github.com/explodinggradients/ragas/pull/615
• fix: node indexing in testset generator by @shahules786 in https://github.com/explodinggradients/ragas/pull/617
• Update ragas-arize.ipynb by @PubliusAu in https://github.com/explodinggradients/ragas/pull/618
• docs: getting started by @jjmachan in https://github.com/explodinggradients/ragas/pull/589
• Added testset generation for bedrock by @VpkPrasanna in https://github.com/explodinggradients/ragas/pull/626
• Made embeddings and LLMs dependent on metric in evaluate function by @peanutshawny in https://github.com/explodinggradients/ragas/pull/628
• docs: bring your own llms and embeddings by @jjmachan in https://github.com/explodinggradients/ragas/pull/631
• docs: fix import in the example code: ContextRelevance -> ContextRelevancy by @izikeros in https://github.com/explodinggradients/ragas/pull/646
• Fix typo: Teset -> Testset by @joy13975 in https://github.com/explodinggradients/ragas/pull/638
• Update ragas-arize.ipynb by @PubliusAu in https://github.com/explodinggradients/ragas/pull/637
• renaming ground_truths to ground_truth. by @alexjyong in https://github.com/explodinggradients/ragas/pull/640
• fix: added types so you can pass langchain llms and embeddings too by @jjmachan in https://github.com/explodinggradients/ragas/pull/652

New Contributors

• @axiomofjoy made their first contribution in https://github.com/explodinggradients/ragas/pull/594
• @PubliusAu made their first contribution in https://github.com/explodinggradients/ragas/pull/618
• @VpkPrasanna made their first contribution in https://github.com/explodinggradients/ragas/pull/626
• @peanutshawny made their first contribution in https://github.com/explodinggradients/ragas/pull/628
• @izikeros made their first contribution in https://github.com/explodinggradients/ragas/pull/646
• @joy13975 made their first contribution in https://github.com/explodinggradients/ragas/pull/638
• @alexjyong made their first contribution in https://github.com/explodinggradients/ragas/pull/640

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.1...v0.1.2

What's Changed

• Fixing typos, vertextai to vertex ai by @mrchtr in https://github.com/explodinggradients/ragas/pull/572
• fix typo by @Kyeongpil in https://github.com/explodinggradients/ragas/pull/570
• fix assertions and quality in prompt adaptation by @shahules786 in https://github.com/explodinggradients/ragas/pull/574
• fix: ensure dict type by @shahules786 in https://github.com/explodinggradients/ragas/pull/578
• style: remove additional comments in prompt.py by @leehanchung in https://github.com/explodinggradients/ragas/pull/577
• Add stepkurniawan community contribution by @stepkurniawan in https://github.com/explodinggradients/ragas/pull/585
• Parse JSON string in adapt() by @mitmul in https://github.com/explodinggradients/ragas/pull/584
• fix: replace string formating by dict by @shahules786 in https://github.com/explodinggradients/ragas/pull/588
• [patch] Reduce NaN Occurrences by Simple Prompt Modification for JSON Output for context_precision by @i-w-a in https://github.com/explodinggradients/ragas/pull/581
• ci: migrate to latest llama-index version by @shahules786 in https://github.com/explodinggradients/ragas/pull/603
• Fix: evolution flows for test generation by @shahules786 in https://github.com/explodinggradients/ragas/pull/602
• fix: llamaIndex latest version by @jjmachan in https://github.com/explodinggradients/ragas/pull/604
• Chore: add language to event tracking by @shahules786 in https://github.com/explodinggradients/ragas/pull/592
• feat(llms.json_load): Recursively load json lists by @pberger514 in https://github.com/explodinggradients/ragas/pull/593

New Contributors

• @mrchtr made their first contribution in https://github.com/explodinggradients/ragas/pull/572
• @Kyeongpil made their first contribution in https://github.com/explodinggradients/ragas/pull/570
• @leehanchung made their first contribution in https://github.com/explodinggradients/ragas/pull/577
• @stepkurniawan made their first contribution in https://github.com/explodinggradients/ragas/pull/585
• @mitmul made their first contribution in https://github.com/explodinggradients/ragas/pull/584
• @i-w-a made their first contribution in https://github.com/explodinggradients/ragas/pull/581
• @pberger514 made their first contribution in https://github.com/explodinggradients/ragas/pull/593

Full Changelog: https://github.com/explodinggradients/ragas/compare/v0.1.0...v0.1.1