6.2.0

What's Changed

WKG-33:

• Fix custom evaluation metrics
• Upgrade litellm to 1.85.1
• Make answer correctness evaluator get_correctness_dict async for consistency
• Make custom evaluator evaluate async for consistency
• Rename llm.py to llm_factory.py
• Improve tests:
  • Fix instructions on how to run the tests
  • Rename all system tests to start with "test_*" for consistency
  • Read the configurations for all system tests in the test methods, not in the file globally

by @nelly-hateva in #81

Full Changelog: 6.1.0...6.2.0

6.1.0

What's Changed

• Statnett-282: Allow matches of IRI discovery with SPARQL Query by @nelly-hateva in #74

Full Changelog: 6.0.0...6.1.0

6.0.0

What's Changed

• Statnett-282: Calculate steps score based on all reference steps, not just the ones in the last reference group & Evaluation of time series and IRI discovery steps by @nelly-hateva in #38
• TTYG-160 Implement custom metric by @pgan002 in #46
• TTYG-165: Upgrade 3rd-party libraries by @nelly-hateva in #47
• TTYG-160 Fix custom evaluations reserved keys by @pgan002 in #52
• TTYG-166 Configurable LLM by @pgan002 in #50
• Statnett-358: Fix evaluation in case an actual sparql query is a DESCRIBE query, and the output is a string by @nelly-hateva in #57
• Statnett-282: Bump vulnerable dependencies and release version 6.0.0 by @nelly-hateva in #73

Full Changelog: 5.3.1...6.0.0

5.3.1

What's Changed

• Update langchain-core due to vulnerabilities by @nelly-hateva in #36

Full Changelog: 5.3.0...5.3.1

5.3.0

What's Changed

• Statnett-278: Take into account the error samples, when calculating averages by @nelly-hateva in #33

Full Changelog: 5.2.0...5.3.0

5.2.0

Statnett-240: Compare SPARQL results with duplicated binding values

5.1.2

Bug fixes

Statnett-142: Cast micro and macro aggregation statistics to dict, so that the yaml serialization works as expected

5.1.1

Bug fixes

Statnett-142: Fix bug in the calculation of the aggregated results in case of an output from DESCRIBE or CONSTRUCT query containing the string "results"

5.1.0

Bug fixes

TTYG-126: Fix ragas errors caused by incompatibility of some libraries
TTYG-126: Rename openai extra and dependency group to ragas

5.0.2

Bug fixes

TTYG-130: Package the prompts as part of the package