Product Update - Online Evaluation
This month we have three exciting updates to share with you!
Online Evaluation
The online evaluation feature has been revamped:
The flow to create an online evaluation rule have been reworked.
You can now run online evaluation on runs (agents, chains, workflows), in addition to generations.
Online evaluation automation rules now supports tag creation in addition to score creation.
Rule params are now editable.
Roles
There are now three roles: Admin, AI Engineer and Domain Expert. We plan to add more based on your feedback.
Improvements
For user feedback or annotations, we now track the user who created the score.
The dashboard was overall improved. There is an additional plot where you can track the usage per agent or chain type.
There is now a default project when onboarding to Literal AI.
Improved SDK: New version of the TypeScript SDK.
New integration with Mistral AI.
Bulk actions to create datasets from generations.
Miscellaneous
We had to change the data model and perform a migration about how we store steps. This unlocks simple step management and better user experience and developer experience.
Better UX for multi-modal input and outputs.
Check out the full release notes: https://docs.getliteral.ai/more/release-notes
Try it here!