LangSmith has released new features for production monitoring and automations. Filtering can be used to inspect data, while automations allow users to act on selected data points. The monitoring tab provides an aggregate view of statistics over time, which can be grouped by metadata attributes. The threads feature allows users to view the entire conversation in a chat application. LangSmith also introduces automations with filtering, sampling rate, and actions such as adding to a dataset or annotation queue. Online evaluation is a new feature that allows runs to be evaluated by an LLM according to configurable criteria. The use case of optimization demonstrates how to combine these features for a tweet styling application.
2 Impressions