LangSmith Tutorial: Bring Your LLM Systems to Production
Learn about LangSmith, a platform for debugging, testing, and monitoring LLM applications. Explore its features and uses for LLM systems. Bring your LLM systems to production with ease!
00:00:00 Learn about LangSmith, a platform designed to help with debugging, testing, and monitoring of LLM applications. Explore its features and uses for LLM systems.
๐ LangSmith is a new platform designed to debug, test, and monitor LLM applications, bridging the gap between prototypes and production.
๐ฅ๏ธ The platform provides a user interface for creating and managing projects and data sets, but it is more efficient to use code for most tasks.
๐ To get started, users need to create a project, retrieve an API key, and set up environment variables. The API key is important for tracking LLM code with sensitivity in mind.
00:02:28 Learn how to bring your LLM systems to production using LangSmith and LangChain libraries. Control and trace your projects with ease.
๐ฆ In order to bring LLM systems to production, we need to install LinkSmith, LangChain, and Python packages.
โ๏ธ By loading environment variables and running a chain, we can get the output from OpenAI.
๐ To have more control, we can use LangSmith and LangChain libraries and create trace instances.
00:04:53 Learn how to use tags to filter and group different steps in your LLM systems using the LangSmith Tutorial. Bring your LLM systems to production with ease!
๐ Tagging LM codes allows for filtering and organizing different steps of the LM chain.
๐ Grouping different LM calls can be done using the Trace Ace chain group function.
๐ฅ๏ธ The project UI provides a visual representation of the tagged and grouped LM calls.
00:07:19 Learn how to list and filter LLM runs in UI and through code, as well as filter runs using metadata. Explore data evaluation using a simple list of tablets.
๐ฑ๏ธ You can structure your LLM calls using tags or by listing them with code.
๐ You can filter LLM runs based on the start time, run type, or metadata.
๐ป You can create a data set and use it to evaluate the quality of an LLM.
00:09:47 Learn how to bring your LLM systems to production using LangSmith. Create datasets, upload CSV files, and evaluate your llms with ease.
๐ The tutorial explains how to create a dataset and upload it to LangSmith.
๐ Different data formats, such as tuples and CSV files, can be used for storing data in LangSmith.
๐งช The tutorial also covers evaluating LLMs using the RunEvalConfig and RunOnDataset methods.
00:12:13 Creating a client to run question answering evaluations with an llm or chain factory. Custom prompt templates can be used for more specific prediction results.
๐ The video explains how to bring LLM systems to production.
๐ป The process involves creating a client, running the data set method, and utilizing evaluation configuration.
๐ A custom prompt template can be created to categorize query, answer, and prediction results.
00:14:41 A tutorial on bringing LLM systems to production, explaining the structure and usage of QA and cord classes. The video also mentions the possibility of future changes in the documentation.
๐ The tutorial discusses the usage of the context QA and chord QA classes in the LangSmith system.
๐ป The q and a class takes an evaluator type, with the llm set to none and the prompt set to default.
๐ The tutorial demonstrates running the system on a data set and viewing the output in the UI.
Want to deep dive into this video?
