LangSmith Tutorial: Bring Your LLM Systems to Production
Learn about LangSmith, a platform for debugging, testing, and monitoring LLM applications. Explore its features and uses for LLM systems. Bring your LLM systems to production with ease!
00:00:00 Learn about LangSmith, a platform designed to help with debugging, testing, and monitoring of LLM applications. Explore its features and uses for LLM systems.
🔍 LangSmith is a new platform designed to debug, test, and monitor LLM applications, bridging the gap between prototypes and production.
🖥️ The platform provides a user interface for creating and managing projects and data sets, but it is more efficient to use code for most tasks.
📝 To get started, users need to create a project, retrieve an API key, and set up environment variables. The API key is important for tracking LLM code with sensitivity in mind.
00:02:28 Learn how to bring your LLM systems to production using LangSmith and LangChain libraries. Control and trace your projects with ease.
📦 In order to bring LLM systems to production, we need to install LinkSmith, LangChain, and Python packages.
⚙️ By loading environment variables and running a chain, we can get the output from OpenAI.
🌍 To have more control, we can use LangSmith and LangChain libraries and create trace instances.
00:04:53 Learn how to use tags to filter and group different steps in your LLM systems using the LangSmith Tutorial. Bring your LLM systems to production with ease!
🔍 Tagging LM codes allows for filtering and organizing different steps of the LM chain.
🔀 Grouping different LM calls can be done using the Trace Ace chain group function.
🖥️ The project UI provides a visual representation of the tagged and grouped LM calls.
00:07:19 Learn how to list and filter LLM runs in UI and through code, as well as filter runs using metadata. Explore data evaluation using a simple list of tablets.
🖱️ You can structure your LLM calls using tags or by listing them with code.
🔍 You can filter LLM runs based on the start time, run type, or metadata.
💻 You can create a data set and use it to evaluate the quality of an LLM.
00:09:47 Learn how to bring your LLM systems to production using LangSmith. Create datasets, upload CSV files, and evaluate your llms with ease.
🔍 The tutorial explains how to create a dataset and upload it to LangSmith.
📊 Different data formats, such as tuples and CSV files, can be used for storing data in LangSmith.
🧪 The tutorial also covers evaluating LLMs using the RunEvalConfig and RunOnDataset methods.
00:12:13 Creating a client to run question answering evaluations with an llm or chain factory. Custom prompt templates can be used for more specific prediction results.
📚 The video explains how to bring LLM systems to production.
💻 The process involves creating a client, running the data set method, and utilizing evaluation configuration.
🔍 A custom prompt template can be created to categorize query, answer, and prediction results.
00:14:41 A tutorial on bringing LLM systems to production, explaining the structure and usage of QA and cord classes. The video also mentions the possibility of future changes in the documentation.
📚 The tutorial discusses the usage of the context QA and chord QA classes in the LangSmith system.
💻 The q and a class takes an evaluator type, with the llm set to none and the prompt set to default.
🔍 The tutorial demonstrates running the system on a data set and viewing the output in the UI.
Want to deep dive into this video?
