LangSmith Tutorial: Bring Your LLM Systems to Production

🔍 LangSmith is a new platform designed to debug, test, and monitor LLM applications, bridging the gap between prototypes and production.

🖥️ The platform provides a user interface for creating and managing projects and data sets, but it is more efficient to use code for most tasks.

📝 To get started, users need to create a project, retrieve an API key, and set up environment variables. The API key is important for tracking LLM code with sensitivity in mind.

📦 In order to bring LLM systems to production, we need to install LinkSmith, LangChain, and Python packages.

⚙️ By loading environment variables and running a chain, we can get the output from OpenAI.

🌍 To have more control, we can use LangSmith and LangChain libraries and create trace instances.

🔍 Tagging LM codes allows for filtering and organizing different steps of the LM chain.

🔀 Grouping different LM calls can be done using the Trace Ace chain group function.

🖥️ The project UI provides a visual representation of the tagged and grouped LM calls.

🖱️ You can structure your LLM calls using tags or by listing them with code.

🔍 You can filter LLM runs based on the start time, run type, or metadata.

💻 You can create a data set and use it to evaluate the quality of an LLM.

🔍 The tutorial explains how to create a dataset and upload it to LangSmith.

📊 Different data formats, such as tuples and CSV files, can be used for storing data in LangSmith.

🧪 The tutorial also covers evaluating LLMs using the RunEvalConfig and RunOnDataset methods.

📚 The video explains how to bring LLM systems to production.

💻 The process involves creating a client, running the data set method, and utilizing evaluation configuration.

🔍 A custom prompt template can be created to categorize query, answer, and prediction results.

📚 The tutorial discusses the usage of the context QA and chord QA classes in the LangSmith system.

💻 The q and a class takes an evaluator type, with the llm set to none and the prompt set to default.

🔍 The tutorial demonstrates running the system on a data set and viewing the output in the UI.