Azure Cognitive Search: Vector Search with Langchain and OpenAI Embeddings

🔍 Azure machine learning service and its different models are explored.

🔗 Vector search support in Azure cognitive search using Langchain framework is demonstrated.

📊 A notebook is used to convert data into vector embeddings and perform searches in Azure cognitive search.

🔍 The video discusses the process of vector search in Azure Cognitive Search using Langchain.

📚 The model catalog allows users to access different models from various companies, view their details, and deploy them.

⚙️ The speaker focuses on using Azure open AI models, specifically the text embedding ada002 model for converting text into embeddings.

🔍 Vector search is a new capability for indexing, storing, and retrieving vector embeddings from a search index.

🔢 The vector search pattern, called retrieval augmented generation (RAG), can be used to prevent model hallucinations.

🗂️ Data from various sources, such as images, audio, video, and text, can be converted into vectors using an embedding model and stored in Azure cognitive search service.

🔍 Vector search in Azure Cognitive Search can be performed using Langchain, allowing users to retrieve matching results.

🔢 The search algorithm used in Azure Cognitive Search is K nearest neighbors, similar to Google search and YouTube search.

💡 Azure Cognitive Search does not generate Vector embeddings by default, but this limitation can be overcome by integrating Azure open AI model and Langchain framework.

🔍 The video discusses a factor search engine called Fast provided by Meta.

💡 Hugging Face embeddings are used instead of Open AI for vector search in Azure Cognitive Search.

📦 Python packages for search, Open AI, and Langchain are installed, and APIs like Open AI embeddings and Azure Search are used to push vectors into Azure Cognitive Search.

🔍 Vector search in Azure Cognitive Search can be used to group cities based on their embeddings.

🗺️ Cities from different regions, such as India, Europe, and the US, can be grouped together using vector representation.

📊 To create vector search, an instance is created with search details, index, and embedding function, and the documents are split using Langchain APIs.

🔍 Vector search in Azure Cognitive Search using Langchain.

🧩 Pushing chunks into the vector store and performing searches based on queries and keywords.

🌍 Ability to retrieve results based on specific countries.