Connect Azure OpenAI ChatGPT with Azure Cognitive Search: A Step-by-Step Guide

AI can create text, image, audio content etc… Developers all across the world are incorporating AI into their applications for the benefit of their users. New AI models and SDKs pop out of the woodwork, it can be challenging for the enterprise to figure out how to get started and use it all especially when their curated content is located within SharePoint, ITSM knowledge bases or internal content management systems.

Microsoft Azure OpenAI offers high-performance AI models at a production scale, boasting industry-leading uptime. Complementing this, Chime provides enterprises with a platform to expediently leverage their pre-existing curated content to train Azure OpenAI models. Chime's chatbot understands natural human language and responds with content tailored to the enterprise, ensuring responsible, specific, and privacy-conscious interactions. Importantly, enterprises maintain complete control over the content delivered to their audience, preserving their unique brand identity and messaging. Here is an example of how Chime uses FAQs as part of a university deployment to populate ChatGPT chat responses as well as display a portal UI.

Step 1: Preparation

Before you embark on this journey, ensure you have the following components ready:

  • Azure Storage Account: This serves as the repository for your data.

  • Azure Cognitive Search Account: The foundation for creating an index and enabling efficient searches.

  • Azure OpenAI Service: A crucial tool to utilize ChatGPT for enhanced responses.

Step 2: Creating an Azure OpenAI Service

Initiate the creation of an Azure OpenAI service, opening the doors to a world of possibilities. You can use the same service for all tenants in various environments or set up separate services based on your unique requirements.

Step 3: Creating a Azure Storage Account

Set up an Azure Storage Account, providing a structured foundation for data storage. You can create multiple storage accounts to segment data for different Chime tenants, but each tenant should have two uniquely-named Blob Containers within the same storage account.

Step 4: Create a Storage Container

Within your storage account, create an empty Blob Container. This container is pivotal for the Azure Cognitive Search setup, and it's crucial to assign a unique name to it.

Step 5: Adding a Seed Content File

To kickstart the creation of essential resources, add a "seed" file to the storage container. This file should be in HTML format and contain minimal content. Without this seed file, the Cognitive Search datasources, skillsets, and indexers won't be created.

Step 6: Creating an Indexer for the Container

Set up an indexer for the storage container. This indexer will periodically check for new documents in the container, ensuring that your knowledge base remains up to date.

Step 7: Creating a Cognitive Search Service

Now, create a Cognitive Search service. You can choose the Basic tier to handle up to seven Chime tenants. This service will be used to build the knowledge base for ChatGPT.

Step 8: Setting Up the Cognitive Search Service

To configure the Cognitive Search service, you can utilize a user-friendly wizard available in the Azure OpenAI Chat Playground under the "Add your data (preview)" tab. Here, you'll need to connect your Azure Storage Account, Blob Container, and Cognitive Search service, streamlining the setup process.

In this configuration, you have the flexibility to enable various search types, including keyword-based searching, semantic search, vector search, and combinations of these. For a comprehensive knowledge base, it's recommended to enable all available options.

Review your selections and save the configuration. The Cognitive Search resources will be created.

Step 9: Verifying Setup

After completing the setup, you should observe two storage containers in your Azure Storage Account: one for raw documents and another for indexed chunks. Additionally, in the Cognitive Search resource, data sources, indexers, skillsets, and indexes should have been created, signifying that your knowledge base is now operational.

Step 10: Configuring Cognitive Search Settings in Chime V5

With the Cognitive Search setup complete, it's time to configure Chime settings to enable this transformative feature.

Configure Upload Storage Account:

Provide the Storage Container name (e.g., example-container) and the Storage Connection String, which can be found in the Azure Portal under the Access Keys tab. Test the connection to ensure its functionality.

Configure Cognitive Search Index Settings:

Provide the Cognitive Search Endpoint (visible in the Azure Portal's overview blade for Cognitive Search Service), Cognitive Search Key (admin key is required for full access), Index Name (use the augmented index, e.g., example-index), and optionally, the indexer name. Test these settings to ensure a successful connection, and then save the configuration.

Step 11: Cognitive Search Status Page

From the Cognitive Search Setting Page, you can access the Cognitive Search Indexing Status Page, which shows an overview of your connection to the Storage Account and Search Index, including the number of uploaded and indexed documents. If configured, it displays the last time the Search Indexer ran.

Step 12: Marking Chime Documents as Indexable

Each content item has metadata attached to track if it should be uploaded. Update content items, and the new version will be automatically uploaded and indexed.

Step 13: Force Reindexing from the Azure Portal

If you want to force the index to update, you can do it from the Cognitive Search Indexing Status page. Manually run the indexer to process new and modified documents in the datasource. Refresh to verify that the indexer has run and indexed the new document.

Step 14: Using Cognitive Search in the Chime Agent AI Chat Assist Feature

With Cognitive Search configured and FAQs indexed, you can enable Chime's Agent Assist feature to use indexed FAQ items in generating suggestions for agents.

Enable Cognitive Search in Agent AI Chat Assist Settings:

The first checkbox adds Cognitive Search to Chat GPT service calls.

  • The second checkbox ensures Chat GPT only uses information from indexed FAQ items. If no relevant information is found, it won't use the base model for responses.

Step 15: Cognitive Search Agent AI Assist in Action

With FAQs indexed, Chat GPT can provide responses that include information from the indexed documents, along with a link to the original FAQ for reference.

Step 16: Using Cognitive Search with the Chat GPT Pipeline Stage

You can also enable the Chat GPT pipeline chatbot to use indexed knowledge base items. With these settings enabled, the chatbot can respond based on the knowledge base, ensuring more accurate and relevant responses to guest queries.

Step 17: Customizing your ChatGPT & Cognitive Search Experience

Set custom parameters from a list of advanced parameters for the Chat GPT Model:

  • Choice Count: Specifies the number of alternate responses to return for a prompt.

  • Max Tokens: Defines the maximum number of tokens to generate in the response.

  • Temperature: Sets the sampling temperature, influencing the apparent creativity of generated completions. Higher values result in more randomness, while lower values lead to more focused and deterministic results.

  • Top P: Governs the randomness of the response. Lower values make the response more deterministic.

  • Frequency Penalty: Regulates repetition in the response. Positive values decrease the likelihood of tokens appearing as their frequency increases, reducing the model's tendency to repeat the same statements.

  • Presence Penalty: Manages repetition in the response. Positive values decrease the likelihood of tokens appearing when they already exist, thereby increasing the likelihood of the model outputting new topics.

Set additional Cognitive Search settings to allow for a custom user experience when referencing your documentation.

  • Restrict Bot Responses to Knowledge Base: When enabled, the chatbot will exclusively suggest responses based on information retrieved from knowledge base resources such as FAQs. If no relevant resources are found, the user will be informed. When disabled, the chatbot can also respond using data from its base model if no relevant information is available in the knowledge base.

  • Strictness: This parameter determines how closely the chatbot matches searched documents to the user's query. Increasing the value raises the threshold for relevance, filtering out less-relevant documents for responses. Setting this value too high may limit the model's ability to generate responses due to a scarcity of available documents.

  • Maximum Retrieved Documents: Defines the maximum number of top-scoring documents provided to the chatbot as background information to generate a response. Increasing this value may enhance the quality of responses, particularly if there are numerous short documents in the knowledge base.

  • Search Query Type: Users can choose between 'Simple' and 'Semantic' search query types.

  • FAQ Citation Link Options: Users can opt to display cited documents as footnotes, where URLs are included at the end of the message, inline within the response, or choose not to link the source documents at all.

Conclusion

Enriching your AI models with your data using Azure Cognitive Search is a powerful method to enhance the accuracy and relevance of responses. By following these comprehensive steps, you can set up the integration of Azure Cognitive Search with Azure OpenAI ChatGPT to create a more informed and capable AI assistant for your Chime V5 platform. This not only enriches the guest experience but also makes your agents more efficient in providing information and support.

bmorris