In this blog post, we will look at how to setup the resources required to programatically call a GenAI model (i.e. ChatGPT) and ask it questions about your own data. This article is intended to support the AI Overview For Sitecore module, but can be used independently to get you up and running with a simple console application (shown at the end).
Getting Started
First things first, you need to deploy some resources in Azure and retrieve the necessary configuration. So, to begin create a new Resource Group and name it accordingly.Create Your OpenAI Resources
Next thing you need to do is to create an Azure OpenAI resource. When doing so, you will be prompted to select a Region and a Pricing tier (Standard S0 is the only option). When selecting a region - select the closest to you to begin with, but you may need to adjust (recreate) to a different region, depending on the availability of models in your region.
From the Azure OpenAI resource (in the portal), you can then select the [Go To Azure AI Foundry portal] button. This will open the AI Foundry, if you then go to the Deployments area, you will be given the option to Deploy a model. For the purposes of this demonstration, we need to deploy two base models. One generative AI model (i.e. gpt-4) and one text embedding model (i.e. text-embedding3-small).
Depending on your region, you may find that some models are not available. Its important to ensure that both of your deployed models are located in the same region and attached to the same OpenAI resource. When deploying a new base model, if the type of model is not available in your region, the GUI will automatically adjust the region and attempt to create a new OpenAI resource… if you see the warning below, then try selecting a slightly different model (worst case, you may need to recreate your resource in a different region).
A new AI resource will be created for your deployment A resource location that supports the model has been pre-selectedThe next thing to look out for, is that some models have a very low available quota in some regions. I discovered this, when trying to test a model out in the playground and repeatedly getting “Rate Limit Exceeded“ errors. In my case, switching from gpt-4 to gpt-4-32k fixed the issue – but this entirely depend on the quotas in your region. From the deployments area, select your GenAI model and take note of the following config:
| Example | Notes | |
| Model Name | gpt-35-turbo | The name you gave when deploying the model |
| Endpoint | https://your-name.openai.azure.com | Root domain from Target URI |
| Key | longstringofcharacters | Key from deployment overview |
Create some storage
The next resource we need is a Storage Account to house our own data. So back in the azure portal, create one in the same resource group (and region). When prompted, select “Azure Blob Storage” as the primary service, Standard Performance and Locally-redundant storage (LRS). When the resource is created, navigate to the storage browser and create a new blob container to house our data. Take note of the following config:| Example | Notes | |
| Storage Account Name | yourstorage | This is the name of the storage account |
| Storage File System Name | yourblobname | This is the name of the blob container you created |
| Storage Account Key | longstringofcharacters | You can find this in ‘Access Keys’ area of Security section |
Get your data into blob storage
Next you need to find a way to get your data into blob storage. Ideally each document you upload should be uniquely named and contain a minimal amount of HTML (i.e. strip out everything that is not basic formatting and content).
Note: If you have installed the AI Overview For Sitecore module, there is a cron task that will upload the data for you.
Index your data
With your data in blob storage, you then need to ingest it into an Azure Search Service. This can be done using the GUI within the Chat Playground (in Azure AI Foundry) or directly into the Search Service (in Azure Portal).To begin, create a new AI Search resource in the azure portal (ensure this is in the same region).
Note: Azure AI Search Services can be expensive, so if you are trialling this in a test subscription, be sure to change the pricing tier to Basic when given the option (FYI – Free tier is not supported) [Basic: £57/month | Standard: £192/month].
Ingesting Data via Azure AI Foundry
To add your data, go to the Chat area within the Playgrounds. Select your deployment in the drop down and give your model some basic instructions (i.e. you are a helpful chatbot). Select the button [Add a data source] in the section below. Then select “Azure Blob Storage” as the data source and add the storage account name and container you set previously. This confirms where our source data is coming from.
To define where we want our data indexed too, select the Azure AI Search resource you created earlier and enter a name for the new index. To improve the speed and efficiency of the data retrieval process, you can tick the box to ‘Add vector search’. This uses the second ‘Embedding’ model that we deployed earlier. The data management section gives options for search type and chunk size – you may want to play around with these settings to see what works best for you (more info here: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/use-your-data?tabs=ai-search%2Ccopilot#search-options).
Finally, select API key method of authentication and save and close. This should then trigger the ingestion process, which will start breaking down the data from your storage account into chunks that can later be searched over. Behind the scenes, the above process creates a new index and indexer in your search resource, which we can call programmatically over the API.
Once complete, you should be able to use the chat session within the playground, to ask the model questions about your data. The advanced settings defaults to only allow responses to be generated from your own data.
Ingesting Data via Azure Portal
An alternative method of getting your data into the search index is to trigger it manually within the search resource.To do this, go to your search resource and click the button to [Import and vectorize data]. Then choose “Azure Blob Storage” and select the storage and container you setup previously. Enter “Items/” (leave blank if not using sitecore module) for the Blob folder and ‘Default’ parsing mode.
On the following screen select your OpenAI service and select the embedding model that you created earlier. Choose API key authentication and proceed.
You can stick with the default options for the remainder of the process, then finally choose a name for your index and hit ‘Create’. This create your new index and indexer and starts adding your data. Once complete, you should be able to run some test queries via the search explorer, which will only bring back response from your indexed data.
Grab the necessary config
No matter which way you decided to ingest the data, you should be left with the same resources in the Azure Portal. Within the OpenAI service you should now have a data source, index, indexer and skillsets.
For the purposes of this run through, we are mainly interested in the search index. This is what are going to connect to our chat client as a data source. Which in turn allows the GenAI model to answer questions about our data. Take note of the following config:
| Example | Notes | |
| Search Endpoint | https://yourresource.search.windows.net | This is the url on the search service overview |
| Search Index | your-index-name | The name you gave to the index |
| Search Key | longstringofcharacters | Found in the keys section of the search resource |
Querying the Chat Client using your own data.
Note: If you have installed the AI Overview For Sitecore module, then you can simply add 'Search - AI Site Search Results' component to a page. If you then call that page and make sure the query is appended to the url with the “q” parameter, then you should see the results of the call.
Use a Console App to call the API
Alternatively, if you want to try calling your model programmatically, then the following console application should suffice:
using Azure.AI.OpenAI;
using Azure.AI.OpenAI.Chat;
using OpenAI.Chat;
using System.ClientModel;
string genAIEndpoint = "xxx";
string genAIKey = "xxx";
string genAIModelName = " xxx ";
string searchIndex = " xxx ";
string searchEndpoint = " xxx ";
string searchKey = " xxx ";
string systemPrompt = "You are a helpful assistant, with in depth knowledge of xxx.";
string userPrompt = "Please supply a maximum of 500 words about the following search term: xxx ";
AzureOpenAIClient azureClient = new( new Uri(genAIEndpoint), new ApiKeyCredential(genAIKey));
ChatClient chatClient = azureClient.GetChatClient(genAIModelName);
ChatCompletionOptions options = new ChatCompletionOptions();
#pragma warning disable AOAI001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
options.AddDataSource(new AzureSearchChatDataSource()
{
Endpoint = new Uri(searchEndpoint),
IndexName = searchIndex,
Authentication = DataSourceAuthentication.FromApiKey(searchKey),
});
#pragma warning restore AOAI001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
var messages = new List() { new SystemChatMessage(systemPrompt) , new UserChatMessage(userPrompt) };
ChatCompletion completion = chatClient.CompleteChat(messages, options);
Console.WriteLine($"{completion.Role}: {completion.Content[0].Text}");
#pragma warning disable AOAI001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
ChatMessageContext onYourDataContext = completion.GetMessageContext();
#pragma warning restore AOAI001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
if (onYourDataContext?.Intent is not null)
{
Console.WriteLine($"Intent: {onYourDataContext.Intent}");
}
#pragma warning disable AOAI001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
foreach (ChatCitation citation in onYourDataContext?.Citations ?? new List())
{
Console.WriteLine($"Citation: {citation.Content}");
}
#pragma warning restore AOAI001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
Console.ReadKey();