When people talk about AI, there is often alot of theory about how it might be used, but not as much real world example showing it in action. In this blog post, I will demonstrate 10 real world examples. Showing how we have incorporated AI into our every day Sitecore workflows.
1. Image Classification
How It Works
In this example, we hook into the item:saved event to process new images automatically. Alternatively, we use a scheduled task if we prefer not to inject a dependency into the save process. We send each image to the Microsoft Vision API. We then use the returned metadata to enrich the image in several ways.
Metadata Enrichments:
- ALT tags: The API generates descriptive ALT text automatically.
- Extended keywords: We get colours, subject matter, and dimension data added to each image.
- Template matching: We compare dimensions against our business requirements. This helps identify images suited for banners, portrait pods, or navigation.
The Result
This metadata makes previously siloed images searchable across our entire media library. A content editor can search "bannerimage + iceberg" and discover relevant images buried deep in another user's folders.
2. Smart Image Cropping
The Challenge
In this next example, we want to create a navigation image for every item in an area of the content tree. Each item uses a variety of templates, with images saved in different shapes and sizes. We need a uniform 200 x 200 px square image that reflects the main subject of the source image.
How It Works
We hook into the publish:itemProcessed event and check whether our 'Smart Cropped Image' field is populated. If not, we check a few defined fields for images. Once we find one, we send it to the Microsoft Vision API with a set of parameters: 200px width, 200px height, and smart crop enabled.
What the API Does
The API picks out the main subject of the image. It applies a crop matching the proportions we specified, then resizes the result. We end up with a smart 200px square image ready to send to Sitecore Search as a navigation image.
3. Keyword Analysis
Generating Keywords
In this example, we use a scheduled task to call a generative AI engine (via API) and ask it to return a comma separated list of keywords for a specific content type (i.e. New Article). A secondary call then asks for that list to be extended (i.e. add variations of keyword - write=> writes, writer).
Matching Content
We then have a custom tool, that when launched does a similar analysis of the given item and breaks it down into keywords. Those keywords are then mapped against the keywords of every processed content type (i.e. news article, banner images, navigation images). The GUI then allows the user to pick articles and images, from the most relevant content based on matching keywords.
4. Textual Analysis
We built a custom Sitecore module called the Language Assistant. It connects the Sitecore backend to a variety of generative AI models. We can perform standard AI tasks, but with the added ability to pass Sitecore item details as context.
Predefined Prompts
Out of the box, the module includes predefined prompts. We can use these to improve punctuation and grammar, for example.
Advanced Options
The advanced section lets us create custom prompts. We can pass in different fields from our Sitecore item for context. We can also reference a data source for background information, such as a list of business rules or style guides to apply to the text.
5. Use you own data - using the AI Overview module
What Is the AI Overview Module?
We built another module called the AI Overview Module for Sitecore. It calls the ChatGPT API, but we limit it to use only our own data.
Building the Data Source
We set up a scheduled task that crawls items of a certain template in a specific area of the content tree. The task trims and formats that data into well-structured HTML, then sends it to blob storage. A subsequent task periodically indexes this data into an Azure Search Index. That index becomes the data source we limit ChatGPT to use when providing answers.
Real-World Example
In the video below, we demonstrate a data source based only on modules offered by our university. We hooked the AI Overview Module up to the site search as a simple interface. Users can ask questions about the types of modules on offer and which courses contain modules on certain subjects.
We could easily create a dedicated component for this instead.
More information about Using Your Own Data can be found here.
In this blog post, we have looked at the first five examples of how we can use AI in our Sitecore processes. In the next article, we will look at another five examples.