This blog post is part of a larger series that looks at the steps to create a
Page Recommendation Engine for Sitecore.
Introduction
Now that we have trained our machine learning model (see this article for more info), we need to select a group of contacts and ask our recommendation service for recommendations.
Projection
To do this, we will create a projection task in exactly the same way as we did on the previous page. However, we will make a slight change to the datasource query, to only bring back interactions for the last 24hrs.
var query = xConnectClient.Interactions.Where(interaction =>
interaction.Events.OfType<Goal>().Any()
&& interaction.Events.Any(ev => _eventsList.Contains(ev.DefinitionId)) // list of goals we are interested in
&& interaction.StartDateTime > DateTime.Now.AddHours(-24) // last 24 hours
);In our InteractionProjectionWorkerOptionsDictionary, we will specify a new IModel and also a new schema generate-recommendation and table name contact (so as not to confuse with the previous training task). The projection task is registered in exactly the same way as before.
var projectionOptions = new InteractionProjectionWorkerOptionsDictionary(
typeof(ContactModel).AssemblyQualifiedName, // modelTypeString
storageTimeout, // timeToLive
"generate-recommendation", // schemaName
new Dictionary<string, string> // modelOptions
{
{ ContactModel.OptionTableName, "contact" }
}
);In our newly defined IModel, ContactModel, we will define a slightly different projection. Because we are only interested in selecting a unique set of contact Ids from the data we take from XConnect.
public IProjection<Interaction> Projection =>
Sitecore.Processing.Engine.Projection.Projection.Of<Interaction>()
.CreateTabular(_tableName,
interaction => new { ContactId = interaction.Contact.Id},
cfg => cfg
.Key("ContactId", x => x.ContactId)
);Merging
Our merge task will be nearly identical to our previous one, but instead we will define a different schema and set of table names to use (i.e. merge all contact tables into the contactFinal table).
var mergeOptions = new MergeWorkerOptionsDictionary(
"contactFinal", // tableName
"contact", // prefix
storageTimeout, // timeToLive
"generate-recommendation" // schemaName
);Recommendations
We call the service in the exact same way as before, by first of all defining a new PageRecommendationWorker (implements IDeferredWorker) with a view to using the RunAsync to do the work.
To register the job, we define a DeferredWorkerOptionsDictionary as below, passing in the new model, schema and table names. Note however that we define a new table to store the results from the recommendation service (i.e. contactRecommendations), we then register the task in the exact same way:
var recommendationOptions = new DeferredWorkerOptionsDictionary(
typeof(PageRecommendationWorker).AssemblyQualifiedName, // workerType
new Dictionary<string, string> // options
{
{ PageRecommendationWorker.OptionSourceTableName, "contactFinal" },
{ PageRecommendationWorker.OptionTargetTableName, "contactRecommendations" },
{ PageRecommendationWorker.OptionSchemaName, "generate-recommendation" },
{ PageRecommendationWorker.OptionLimit, "5" }
}
);
var recommendationTaskId = await taskManager.RegisterDeferredTaskAsync(
recommendationOptions, // workerOptions
new[] // prerequisiteTaskIds
{
mergeTaskId
},
taskTimeout // expiresAfter
);Again the bulk of the work is done in the RunAsync method of our custom model. Here you can see we read each contact Id from the contactFinal table, and call the ML predict method with that contact ID and each available page (stored in _pageIds - which is not shown for berevity). These results are then sorted and the top 5 taken for each contact. That data is then stored in contactRecommendations table for future use.
public async Task RunAsync(CancellationToken token)
{
var sourceRows = await _tableStore.GetRowsAsync(_sourceTableName, CancellationToken.None);
var targetRows = new List<DataRow>();
var targetSchema = new RowSchema(
new FieldDefinition("ContactId", FieldKind.Key, FieldDataType.Guid),
new FieldDefinition("PageIds", FieldKind.Key, FieldDataType.String),
new FieldDefinition("Score", FieldKind.Key, FieldDataType.String)
);
while (await sourceRows.MoveNext())
{
foreach (var row in sourceRows.Current)
{
var results = new List<Result>();
foreach (var crseId in _pageIds)
{
var result = new Result() { pageId = crseId.ToLower() };
result.score = _machineLearning.Predict(row["ContactId"].ToString().ToLower(), crseId.ToLower());
results.Add(result);
}
foreach (var page in results.OrderByDescending(x => x.score).Take(5))
{
var targetRow = new DataRow(targetSchema);
targetRow.SetGuid(0, new Guid(row["ContactId"].ToString()));
targetRow.SetString(1, page.pageId);
targetRow.SetString(2, page.score.ToString());
targetRows.Add(targetRow);
}
}
}
// Populate the rows into the target table.
var tableDefinition = new TableDefinition(_targetTableName, targetSchema);
var targetTable = new InMemoryTableData(tableDefinition, targetRows);
await _tableStore.PutTableAsync(targetTable, TimeSpan.FromMinutes(30), CancellationToken.None);
}Store Recommendations
Finally we register one final task to store the recommendations. In a similar manner to the above we create another custom RecommendationFacetStorageWorker (implements IDeferredWorker). Which in turn calls the RunAsync method below (as expected the table and schema are passed in via the usual method).
Here we call XConnect to get each contact. Check its current recommendation facets, remove the oldest if more than 4, then add each new facet. If any recommendations exist already, but have a different score, we update.
public async Task RunAsync(CancellationToken token)
{
var rows = await _tableStore.GetRowsAsync(_tableName, CancellationToken.None);
while (await rows.MoveNext())
{
foreach (var row in rows.Current)
{
var contactId = row.GetGuid(0);
var pageId = row.GetString(1);
var score = row.GetString(2);
var contact = await _xdbContext.GetContactAsync(contactId,
new ContactExecutionOptions(new ContactExpandOptions(PageRecommendationFacet.DefaultFacetKey)));
var facet = contact.GetFacet<PageRecommendationFacet>(PageRecommendationFacet.DefaultFacetKey) ??
new PageRecommendationFacet();
if (facet.PageRecommendations.All(x => x.PageId != PageId))
{
if (facet.PageRecommendations.Count > 4)
{
facet.PageRecommendations.OrderBy(x => x.DateRecommended);
facet.PageRecommendations.RemoveAt(facet.PageRecommendations.Count - 1);
}
facet.PageRecommendations.Add(new PageRecommendation()
{
PageId = pageId,
Score = Double.Parse(score),
DateRecommended = DateTime.Now
});
_xdbContext.SetFacet(contact, PageRecommendationFacet.DefaultFacetKey, facet);
await _xdbContext.SubmitAsync(CancellationToken.None);
}
else if (facet.PageRecommendations.Any(x => x.PageId == pageId) && facet.PageRecommendations.Where(x => x.PageId == pageId).FirstOrDefault().Score != Double.Parse(score))
{
var toRemove = facet.PageRecommendations.Where(x => x.PageId == pageId).FirstOrDefault();
facet.PageRecommendations.Remove(toRemove);
facet.PageRecommendations.Add(new PageRecommendation()
{
PageId = pageId,
Score = Double.Parse(score),
DateRecommended = DateTime.Now
});
_xdbContext.SetFacet(contact, PageRecommendationFacet.DefaultFacetKey, facet);
await _xdbContext.SubmitAsync(CancellationToken.None);
}
}
}
await _tableStore.RemoveAsync(_tableName, CancellationToken.None);
}Summary
With this task complete, we should now have a selection of contacts in XConnect with custom Recommendation Facets. This data can then be used to personalise components on the website in the usual ways.
Thanks for taking the time to read my series of blogs on How to create a Page Recommendation Service for Sitecore. If you are interested in the code that accompanies this series of blog posts, you can find it on my GitHub Repository here:
https://github.com/deanobrien/page-recommender-for-sitecore