This is the first in a series of articles that will explore the idea of creating a headless application, without the luxury of Sitecore Headless Services Module (Sitecore JSS). The main driver for doing this is that the module is not included in the perpetual license and our institution cant justify paying for it right now…. yet I am interested in the advantages that Headless might offer.
In this first article we will look at creating the layout service. This will serve as the only connection between our headless application and the data stored within Sitecore. To facilitate this we need to provide some endpoints that can be consumed in the front end application layer.
Creating the project
The layout service will we housed in a standard MVC project, full details of which can be found here: https://github.com/deanobrien/custom-layout-service-for-sitecore
The layout service comprises an API Controller with the following actions
- [HttpGet] Index (accessed by route /sitecore/api/layoutservice/get )
This is the main endpoint for the layout service – providing data for all routes. - [HttpPost] Secure (accessed by route /sitecore/api/layoutservice/secure )
This is provides the same info as the ‘Index’ route, but is only accessible to logged in users. The bearer token on each request enables user identification, so personalised info can be retrieved. - [HttpGet] StaticPaths (accessed by route /sitecore/api/staticpaths/get )
This is endpoint outputs a list of all paths using predefined page templates.
In order to make these available, we need to first register the routes. You can do this creating a processor as below:
public class RegisterHttpRoutes
{
public void Process(PipelineArgs args)
{
HttpConfiguration config = GlobalConfiguration.Configuration;
config.Routes.MapHttpRoute("LayoutService", "sitecore/api/layoutservice/get", new
{
controller = "LayoutApi",
action = "Index"
});
config.Routes.MapHttpRoute("SecureLayoutService", "sitecore/api/layoutservice/secure", new
{
controller = "LayoutApi",
action = "Secure"
});
config.Routes.MapHttpRoute("StaticPaths", "sitecore/api/staticpaths/get", new
{
controller = "LayoutApi",
action = "StaticPaths"
});
}
}And registering it with the following config:
<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
<sitecore>
<pipelines>
<initialize>
<processor patch:after="processor[@type='Sitecore.Pipelines.Loader.EnsureAnonymousUsers, Sitecore.Kernel']"
type="DeanOBrien.Feature.LayoutService.Pipelines.Process.RegisterHttpRoutes, DeanOBrien.Feature.LayoutService" />
</initialize>
</pipelines>
</sitecore>
</configuration>Building out the main endpoint
[HttpGet] Index
The Index endpoint is intended to provide general layout information about all paths within the sitecore content tree. The following parameters must be provided:
- language
- site
- path (to sitecore item)
- apiKey
A successful call to this endpoint will result in a LayoutServiceResponse object being returned. Within this is a Route object with the following properties:
public class Route
{
public string Name { get; set; }
public string DisplayName { get; set; }
public string ItemId { get; set; }
public string TemplateName { get; set; }
public string TemplateID { get; set; }
public string Path { get; set; }
public List<SimpleNavItem> Parents { get; set; }
public List<SimpleNavItem> Children { get; set; }
public List<SimpleNavItem> Siblings { get; set; }
public ExpandoObject Fields { get; set; }
public List<Component> Components { get; set; }
public Component Component { get; set; }
}As you can see, as well as the basic info for an item (i.e. DisplayName / Template etc), it also contains a collection of *Fields (i.e. page item fields) and Components (i.e. renderings).
* Note: The Fields are transformed into an ExpandoObject, as this makes it simpler to reference in NextJs application (i.e. pageItem.Fields.ItemX.FieldY rather than having to filter an array).
A variety of optional parameters are available to tailor the return. The intention with these is to allow the caller to reduce the size of the response to only what is required. Parents, Children and Siblings are return only if an optional flag (i.e. includeParents – default false) is included in the call. In a similar manner, the Fields property can be ommitted if the optional includeFields (default true) is set to false. Finally, if componentId and componentDataSourceId are included, only the specified component is returned.
How does it work?
Basic Info
To begin, we initialize our settings by taking the ‘site’ parameter and building a SiteContext. From this, we can scope to the correct database and also find the root of the site (i.e. StartPath).
_siteContext = SiteContextFactory.GetSiteContext(site);
_database = _siteContext.Database;
_root = _siteContext.StartPath;
_rootItem = _database.GetItem(_root);We then combine the _root with the path parameter to get the target item.
var item = _database.GetItem($"{_root}/{path}");Note: by doing this we restrict access to items that are children of the start path (i.e. protecting system items)
The AddRoute() method then instantiates the new Route object and populates the basic information from the returned item. If any of the optional flags are set for Parents, Children or Siblings, we grab the relevant items and transform them into a collection of SimpleNav objects (we do this to reduce the amount of data returned – also if we tried returning a sitecore Item, it would result in a stackoverflow exception during serialisation).
Where to find the data
Once our response object has been setup and basic info added, we can start to populate the fields and components. In sitecore, these values can come from a range of places, depending on how the template for the page item has been created. For each of the locations below, we check and add components and fields to our collections
- First we look at each of the base templates for the pages template
- check its standard values
- check the base template item
- Then look at the page templates standard values
- Finally we look at the page item itself
As we iterate through each of the locations above, if a field or component has already been added, then the value or its properties are updated. By following this pattern, we mimic Sitecores own logic, so that fields set at page level have a higher precedence than those set in a standard value or base template.
Fields
The Fields property of our response object is an ExpandoObject. These types of object are very useful for defining something which we dont know the structure of at build time. By using one of these, we make the consumption of the object far simpler in the presentation layer (i.e. pageItem.Fields.ItemX.FieldY rather than having to filter an array).
The AddFields method takes in an ExpandoObject and casts it to a IDictionary so that we can use Add, ContainsKey and Remove methods to manage the collection. It also takes in a sitecore Item as an input parameter and cycles through each item in the Fields collection and updates the IDictionary accordingly. At its simplest, this process checks to see if the key exists and removes it. Then adds the new value with same key – thus overwriting values coming from sources with less precendence.
However, if the type of field being processed is one of a special type (i.e. general link, image or droplink – these are currently the only ones covered, need to add more), then a new ExpandoObject is created to house the special field to accomodate the additional list of properties. In the case of a droplink, this process could continue indefinitely.
private static void AddFields(Item item, ExpandoObject result)
{
if (item !=null && item.Fields != null)
{
var fields = result as IDictionary<string, Object>;
for (int i = 0; i < item.Fields.Count; i++)
{
var key = item.Fields[i].Key.Replace(" ", "");
if (ignoreList.Contains(key)) continue;
var value = item.Fields[i].Value;
if (item.Fields[i].Type == "Image" || item.Fields[i].Type == "Droplink" || item.Fields[i].Type == "General Link") {
var newValue = new ExpandoObject();
var f = newValue as IDictionary<string, Object>;
var newInnerFields = new ExpandoObject();
if (item.Fields[i].Type == "Image")
{
var imageFieldItem = (ImageField)item.Fields[i];
if (imageFieldItem?.MediaItem == null)
continue;
AddPropertyToExpando("Url", MediaManager.GetMediaUrl(imageFieldItem.MediaItem), newInnerFields);
AddPropertyToExpando("Alt", imageFieldItem.Alt, newInnerFields);
AddPropertyToExpando("Height", imageFieldItem.Height, newInnerFields);
AddPropertyToExpando("Width", imageFieldItem.Width, newInnerFields);
}
else if (item.Fields[i].Type == "General Link")
{
var linkField = (LinkField)item.Fields[i];
if (string.IsNullOrWhiteSpace(linkField?.Value))
continue;
AddPropertyToExpando("Url", (linkField.TargetItem != null) ? LinkManager.GetItemUrl(linkField.TargetItem) : linkField.Url, newInnerFields);
AddPropertyToExpando("Text", linkField.Text, newInnerFields);
AddPropertyToExpando("Target", linkField.Target, newInnerFields);
}
else if (item.Fields[i].Type == "Droplink")
{
var linkedItem = _database.GetItem(value);
AddFields(linkedItem, newInnerFields);
}
f.Add("Fields", newInnerFields);
newValue = f as ExpandoObject;
fields.Add(key, newValue);
} else {
value = CleanIfRichText(item, i, value);
if (((IDictionary<String, object>)fields).ContainsKey(key))
{
((IDictionary<String, Object>)fields).Remove(key);
}
fields.Add(key, value);
}
}
result = fields as ExpandoObject;
}
}Components
In order to extract and add the components to our collection, we need to look at both the renderings and final renderings fields (in each of the locations specified above). These fields store the component data in XML format, so to access this information we use System.Xml.XmlDocument object and its related classes.
A typical XML document might look like this:
<r xmlns:p="p" xmlns:s="s" p:p="1">
<d id="{FE5D7FDF-89C0-4D99-9AA3-B5FBD009C9F3}">
<r uid="{913C47A1-AF13-434D-9F59-3B00C16138C4}" p:after="p[@uid='{7945BDB1-E899-4C42-B876-7214BC1F9351}']" s:ds="{93096042-A629-4E75-BED9-472E0903981E}" s:id="{6D518D1D-8137-4FF8-AB0F-7670EAB00375}" s:pt="{313020D7-69ED-414C-A76C-A46E72207A96}" s:par="Title&Subtitle&Description&Header Links&Show Divider&Number Of Columns&Spacing Between Columns&Customise Layout&Override Width&Image Priority" s:ph="/Content/Section-{16523EC7-7BCB-4209-88D0-1099A80DCDA8}-0/Row-{C9EEAE74-E78D-4D5D-9CCC-BB00C28DA7AB}-0/Column-{361E5664-E300-481F-9490-EF0C1310E2B7}-0" s:ccb="Clear on publish">
.... many more .... The first line outlines the names spaces being used within the document and the second line specifies which device (device id links to device in content tree) its child nodes are being used to define presentation for. All direct children (starting ‘r’) are then the renderings for the page. The order of these renderings is important, if one comes before another and they are to be displayed in the same placeholder, then it will be displayed first (unless the p:after attribute is defined, in which case it is displayed after the rendering with that uid). For the purposes of this service, we are only interested in these renderings (i.e. direct child nodes).
Note: These nodes can have child nodes themselves, but they are used to define things like personalisation rules etc, which we are not focusing on at this point in time.
The ExtractComponentsFromXML method extracts a list of renderings from the XML and cycles through them. For each rendering, it creates a new Component object and adds the following properties (if populated).
- uid – this is the unique identifier to a component (rendering)
- id – this is the id of the rendering item (i.e. controller rendering) in sitecore. (referred to as s:id in final renderings)
- ds – this is the datasource id for the component (referred to as s:ds in final renderings)
- ph – this is the placeholder in which to display the component (referred to as s:ph in final renderings)
- par – this parameter stores the parameters in query string format (referred to as s:p in final renderings)
- p:after – this is used to augment the natural ordering of the renderings
After extracting these values, we populate the name field by finding the display name of the component definition in the content tree and removing any spaces. We then check to see if a component with the same uid exists. If it doesnt exist, we add it – otherwise we add/overwrite the properties above. Finally, we check to see if the p:after property has been populated and adjust its location within the collection.
Further defining components
The ExtractComponentsFromXML method has been ran from each location and our collection of components has been populated and ordered. We then look to add further details to allow the component to be displayed on the front end.
For each component, we check to see if a datasource has been defined. If so, we then create an ExpandoObject to house all of the fields from the datasource item (as we did for the page level fields).
Each Component object also contains a CustomViewModel property of type object. The purpose of this property is to pass additional information that the previous MVC implementation required to the view. We can use a ComponentHelper to check the id of the component (i.e. the rendering definition id) and conditionally retrieve data (most likely using the code from previous implementation).
Examples of when you might want to use this might be:
- Navigation component – that uses a variety of lists obtained from across the site – this information may not be so easy to populate using the standard datasource approach.
- Custom data component – a component that relies on external data
- Secure component – for example something that displays user specific data (i.e. messages – see: Secure endpoint)
With this additional data added to the response object, the data is then returned to the caller in Json format.
Additional Endpoints
[HttpPost] Secure
The main difference between the main Index and the Secure endpoint is that the later is labelled with the [Authorize] attribute. By adding this attribute, we restricts access to only those logged in via Identity Server.
In order to call the endpoint, the calling application needs to first authenticate against Identity Server and retrieve an access token (there are a couple of ways of doing this – which I will elaborate on when discussing the front end NextJs Application). That token needs adding (as bearer token) to future requests to confirm they are authorized to access the resource.
A successful call to the endpoint will populate a SitecoreContect.User.Identity object, from which we can access the claims sent in the bearer token. Example below:
if (Sitecore.Context.User.Identity is ClaimsIdentity)
{
var claimsIdentity = Sitecore.Context.User.Identity as ClaimsIdentity;
return claimsIdentity.Claims.Where(c => c.Type == "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier").FirstOrDefault().Value;
}Using the above method, we can confirm the identity of the caller and use that information to populate user specific data. As discussed this Secure endpoint follows the exact same steps as the [HttpGet] Index version, but when it comes to the final stages of populating the CustomViewModel, we have a populated User.Identity which can be checked within the ComponentHelper and access to the correct information provided.
[HttpGet] StaticPaths
This last endpoint has been specifically created to support the development of a NextJs application. By providing a list of all the possible paths that a user might navigate too, the NextJs application can fetch the data for each path at build time and statically prerender the pages (there are a variety of ways to refresh / revalidate this data if required).
The code for the endpoint is very straight forward. We simply trawl through every descendant of the root item and select every one that uses one of our predefined page templates. We remove the unnecessary part of the path and then add it to our response object (list of strings). That is then returned as Json.
Summary
In this article, we have looked at how you might build a custom layout service. Pulling in all the information about a page item and the different renderings stored in the presentation details. With this information, we should be able to the build a front end application to consume the layout service.