CRO with Sitecore

So you have Sitecore. At some point in the past, your organisation bought into the inspiring vision of an all-singing, all-dancing marketing platform. With in-built personalisation, analytics at your fingertips, and much more, it was an exciting investment.

Now fast forward X (insert your number) years and maybe you haven’t managed to fully take advantage of the marketing features that originally sold the platform to you. There’s pressure from above to show ROI, but you are not quite sure how to deliver.  

If any of this sounds familiar, then read on. We’ve got your back.  Our approach will help you to: 

• Increase speed of testing and experimentation 
• Quickly spin up new components and features on the site in hours rather than days 
• Free yourself from the limitations of page and component structures 
• Focus development spend and reduce costs 
• See rapid ROI gains 

Now before we get started, full disclosure, we love Sitecore and have been working with it for a long time and it’s one of the strings to our bow. We’re also very good at conversion rate optimisation (with an experiment win rate 4 x the national average). We’ve spent a lot of time refining our approach to CRO on Sitecore websites.  

So, what have we found?
 

Our two-track CRO approach to delivering rapid ROI with Sitecore

We’ve had great success delivering ROI on CRO programmes on Sitecore websites and believe that a two-track approach brings the greatest (and fastest) ROI and reduces development spend. Through this approach, we combine client-side A/B testing along with server-side A/B testing and personalisation. 

Client-side testing is faster and cheaper, so we use it to validate ideas and making the business case for development before building them within the CMS. Server-side testing is then used to refine those ideas further, apply personalisation, etc. The methodology behind the two tracks is similar, but each plays to the strengths of the testing platforms being used: 

Track 1: Client-side testing

• Hypotheses backlog creation 
• Rapid delivery of simple A/B tests into digital channels regardless of existing page structures 
• Refine in-flight and report learnings 
• Successful experiments scoped and built within Sitecore and moved to Track 2 
• Continuous experimentation in fortnightly or monthly sprints 

Track 2: Server-side testing

• Hypotheses backlog creation 
• More complex A/B and personalisation testing, leveraging customer profiling and other data sources to experiment with more targeted experiences within the available page structures and components 
• Refine in-flight and report learnings 
• Continuous experimentation in fortnightly or monthly sprints 

But what do we mean by server-side or client-side testing, I hear you say? 

What is client-side testing?

Client-side testing involves a third-party tool over-writing the content served by your CMS with alternative versions. Examples of these tools include Optimizely, VWO, convert.com, AB Tasty or one of the many other testing tools. Sitecore also has its own platform called “Personalize” which is a CMS-agnostic CRO tool, separate to the in-built XP functionality. To do this, you include a small JavaScript code snippet on the pages that you want to test.

What is server-side testing? 

While client-side testing happens in the browser via JavaScript, server-side testing (and personalisation) is handled at the web application/server level, making the changes before the user receives the content in their browser.  

Should you choose server-side or client-side A/B testing?

Each technique has its place, and they have different pros and cons.

Pros of client-side testing

• Quick and easy to setup 
• Lower investment needed 
• Great for testing look and feel, CTAs and smaller elements 
• Can run tests without being constrained by component limitations, so can test ideas before investing in the development of new components and features 
• Can often be created without the need for a developer, therefore reducing costs and time 
• Can often achieve a higher test velocity as it is really easy to set up lots of tests

Cons of client-side testing

• Requires a third-party tool (although some have free accounts to get you started) 
• As the JavaScript needs to load before the change can be seen by the user, it can add a small amount of latency, often called the flicker effect. There are things that can be done to limit this though  
• Limited to UI/UX testing 
• Can impact page speed if the JavaScript is added to every page 
• Operates outside of governance workflows for other content  
• Typically relies upon the HTML structure of the page remaining the same; so, changes to content mid-way through a test can break it. 

Pros of server-side testing

• Can be used for more complex use cases and can theoretically ‘test anything’, including features that rely upon APIs like for pricing, changes in back-end logic, for example product recommendations or search algorithms.  
• In a properly implemented component structure, can personalise any part of the page 
• Can run omnichannel experiments 
• No flicker effect as the code is handled at server level, not browser, so can offer a better user experience. 

Cons of server-side testing

• Often needs to be supported by a developer (e.g. custom rules, data connections or creation of new components), therefore can be more expensive and slower to setup 
• Can be subject to your code release/deployment cycles 
• As the marketers sometimes can’t create the tests themselves, it can make smaller or more short-lived changes harder to justify. 
• Can have a lower velocity testing programme and it can take longer to build the tests. 

So you can see that neither server-side nor client-side testing offer a CRO panacea. They are both complementary and we believe there is a strong case for using both in parallel.

Ratio’s approach to CRO with Sitecore

Following a full discovery and data review, we will create a backlog of test ideas. You can read more about that here.Then, with the client-side tool such as Optimizely, VWO, convert.com, Sitecore Personalize etc: 

• Quickly build and deploy UX/UI client-side tests. We can use this to refine the user journey and reduce friction points, all with the aim of increasing conversion rates (or whichever metric you are targeting) jumping straight into delivering that precious ROI as soon as possible.  
• We can also test new business priorities that you have received to sanity test, refine changes and predict the longer-term ROI before you go to the expense of having them developed.  
• The test metrics and results can be tracked, and ROI measured allowing you to make decisions about which tests you go on to have developed and added permanently to your site.  

As these tests are generally easy to implement, we can build up a good test velocity helping to increase your culture of experimentation. 

Alongside the above, we use Sitecore to: 

• Implement more complex testing and personalisation in existing components 
• Integrate data sources, e.g. CRM to drive more targeted personalisation.
• Do the development for the successful tests from both the client-side and server-side experiments. 

Throughout the programme we track and measure success and support you in reporting the improved ROI upwards.  

See? Told you we had your back.  

Our two-track approach is a real winner which:

• realises the benefits of both server and client-side testing 
• delivers a more efficient, low-cost approach 
• allows you to rapidly test new features to drive ROI 
• allows you to better focus development spend, making your budget work harder for you 

Who wouldn’t want that? 

Want to talk further?

If you’d like to talk to us about how you can introduce an experimentation programme to your business, then give us a shout.