Engagement - if you need a definition, here it is

Engagement has been a buzzword in web analytics for probably a few years now. It came along with those other buzzwords like Web 2.0 , RSS, Rich media etc.

Currently people still seem to be banding the term around and trying to find a definition for it so they can impress their bosses. So here it is.

Measuring engagement is the process of segmenting your customer base by the activities they perform and counting how many customers fall into each segment.

There, that wasn't so hard was it?

The key issues so far with engagement has been similar to those that have bedevilled other terms in the online arena, simply that for different websites, different things will be positive and negative. Examples of this would be "stickiness" and "Time spent on site" for which an increase in either can be a positive or negative metrics depending on the goal of the site.

So why has engagement hung around and is it really different as a concept to anything else before it?

I dont think its new in marketing but one thing I would say is that it is something new for the web analytics vendors out there and thats why its creating a buzz.

On the whole the web analytics vendors are still used to dealing with metrics based on site activity - X many visits to this section, Y number of purchases in that category etc. What they are not used to is measuring things from a customer or user perspective and thats what engagement is all about. For example, knowing that you get 12 leads in 100 visits is not the same as knowing that you have two customers who provided 6 leads each.

So lets look at that definition again.

Measuring engagement is the process of segmenting your customer base by the key activities they perform and counting how many customers fall into each segment.

Lets hit the main problem head on. The activities you chose to measure within engagement will depend on your site, your goals and your business - I cannot tell you them, other than to say you will probably have defined what these should be in previous discussions regarding the equally vague "success events".

Next problem is how do you segment your user base? Well again, you can do all sorts of clever stats involving dozens of metrics and fancy algorithms talking about "distance from cluster centres", "neighbourhoods of attraction" etc. here but lets keep it simple.

You could for a given key metric

a) Set some arbitrary thresholds up, such as "any customer who posts more than 3 times is call an engaged user"
b) Look at the spread of activity and split customers into significant groups such as "we have a 100 customers who post anywhere between 0 and 100 times a month, lets sort the customers by their posting activity and split those customers into quartiles (4 equal groups) and then see what the thresholds exist and label the groups Highly engaged, Quite engaged, Slightly engaged and Not engaged at all" - (note if this example were real I would probably have a separate segment for all those customers who never post and then split those who posts between 1-100 times into the quartiles)

If you have multiple metrics, then simply add them together for each individual customer creating a score and then again chose approach a) or b).

Once you have your groups defined and labeled what next? Yes, the counting. Essentially, over time you need to see whether more customers are falling into the higher "engaged" segments. If they are, then great! They are more engaged and you have succeeded! If they haven't then you need to look at what things you can do to encourage customers to move up into a more valuable segment.

The next step after measuring engagement would be to look at "engagement over time" which essentially means tracking customers as they move from one segment to another and trying to see whether you can define their expected engagement over their complete customer lifetime.

The key thing is "engagement" is all about measuring things from the user perspective, not the site. This is something understood in other areas of customer analytics but we are still waiting for some bright spark to realise this and market it under some other clever buzzword (like "personas") and then for the web vendors to catch up.

No customer has ever asked me to validate their business plan

When I go and meet customers and try to gather requirements I always ask questions like "What is the web site for?" and I get a variety of answers but never a clear predefined plan of what is meant to be achieved.

I find this bemusing. I believe that every business venture requires a business plan and yet I see neither hide nor hair of them in my day to day dealings.

If we are talking about a whole site, then obviously business cases would be proposed at a high level "increased sales by X" etc. but I rarely see these plans and if they exist at a 'C' level they rarely filter down to the people who have to implement these things and therefore rarely by default into the analytics requirements.

This isn't just a high level problem. Whenever a new piece of functionality has to be added to a site, someone, somewhere has to make a business case for it but again, so rarely do I see this case when requirements gathering. Admittedly I work for analytics vendor, it could be argued I don't need to know, I just have to explain to the client "how" to measure things with the tool my company provides but I don't think thats the case. More often than not I'm asked my opinion on "what" should be being measured implying there isn't a clear business case stating what hopes to be improved by the additional functionality.


I do think there needs to be more clarity on higher level goals within organisations in general. Every time there is a planned change it should be communicated across the company whether it hopes to improve sales, leads, "engagement" etc. and over what period of time the improvement is being compared to. When doing this we must always take into account the seasonal difference, market differences, increased competition etc.

One of the other facets of this lack of clarity is that a site or a piece of functionality isn't usually designed to increase a single metric. Its usually there to attract and engage with a particular type of customer - maybe it's there to appeal to a younger audience, maybe mainly women, maybe mainly elderly couples looking for that cruise of a life time, whatever, there is usually a concept of any ideal customer.

The point is, if such attributes have been made clear in building the product or functionality, consideration needs to be given to how you measure the targeting of these populations. Is the site really attracting the kinds of people you expect for your product? Are they really performing as you'd expect? It amazes me how often these kind of considerations get lost between the creative product development department and the department in charge of actually making things happen online. I think this is often a reflection of the business goals not being communicated around the company and the creative team not taking responsibility to prove the benefit of their fabulous new ideas.

So what things should people be considering? Well often its a question of "To what segment is this new thing supposed to appeal?" and the challenge is to get a baseline of that "segment" performance now and after go live of the new technology. (I'm ignoring the concept of MVT right now because you dont always have the traffic or capability to test everything in isolation).

In order to get information on that segment think about the following

• Look at the demographics of people purchasing/registering (i.e. use the existing methods of data collection for analysis.
• collect ad-hoc survey information and tie that into user behaviour.
  
• Implement some additional functionality that encourages input of information you know is correlated with your segment. (i.e. teenagers are unlikely to respond to competitions for stairlifts).
• Analyse targeted marketing of your segment and compare to the random sample of marketing
• Buy data from other agencies for enrichment if it can be tied to your own.
  

These are all things that in my opinion should be considered from the beginning of any new venture. Remember, the simple question is

"What are we hoping to achieve and how are going to measure it??

And once your organisation asnwers that for any piece of functionality this should be communicated to all involved in the venture so that even if the specific details are not there all those involved are aware of the goal they are working towards giving much more focus to the
overall project.

How do you encourge your staff to improve your website?

Pay them to do so of course.

Last year I read an interesting book, Freakonomics where the author makes explicit many of the things regarding human motivation you probably implicitly knew in a very clear and entertaining way. This lead me to thing about the way my salary had been set in the past along with the targets of my colleagues.

One of the problems with asking your staff to improve your website is that just as you have trouble defining what is regarded as KPIs for your organisation, you then have to try and tie these things to individuals salary and targets.

An example of one thing I once saw was that the online sales people were given targets to increase sales and conversion as well as continuing to maintain the site with the regular BAU changes.

Increasing sales could be tough. In this situation, sales were separate to marketing and also to product marketing. All they really had within their immediate power was to change the website. One would think this a good thing, for them to increase sales there would need to be enhancements to the website.

However, two things were noticed by these individuals

• whenever a product was competitively priced in the market place, the online sales and conversion would dramatically increase regardless of changes to the website.
• whenever marketing made a big splash, sales would also increase but usually to the detriment of conversion.

The upshot of this was that they concentrated more on lobbying product marketing to produce "competitive products" as it was simply the easiest way for them to hit their targets than almost anything they could do onsite. If product marketing shifted in their favour, they would easily hit their targets, if they didn't, well they could hope that the general trend for increasing sales online would continue or else blame the "competitive landscape" and hope things would be better next year.

The point I'm getting to is that though their targets were directly tied to a key "KPI" for a retail website - its sales (and even its conversion), nothing about that KPI actually encouraged this team to do anything to improve the company website - nothing spelled it out for them.

Had it been my team, I would have liked to acknowledge that product sales can be price sensitive and also sensitive to marketing however this should not have stopped the team from being encouraged to change their site as much as possible during the year. I would have set lower thresholds for sales and conversion i.e. they must not go below X where X is linked to a combination of previous sales in previous period, offline sales of the same period (to provide indication of market performance) with the empahasis that they must have conducted a series of Y trials during the period as well as BAU changes.

The size of the trials is probably up for debate, obviously a bigger trial (on the homepage or registration form) could have a bigger negative impact on sales than a smaller piece of functionality but I would actually wish to encourage calculated risk taking through a formal test process and so would set an individuals size of trial and sales thresholds dependent on what contribution I believe their area plays on sales.

I would then assess the staff additionally from the perspective of the ease/number of tests
they performed, i.e. are they doing it effectively and efficiently combined with the revealing of any particularly useful insights. (things to consider would be number of tests, size of trials, impact of area changed)

So what I would be doing is encouraging a workplace where new ideas go hand in hand with accountability and individuals feel empowered to make a difference rather than having a lack of focus to update and improve their website.