Engagement - if you need a definition, here it is

Engagement has been a buzzword in web analytics for probably a few years now. It came along with those other buzzwords like Web 2.0 , RSS, Rich media etc.

Currently people still seem to be banding the term around and trying to find a definition for it so they can impress their bosses. So here it is.

Measuring engagement is the process of segmenting your customer base by the activities they perform and counting how many customers fall into each segment.

There, that wasn't so hard was it?

The key issues so far with engagement has been similar to those that have bedevilled other terms in the online arena, simply that for different websites, different things will be positive and negative. Examples of this would be "stickiness" and "Time spent on site" for which an increase in either can be a positive or negative metrics depending on the goal of the site.

So why has engagement hung around and is it really different as a concept to anything else before it?

I dont think its new in marketing but one thing I would say is that it is something new for the web analytics vendors out there and thats why its creating a buzz.

On the whole the web analytics vendors are still used to dealing with metrics based on site activity - X many visits to this section, Y number of purchases in that category etc. What they are not used to is measuring things from a customer or user perspective and thats what engagement is all about. For example, knowing that you get 12 leads in 100 visits is not the same as knowing that you have two customers who provided 6 leads each.

So lets look at that definition again.

Measuring engagement is the process of segmenting your customer base by the key activities they perform and counting how many customers fall into each segment.

Lets hit the main problem head on. The activities you chose to measure within engagement will depend on your site, your goals and your business - I cannot tell you them, other than to say you will probably have defined what these should be in previous discussions regarding the equally vague "success events".

Next problem is how do you segment your user base? Well again, you can do all sorts of clever stats involving dozens of metrics and fancy algorithms talking about "distance from cluster centres", "neighbourhoods of attraction" etc. here but lets keep it simple.

You could for a given key metric

a) Set some arbitrary thresholds up, such as "any customer who posts more than 3 times is call an engaged user"
b) Look at the spread of activity and split customers into significant groups such as "we have a 100 customers who post anywhere between 0 and 100 times a month, lets sort the customers by their posting activity and split those customers into quartiles (4 equal groups) and then see what the thresholds exist and label the groups Highly engaged, Quite engaged, Slightly engaged and Not engaged at all" - (note if this example were real I would probably have a separate segment for all those customers who never post and then split those who posts between 1-100 times into the quartiles)

If you have multiple metrics, then simply add them together for each individual customer creating a score and then again chose approach a) or b).

Once you have your groups defined and labeled what next? Yes, the counting. Essentially, over time you need to see whether more customers are falling into the higher "engaged" segments. If they are, then great! They are more engaged and you have succeeded! If they haven't then you need to look at what things you can do to encourage customers to move up into a more valuable segment.

The next step after measuring engagement would be to look at "engagement over time" which essentially means tracking customers as they move from one segment to another and trying to see whether you can define their expected engagement over their complete customer lifetime.

The key thing is "engagement" is all about measuring things from the user perspective, not the site. This is something understood in other areas of customer analytics but we are still waiting for some bright spark to realise this and market it under some other clever buzzword (like "personas") and then for the web vendors to catch up.

No customer has ever asked me to validate their business plan

When I go and meet customers and try to gather requirements I always ask questions like "What is the web site for?" and I get a variety of answers but never a clear predefined plan of what is meant to be achieved.

I find this bemusing. I believe that every business venture requires a business plan and yet I see neither hide nor hair of them in my day to day dealings.

If we are talking about a whole site, then obviously business cases would be proposed at a high level "increased sales by X" etc. but I rarely see these plans and if they exist at a 'C' level they rarely filter down to the people who have to implement these things and therefore rarely by default into the analytics requirements.

This isn't just a high level problem. Whenever a new piece of functionality has to be added to a site, someone, somewhere has to make a business case for it but again, so rarely do I see this case when requirements gathering. Admittedly I work for analytics vendor, it could be argued I don't need to know, I just have to explain to the client "how" to measure things with the tool my company provides but I don't think thats the case. More often than not I'm asked my opinion on "what" should be being measured implying there isn't a clear business case stating what hopes to be improved by the additional functionality.


I do think there needs to be more clarity on higher level goals within organisations in general. Every time there is a planned change it should be communicated across the company whether it hopes to improve sales, leads, "engagement" etc. and over what period of time the improvement is being compared to. When doing this we must always take into account the seasonal difference, market differences, increased competition etc.

One of the other facets of this lack of clarity is that a site or a piece of functionality isn't usually designed to increase a single metric. Its usually there to attract and engage with a particular type of customer - maybe it's there to appeal to a younger audience, maybe mainly women, maybe mainly elderly couples looking for that cruise of a life time, whatever, there is usually a concept of any ideal customer.

The point is, if such attributes have been made clear in building the product or functionality, consideration needs to be given to how you measure the targeting of these populations. Is the site really attracting the kinds of people you expect for your product? Are they really performing as you'd expect? It amazes me how often these kind of considerations get lost between the creative product development department and the department in charge of actually making things happen online. I think this is often a reflection of the business goals not being communicated around the company and the creative team not taking responsibility to prove the benefit of their fabulous new ideas.

So what things should people be considering? Well often its a question of "To what segment is this new thing supposed to appeal?" and the challenge is to get a baseline of that "segment" performance now and after go live of the new technology. (I'm ignoring the concept of MVT right now because you dont always have the traffic or capability to test everything in isolation).

In order to get information on that segment think about the following

• Look at the demographics of people purchasing/registering (i.e. use the existing methods of data collection for analysis.
• collect ad-hoc survey information and tie that into user behaviour.
  
• Implement some additional functionality that encourages input of information you know is correlated with your segment. (i.e. teenagers are unlikely to respond to competitions for stairlifts).
• Analyse targeted marketing of your segment and compare to the random sample of marketing
• Buy data from other agencies for enrichment if it can be tied to your own.
  

These are all things that in my opinion should be considered from the beginning of any new venture. Remember, the simple question is

"What are we hoping to achieve and how are going to measure it??

And once your organisation asnwers that for any piece of functionality this should be communicated to all involved in the venture so that even if the specific details are not there all those involved are aware of the goal they are working towards giving much more focus to the
overall project.