Fallout, Pathing & Clickstream over multiple visits

I think I need a bit quicker turnaround on my blog posting.

So here's a quick one.

Clickstream analysis or fallout report is very useful on a specific set of sequential actions but it is overrated in many of its applications in web analytics tools.

When analysing your clickstream data it should be at the visitor level with the option to look at the visit level if need be.

By looking at visit level data, yes you will be able to understand what parts of the process are letting you down but you will struggle to place an accurate value on this fallout because you will likely be ignoring many of your customers future actions.

Retail example

A good example of this is where a customer is asked for some information they don’t know off hand and their session times out before finding it. They then have to go through the complete process again to complete it. That first fallout was not a “true” fallout and it helps to understand this value in comparison to “regular” fallout.

Knowing you have a 50% fallout rate at a given point means different things if you consider two scenarios 1) where 100% of the people come back to complete the process end to end again 2) where only 10% of people come back and complete the process end to end.

In 1) It’s a temporary glitch in the purchasing process and it might make a lot of sense to develop some save and return functionality to make it smoother for a quantifiable portion of your customer base in 2) its indicative of a hurdle people don’t wish to get over and you need to seriously ask yourself why you are putting your customers through this pain.


B2B example

I’m an executive who downloads a PDF 5 times before becoming a lead. The fact I downloaded the PDF many times is in fact irrelevant (I just happen to be forgetful of where I download it to), the act of downloading is the key action that needs to be connected to the lead. This means for me you only see a lead created 1 in every 5 downloads (really poor lift/high fallout – 20%/80%) whereas in my case the connection between download and lead happened 100% of the time. Multiple me by all the other forgetful executives there are and suddenly you’ve lost insight of the power of one your greatest lead generators!

Hope that helps.

Omniture acquires Visual Sciences

They've done it again.

They've gone and spent a load of cash and I didnt even get a phone call.

So, Visual Sciences is planned to become part of Omniture. Interesting...

We all know Visual Sciences have been on the market. As such their customers have been in a position of uncertainty for a while now and this has also hampered their competitiveness in the market place. My first thoughts were really not

1. "Wow, we just got some cool technology" or
2. "Great, we just gained a nice bit of market share" but actually
3. "Its probably a good job a BI/CRM vendor who concentrates on the offline world didnt buy them with the goal of turning their offerings into actionable insight from a proper 360 view of customer"
   

Taking those points in reverse:-

(point3) The NetGen buyout by SPSS was conceptually quite ahead of its time, but for reasons I'm not going to go into now it failed miserably. This wouldnt have stopped someone now, over half a decade on from making a success of combining an online analytics/operational platform with a offline one. This would have created serious competition for Omniture and indeed all of the online web analytics vendors as well as few offline ones. No company currently has strengths across all channels in both analytics and operational optimisation but it has been interesting to note the slow but gradual improvement of the offerings from some of the offline vendors in this area (Unica, SAS+Speedtrap etc.)

(point2) Omniture just bought itself a lot of customers. 400 million dollars worth of customers? Thats not clear to me, but I dont really tread in those financial circles so find it hard to comment. I expect their current customer base isnt anywhere near that but the removal of competition from future deals and developments probably amounts to an awful lot in this ever increasingly valueable area of business. Had noone purchased VS I would have expected them to have gradually have lost market share anyway with a combination of Google Analytics and Omniture sharing the spoils dependent on customer size.

(point1) I'm not going to claim to be fully versed in their product suite but VS clearly had some cool technology. Will this be incorporated to Omniture? Is that why Omniture purchased them? I expect not - in the realm of real time reporting or making actions based on real time data an application is very dependent on its data architecture and the way data gets into the system. I find it hard to believe Omniture would adopt a completely different architecture in order to benefit in what really amounts to only a few specific bits of functionality, its probably quicker and easier just to development them internally. That being said, there may be some interesting areas of cross over (Omniture Discover vs. Visual Site) and also useful technology from the approaches VS made in the other channels (Visual Call and Visual Mail).

However you look at it, these are interesting times and I expect the consolidation of vendors in the online area to continue (site analytics, adserving, onsite search, email, online marketing etc.).

So, my question is "who's going to buy who next?"

Omniture acquires Offermatica

Wow.

I go away for a fortnight and Omniture goes and buys a company behind my back. I guess that shows how much they consult me!

OK. Without wanting to sound the corporate lackey, I have to say I think this is a smart deal. Omniture appear to be moving towards the wonderful "site optimisation platform" that they've been looking for.

What have they bought? Its a company with proven success at creating and selling an application that wraps up the natty problem of how to make actionable analytics.

So where does that leave Omniture? Well marketing have come up with a wonderful tagline (que image I dont have yet)

AB + MVT + BT / Web Analytics = ROI.

Firstly, lets establish what (AB+MVT+BT) provides a customer. Lets ignore the AB bit and just talk about its more sophisticated cousin MVT and the different approach of BT and where you would apply either.

Broadly...

BT is based upon customers behaviour and aims to optimise a key tagline, banner, piece of content. You use it when you're really not sure what content you should be targetting at any one time.
MVT is based upon customers reaction to a single page. Its optimising the arrangement of content or indeed subtle differences in a tagline. Ideally, it's used when you know exactly the product, process or content on a page you need, you're just trying to refine it to make sure its presented in the best way - forms is an ideal situation, becuase by that point in a "conversion" process all customers are likely to behave similar hence the lack of need to segment the customer base.

Between them, that covers a lot of onsite optimisation (I'm ignoring campaign optimisation for the sake of this blog). The main areas left for onsite optimisation that I reckon Omniture need would be a rules engine and a recommendation /onsite search engine.

What applications would I buy next if I was Omniture trying to improve the Site Optimisation portfolio?

Rules engines are required because having smartly applied rules that can be deployed easily, understood clearly by business and provide an awful lot of benefit. (In an Omniture situation, a rules engine would be performing a similar function to a complicated CMS)

For instance, if you are launching a new product, have new content related to that product and have marketing spend related to that product, you do not need to waste any time "training" an analytic model. It just makes sense that things line up, have people coming from a certain piece of marketing going to a certain piece of content. In fact, in many "straightforward" businesses you would find examples of marketing activity and site content rules that would get you an awful long way.

This is actually something I've found in lots of work done with offline companies where the application of sophisticated analytics is a little more mature than the online environment. Rules, business knowledge and good management will get you an awful long way and often analytics is really best placed on top of the rules in one of the following ways:

1. to optimise, enhance or make incremental improvements on existing business processes
2. to discover new, as yet unknown patterns in customer behaviour which ideally are feedback to guide business strategy.

Obviously, if you have 50,000 peices of marketing activity, and 10,000 products, managing rules for all of that would seem impossible - but if you can segment the products and marketing activity in 20 segments with clear goals and an estimation of value, suddenly the task doesn't seem that hard. With these segments not only would you have less rules to manage but you would have a clearer understanding of marketing and product strategy that would have a dramatic step change in effectiveness.

Search/Recommendation Engines : I've already mentioned recommendations in this blog so wont go too far into this - The reason I have grouped these things are that they simply return a rank set of results based on a customer interaction. In the case of search engines, its a list of ranked content based on a given keyword. In the case of a recommendation engine, its usually a list of ranked products or content based on some website activity (like looking at another product). The main difference is how you do the ranking of delivered content. In a search engine, a user often has a specific thing in mind, the search engine must return as close a match to that thing as possible (PULL activity). In a recommendation engine, its usually more a "push" from the organisation and the results returned are not usually so directly matched, its usually more about making connections based on historic behaviour (behaviour and product activity + customer segment) (PUSH activity).

Heres an example - if I search for Canon IXUS on Google, I want to find things about exactly that, most probably a review or site with the cheapest price in my case, but still, things that have a direct relationship with the terms used.
However if I've just added to my basket a Canon IXUS on a site I dont really need to be recommended a Canon IXUS by the engine, I've seen that already. Better to recommend complimentary products (possibly bought by other customers with the Canon IXUS) or alternative products. Anyway - lets not get into how we do this ranking of content or products in either search / recommendations, the point although the algorithm is different is the mechanism for deployment (action -> return results -> customer selection) is exactly the same and that is something I reckon Omniture should be getting into.

ttfn

Web Analytics Skills Snobbery

"Web analytics" as an industry is funny.

http://blog.jimnovo.com/2007/07/19/aberdeen-web-analytics/

"We’re already seeing web analytics job postings with phrases like “strong knowledge of SAS and SPSS highly desirable” meaning employers are looking for cross-platform, cross-tool, cross-channel analysts. The folks with this cross-knowledge set who can also “speak business” are going to be a very hot commodity going forward. Fortunately, most web analysts already “speak business”, it’s part of the WA culture - and speaking business is the hard part for most analytical minds....."

When I started in 1999 you couldn't interest anyone in a spot of proper statistics (or data mining as it had been rebranded in the mid 90's - since then its going through a few more names like "predictive applications") and now we've come such as long way that people are actually recommending the skills! I'm astounded and hope this means I'll never struggle to find work again!

Suggesting that people with analytic competency struggle to "speak business" is down right insulting. I suspect Jim is playing on the rather archaic stereotype of the speccy techie analyst that do indeed still exist - probably all too often - but for every non-"business speaking" techie, I'll show you a non-"tech speaking" bullshitting business speaker which could do a company just as much damage. Unfortunately quality is not the preserve of any particular domain of business.

"Like I said, the data is data, the behavior is behavior - and the tools are just tools. Web analytics is patient zero, infecting the corporation with a proper analytical culture....."

I have always found it amazing how those people who classify themselves as in the Web business are so amazingly confident in their own abilities and so dismissive of those in the rest of business. Admittedly, at lot of new online businesses have been created in the past few years, some with new and unique business models. A lot of new people are being introduced to web analytics but to suggest web analytics is to infect the larger organisations with the principles of analytics is ridiculous, snobbish and reflective of the superiority complex the channel has had since the late 90's.

When you consider the current money spent on campaign optimistation, fraud detection, credit risk scoring, direct marketing, market analysis, market research, usability, call centre analytics, data warehouse, operational optimisation etc. it's a little brave to suggest that these web analyst souls are going to be treading on new ground, exploring the great unknown in terms of how you go about doing things in an analytic manner.

Its a good job Jim goes on to point out that data is in deed, data, and that behaviour is indeed behaviour otherwise I might have been more scathing. This is something that many "web analysts" have long overlooked. The majority of web analytics has all been done before in other channels however these web analysts have been too snobbish to appreciate it.

When you consider that most web analytics tools have in the past had a very limited drill down capability, little data import or export functions, little ability to tie information to other channels and on the whole little flexibility on what they record, its remarkable they've got where they are.

There are, no doubt, areas where web analytics differ to other channels, but to be honest these are mainly in the areas of data collection and the speed at which it is possible to turn around analytics into action - however, sadly, action is still often far from the minds of the majority of users in this area - and this is mostly because the analysts themselves are busy with the other parts of their roles and the tools they use do not lend themselves to much more than reporting.

And this is where other channels can still lend web analytics a hand. The number of applications out there, utilising analytics and business rules in an offline environment is huge - and web vendors can still learn an awful amount if they open their eyes and adjust their product plans.

"If you’re a web analyst and are offered a chance to do SAS / SPSS / Business Objects / etc. training, I would jump on it."

As for jumping on courses I would say this. If your organisation has the foresight to invest in your skills in any way, you should take it.

I would not limit it to SAS, SPSS and Business Objects. Much as I love the analyst world, and amongst it are some very talented people, going on a course will teach you little more than the ability to punch buttons, write code or both - and if you do something like a SAS course you'll be writing a lot more code than you thought!

If you want to learn coding, I'd pick your language carefully based on your organisation and your goals and learn that, rather than thinking knowing a bit of SAS syntax is going to make you analytic genius. (At least with SPSS the tools have a greater emphasis on usability and the novice can get past the "button pushing" stage and into analytics quicker). Simply put, spending time with individuals who have experience in these tools in your organisation is always going to be a lot better than simply going on a course.

Personally, I'd much rather have someone in my team who is a whiz in the Microsoft office products (namely Excel and Access) or perhaps an SQL coder. Someone who has the familiarity with regular business reporting and their data structures, linking data together and capable of understanding the impact of analytics/figures on an organization would always be my preference - so consider that the next time you have courses to chose from.

Remember. The web is usually only one channel or touchpoint of your business - in the long run the key is going to be pulling these channels together and any skills that will help that should be encouraged.

Interesting Rand Schulman Article

Interesting article from Rand Schulman and some of his time in the industry. Interesting to note some of his comments on WebSideStory and the subsequent Visual Sciences stuff. Not only that but also making a case for cross channel applications and optimisations with his new company Unica.

A few interesting snippets from the article, enjoy.

"People woke up and started to understand that the online business was really very much the same as the offline business."

I think people are still coming to terms with this in the larger organisations with an offline history. If you were to look at the numbers of people supporting / analysing the online channel compared with those in the offline world within an organisation and take into account the money the channels bring in, I think the web is still rather undervalued even if you believe that the web channel should be a more efficient tool for reaching customers.

"One of my major contentions was in 2003 that web analytics is not inherently a stand alone application. You can’t just look at the numbers, but you must take action on those numbers."

Again I think this is still a problem. Many organisations have never got to a point where they have a good enough grip on there numbers to feel empowered to take actions. In my experience, those that do are often the marketeers spending those large sums online who are on average doing the most in this area, not the site analysis /content /conversion people. Lets face it, its the marketeers who have the majority of the money and its usually easier for them to switch on and off campaigns than the site based people. Whether they are taking good actions on the numbers they see it always arguable :-)

"I think at the time WebSideStory was the first mover around the integration of actionable applications, and the creator of the first suite of standardized and public APIs"

First mover in the online space perhaps. Other vendors had for some time been trying to open up systems and databases to enable application development.

"but marketing people just don’t need the infiniteness of the slicing and dicing of real time data"

Thanks goodness someone said it.

The reason why you do need the slicing and dicing is because people havent figured out a simple way of representing the data that is needed so give you access to all to cover their bases.

However, that doesnt mean that marketeers shouldn't slice and dice. They should have applications that do the slicing and dicing automatically for them to come up with the best results, but then marketeers should have the key figures presented to them in an easy to understand manner with explanation of how the figures got there. (The last thing I want to see is millions of marketeers all slaviously following the output of black boxes.)

"If you agree with my primary contention that web analytics products collect data, and then they trigger other applications, there is no “visual” requirement about a triggering mechanism whereby you threshold the business rule, and you take the actions."

See note above about black boxes. The problem with people who build clever applications is they often dont think about the users and the feedback they require to either

a) make sure the app isnt going off the rails
b) gain insight to make the app or other sections of their business better in some way

"The market is going full circle to encompassing cross-channel marketing and cross-channel action, not just cross-channel reporting, but cross-channel action, and combining online and offline action."

360 degree view of the customer is not news.The thing is vendors are only now building the tools to cope with it properly and all vendors have their weakness, yes, even Unica...

"But, all the vendors are talking about this “cross-channel” thing now, and what they are really talking about is cross-channel reporting."

Again, thats what a lot of the web analytics people are saying, but look at people like Omniture and our Touch Clarity solution? What about SPSS and their predictive applications? Its pretty simple, companies whose emphasis is one reporting will continue to have those leanings, but that doesnt mean there arent a lot of application builders out there.

"That’s going to be a fundamental shift back away from the client side to the server side, where log files and tag information will come together."

Very interesting. It will pose difficulties for those vendors who are not used to dealing and integrating with different data sources. Many of the web analytics vendors have had data processing relatively easy as web logs have been of a relatively standardised format for years and if you're tagging you can make your own data structures no problemo. New systems, new data sources, then integration gets really interesting...

"Secondly, we are going to have to have a notion of the correlation of orthogonal web-based events” versus the linear display of a series of page view- based events"

Not sure what that really means. Orthogonal in my book is perpendicular, correlation implies they are lined up - orthogonal != correlated !!

I'll cut him some slack as I really like the article. I suspect what it means is a move away from pages that can very easiloy be linked together and causaly related to things that operate a little more independently. I dont expect this will be unsurmountable its simply that vendors will need to organised their data into a specific level and report on it, like say customer level. I expect data mining type stuff to become more prevalent.

"Some of these are finding out that the client’s greater engagement on the website means lower conversion to their store."

Needs to be said to a lot of people. There's process and engagement. You want to be able process as quickly as possible. You want to be be ableto engage as long (or as often) as possible. The same website can often have both and depending on who you're speaking to in an organisation, both can be important.

"Yet we have these huge and vast stores of personal information that are going to be employed for the benefit of us all as individuals. Those benefits will help us live richer lives."

And lets not forget the corporations who own the data and the people who build the apps...

What's happening at Visual Sciences?

I guess I should state for the record before we go any further together, I'm currently an Omniture man so I'm just going to state what facts I can and then speculate wildly.

Visual Sciences has just announced a poorer than expected set of earnings. Not terrible by any mark except perhaps by those that really do matter in the city.

Is the a down turn in Web Analytics or something specific to Visual Sciences? I suspect it is the latter, suffering in an ever more competitive market. There is the interesting comment that there "has been approached by a number of strategic buyers"

Who would this be then? I don't suppose it would be another vendor in exactly the same space but would expect a vendor who is trying to complement their offline offering.

Google of course has its free web analytics tool, so one would expect Microsoft to naturally be interested but I would be extremely surprised if they got involved despite any rumours (doesn't every tech firm have a rumour about Microsoft wanting to buy them?)

However I could see someone in the other areas such as Oracle, SAS (who currently partner with Speedtrap, a very cool piece of UK web analytics technology), IBM, Business Objects having a sniff. As the online channel becomes more and more important the lack of ability to collect, report, support and deliver applications on the online channel is becoming a glaring hole in the portfolio of these companies despite most claiming to already have 360 view of customers and various ways to interact with a customer in their value lifetime.

I dont know Visual Sciences intimately but do believe they have some cool technology. I've always been a believer their products dont represent a paradigm shift, in particular the Visual Sciences product is really spit and polish on an old OLAP web reporting paradigm but punters do like that kind of thing, flashy graphics etc. so its always likely to sell - to repeat a point, its a technology that many are envious of and I can only hope for their sake that any integration in the future would go a lot better than that of others in history of on/offline mergers, particularly in the analytics space.

End Searcher Optimisation? Are we being Short Changed by Search Engines?

A recent post on clickz by a gent named Bryan Eisenberg wrote an article call End Searcher Optimization (ESO).

Its focus, in my opinion, is simply on the thought that we are going to have different kinds of results from search engines and therefore will have to start marketing to different segments in different ways, rather than "just" optimimising the text content. I find that somewhat suprising as I would hope anyone spending a moderate amount on search ppc are already considering how they target the keywords.

Bryan goes on to comment about how one part of the population might prefer an executive summary whereas another might prefer more detailed analysis. Not only that, but it we consider in advance the different kinds of content they might like we can arrange our marketing strategy accordingly.

I dont think thats really very shocking is it?

Also, note how Bryan approaches the thorny problem of what segments / personas actually exist for a company, in his own words
"we went through all the content we'd planned for each of the personas and decided which piece of content had to be delivered as an article, a blog post, an image, a video"

For a man professing analytics, "went through" doesnt sound very "analytic" does it?

However, aside from drawing attention to Bryan's weakness in segment creation - and I appreciate customer segmentation is still an area very much driven by the fluffier side of marketing - I think we have been getting a rough deal from search in this area for ages, and I think only now that keywords are really beginning to rocket in prices will we begin to question their real value.

Think about it. When was the last time you went to your advertising agency and placed your various banners randomly across the web and then had to monitor the results yourself to work out whats best and what segments you might have?

Dont you normally highlight to your agency "we'd like to target 3 particular groups with the following three campaigns of content, please place them on suitable sites". Regardless of whether the ad agency does a good job or not, they offer the hope of targeting the content for you, they take their expertise of the network they possess and hope to apply those things to give you a good quality customer. The key point there being that you pay them for their expertise in their domain.

What do search engines do for you? They let you try individual keywords and then let you do the hard work of determining how well they perform. If they provide more ways your content can reach the customer through pictures and video then great but you've still got the overhead of trying to figure out what works and what doesnt.

Now, let see what the search engines do know that they dont tell you.

• When a customer comes to you they dont tell you what the person has previously searched for either in the same visit/day/month etc.
• They dont tell you what type of site they've previously delivered that customer to

Think about those two.

Knowing the first will give you an idea of what are the current intentions of the customer.
How important could that be? Are they shopping around? Are they refining their search? are they ready to buy?

Knowing the second will give you an idea of what kind of customer segment they belong to.
Imagine that, a web equivalent to a Mosaic segment. Imagine if you had Mosaic-like segments that you could apply to the keywords and indicate what kinds of people used what kind of words, giving you an immediate feel for the value of the traffic.

Then, if you imagine that rather than PPC, the search engines move to a Pay-Per-Purchase model that records whether customers buy after purchasing.

Add that to the previous points and you have a picture of a customer, you have intentions, customer segment and even likelihood to buy. You find yourself looking at keyword value with some depth of customer value behind them.

So many web analysts are beginning to try and do this with there own site, i.e. what multiple media are the customers seeing, what purchases have they already made, what keywords have worked for them but they are limited as they only know what happens on their site.

So, my call is for search engines to start releasing more information about the customers that they serve and delivery to their paying customers - currently they (google mainly of course) have it too easy as we dont question the value they bring or have high expectations of service.

All Change at SPSS!

As an ex-SPSSer of many years I do like to follow the fortunes of the company. It has a very interesting history (one of the oldest in software) and still has some bleeding edge technology
however it does seem to have misfortune when plotting its strategic goals. (SPSS bought NetGenesis at the peak of the dot com boom for any of you web analysts how don't know who they are. What? You've never heard of Net Genesis, hmm it has gone quiet hasn't it....)

Anyway, big changes there, according to yahoo Jon Otterstatter and John Shap have left. These are two key figures in recent SPSS history, Otterstatter being given very nice package and role after the "interesting" takeover of Showcase (large amount of shares that I assume he still owns) . Internally, it was largely assumed Otterstatter was CEO in waiting - well I guess he's not waiting more more. John Shap was a bullish head of sales who pulled SPSS sales into shape (somewhat at the cost of the services organisation) but certainly someone who shook things up.

Between them, these two C-level execs dramatically changed the landscape of SPSS internally and its interesting these two strong figures have now left. This leaves long standing CEO Jack Noonan to lead with a new set of supporters - I do wonder where this will take them next.

Netflix and Recommendation engines

Given I was talking about recommendation engines I should really mention one of the most interesting of the moment . This is a big prize where academic and small start ups attempt to beat the internal teams recommendations. I mention those to groups as the larger companies are most likely not paying any attention as the prize involves handing over all methodology to Netflix, something the larger companies would like to keep for themselves thank you very much. (a million dollars isnt worth SAS giving away their trade secrets).

The problem with the way netflix set this up is that it's all about horsepower, its all about trying to show that netflix can be outscored with some tremendous algorithm, however what I suspect the competitors are finding is that much like most data mining situations, its not the power of your algorithm, more your understanding of the industry and hence the underlying data. There will be very little between many of the main algorithms, certainly not the 10% Netflix is looking for, it all comes from knowing the area. I think this is what the comment “One of the reasons of our current leading position is that we keep on trying to implement new tricks and ideas, and a reasonable portion of them works” really alludes to.

What will happen is that the solution will probably be one very fine tuned for Netflix (good for them) but that it isnt totally translatable and marketable to other industries. However I'm sure whatever architecture they come up with will indeed be a good starting point and I think its really good Netflix have open this up.

Why does anyone need a recommendation engine?

Well in Netflix case its all about making sure people select products and as many as possible - making sure the recommendation precedes a selection at all costs (in the case of Netflix I bet I could get 70% of there score by clever customer segmentation and only 50 of all of their products - but I've not done this so you can chose to ignore what I say on the basis I'm an armchair analyst)

Is this the only goal of a recommendation engine?

Not really. Something like a public services broadcaster should not ultimately be about maximum purchases but about delivering appropriate selections that open up the various offerings to a previously unknowing general public. Particularly in the days of the long tail and thousands of products, it can be about making people aware of what you offer. This is done in conjunction with your regular campaigns to make sure you dont lose the business of people unwilling to take a chance but that those that like a varied mix are dealt with accordingly. In doing this a company would take a hit immediately on the netflix score but over the course of time gain the trust of customers who will eventually pay them back several times over.

How do you go about making recommendations?

Well traditionally there are five different ways I'm used to doing it, although I'd love to know more.

Business Rules - you tell the customer what they want and what you are prepared to give them based on business knowledge and current strategy.

Counting - you find out what is most commonly purchased with the other products bought by a customer

Predictive algorithms - you say "what are the characteristics that define a person who buys this product" and then score all the products against each other.

Customer Segmentation - the customer looks like they belong to group X. We know that X's most like children's movies

Trending/Time series - "we know that action movies sell better on fridays whereas cerebral offerings tend to be earlier in the week"

Sequences - Hmm. A person has bought the first 4 series of "24", I wonder what they'll want...

What factors influence these approaches?

Trigger events - "we know loads of people seem to get romantic movies around mid february"

Discounts - People love discounts

Whos buying - People dont always buy for themselves.

Service levels - not point in recommending stuff thats out of stock, takes time to deliver or even costs more than a person would likely pay.

Current Behaviour - often called Scenarios by a lot of people nowadays. If a person is looking at horror movies, even if they have a high propensity for fluffy romantic numbers based on previous behaviour, its probably best you stick to recommending the horror genre.

Its a new product! - I have no data, how do I recommend it!

So how does all this come together?

Well thats an interesting point, more often than not, one approach will get selected and then optimised and squeezed until it reaches a plateau, few vendors have the time to invest on a complex architecture. I am a great believer in Ockham's razor and as such selecting one approach that has clear feedback to a business on how it works is in line with a lot of my thinking.

In fact, one of the main points that suspect will be missed by this scoring competition - any business needs to understand why something gets recommended so it can plan and structure itself to meet these needs better - a black box is good in the short term, bad in the longer and the better something gets at predicting activity, the more specialised and harder to understand it becomes.

However, acknowledging Ockham's Razor, I think that one of the main things with producing a good recommendation engine is to have a combination of all approaches (each has weakness in different areas that I'll go into at another post) so that the system can be adjusted to reflect the way a customer does business - for instance, trigger points will have different impacts in different business, current behaviour will also be more relevant in some than others., in some industries trends will be more prevalent.

As I highlighted to begin with the goal of the engine may be different, and within the same organisation it could be to push the top ten products in one channel, but to be more exploratory in another - and as long as the engine can provide feedback on the cost of those decision to the business this can then alter strategy going forwards.

Why am I interested? well it is fascinating stuff and it has potential for so many areas. Anywhere where recommendations or advice can be offered could stand to benefit from greater expertise in this area. For my own part I'm looking at content and product delivery on websites currently, but it certainly doesnt stop there.

Anyway, enough for today. ttfn.

NB. I think I need more pictures, this is a horrid wall of text....

Guh, Enough about A/B testing already..

Sorry, I cant stay away from it. I've seen a few articles over the years, this one is still one of of my favourites by a chap called John Quarto-von Tivadar. Very technical, but to me it gets many point across quite clearly.

I like a/b or Multivariate testing. I enjoy the fact its brought a feeling of scientific methodolgy into developing websites.

The 2 main problems for me has always been that

a) It only works for the average customer - you may be able to increase revenue through testing butit always work on the basis you have one type of customer (or customers all centred around one type of behaviour) - you have to set the test up very cleverly to be able to understand why different customer segments work in different ways.

b) You cant polish a turd - If a design is bad, small incremental modification is not going to make the great leaps you require - how do you get great leaps? combinations of usability studies, common & business sense, understanding what your customer wants and seeing what your customer does.

When I look at the way the mainstream vendor technology has changed over the years, its interesting to note evolution in this area. I do think its worth highlighting that these testing techniques was snapped up by the traditional web vendors, those whose software only really analysed per page, and not per customer (ie. the data is generally very OLAP-y and only structured for reporting in certain dimensions to be drilled into rather than on a complete customer usage basis). There is a world of clever statistics and data mining that has been left behind, even though some vendors talk the data mining / analysis talk most of their tools do not have data architectures that would support smarter, cleverer, customer centric view needed to move on. Not only that, apart from multivariate testing, most vendors have no clear way of deploying and making a difference with all those lovely metrics they provide you with, let alone any that are provided by fancy analytics. There is, for me, a clear gap in the building of applications in this area.

I think I may elaborate this in the next post. One of the current hot topics that I have some experience in is recommendation engines. These have previously been based on things either too black-boxy or without the fundamental emphasis on feedback on the calculations they perform to help and aid a user make their business better - algorithms and applications are great, but without providing feedback on how they work, its like entrusting your business to an idiot savant, pretty impressive at somethings but not something that copes with a lot of variety.

Also they are usually pretty limited dimensionally - what's the products bought, what's the products seen - not usually much of a concept on customer segment, customer actions - What they need is an architecture that combines the clever counting most current recommendations engines do with some insight into customer behaviour and business segments, and then a healthy understanding of the risk and impact of offering any one product over any other. This is not easy, because you'll have many different things impacting on the final recommendation, an application has to take on board the risk level the business is prepared to accept - but for me the most important thing is feedback its success in as clear a manner as possible. To make things more complicated there then has to be a layer on top of all this which is business rules required (to push stock, react to triggers not captured in models, reacting to new circumstances) but again, the business has to see the impact of the rules they deploy.

Actually, I'll stop there, this has already go on too long - I'll pick up with this another time as I reckon I'm going a bit off topic for what I planned to be a quick post. Toodle-loo.

Will this be the start of my real blogging?

So, in the words of Led Zeppelin, its been a long time. I started this blog, not really knowing what I wanted to do with it, but realising if I didn't get some of the experiences out of my head and onto paper I was going to miss out on the internet bandwagon - as I almost did in 2001.

Given that I'd been doing reporting, statistical analysis, data mining etc in the on and offline channel since 1999 I figured I finally had a few things it would be good for people to learn from - if they wished.

And then what happens? I go and get a job in a bank leading a team of analysts on the web channel that took all my time up and I never got back to sharing insights.

So why am I back here now? well, I just got a new job with a web analytics company in a post sales consultancy position and find myself not with anymore time, but at least a greater incentive and desire to share what I know.

It's amazing how meeting a few different clients in different situations makes you think - wow, there really needs some better guidance out there.

However, I may have to tread carefully, I think my contract is roughly a form of gagging order and that I'm not really meant to talk about the things I do and the company I represent. Which is fine at the moment as most my insights so far were gained elsewhere anyway..

But what will be my topics if I'm not allowed to talk? Well, I guess I should say that nothing I say is representative of the views of my employer. I've found an number of sites/blogs online that offer an awful lot of help in web area and I guess I'll be critiquing these (should be plenty of material there...) - look out web analytics!! :-)