SimonWaldman.net

UPDATE 31 JAN: I have now added some Hitwise data which shows that even by this metric, if you combine our jobs data with the core site (and why wouldn’t you?) - the Telegraph is no longer number 1.

First, apologies for the web-metric-fest that follows. The truth is I am something of a web metrics nerd. It’s not something I ever planned, and it’s not simply part of my slightly geekish tendencies. Given the enormity of the decisions we all have to make at the moment - and the leaps we are often asking advertisers to make, transparent, accurate and well understood metrics are absolutely critical, and all of us in the market have to be very careful about what we use.

Given this, and after a decade of looking at stats in our market, imagine my surprise when I saw a poster campaign from The Telegraph announcing they were the most visited UK quality newspaper website - based (as it explains in the teeny weeny small print barely visible on a 48-sheet poster) on some Hitwise stats for Jul - Sep 06. And their site now carries the legend ‘Britain’s number one quality newspaper website’).

Initially, I thought this was just a marketing wheeze - just to put themselves back in the frame. But word gets back that they really believe it. There has been a complaint to the ASA (not from me, I should add), and ultimately it will be their call whether this is legal honest and decent etc. But, in the meantime, I think it probably makes most sense to have a look at the figures in a bit of detail here and let people make their own mind up.

And yes before I get going, the big question is does it matter anyway? But I’ll get to that at the end.

View 1: Nielsen NetRatings

Given the fact that we don’t all audit our UK users every month (see below) the best way to get comparative data is to go to Nielsen, Comscore of Hitwise. There is plenty written on the difference between them (user centric vs network centric if you’re inerested). My personal view is that the absolute numbers from all these services are less than the sense of proportion and general trends they show in the market. So, let’s see what they say.

We are not a Nielsen client, so thank you to a kind friend for providing me with this. But this is their view of UK reach among the main newspaper sites.

Which I think gives a slightly different picture. In fact, the sense of proportion between us is pretty much much the picture that anyone who operates in this market (including I expect News International) knows to be true. But Nielsen might be flawed. So let’s take another view.

View 2: Comscore

Now we are a Comscore client - frankly because we feel their panel has both the right scale (around 29,000) and takes the most scrupulous approach to weighting (in other words to ensure that their data is representative of the audience as a whole, rather than simply aggregating ISP data as Hitwise does). Again, they focus only on UK users (for the purpose of this chart anyway), and importantly they have very good penetration with ad agencies.. So let’s take a slightly longer view of what’s happening in our market.

OK. so this goes back to June 05. And what does it show? Well yes, we’re still quite some way ahead although the Sun over took us briefly in June (World Cup? Big Brother?). The Times has shown steady growth. And the Telegraph? Well, their UK reach has actually gone down quite steadily since March 06.

Now, we can debate the comparative figures and methodologies between Nielsen/ Comscore and Hitwise etc till the cows come home. But I think that if we’re talking about the proportion of audience (and let’s not forget that is the most important thing in normal matters), again, the general order is ourselves, The Times/ Sun (they can argue that one out inside News International), and then The Telegraph.

View 3: The ABCe

But what about the ABC, I hear you ask? Well this is going to be a little trickier - partly because the Telegraph doesn’t get an audit every month. And also because at the moment, no-one gets their UK audience or traffic broken out from the total in their audit.

Now, I’ll show you what we publicly state about UK/ international audience in a minute - but it’s probably worth going through the audits in a bit of detail to see what they show us.

Monthly unique users

First the long term view on unique users. (We only started auditing them monthly in the middle of last year) Well, when the Telegraph last got an audit they were up to around 6m, about 2m behind the Times and at about half of Guardian Unlimited.

Monthly page impressions

Yes, yes - the page impression is dead. But it provides a sanity check against the user figures. Perhaps we have all those users, but The Telegraph has a smaller much more loyal gang. Well, it’s quite hard to read this one, as the Telegraph reports too infrequently for Excel to draw a line, but the answer seems to be not. It shows that the Times and Telegraph are both hovering at around 60m page impressions, again, about half of Guardian Unlimited. Interesting that their users have gone up, but traffic has stayed flat.

Daily unique users

I’ve included this because it’s an often ignored part of the ABCe certificate. What if the Telegraph actually had more people coming in every day, but a very low churn rate (ie the same 6m) while the Guardian and/ or Times had a relatively low number each day, but very high churn (ie a completely different 400,000 every day for 30 days). Well, actually that’s not the case. Yes, on a good Sunday the Times might beat us - but on the whole the spread of users throughout the month is consistent .

Actually, I think we should be using these figures much more often. For the record, the average daily user figures (perhaps the closest we can get to a print ABC) are:

Guardian 750,794
Times 476,255
Telegraph 350,536

Obviously, you might want to check those figures as a proportion of newspaper ABC to see how effectively a brand has projected itself online. But, that’s for another time.

But what about the UK?

OK - so, on our advertising information site, we break out both our UK traffic and users - they are as follows for Dec 06 (and let’s remember Dec is never a great month).

Total traffic 116,430,102
UK Traffic: 49,060,020
Total Users: 13,168,126
UK Users 4,206,160

Yes, the sharp eyed among you will notice that the UK numbers aren’t audited - but frankly these days the %age difference between our internal stats and final audit tends to be within a fraction of a %age point, so they are there or there abouts.

Perhaps the Telegraph does the same? Tragically not - in fact their ad site is a little too out of date to be of much use in this (strangely, their site is down as I’m typing this - but fortunately I took a screen grab last night)

Similarly, the Times (almost as out of date) ad site doesn’t break out their UK audience either

So let’s look at the US data

So, if there was someway that the Telegraph had more UK visitors than us, one way to check would be to see that they have an exceptionally low volume of US audience and therefore a disproportionately high (ie 80% plus) UK audience. Now, if only there was an easy way to look at this. Oh, did I hear someone mention Quantcast? This compares the Guardian, Times and Telegraph with the NYTimes and LATimes (partly to sanity check, partly to adjust the scale and avoid this mess)

Well, looking at this table of activity over the last year and looking at our individual pages (Guardian , Times, Telegraph ) it seems our US audience is pretty much in proportion with overall activity - in fact, some days the Times beats us and yes, the Telegraph isn’t that far off on some days. So - lacking in US audience they certainly are not.

The wrong type of users?

Ah yes, but perhaps we should be talking about quality, rather than quantity of users. OK, so why don’t we look at Forrester’s survey of Affluent internet consumers from September 2006.

This shows two things - first the top newspapers read by these affluent online consumers, and next the websites they use. Perhaps things have changed since September 2006 - but as this falls into the same period that the Telegraph were promoting in their advertising, it is still relevant to our overall story.

So what about the Hitwise stats and that number one claim?

So, let’s get down to the data in question. This measures proportion of visits - not the number of users. Having seen the table (I am waiting for another friend to provide one I can put up here) yes, The Telegraph did come at the top of the list, although The Guardian site on hitwise is broken into bits (Jobs gets a different line, for example - while the Telegraph is all bundled together), put all our stuff together and I suspect the lead is marginal at best . and you find that the Telegraph is no longer number one. Their share is 5.23% while ours is 5.7%. This is for a different period (30 days to 20/1/07) - but a) it was the true story while the posters were running and b) I doubt there is that much of an overall shift in trend between periods.

But more importantly, I think that if you look at the stats provided above, they’re probably taking things just a teeny bit too far to start calling themselves the UK’s number one quality newspaper website. On any front.

I should add - I don’t think we have any God-given right to being number one for ever. I know that everyone within Guardian Unlimited is all too aware of that - and we continue to improve and innovate: editorially, technically and commercially. Maybe the Telegraph will fly past us and the Times. Maybe the Times will rocket ahead. But not if we can prevent it. And the data shows no sign that it has happened yet.

So what?

So what indeed, I hear you ask. Well, some general points.

1. In the grand scheme of things it doesn’t matter
Actually, while there’s nothing the UK newspaper market likes more than a good old scrap, this sort of tussle between old enemies is sort of irrelevant in this new media landscape. First because advertisers buy campaigns - not the entire audited audience. But Second and more important, because we are all much smaller fish in a much bigger ocean. Two things I think will really matter.
* Putting in place the strategies to start to see the sort of growth levels being experience by non-newspaper sites
* Genuine understanding of the unique nature of our audiences - across print, online and whatever other media we inhabit in the future.
* Offering smart commercial solutions that make the best use of all available channels.

2. Auditing less than monthly is shocking
I don’t think I need to go into this. I suspect the Telegraph will move to monthly audits very quickly given how seriously they now take this medium. Monthly audits will mean that we can compare independently verified stats on actual usage of our site (as opposed to panel based activity). And frankly, we should really be talking about average daily figures, not aggregate monthlies (a trend started in the early days of the web, when we needed to make our user figures seem big enough within our organisations).

3. UK figures need to be broken out
Why not? Surely no-one’s hiding anything. Basically I don’t think anyone can disagree with the fact that more transparency should be a good thing. I look forward to both the Times and Telegraph telling us regularly how much of their traffic and user base is from the UK.

4. As an industry, we’ve agreed on users. Let’s stick with it.
There is little that’s perfect about measuring Unique Users. It’s not the same as people. But we have all (including the Telegraph, indirectly) agreed through Jicwebs that audited unique users are the way forward. At least it is consistent and frankly, our industry looks a shambles if we keep hopping from one metric to the other just because it suits us.

Perhaps we should start shouting about the fact that we have (at the time of writing) some 32,000 inbound links to our blogs compared to the 5,000 for the Telegraph’s. But no, that would be too petty by half.

Anyway - if you’ve made it this far well done. I will go back to blogging about rural life in Surrey very shortly.