LinkedIn discussion - "Measuring Job Boards traffic - which one do you believe?"

Tony Harding started a discission on the Recruitment Futurology group on LinkedIn

He asked "Having worked as a media professional for 20 years it amazes me that their are so many different ways in which recruitment job boards measure their own traffic and as such it means that the only way you can really judge is by actual applications. NORAS, ABC, Comscore , Google - in other words pick the one which works best for your stats - you cant buy media on the back of different scales of traffic as you will never know the correct figure. I would say that applications are the only worthwhile judge, and in short what then happens to them wastage/interview/offer - as clients we really need to savvy up on knowing what these stats are - what do you think?"

Quite a few comments ensued (go have a look if you have access to the group or want to get it) - but I thought I'd bring his question and my response over here and see if any readers of this blog had any thoughts. Well do you? ;)

My thoughts:

Job Board Traffic metrics are so open to varying degrees of interpretation and manipulation (both pre & post reporting) it's a close run thing to being no more, or even less, use than gut feel TBH.

The real reason metrics are needed? To cover the arse of the person who's consulting on or buying the media. People are desperate for certainty in an uncertain world, a fast moving world where so many factors influence response - so it's most often about making sure you've got a few figures to back up your thinking and you're safe from too much criticism if it doesn't pan out.

I've asked many media in the past why they subscribe to NORAS, which IMHO added little more of real value beyond the ABCe certification which they could have got anyway (although top level user activity behaviors are interesting). Whilst they always seemed to agree with my perspective, the reason they paid the extra for inclusion was simply that many clients they sold too took a disproportionate degree of comfort from having a logo to back up their advice / decisions - irrespective of what it stood for and/or the integrity of conclusions.

To twist a well known saying - "there are lies, damned lies and web traffic statistics", which means there will never be a harmonious agreement on which metric is the most valuable or which methodology / traffic monitoring organisation has the most true reflection of reality. It's only natural that every commercial entity when promoting themselves will only want to use that metric which is flattering to them when compared against who they view as their competition.


So the onus lies with the person collating and processing that response - and rightly so. Location, salary, perception of employer brand, time of the year, how you've written your ad, general state of the market, application process - all these things (and many more) affect response (both quality and quantity of), so unless you have in place a facility to report with maximum integrity where your traffic and, most importantly, interviews and hires came from in previous activity, then gut feel is pretty much as good as anything you've got.

And what should you look for in a gut feel assessment of a job board?
- Are any of your colleagues in that function / role aware of it?
- What's it's online (and potentially offline) profile like - where does it generate it's RELEVANT traffic from?
- How easy is it to use - putting yourself in the shoes of the candidate?
- Are there similar roles to yours advertised already? (great if they can provide info on these or similar too)
- And then ask how many relevant people they have subscribed to alerts and / or in their CV database.


The future has to be for recruiters to pick their ATS' with integrity of metrics and MI reporting as a fundamental underpinning requirement, taking responsibility for better knowing themselves what works and what doesn't - and I'm sorry, but reliance on "what media did you come from" drop down options is simply not good enough. (Note - am consciously staying away from pitching our solution here)


And I don't think this is really clients keeping this info close to their chest Wendy - the truth is that it's because they don't actually know themselves because of rubbish ATS. However you also need to be aware that, even where an ATS would allow seamless tracking, the JobServe application set up actually creates such a negative application and (from my personal & previous often discussed) direct client recruiter experience, that your metrics of ROI are going to be as flawed as anyone's (my reasons/experience set out here: http://www.3dmarcomms.com/blog/2010/01/new-year-new-recruit-some-clear-mi/ - which also includes why I found the PPA experience pretty rubbish too I'm sorry to say Nicole).


But, for what it's worth - ABCe was always my preferred "go to" for verified stats when I was advising / buying on behalf of clients.

Comments

[Melanie]

This is a very interesting post, having previously worked as a Marketing Manager for a recruitment consultancy, we would investigate the performance of every job board we used in great detail to gain a true picture of results. For us it was about two things:
1. The quality of the applications received
2. Successful placements

Representatives from the job boards would come in to see us armed with their reports and advise that were were getting fantastic results, based purely on the number of applications that had gone through their system.

However when it got to our side it was a different story, all applications were coded for the source and tracked over a six month period. The applications as you mention above must be relevant and surely by tracking the number of appropriate applications, followed by interviews and then placements will only give a true picture of success and ROI?

During the recession people were applying for anything they could. We received 50 applications in 1 day from 1 candidate directly on to our own website, this candidate was desperate for work and willing to apply for any job. We could not help him, so whilst our applications and traffic report looked good, on closer inspection it clearly wasn't.

[Alex Hens]

I'm afraid to say, Melanie, from my experience you were in the minority in terms of your diligent tracking - be that as a Recruitment Consultant or as Direct Recruiter. But it is great to hear of people doing it properly and the benefit that ensued, because I think it's mad not to (and if you're employed specifically as a Marketeer then it's quite simply negligent not to).

As I'm sure we've all heard said (on googling it seems it may have been said first by Lord Levehulme): "“I know that half of my advertising budget is wasted, but I’m not sure which half” - and that is all too true in recruitment budget spend (although I fear just being half could be a little flattering).

A little bit of extra diligence and effort in tracking can reap massive ROI in itself - but this is all too easily forgotten or not grasped at all.

I do feel sorry for the Job Boards (to an extent) - they have to use something to try and sell against competition, but they know that the metrics they use are 1. pretty irrelevant or 2. probably flawed. But until there's transparency then what's a media industry to do? The job boards that really work would like nothing more than transparency of course - because it is they who have nothing to hide.

At least technology (like ours ;) is coming through that will make it pretty painless for people to see what's working for them - because that's the key: make it easy (or even better make it happen without any effort at all) and then there's simply no excuse. At that point - everyone gains (apart from the job boards that really don't work of course).

thanks for the comment :)

[Ciaran O'Sullivan]

At the moment comScore sees traffic on many of the key job boards including the aggregators, so if you want to take a peek at the Jobs market feel free to give me a shout. As you know, a handful of the major job boards are clients. We can work closer with job boards by engaging them in our Hybrid 360 Methodology. The core of the comScore methodology is panel based where we enumerate panellist activities and their behaviour to represent the population of the 38 countries we are report on. (2mil panellists worldwide) To engage in Hybrid 360, sites must implement comScore beacons which allow us to see every instant their site is fired. Then we filter and report on that traffic and unique users accordingly within our client facing interface. It’s like having a referee on the pitch and video ref at the same time. (Speaks volumes if you’re an Irish football fan) Hybrid also allows us to see who is accessing a site from work, home, wifi/public access and mobile. For those purely interested in mobile, you might not be aware of the GSMA initiative. In summary, it’s the first full census of its kind!! A huge potential for your market in 2010 I’m sure you’ll agree. The main challenges as always are tracking niche jobsites online (Mobile should be covered).

[Gareth Jones]

Hi Alex

I think you are right - very few recruitment consultancy databases are credible enough to be able to produce the data, its just not a priority for most. Melanie is indeed in the minority.

This had been a long term issue for me until last year when Marketing and technology became my responsibility. The first thing on my agenda was to understand which of the job boards were really performing as we didn't have a clue beyond numbers of applicants. As a large business, we were talking serious amounts of money.

Luckily, we have our own proprietary system here and data integrity is a big deal so all we had to do was tinker with the back end and some reporting to get the numbers. It took 3 months but it was worth the effort. I blogged the story with results - link below - but i have reproduced the headlines for you:

The stats over the period we measured for this initial review (8 months) were as follows (Marketing Only):

* Total Applications generated = 15,118
* Total applications considered relevant = 3,308
* Total applicants interviewed by client = 75
* Total applicants placed = 16

These are the totals - we had the data for each individual board. On the basis of these numbers we reduced our job boards from 7 to 3 immediately.

I picked up another conversation on LinkedIn about the very same thing and decided to publish our numbers which caused a bit of a stir. I have been inundated since with requests to share this stuff on a wider basis.

We are now digging deeper having run these stats regularly and they are making us question the validity of investing so much - time and money - in job boards. We are also looking deeper into our own internal practices as a result. To be fair to job boards, the gift of measurement relies heavily on recruiting firms having robust measurement ability in place, which many don't.

My original post with the whole story is here:

http://garethmjones.wordpress.com/2009/10/29/job-board-performance-stats/

[Chrismuktar]

This is really interesting actually. I know for a fact some websites completely fabricate their traffic, and it is easy to demonstrate. It seems from our experience of trying to market WikiJob (http://www.wikijob.co.uk) graduate job advertising to recruiters/agencies, nobody is much interested in traffic, and if they are, they are more than happy to take our word for it.

This is all well and could- until some of our competitors start to make up stats to compete with us. We worked extremely hard on building up traffic so it pains me to see this happen.

We would definitely get an ABCe audit if clients started requesting it, but we've billed over £100k and nobody's asked...

I for one would like to see more accountability when it comes to traffic.