Alas, measurement is the elephant in the room. While there are a couple of initiatives such as http://advertising.microsoft.com/europe/ResearchLibrary/ResearchLibrary.aspx?Adv_ResearchReportID=1031 and http://www.avc.com/a_vc/2008/12/display-adverti.html we are still quite a long way away - particularly with JICIMS now scaled back.

Looks like it will be lolcats and DR for the foreseeable...

"That is not a return on investment, that is the sort of haphazard disregard for accurate accounting that precipitated the current global financial crisis"

Such sweetly caustic and dreadfully needed words, Richard! Brilliantly said. I pray more people in digital read your words than not!

I hate to admit it, but I too have been guilty of the crimes as state. But anyone in digital likely has been as well. We have to change this if we're going to build any respectability!

strikes me that the intermediate measures used to judge effect are media measure, similar to circulation and readership measures for print, or viewing audiences for TV.

This data is the currency of the media world and I guess that's where the digital agencies are showing efficiency(as you say not effect!). Even then do the arguments they stack up, do they delivering a lower cost per thousand than TV for example? Sure the data is more robust but do they deliver the numbers?

And given TV's proven effect (20 years of IPA)albeit on the wane, do they have the real business results to challenge?

I'm with you, if we're talking effectiveness then we're obliged to talk about the effect on business, not clever intermediate measures.

Could be just the point of view of a client side marketing handyman but I do not see a contradiction, rather it is a typical development path.
Big Ad guys in time become very expensive and not so effective (and maybe it is not just down to lazy advertising...) so the new digital boys, who can at least track more precisely HOW the money spent worked (efficiency), become darling.
So perhaps the blame stays with client side people looking for the easy way out....
However, what about a bit of good old common sense and say: let's try and get the strongest effect with the most efficient use of money, i.e.
the REAL catch is the search the optimum channel mix (ALSO considering variables such as the macro/industry environment, initial competitive position, effect of distribution etc.).
Am I talking the blatant obvious? In which case I am looking forward to hearing from an agency that is already on that quest and can show how far they have gone...

Richard - you forget that digital agencies, rightly or not, still don't command budgets that are likely to make a statistically relevant dent in those top line figures - we're talking total production + media spend of 250-500k if you're lucky, versus £5-10m for TV ads. Which makes it pretty hard to take the top down approach you advocate.

Where's Mr Binet when you need him?

"Indeed, any effectiveness metric that does not tell you the incremental profit that your marketing activity generated for the business isn’t worth the paper it is written on."

Hi Richard, I have just the site for you!

http://www.wsu.edu/~brians/gradgrind.html

I'm quite sure you're about 80% right. I only say that because by the tine I figured out I wasn't really certain what I thought on the topic, my Taxi had overshot it's destination and I was in digital efficacy La-La land on my mobile phone reading this post.

I'm pretty sure it's an important line in the sand. It definitely raises even more difficult issues than it slays.

Good work. As ever.

Interesting.

I think I might have fallen for a certain amount of "sleight of tongue" from the pure play digital chaps as I always heard "effectiveness" (in an ROI sense) when they said "accountability". Now you point it out, this isn't the same thing at all.

I'm also wondering if this lack of "commercial value delivered to client" accounts for the generally low net profit margins in the digital industry (typically less than 8-10%, and that was '06 and '07)

I had previously thought that they were rather inexperienced managers who couldn't run businesses well (contrast that with media companies) but it may be that clients are not getting the value (in a CEO/ROI sense) from this industry.

By the way, I've put a somewhat provocative piece over at my blog ("PRliterate" style?).

It is for our Marketing team over here (hence is rather biased) but underneath the bombast you might like some of the data i have been digging up recently.

Your piece is an interesting one, but there is no no disjoint between efficiency, effectiveness and accountability. This ideal combination results from sound account planning at a strategic level (the role of traditional planning) for the first two and from pro-active tactical analysis & planning (the reality of strategy in a dynamic digital world) on the other. As proof, take a look at VW's Best Digital case study in Adworks 15 the IPA effectivess awards 2006. And in regard of the executives who aren't interested in how accountable the people who use their money are - they've probebly been sacked since you wrote this piece.

well its such a nice post,
thanks for the information,

well that's that then

Insightful post. I'd stretch it a bit further, making a distinction between marketer types.

A productivity-based approach to marketing, Mr. Huntington's "effectiveness", is not uncommon in pharma companies and, previously, auto manufacturers where R&D, production and distribution costs can dwarf ad spend. At that point marketing ROI is calculated using a more comprehensive set of product costs.

Unfortunately, productivity-based planning breaks down in less capital-intensive markets where ad spend efficiency (between mediums, channels, formats even individual ads) is critical to overall profitability.

I love this post, Richard, as I have already told you. Representing, as I do, the commercial TV industry, the conflation of 'accountability' and 'effectiveness' has been a curse on us all.

No-one's been wicked; 'digital' and DM specialists have often had little choice but to work in a silo, but that's changing now. Holistic effectiveness analysis really is the only solution.

But I just wanted to respond to Sean Howard's comment about TV's proven effectiveness being on the wane. Actually the reverse is true. Les Binet and Peter Field, using the IPA Datamine, found that TV is more effectiveness now than 20 years, which they attribute to better targeting
from the growth of multi-channel and deflation.

Of course, the most effective combo is TV + online, research which Thinkbox and the IAB worked on together to prove in 2008.

Can you have a pop at ROI next please, Richard?

Apologies. I've just sussed out the signatures. It was jemster not Sean who made the comment.

Hi Tess

anything wrong would have been Sean's, anything right would have been me ;)

A well-aimed hand grenade, Richard.
Imagine if the roles reversed, so digital case studies told you how an interactive campaign had influenced sales and brand growth and advertising awards told you how many people saw a film?

I agree with your general criticisms, but not some of the nitty-gritty.

For one, ROI to me is dollars in per dollar out. I could be wrong, but I don't see it as a comparative or incremental thing.

Also, yes, it should be measured in terms of revenue/profit instead of impressions, click-throughs, conversions, dwell time and the like, which are all nothing but means to an end. But the same can be said of reach, frequency, ratings and even awareness and recall in traditional media.

That said, I love "it measures efficiency not effect." Couldn't have put it more sweetly.

If a digital agency is using click-through rates as the measure of success, then that's of course a waste of everyone's time. The only thing that matters in any business is profitability. Not click-throughs, not awareness, not impressions, not even revenue. Profitability.

However, and this is where you and most everyone else here can attack me, digital marketing offers a much clearer path to determining effectiveness than any other medium.

In much the same way that traditional advertising can say "your profits rose X percentage during the period where we spent Y amount of your dollars", digital can say "your profits rose X percentage during the period where we people did ABC on your website."

There are two points to take into account here.

First, that digital can measure any sales made directly online. This is not true for all businesses, since not all businesses can sell online. But for those who do, there is a way to very accurately tell at least a portion of your profitability story.

Second, as Robin Grant pointed out above, it's difficult to correlate profitability to your online budget if you're only spending 5% online. Conversely, it's very easy for traditional media to lay claim over any sales that online helped drive unless those sales can be directly traced back to online. (Which is another reason digital agencies should focus on direct response - no one can argue with us if we made those sale.)

Now, I'm not suggesting that all digital work should suddenly become cheesy "Buy now!" ads. What I am saying is that step 1 is coming up with the big idea and deciding how to execute it, etc etc. That hasn't changed. But step 2 is figuring out all the different ways we can measure that so that we have more reliable ways of tying profitability back to the work being done.

So yes, your "top down" approach is the way to go. Profitability is king. Producing great work must be central. But the only way to make it from the top (profitability) all the way down the actual things you've been spending money on is to measure them everywhere you can. Otherwise you're pissing your money away, and hoping to the gods that it all works out.

Very much agree with the initial post - the role of the digital agency can only grow once it begins to show it’s value from a broader business perspective.

However, efficiency still has a place where measurement of footfall & spend is needed to define the CPA value of using different channels, messaging, offers etc. This is an integral to effective campaign planning.
Yes, digital has a number of benefits - enabling instantly trackable results & offering the ability to follow relationships at an individual level. However, for some markets (i.e. higher value purchases or B2B relationships with a longer sales funnel), these need to be need to be measured as part of a bigger picture, where brand awareness & other marketing / sales / PR presence plays a considerable role.

As a follow on to Robin Grant’s note re. scale, I agree that even the most integrated agencies can be restricted by silo client responsibilities, having an impact on one channel alone as opposed to the growth of the business. It’s when clients are open about their REAL objectives we are really able to create something.

I actually think that true ROI can only be achieved when agencies are able to steer marketing activity as a whole (brand / direct, online / offline, tactical / CRM, warm / cold). Only then can the combined activity’s impact on revenue growth be determined & long term views on customer value be determined alongside initial gain. Test structures (reviewing acquisition rates & associated customer value), buzz monitoring & online search levels can all play a part in interpreting the impact of the various marketing elements – however, these only prove their value to the business when interpreted alongside overall growth in profitability for the relevant customer or business sectors.

I think online/offline integration is the key here – just because the advertising industry makes the distinction between the two doesn’t mean consumers do. Ultimately, most brands are no more (and often less) than the sum of their marketing channels. It’s only when the entire comms output is integrated and driving towards a higher goal that a brand can become more than that i.e. focused and well defined (I’m thinking o2/tescos here).

From a digital perspective, I think this is what we need to aim for: proving digital's value as a stand-alone or central medium to brand development. Offline can support the online work – something it is well positioned to do (much better, in fact, than the other way around). Great digital work is about creating content, something that engages in the true sense of the word, something that a poster or tv ad can’t so readily do.

It’s no more than history prescribing that digital agencies aren't the ones to define brands at the top level. As for our metrics proving efficiency over effect, what else can most digital agencies do right now? We can’t exactly look at the change to a client’s bottom line after every single-channel campaign.