Metrics and Measurement – 1-3:30
Panelists:
- Bruce Leichtman, Leichtman Research Group
- Stacey Lynn Schulman, Turner Broadcasting
- Maury Giles, GSD&M Idea City
- Jim Nail, Cymfony
Description:
As media companies have come to recognize the value of participatory audiences, they have searched for matrixes by which to measure engagement with their properties. A model based on impressions is giving way to new models which seek to account for the range of different ways consumers engage with entertainment content. But nobody is quite clear how you can “count” engaged consumers or how you can account for various forms and qualities of engagement. Over the past several years, a range of different companies have proposed alternative systems for measuring engagement. What are the strengths and limits of these competing models? What aspects of audience activity do they account for? What value do they place on different forms of engagement?
—
Notes:
Jim Nail – Cymfony is a brand monitoring company – tell enterprises what users are saying about them.
Maury Giles – GSD&M Idea City – ad agency / interactive agency in Austin. Background in political campaigns, where measurement is paramount.
Stacey Lynn Schulman – new to Turner. Previously at Interpublic group – consumer experience practice. Measurement in the old media are well understood and stable. Walks through history of shifts in measurement – movement into multi-network world (cable), move to “people meters” in households, etc.
Bruce Leichtman – based in Duram NH. Boutique analyst firm focused on future of entertainment. To understand the future we need to begin with the present. Talked about needing to avoid the sample of one problem. We don’t represent the masses – need to focus on quantitative research across broad audiences.
—
SF – Place to start. The Writers’ Strike.
BL – The writer’s strike over the 4.6 billion in revenue that could occur – but the hockey stick curves aren’t real yet.
SLS – We don’t know how big the pie will be – but that doesn’t mean that writers should have a piece of the pie. We have difficulty really quantifying this stuff – especially when it comes to fusing samples across media. People starting online then watching tv, rewatching things they downloaded, etc – we don’t have any way to capture this information reliably across channels.
MG – For me it comes down to how you measure success. Are we going to stick with eyeballs, audience size, etc., or can we adjust to a different way of measuring to understand the control users have. The old paradigm, based on eyeballs, is falling apart – rather than tracking the diffusion of media throughout channels, we focus on what is enabled by all these niche audiences. If we focus on the impact of content on niche audiences rather than mass media – it’s not about how many people we reach as opposed to our impact on niche markets.
JN – the challenge is that those hockey stick graphs are just opinions expressed in numeric form – the real discussion should be not about the size of the chart, but about what assumptions are made to generate them and what direction they indicate things are changing. But we cannot forget about the consumer and how rapidly they change, which is a slowing effect on change, no matter how much the technology changes. This has to come down to the number of times something is viewed, downloaded, etc – not a flat fee since we don’t know how much revenue this will generate.
—
SF – Is it someone’s fault that their isn’t a viable revenue stream?
JN – The networks have been in control for 50+ years. As their control and revenue stream erodes, they are struggling. It isn’t anyone’s fault it is just a fact of life.
BL – 6 minutes of video/day is the mean number in terms of what users are viewing. People talk alot about the YouTube phenomenon, but not much about “Don’t Forget The Lyrics” – but that is still something which got more eyeball time than YouTube did. It’s more about evolution than revolution.
SLS – When I was on the agency side, clients just wanted to be on the next big shiny object.
JN – I call that the GMOOT – get me one of those.
SLS – What happens is that the industry gets this sense that everybody is in these spaces and that they have to be in these spaces. But that’s because 80% of their mindshare is on that big shiny object. But the reality is that 80% of their dollars are in that traditional media, because thats where the audience is. They want to see these new bright shiny objects expressed in terms they understand – which means they want the market numbers they get for traditional media.
JN – During the bust, people were pointing at companies which spent money online going out of business failing and saying see – online advertising doesn’t work. At the same time, however, the % of time people spend online keeps increasing – the percentage of consumers media consumption online outpaces the marketing spend.
SLS – the flipside of the bright shiny objects crowd is the lean back arms crossed posture – the marketing folks who don’t even believe anything new is important or signficant. Yeah, but everytime I put that commercial in old media I sell X amount of Y.
BL – but as we see these things evolving, old media is not dead. We just saw the largest cable event ever in history – high school musical 2. Audience segmentation is important, and we can’t think that we are the audience.
JN – Appointment TV isn’t dead – it is just that the user is setting the appointment.
—
SF – Jericho / CBS. Fans rallying for a show. But also fans saying we’re watching, how come our eyeballs don’t count?
SLS – Engagement is the beginning of that. Trying to determine how much people like a show based on how much they talk about it. When Lost first was being talked about, everyone thought it wasn’t going to work – but our analysis of buzz said that it was going to work. In that case it turned out to be right. But there are also small, highly engaged audiences in some cases – Veronica Mars, The Office, Friday Night Lights – these are shows which ranked very high in buzz, but small in audience. The small engaged fan cultures are something we should be looking at. We also can’t forget that consumers are themselves channels – they are distributing content as well.
MG – The content seller has a need to validate the value of the content. What we’re trying to do is measure engagement in a context – what role that engagement has in the decision cycle of the consumer. Is it having an impact on how they purchase?
BL – if it doesn’t sell, today, tomorrow, or at some later point, it isn’t worth it for the agency.
JN – but branding does work. People deny it, but we’ve seen it time and time again – they may see an ad on toilet paper, and then later they pick that brand in the store – without even knowing it.
—
JN – people don’t like to talk about advertising. But they do talk about what is important to them, and how they talk about what’s important to them, it helps you figure out how to engage with them and how to position your products and where to position your products. Criticism is a useful metric because users who are critical of your product they tell you that because they want you to get better. Engagement is about also listening – you have to let go of that total control and develop a relationship with consumers where they help create.
SLS – We need to question this notion that as a marketer you have a portfolio of brands. What if we thought instead about having sets of consumers whose needs were meeting. What we have, what our asset is, is the consumers we are serving, not this portfolio of products we’re trying to sell.
JN – the control marketers or advertisers ever had was always a myth. We never had the control we classically thought we did. Now we can see that co-creation of meaning happening in much clearer ways. You cannot just surround people with integrated marketing messages and think that we control the conversation.
BL – Between DVR and On-Demand, about 5% is when the user wants it – the other 95% is viewed on the schedule created by the networks. Even that push is due to it being pushed by providers (cable box integration, dish integration) not end consumer demand (stand alone TiVo box). Even as the number of DVR’s grow, the % of viewing which is time shifted, it will still be only 15% of all viewing time even when we have 50 million DVRs in households.
—
Audience questions:
Q: When you make predictions about audiences over time, how do you account for the aging of the audience over time as well?
BL – My forecasts are based on demand and supply – in a 3-5 year time frame those issues don’t impact as much.
SLS – But it isn’t always about studying a single generation across time – the millenials have an impact across time, but when you project their teenage behavior over time, don’t assume they don’t change.
JN – it’s valuable for how to connect to them now, not what they will be like in 30 years or even 10.
—-
Q: What about the kids market? What kind of research are you doing in terms of how to reach that audience?
SLS – More difficult because there are restrictions and regulations about doing research with children, especially in the context of trying to sell them stuff.
—
Q: What about users recommending things to each other and how you can track that?
SLS – recommending products is something being enabled in facebook. It isn’t about the reach of one distribution mechanism but the reaggregation of all the various sums.
JN – 80% of word of mouth is still offline, so even measuring online word of mouth is only a proxy for the recommendations people make. If you think about the never ending friending report about MySpace, the revelation in that report to me was the importance of the widgets and portability – people putting that widget in their profile is so much more important than having your own brand page or banner ads.
MY – You also have to be very careful about that “facilitate” role – if you’re actually creating it and pretending people popularly / virally created it you’ve got a problem. I love the Nike/Apple iPod integration example – if we can provide a real service that happens to also be branded that is the loyalty solution. Facilitating the experience in order to drive to real results. The goal of the campaign is to have a specific impact on consumer behavior and that behavior might include telling friends about something, subscribing to a feed, etc.
SLS – Mass media isn’t going anywhere, even as we hear alot about fragmentation.
BL – We hear about how cable is beating broadcast – well, there are 4 broadcast networks and 100 cable channels – shouldn’t cable beat broadcast? Those 4 channels are still very dominant and that reflects something about human nature and centrality of shared experiences.
—
Q: Other than word of mouth, what other engagement metrics do you see.
MG – Some of the softer, traditional metrics from branding and advertising – it’s about what makes people think, feel, and act – and thinking and feeling are hard to measure, especially when the “act” comes much later.
JN – It’s about return on marketing objective. The right measurements are different in different places – growing awareness, repositioning a product when it relaunches, etc – those are valid metrics in different cases. It isn’t alwyas about specific ROI – there are things you do in marketing which lead to future sales, which you should do, and you have to do them whether they can be directly tied to sales or not.
MG – The thing that fascinates me currently is using complex scientific approaches to create virtual environments and test in them based on metrics tracked over time – you create virtual agents and introduce different stimulous and then see what emerges. Basically become predictive, rather than reactive – it isn’t just about measureing how effective this last campaign was, but predicting how effective the next one will be.
—
Q: What about an open, transparent approach to measurement? It is frustrating that we (users) have no access to how things are measured?
Great idea, but unlikely to happen – lots and lots of money in this space and lots of investment in how it is done today.
—
Q: Consumers as a channel for us to think about – what about Bebo Channels? Couldn’t the revenue in that space be shared with the writers? (Back to the WGA strike).
SLS – In terms of the writers strike, it is very layered here. It is just more complicated than simply saying because ads are there it is therefore profitable. It’s all too all over the place at this point to know what we can and can’t support from a cost/revenue perspective. Even the sites the networks are building have a hard time competing with bittorrent, file sharing, and other mechanisms out there which provide more control – so they are having a hard enough time creating the ability to actually get online distribution they control rather than the distribution users control, let alone worrying about paying more to be able to do it.
BL – but it also isn’t necessarily all incremental revenue – is this in place of other syndication later? Does the value of the show in broadcast, in rerun, in syndication, diminish as it is spread more broadly online?
—
Q: Are there any specific metrics you’ve seen advertisers believe which demonstrates people paying attention to ads?
SLS – IAG and the rewards tv model – there is a measurement here in which the user needs to recall copy points, but is the expectation it is setting real? This is an example of a metric the industry has accepted largely accepted because there is nothing better.
MG – What we’re trying to do is connect to the metric which really matters – incremental improvement in revenue. Skin in the game, tying marketing/advertising to how the company actually does – if we fail to have a positive impact on your revenue that puts us in a different position to worry about these other intermediate metrics which ultimately connect to improved company performance.
—
Q: What about online versus offline: we tend to think of offline as about brand awareness and online about direct action – but online can also be used to build brand awareness, can’t it?
MG – the interactivity of being online can still be a brand building experience – so the actions users take (click here, send this to a friend, whatever behaviors you offer) *are* part of building awareness and brand recall.
JN – TV is still very influential. What online can ad is reach to the lite tv viewer once you pass the point of inefficiency in tv.
SLS – Although TV is seen as the reach medium, remember that things are changing. In an on demand environment more options are available – TV may be growing into a medium which offers more interactivity . . .
JN . . . and those studies were all done on banner ads – as we get more online video, so both are evolving toward each other in terms of capabilities.
—
Q: Any case studies which surprised you about the impact of various kinds of media? Times where what happened was unexpected?
BL – High School Musical. The mass still does exist. How Disney was able to move that across media – an album, a show, a skating tour, etc.
JN – I’ll give you three: MySpace, YouTube, FaceBook. All of these really took the world of tracking influence by storm as places for people to express how they feel about various products and ads. A fourth is the people talking about their ads in advance of the superbowl – the tradition was to keep things quiet and try to make this big surprise. Instead, as folks were sharing info about their ads in advance of the big show – but we found that they still had the same influence.
BL – AppleTV as a great case study. 2 million by the end of the year? Now they won’t talk about it. Now it is Steve Jobs hobby – the case study was already written – people don’t want a standalone box.
—
Q: Any other outside examples in terms of measuring engagement, which didn’t originate in media but come from other fields?
SLS – check out a company called Neuro focus. Measureing brain waves to measure engagement.
MG – swarm theory, chaos theory – these worlds are increasingly relevant. Studies of complex biological systems and how they evolve – marketing is increasing like an organic system.
—
Q: Problem established. Tell us how you are addressing it
BL – not my problem. I’m trying to help understand the consumer and clearly define where we actually are not just where we are going.
SLS – our biggest challenge is trying to figure out how to keep commercial minutes relavant to content minutes. (New ways to get advertisers involved in the content, new ways to keep consumers engaged and get them to see messages from advertisers without interrupting your primary reason to be there).
MG – we’re focused on studying how the consumer engages with the product. Dynamics, triggers, stages of decision making – looking in depth at what “reachable moments” exist to influence that behavior.
JN – one of my notes from this panel has been SLS on the reaggregation of meaningful sums. In the future it isnt going to be who is the audience of this tv show – at the end of the day it is about reach and impact, regardless of the channel or mechanism. Advertisers want to reach a certain number of certain kinds of people in a certain timeframe with a message – they don’t care what *channel* is used – so maybe again it is aggregation from a lot of smaller more passionate audiences.
1 Comment
Comments are closed.