You can think of the Super Bowl as just about our last grand-scale regularly scheduled national mass-media event. Now that the nightly network news has been supplanted by blogs and RSS feeds, the late-night talk shows have given up the ghost, and substantial portions of the prime-time audience have time-shifted themselves out of the real-time reach of advertisers, the Super Bowl is marketers’ best chance to reach audiences with traditional broadcast creative messaging.

Forget about appointment TV: Even the water cooler has been moved on these marketers. Once, the branding aim of marketing messages during these big broadcast events was to win the buzz competition the day after and dominate the Monday-morning conversations. But no one waits for Monday anymore.

Interestingly, the same digital media that are posing a threat to the mass audience for big TV events may offer a tool for measuring the ROI on ads placed in those broadcast spectacles—or at least an index of the buzz they’ve gathered.

Two ad agencies chose to watch and analyze Twitter traffic during the Super Bowl, as a kind of demo project for their social media monitoring platforms. In the case of Boston-based Mullen, this was actually more of a public beta; the company had tracked tweets during last year’s game, but mostly to prove out the concept to itself. This year the BrandBowl results were made broadly public.

“Last year was really just a big party we hosted on Twitter during the game,” says Mullen chief creative officer Edward Boches. “Firstly, Twitter itself was much smaller then—about 2% to 5% of the size it is now. And our community was smaller, too. I had 800 followers on Twitter rather than 10,000 now, and Mullen has none.” The agency’s @MullenUnbound account now has about 2,000 followers.

Mullen and Radian6 developed a set of patented algorithms that could track almost every tweet that referred to a Super Bowl ad posted during the game broadcast. The platform used keywords to track the total number of tweets about each brand as the ad appeared, and then looked at the sentiment expressed in those tweets to figure out a “net sentiment” score, subtracting negative comments from positive about a brand and dividing them by the total tweets for that brand. About 100,000 ad-related tweets were thus captured from the wild during the game.

“We wanted to be conservative,” Boches says. “So if a tweet simply said, ‘Well, that sucks,’ we didn’t include it, because there’s no way to tell what the poster was referring to.” Posts that just said “That’s bad” were also dropped—no way to tell these days if “bad” is ironic for “good”, or just BAD.

Marketing community users could also make their voice heard during the BrandBowl exercise by tweeting using the #brandbowl hashtag. Approximately 8,000 users did, posting 15,000 to 20,000 tweets.

Finally, Mullen and Radian6 produced the BrandBowl overall scores by adding each brand’s positive and neutral tweets as a percentage of all the Twitter traffic about Super Bowl ads.

The results, according to Mullen and Radian6: Doritos came out on top, with 17,940 posts and an 81% positive sentiment. In second position—but way back in terms of Twitter traffic—was Google’s initial foray into Bowl ads, with 10,341 posts and a 75.3% positive vote. In show position came Focus on Family, with 7,248 posts and a 71% positive rating.



Mainly because they so dominated the Twitter conversation while keeping their net sentiment rankings high, those three headed Mullen’s list of “most effective Super Bowl brands”. McDonald’s, Dr Pepper and Universal earned the best sentiment numbers among the game’s marketers but didn’t generate enough Twitter traffic with their ads to knock Doritos or the others out of the top three overall ranks.

Meanwhile Minneapolis agency Colle + McVoy took its own proprietary Twitter measurement tool, Squawq, out for a spin during the game broadcast, to see the volume of what was being in real time said about the brands showing up in the ad breaks.

Like Mullen’s tool, the Squawq platform was also initially rolled out last year but enhanced for deployment at Super Bowl XLIV. Squawq’s algorithm used keywords, hashtags and URLs associated with the Super Bowl to discover game-related tweets, and then overlaid keyword sets specific to each of the game’s advertisers: “Coke”, “Coca” and “Cola”, for example, as well as references to the Simpsons.

Colle + McVoy did not tally the sentiment behind those tweets, so theoretically the ads that got particularly heavy pickup in Twitter might simply have been the most egregious and that traffic all negative. Since Squawq named Frito-Lay’s Doritos as the most-tweeted brand, that seems unlikely. (The agency is working on incorporating sentiment evaluation in the future, it says.)

But the agency produced one yardstick that marketers may eventually find a very useful way to measure at least the microblog portion of their social media campaigns: It correlated the volume of Twitter traffic to the presumed cost of the ad or ads, producing a kind of rough and ready ROI for Bowl buzz.

For instance, Doritos was the brand that stole most attention during the game and got 21% of all the tweets,” says Chris Lawrence, director of client services for Colle + McVoy. “A snack-food competitor of theirs, Diamond Foods, got far less. So we calculated the cost per tweet based on their investments in the Super Bowl. For Doritos, it was $234 per tweet, while for Diamond Foods, it was $3,188 dollars. That’s an interesting way of evaluating the return on those brands’ investments during the game.”

The results are laid out in an interesting interactive graphic display at http://squawq.com/superbowl/.Anheuser-Busch came in second, according to Squawq, with 13% of the Twitter chatter for its three Bowl ads for Budweiser, Bud Light and Select 55. (Cost per tweet, by Colle = McVoy’s reckoning of A-B’s costs: $1,021.) And Coca-Cola rounded out the top three brands with 6% of the ad-related tweets during the game ($522 per tweet).

“These are conversations that were already happening among a brand’s audiences, but we [brands and agencies] didn’t have a way to participate,” Lawrence says. “Now Squawq can serve as a lens to better understand what’s going on and to use those results as a guide for effectiveness.”

Obviously, these Twitter-based metrics are very much works in progress. For one thing, even given the new opportunities of real-time feedback, brands might not be interested solely in how they’re being received on Twitter; conversations about the ads extended for at least a few days beyond the Super Bowl, as people viewed and shared the clips online. That buzz counts too.

Then again, Twitter is not every brand’s demographic cup of tea. It doesn’t have much traction among older consumers, and it may not reach adolescents, who have other preferred ways of instant-messaging. If you’re using event TV to sell retirement plans or tween movies and music, you may be much less interested in Twitter performance than in Web traffic or Facebook friends.

But Mullen’s Boches makes an interesting larger point about brands’ participation in a social conversation, one that can be extended to both of these platform efforts.

“You can go out there and you can listen and be on Twitter and Facebook and have conversations and gather your consumers and have various kinds of engagement,” he says. “But I think one of the great opportunities in social media is to create applications, utilities and experiences that in and of themselves become buzz-worthy.

“The amount of attention and chatter that we, Mullen and Radian6, got, as two little companies compared to the brands advertising in the Super Bowl, is unbelievable. By creating an experience, inviting people to participate, sharing results and then being able to capture everyone who came to join us for future contact, that’s a clear opportunity for all brands.”

You can think of the Super Bowl as just about our last grand-scale regularly scheduled national mass-media event. Now that the nightly network news has been supplanted by blogs and RSS feeds, the late-night talk shows have given up the ghost, and substantial portions of the prime-time audience have time-shifted themselves out of the real-time reach of advertisers, the Super Bowl is marketers’ best chance to reach audiences with traditional broadcast creative messaging.

Forget about appointment TV: Even the water cooler has been moved on these marketers. Once, the branding aim of marketing messages during these big broadcast events was to win the buzz competition the day after and dominate the Monday-morning conversations. But no one waits for Monday anymore.

Interestingly, the same digital media that are posing a threat to the mass audience for big TV events may offer a tool for measuring the ROI on ads placed in those broadcast spectacles—or at least an index of the buzz they’ve gathered.

Two ad agencies chose to watch and analyze Twitter traffic during the Super Bowl, as a kind of demo project for their social media monitoring platforms. In the case of Boston-based Mullen, this was actually more of a public beta; the company had tracked tweets during last year’s game, but mostly to prove out the concept to itself. This year the BrandBowl results were made broadly public.

“Last year was really just a big party we hosted on Twitter during the game,” says Mullen chief creative officer Edward Boches. “Firstly, Twitter itself was much smaller then—about 2% to 5% of the size it is now. And our community was smaller, too. I had 800 followers on Twitter rather than 10,000 now, and Mullen has none.” The agency’s @MullenUnbound account now has about 2,000 followers.

Mullen and Radian6 developed a set of patented algorithms that could track almost every tweet that referred to a Super Bowl ad posted during the game broadcast. The platform used keywords to track the total number of tweets about each brand as the ad appeared, and then looked at the sentiment expressed in those tweets to figure out a “net sentiment” score, subtracting negative comments from positive about a brand and dividing them by the total tweets for that brand. About 100,000 ad-related tweets were thus captured from the wild during the game.

“We wanted to be conservative,” Boches says. “So if a tweet simply said, ‘Well, that sucks,’ we didn’t include it, because there’s no way to tell what the poster was referring to.” Posts that just said “That’s bad” were also dropped—no way to tell these days if “bad” is ironic for “good”, or just BAD.

Marketing community users could also make their voice heard during the BrandBowl exercise by tweeting using the #brandbowl hashtag. Approximately 8,000 users did, posting 15,000 to 20,000 tweets.

Finally, Mullen and Radian6 produced the BrandBowl overall scores by adding each brand’s positive and neutral tweets as a percentage of all the Twitter traffic about Super Bowl ads.

The results, according to Mullen and Radian6: Doritos came out on top, with 17,940 posts and an 81% positive sentiment. In second position—but way back in terms of Twitter traffic—was Google’s initial foray into Bowl ads, with 10,341 posts and a 75.3% positive vote. In show position came Focus on Family, with 7,248 posts and a 71% positive rating.



Mainly because they so dominated the Twitter conversation while keeping their net sentiment rankings high, those three headed Mullen’s list of “most effective Super Bowl brands”. McDonald’s, Dr Pepper and Universal earned the best sentiment numbers among the game’s marketers but didn’t generate enough Twitter traffic with their ads to knock Doritos or the others out of the top three overall ranks.

Meanwhile Minneapolis agency Colle + McVoy took its own proprietary Twitter measurement tool, Squawq, out for a spin during the game broadcast, to see the volume of what was being in real time said about the brands showing up in the ad breaks.

Like Mullen’s tool, the Squawq platform was also initially rolled out last year but enhanced for deployment at Super Bowl XLIV. Squawq’s algorithm used keywords, hashtags and URLs associated with the Super Bowl to discover game-related tweets, and then overlaid keyword sets specific to each of the game’s advertisers: “Coke”, “Coca” and “Cola”, for example, as well as references to the Simpsons.

Colle + McVoy did not tally the sentiment behind those tweets, so theoretically the ads that got particularly heavy pickup in Twitter might simply have been the most egregious and that traffic all negative. Since Squawq named Frito-Lay’s Doritos as the most-tweeted brand, that seems unlikely. (The agency is working on incorporating sentiment evaluation in the future, it says.)

But the agency produced one yardstick that marketers may eventually find a very useful way to measure at least the microblog portion of their social media campaigns: It correlated the volume of Twitter traffic to the presumed cost of the ad or ads, producing a kind of rough and ready ROI for Bowl buzz.

For instance, Doritos was the brand that stole most attention during the game and got 21% of all the tweets,” says Chris Lawrence, director of client services for Colle + McVoy. “A snack-food competitor of theirs, Diamond Foods, got far less. So we calculated the cost per tweet based on their investments in the Super Bowl. For Doritos, it was $234 per tweet, while for Diamond Foods, it was $3,188 dollars. That’s an interesting way of evaluating the return on those brands’ investments during the game.”

The results are laid out in an interesting interactive graphic display at http://squawq.com/superbowl/.Anheuser-Busch came in second, according to Squawq, with 13% of the Twitter chatter for its three Bowl ads for Budweiser, Bud Light and Select 55. (Cost per tweet, by Colle = McVoy’s reckoning of A-B’s costs: $1,021.) And Coca-Cola rounded out the top three brands with 6% of the ad-related tweets during the game ($522 per tweet).

“These are conversations that were already happening among a brand’s audiences, but we [brands and agencies] didn’t have a way to participate,” Lawrence says. “Now Squawq can serve as a lens to better understand what’s going on and to use those results as a guide for effectiveness.”

Obviously, these Twitter-based metrics are very much works in progress. For one thing, even given the new opportunities of real-time feedback, brands might not be interested solely in how they’re being received on Twitter; conversations about the ads extended for at least a few days beyond the Super Bowl, as people viewed and shared the clips online. That buzz counts too.

Then again, Twitter is not every brand’s demographic cup of tea. It doesn’t have much traction among older consumers, and it may not reach adolescents, who have other preferred ways of instant-messaging. If you’re using event TV to sell retirement plans or tween movies and music, you may be much less interested in Twitter performance than in Web traffic or Facebook friends.

But Mullen’s Boches makes an interesting larger point about brands’ participation in a social conversation, one that can be extended to both of these platform efforts.

“You can go out there and you can listen and be on Twitter and Facebook and have conversations and gather your consumers and have various kinds of engagement,” he says. “But I think one of the great opportunities in social media is to create applications, utilities and experiences that in and of themselves become buzz-worthy.

“The amount of attention and chatter that we, Mullen and Radian6, got, as two little companies compared to the brands advertising in the Super Bowl, is unbelievable. By creating an experience, inviting people to participate, sharing results and then being able to capture everyone who came to join us for future contact, that’s a clear opportunity for all brands.”