Reaction From MSNBC.com Senior Producer for Broadband Productions Ashley Wells
Here's my reaction to your write-up on the Big Picture:

It's important to note that our format for doing these has changed significantly in design following the Oscars 2003 show.

In your study, you say 5 of 6 viewers opted to control the flow of the show. Our own internal tracking shows that only 40% of all viewers used the left-side navigation. That's significantly less than your sample group suggests.

The observation about viewers looking at the control bar and time remaining display is interesting. I wonder if they were more inclined to look at that because they were in a controlled environment, or perhaps because by the time they were watching this, the Oscars had probably already been awarded. I always thought that people might look at the time remaining and instinctively figure they didn't have time to watch instead of giving the content a chance to convince them to make time. I do think it's important to give some indication of how long a presentation is. But perhaps that could be more subtle. So our newer Big Picture format does not have a time remaining indicator. Instead, the control bar is longer, making it easier to visually get a sense of how quickly a segment is progressing without saying "this will take 3 minutes of your life." In comparing the overall usage stats between the Oscars 2003 package versus Oscars 2004 in the newer format, it would be hard to attribute the impact of eliminating the time remaining indicator, other than to say that since there was an increase in the overall time spent for the latter, there doesn't appear to be any obvious down side to eliminating it.

Your breakdown of how long the test subjects spent with the presentation roughly reflects our own tracking. There seem to be three types of viewers: Actives, passives, and samplers. Samplers don't stay long -- less than a minute -- but tend to click around a little before leaving. We don't quite know why, though I suspect it's a combination of people who are too busy and people who don't have the computing power or bandwidth (or even audio) to run the presentation smoothly. Learning how to convert more of those viewers into actives or passives is a key goal of mine. Active viewers spend a moderate amount of time on these presentations and seem to find the topics they're most interested in quickly. They don't watch everything, but that's okay. They "get" the interface and will use it. Passives sit back and let it play like a TV show. They tend to watch almost everything (god bless 'em). But they don't always interact when prompted as they may not entirely understand how. Roughly, each category is about a third of our Big Picture audience. When we redesigned the format, we did so with an eye toward helping all three of those types of viewers consume the content. So if you had three different heatmaps showing where each type looked, that would be really interesting to me.

There has been a lot of debate here about whether having our host appear visually is distracting to viewers. Your analysis says that those who watched more moved their focus between the big screen and the small screen. I'm curious as to whether this shows she's a distraction or whether it shows she's useful. Either way, our new format allows viewers to hide all of the extraneous stuff at the bottom if they find it overwhelming. And I guess that's the kind of solution I prefer. Eyetracking and data tracking are useful, but they ultimately show that different people do different things. So why use that information to settle on one approach for an audience when the beauty of the online medium is that you can create formats that allow individuals to customize their own experience?

That people are looking at the sponsor's logo would be the single most important finding to me. If only we had the newer format ready for your testing cycle! This is the kind of brand impression information that I cannot get any other way. And it's centrally important to our cause here because Big Pictures these days must be sponsor supported. It's literally information that could ultimately sway the case as to whether we continue to do these or not.

Ashley Wells
Senior producer, Broadband Productions