Eyetrack Is Not a Solution
By Howard Finberg

Research is a tool, not a solution.

Eyetrack III is a tool, not a solution.

Eyetrack is an effort to show how online news users process information on a webpage. We look through the eyes (literally) of a group of consumers as they view sample news websites, multimedia editorial content, and advertising. This is the third eye-tracking study conducted by Poynter since 1991.

For media company managers and staff, this Eyetrack research will provoke important discussions and may help guide sites in conversations about redesign and navigation decisions. It is likely that there will be some findings that will raise questions and concerns.

Fundamentally, however, the Eyetrack results are just one more tool to help journalists do their jobs better.

Research tools, like the garden tools lying in the backyard shed, don't really mean much without two important factors:

• A strategic vision of where you want your site to go.
• The integration of all other elements -- your market research, your technical abilities, and content resources.

Finally there's the sweat and hard work to make it come together. It is your job to put all of the factors -- research, resources, vision -- to good use.

We work in a society that often seems like it is governed by research. There are statistics for just about any topic you want to address. And there is research that can pretty much prove any point you want to make.

For many news media managers, research is a strange and unknown commodity. We embrace it when it proves our point; we dismiss it when it makes us uncomfortable. Research that breaks new ground often challenges our preconceptions. And to that I say "Hooray."

What research should not do, however, is govern our actions. Research can not tell us what to do any more than the Magic 8 Ball can provide managerial decisions.

Jakob Nielsen, principal of Nielsen Norman Group, a user research consulting company, has been creating and studying Web usability research for years. He agrees that research can only highlight an issue or problem.

"Research is a reality check. It tells you what really happens when people use computers. We can speculate on what customers want, or we can find out. The latter is the more fruitful approach," he said in an e-mail interview. "Unfortunately, research doesn't tell you what to do. It shows what works and what causes problems, but it doesn't say 'this is how to build the next release of your website.'"

Mike Donatello, director of Survey Solutions for comScore Networks, Inc., has this caution about any eye-movement study: "Folks ought to realize that it's very granular measurement that indicated how readers' near-automatic parsing of on-screen information is affected by presentation elements. Unless attitudinal or recall data are included in the study, no one should necessarily make the leap that just because folks read/glance at a page in a certain way means that they remember or comprehend the information any better," he said in an e-mail interview. "Studies like this are best used in a diagnostic capacity or to provide a general framework, rather than strict guidelines or rules."

But what happens when you disagree with what's being presented?

Rusty Coats, director of New Media at MORI Research, a company that works with many newspaper websites, likes the idea of disagreement.

"Disagreement can be the best thing for research, because it forces us to look harder at the facts rather than preconceptions and emotions. First, make sure the data is sound. Research is like cooking a cake: There are a lot of ingredients, and they all have to be exactly right. If you disagree with the data, dig into it," Coats said.

He urges examination of research in the same manner as an objective reporter looks for truth.

"Sometimes, people disagree with research because of a belief they've had, a dogma they've established, a purely emotional response. In any business, these can be false idols," he said in an e-mail interview.

The issue of emotional response to new research is an important one. Journalists advance through their careers by learning the stories of their organizations. We learn the mythology of the places we work and of the leaders we follow. This mythology shapes our beliefs and values.

Sometimes it is very hard to surrender those beliefs when new data arrives.

"If your intuition disagrees with research findings," Nielsen says, "you should view this as a learning opportunity to improve your insights in the future. Design is not religion. You don't have to defend the beliefs of your forefathers to the bitter end. Design is a business decision, so you should follow the data and do what works best for your company."

Coats looks at the disagreement issue this way: "No one is forcing anyone to agree with the research. If the data is sound and you still disagree, that's certainly your prerogative, though ultimately it might not be a very wise one. There's no profit in making customers wrong, though it can be very tempting at times."

One of the biggest challenges to readers of the Eyetrack III research will be the volume of information available. And while the authors have done a great job in organizing the material, there's a lot to absorb. Here are some suggestions from Nielsen, Coats, and myself on how to cope with this problem:

• Process the information in small bites.
• Take a finding and look at other sites, at first, rather than your own.
• Give yourself time to absorb the information before acting or reacting.
• Print out a copy of your homepage and circle design elements that contradict a research finding.
• Assemble a diverse group of staff members and have them discuss different findings.
• Find similar research; compare and contrast.
• Join the online discussion at Poynter Online and ask questions.
• Assume nothing; start from a position called "zero sum budgeting" where every rule is open to challenge and change.
• Ask for help from your company's research department.
• Write down questions and then look for answers within the data.
• Go back and read the research again.

Coats says one of the best places to look for help is "a study's authors and authors of similar studies. And then ask everyone -- all the wickedly bright people you know -- 'What do you think of this? How are you applying this? What makes sense to you?' Sometimes the best thing research generates is conversation."