For the Eye Tracking Skeptic, Seeing is Believing

skeptics and believers

When evaluating new technology, skepticism is a useful reflex. Are you sure it works? How do you know? Where is your evidence? Such questions help to weed out the ineffective tools and improve the ones that show promise. In this way, skeptics guide the evolution of the very technologies of which they are skeptical. And if you’re skeptical of that conclusion, just look at eye tracking. Today’s excellent visual behavior analysis tools are in part the result of a century and half of skepticism regarding the accuracy of data collected, the applicability to different research areas and the realism of the testing setup. For example, if no one had ever said, “Hmm, that chin rest and bite bar sure do seem to distract the participant,” then progress toward systems with head movement correction would not have been as swift.

At this point, the technical quality of eye tracking is well-established. Most serious researchers accept that the data collected is accurate and realistic, but that doesn’t mean that the skepticism has disappeared. The main challenge that we currently hear from skeptics is one of utility: “Do I really even need eye tracking?” It’s an important question, one that drives us to constantly improve our services and software. However, as practically useful as this question may be, it isn’t a simple one to answer, at least not in a way that will satisfy the true skeptic. I can list the ways in which eye tracking is beneficial, but that’s just marketing. I can provide references to studies that have effectively used our technology, but those are just someone else’s findings. To the skeptic wondering if eye tracking offers them any real advantage, seeing is believing.

Here’s what I mean. To combat skepticism in website testing, I have two options: I can say “evidence suggests that traditional usability research does not fully account for the richness of the user experience” (and then listen to the crickets), OR I can simply show this video…

As you might expect, the video is the more effective means of conveying the value of eye tracking. It’s not about replacing traditional methods; it’s about using all of the tools available to describe the interaction between user and site completely. If you are only capturing outward behavior, then you aren’t getting the whole picture. You’re watching the highlight reel instead of seeing the whole game. You’re skimming the blurb on the back cover instead of reading the whole book. I could probably rattle off a few more analogies, but again, I think the video does a much better job of turning eye tracking skeptics into eye tracking believers.

Featured image from Unsplash.

Even Eye Trackers Have Blind Spots

Blind spots

“To the man with a hammer, everything looks like a nail.”

The origins of the preceding quotation are unclear – most likely Kaplan or Maslow, but some argue that Mark Twain said it first. No matter the author, this analogy is apt to describe a current trend in our industry. After roughly a half-century of amazing technological advancements and staggering feats of R&D, eye tracking researchers have created some extremely useful hammers. We have hammers that measure every fixation, saccade and flicker of your pupil. We have hammers that sit on your desk and hammers that rest unobtrusively on the bridge of your nose. We have hammers that can track the eye of pretty much anyone pretty much anywhere doing pretty much anything. I am referring, of course, to our eye tracking hardware systems, which seamlessly translate raw physiology into accurate visual behavior data. Regardless of the source of this well-worn quote, the point can be easily applied to our own high-tech tools.

Eye tracking researchers must resist the temptation to approach every study objective exclusively with eye tracking. Although there is a treasure trove of valuable information available through this methodology, there are many questions that analysis of visual behavior alone simply cannot answer. Eye tracking cannot tell you for certain which item a shopper will purchase. It provides no means of divining click or scrolling data. Most glaringly, there is no configuration of cameras, software and infrared lights capable of capturing the thoughts, expectations and perceptions of the consumer. When planning a research study, it is important to ask yourself the following two questions: Which of my objectives can be addressed by eye tracking? And what other methodologies might I employ to fill in the blanks?

This may seem like common sense, and yet the reach of eye tracking is sometimes overstated by its practitioners. For example in the field of web usability, there are researchers who suggest that heat maps and gaze plots will tell you virtually everything you need to know about usability. Obviously we agree that the eye of the user is an indispensable resource for answering a great many questions, but we only recommend it as a standalone approach when the goal of the research is extremely simple. In addition to eye tracking, any comprehensive evaluation must include analysis of click patterns, pages viewed, time on page, usability errors and scrolling. Websites are not static test stimuli; they are dynamic multilayered interfaces, which cannot be assessed without considering visual and navigational behavior.

Perhaps even more critically, there is the subjective component. Everyone knows that analyzing qualitative data can be somewhat messy. That doesn’t mean it isn’t useful. In our experience the best way to understand the implications of eye and click data is to ask the user to explain it in an eye tracking-enhanced post-testing interview, in which the eye movements of the user serve as a powerful memory cue. A gaze plot will tell you exactly what he/she looked at during a given task, but unless you ask ‘why,’ you can only guess at underlying motivations. Yes, interview data can be unreliable. Yes, the process can be inefficient and the data difficult to quantify. However, if you ask the right questions and interpret the answers carefully, your understanding of the user experience will undoubtedly be enriched by your qualitative efforts.

Questionnaires, physiological sensors, task success metrics – there are too many other methods to mention, most of which compliment eye tracking nicely (and are supported by EyeWorks). The best approach is one that pairs each research question with the appropriate research technique, for example…

  • How quickly do users notice the left navigation? Utilize eye tracking.
  • How often do users click links in the left navigation? Track navigational behavior.
  • Do users like how the left navigation is organized? Interview the user after the session.

Eye tracking is an unparalleled research methodology with applications in a wide variety of different fields. Nevertheless, that doesn’t mean it must stand alone. Don’t limit yourself. Use any and all tools necessary to meet the specific objectives of your study. If you can do that, there’s a good chance your results will hit the nail on the head.