Apart from my main goal, designing easy to use web sites and other interactive products, I also help my clients evaluate their IT systems in usability testing. This means, basically, placing a person in front of the system, present them with a scenario and some tasks for the to perform and then stay out of the way. The users will then, by their actions and their expressions, tell me whether the system works as intended or not.

The last few years, I have started working with eye tracking, a form of usability study, where I use an eye tracking camera to be able to see what the users see. This technology provides even greater insights than traditional usability testing. The possibility to follow the eye movements makes it possible to understand what the users are thinking.

For my studies I use is sophisticated eye trackers from Tobii Technology and during the last 3-4 years I have developed ways to analyze the eye-tracking data in a more convenient way than normally used in these kind of tests, without compromising the quality of the test result. More specifically, I have developed a technique where the analysis is made in collaboration with the client in a transparent way making the client more involved in the evaluation process. Being more involved in the process, the client get a deeper understanding and thereby maximizing the value of the study.

My work with this methodology has been recognized in the international eye tracking community, and I was invited to present my method at the annual International eye tracking conference in Leuven in 2010. Further more, I am honored in being recommended by Tobii as a usability consultant.

Preparation and execution

Each study is unique since the preconditions vary. When it comes to preparation and execution I generally follow these six steps.

1. Startup workshop
The first stage is a workshop where I sit down with the client and go through the site. Together, we agree on the main question/problem to investigate and what kind of users to invite to the test. It is very important to select test persons with matching customer profile. The result of the workshop is a target group definition and an adequate scope for the study.

2. Creating the scenario
I order to create a credible starting point for the user during the test, I use a test scenario. This little story is used in the test session to put the respondents in the right mood and at the same time present them with the main tasks the client wishes to evaluate. The scenario needs to be focused enough to get the answers but still allow some flexibility for the users to explore.

3. Inviting the respondents
Finding the right test users is somewhat of a trade secret. The important thing is to be able to find users with the correct skills and socio-economic context. If the client has demanding participant profiles, I sometimes consult dedicated recruiting companies to help me with the selection. In my opinion, it is better to have a smaller test group but test more frequently than to spend all of the budget at once. As a matter of fact, Jacob Nielsen points out that a group as small as five people may still provide valuable insight. I usually go for 8-10 respondents as a minimum in order to get reliable eye tracking data, to use for visualizations later in the process.

4. Conducting the test sessions
I conduct the test sessions in an environment the test subjects is familiar with and where they feel comfortable. I generally prefer to conduct the session in their own home or office rather than in a dedicated usability lab. In my opinion I get more reliable results when the users are as relaxed as possible.

5. Inviting client representatives
Too many people present in the room makes the user less comfortable and thus, the test result less reliable. However, I have found that including one representative from the client in the actual test session, give the client a more profound experience than just reading about it in a report. I therefore invite the client to have one representative present at each test session, which give several client representatives the opportunity to participate and get a face-to-face, emotional connection with their users. Taking part of a test session is usally an eye opening moment for any client’s representative:

To witness a user struggle for several minutes with a problem we did not even know existed turned out to be equally painful and rewarding. I now feel that I know our users a thousand times better than before.

Maja Lindström, Project manager, Rebtel

6. Publishing the video material and the visualizations
The raw material recorded by the eye-tracking camera showing how the test subjects navigate the web page. After finishing all test sessions, I export whole and unedited movies and make everything quickly available for the client in an online information system accessible for all project members.

One of the coolest things with eye tracking is that the information about where the user is focusing their eyes and for how long, gives us the opportunity to create images that graphically shows how each part of the system is perceived by the users. For example a heat map visualization. This visualization presents the users fixations as colored areas and highlights problems. The images created based on the eye tracking data is immediately published together with the video material to further complete the picture.

This makes the process totally transparent. It also makes it very convenient for clients to look at the material, which they always find very interesting and valuable. They often discover important new facts about their customer’s behaviour just by going through the movies. My methodology is different from that of other usability consultants, who only publish selected parts of video material or visualizations, making the process less transparent and increasing the risk of missing important new findings.

Analysis

With the raw material shared with the client I initiate the detail test analysis. I have found that the best way to deliver bad news is to let the client follow each step of the analysis. This way, they start absorbing the findings earlier, a process which takes a lot of time for most companies. The analysis is the most crucial of all stages:

• First, I make a detailed second-by-second analysis of what is taking place in all the test situations. This makes each test subject’s experience clear and visible.
• The analysis is written down, point by point, for each part of the system tested
• I name and describe the identified problems and suggest alternative solutions, which can be anything from minor improvements, like adjusting color, size, language, to major structural changes on the site.
• I then organize the findings into focus areas, present the main conclusions and point out where the client should focus the resources.
• In cases where the client after the presentation questions the result, they can always go back and check the raw data and the findings. This is one of the main advantages of being completely transparent throughout the process.

Presentation workshop

When the analysis is complete, I organize a presentation workshop with as many key stakeholders from the client present as possible. The findings are presented and the problems and alternative solutions can be discussed in detail. An action plan can then easily be formed, stating the measures that need to be taken and allocating the adequate resources for seeing them through. Such an action plan is often times very appreciated by the client:

Mårten’s study report was presented with all suggestions of improvements arranged in a step by step fashion, almost like a how-to-guide on how we could improve our site.

Jonas Persson, CEO, Box Experience

What should be expected from an eye tracking study?

An eye tracking study usually identifies three types of problems that need to be resolved in order to improve usability and user experience:

Type 1: Things that are easy to change from a technical/practical point of view
Minor improvements such as changing the wording/language, the placement and size of objects and adjusting colors can have a great and instant impact.Minor changes in small details sometimes makes all the difference in the world:

Terminology we use in the office among ourselves and that we also used on the site, the users found hard to understand and was replaced with an easier language. For example the term “Destination”, which was confusing for the users, was simply replaced with ”Place”. All these minor changes combined resulted in an immediate improvement of the user experience.

Anna Madsen, Marketing director, Holiday Autos

Type 2: Crucial functionality of the site does not match the user’s reality
Sometimes, an eye-tracking test provides the evidence that key functions of the site do not correspond with how the users would like them to be. When the users have to struggle too much due to poor usability, major structural changes on the site or parts of the site should be considered. The test result facilitates the client’s decision-making. The conclusions – supported by the test movies – are well-founded, easy to explain and easy to prioritize. Implementing these type of changes often results in significant improvements in terms of user experience and actual gains. Needless to say, clients find this extremely valuable.

I couldn’t believe my eyes. I knew from experience that we had some usability issues. I had no idea the users had this hard time using our site.

Fredrik Lenström, Former Technical Manager, Ginza

Type 3: Graphic design issues
The eye tracking data painfully obvious when a certain graphic object is not received as intended by the users. In fact, Eye tracking is the only way to make sure that the users put their attention on the correct object on the screen in the correct order as intended by the designer. Systems used for web traffic analysis and even mouse tracking is very useful but can not be said to have this ability. It is scientifically proven that the eyes of the test subject does not correlate with the mouse movements to any large extent. In fact, in a study conducted by Google, less than 20% of the users showed any sign of moving the mouse in relation to what the eyes was focusing on.

Eye tracking is one of the most powerful evaluation tools out there

With an eye tracking study you get to see what the users actually think about your site. The way I see it, eye tracking is the complete package. You get the opportunity to meet users from your target group face-to-face. You get the opportunity to watch them succeed or fail to use your product. You even get to see what they think when they do it. After the test is completed, you get a “detailed manual” on how to improve even the graphic design of you site.

So, if there is bad news to be told about your site, this is in my opinion the best way to get to know about it.

Don’t hesitate to get started right away by contacing me here!