A formal or informal expert usability review is a common starting point for most of the projects we undertake in-house. Oddly enough it’s also one of the only methods in a user centred design process that doesn’t directly involve people who will use the product or service.

That said, we still find it a useful way to capture our “initial thoughts” when we use a website for the first time. This is particularly useful for longer term engagements when you’re likely to be collaborating directly with an in-house team.

What is an expert usability review?

An expert usability review is a method used to assess how usable a website is from an end user(s) perspective, in the absence of usability testing.

The focus of an expert review is to identify issues people are likely to encounter when they use a digital product or service.

In short, an expert usability review should:

Even though an expert review cannot offer the depth or confidence that observing real end-users of a product or service can. It is still a useful method for rapidly (and cost-effectively) sharing independent advice throughout a project’s lifecycle. Hence, its popularity.

usability wordle

How to conduct a useful expert usability review?

Best practices are just the start

An expert review should be informed by heuristics and human computer interaction. However, we believe it should go beyond this too.

Take a user centred approach

A good expert review shouldn’t just rely on best practices it should also be informed by who the specific users of the product or service are/will be. These should be segmented by intent or behaviour and where possible you should try to define their likely:

Understand the business

In addition to this the consultant must have a sound understanding of the:

Involve multiple consultants

At Border Crossing UX we always use at least two consultants to conduct and document an expert review. Why? Given the method, it is absolutely critical that you get more than one consultant to conduct a review. No matter how good the consultant it’s impossible for one person to find all of the different issues a user may encounter whilst using a website.

Complete the following steps

  1. Understand (and segment) who the users of the product or service
  2. Use the website as if they were a prospect or customer
  3. Leverage their knowledge and experience of testing websites with users
  4. Identify issues and recommend changes to improve usability.

If budget allows, we would always recommend that you replace or supplement an expert review with first-hand user research and usability testing.

Our process for conducting an expert usability review

Oddly enough the best thing you can do when starting an expert review is to NOT look or use the website you’ve been tasked with reviewing.

This is critical as you only have one chance to see it, and truly think like, a first-time user. Instead we’d recommend that you:

How accurate are expert usability reviews?

There has been extensive research conducted on the accuracy and veracity of expert usability reviews and usability testing.

Studies indicate that there is up to a 41% overlap in the problems highlighted when conducting an expert review and usability testing, (Fu, Salvendy and Turley).

Expert usability reviews tend to find more problems compared to usability testing. But note that an expert review may still miss serious usability issues that are more likely to be found if testing with users.

What are the benefits of an expert usability review?

What are the drawbacks of an expert usability review?

When to consider an expert usability review?

One of the biggest benefits of an expert review is that you can do it any time.

Developing a new product or service? Commission one to inform the development of your wireframes or front-end designs.

Trying to improve an existing product or service? Commission an expert review to augment usability tests or identify what to test first.

When to consider usability testing?

As we’ve mentioned an expert usability review is great for finding high-level issues. But usability testing is certainly our preferred option if there is:

This is because an expert usability review can never compete with insights generated from (remote or on-site) usability tests with real end-users of the product or service. This is particularly true if a product or service is tightly focussed on the needs of domain experts and their specific task flows. As in these instances it’s nigh on impossible for a consultant to act as a representative user.

Should I combine an expert review with usability testing?

Yes, if the budget and time is available. This is because, research by Law and Hvannberg (2002) has shown that usability testing and expert reviews tend to highlight quite different issues.

Expert reviews tend to highlight more problems with labelling, design and consistency. Whereas usability testing tends to highlight issues with task flows (as in the order a user would expect to complete a series of steps). What’s more further research by Rasmussen indicates that expert reviews identified significantly more skill-based and rule-based issues. Usability testing identified more issues at the knowledge-based level of behaviour.

This should be altogether surprising. As knowledge-based challenges arise when users are learning or modifying their mental model while completing a task.

The conclusion that Fu, Salvendy and Turley (2002) came up with is that the most effective approach is to integrate both expert reviews and usability testing at different times in the design process. Their analysis indicates that an expert review is best applied first or early in the design process. Once these issues have been resolved, usability testing can be used to focus on the higher-level issues that may be missed in the course of an expert review.

Their research also indicated that if:

How should I evaluate my website?

Well it depends. Here are a few parameters that should help guide your decision on the best way to evaluate your product or service.

a) Specialists do them

b) Limited domain expertise is required to complete the tasks to be assessed

c) You already have internal buy-in to fix the issues highlighted by the report.

Want an independent evaluation of your website?

Could an expert review or UX audit help you benchmark your performance and help you focus on what matters most?

Get in touch to discuss your objectives and how we can help you achieve them.

François Roshdy

François is director of user experience at Border Crossing UX. He specialises in helping clients continuously improve the experiences they deliver.