Why were customers not using our products latest feature?

Summary

The product had a limited number of features when developed first. But over time extra functionality was added without an overall plan, this lead to features placed in unsuitable areas and was difficult for users to find.

This was obvious by looking at the analytics of the latest feature 'App Control', we could see that they were not clicking on the correct tab.

Goal

Increase usage of the new 'App Control' feature

Hypotheses

Users are not using the 'App Control' feature because they can not find it

Introduction and context

Users were not clicking on the right part of the U.I. I did not understand why? To find out, I did a usability test, I gave users a task and observed them completing it.

Test results highlighted two main problems:

1. The content was poorly organised it took too long for users to find what they were searching for and
2. Category labels were not meaningful or distinct. They did not create enough context for users to understand the product

Solving the problem

The problems were clear users could not find what they were looking for ('App Control' feature) and the category labels were not clear.

The solution was to organise the product so that it made sense to users. Or to say it another way the information architecture had to improve.

Information architecture is the design of information environments; organising and labelling software to support usability and findability. I adopted two methods to improve the I.A. 1. Card Sorting and 2. Tree testing.

Use card sorting to organise related concepts

If users can not find what they are looking for quickly, they will go somewhere else. To prevent this from happening you need to understand why people use your product? What is the job to be done? Card sorting is a method to help determine how users organise content, label categories and concepts in their heads.

The team wrote down on cards all the concepts and functionality within our system in plain English (roughly 25 cards). I asked users and key stakeholders to organise the cards into groups that made sense to them. I then asked them to label each of the groups they had created.

Where possible we did this in a room together to hear 'the why'. Why people thought certain concepts belonged together and the rationale for the labels they had chosen. Using a label already familiar to the user to explain a concept will reduce the amount of thinking they have to do. Using labels for categories in the U.I. that relate to the task at hand will result in users having a better understanding of the application 'Software concepts (objects and actions) should be based on the task rather than the implementation'.

Use tree testing to measure the findability of topics

Once card sorting was complete I had the basic structure of the system. Then I tested if the structure made sense to users; can they find what they are looking for (App Control feature)? Did the category labels and how the content had been organised align with their mental models?.

Tree testing is a technique that can answer these questions. It involves writing very simple, realistic tasks. Presenting the tasks to users and measure if they find what they are looking for. Users complete tasks using a text-based version (information architecture tree) of the application.

Tree testing results

Above, you can see an example of how all users performed on a task. Highlighted 'First Click' or 'Visited First' is one of the most interesting metrics because it is a good indicator of the suitability of the category label. Based on the results, there was some iterations of the category labels and reorganising the grouping of content within the tree. This process was repeated until I was satisfied that all the key tasks were quickly and accurately completed.

Visual design and retestings

Once the information architecture tree was finished it was time to start the visual design. I created a prototype that matched as closely as possible the final look and feel of the product. Using the same tasks and metrics, I tested the prototype again to establish if there has been an increase or decrease in findability.

There were big improvements in findability for all of the key tasks, the security functionality saw the biggest gains. I moved all of the security features (included 'App Control') from under the 'Account' tab to a new top-level tab called 'Security'.

Outcomes and results

Reorganising the information architecture was a big undertaking but was worth the effort:

1. The Time to find the new feature 'App Control' improved by 97% (from 5m 40s average to 10s)
2. The system was now based on the users' tasks and used labels for categories that were already familiar to them. This led to significant usability improvement
3. I accomplished my goal by restructuring the system and improving the information architecture. Usage of the 'App Control' feature increase
4. Stakeholders were part of the process, and evidence form testing helped convince the development team to refactor the whole product