Testing a concept is a valuable resource for product development.

Reading time: ~7 minutes (but the juicy bit is about 4).

Intro

The team came up with an idea for a new feature, that is unheard in the industry. There is excitement, and a lot of conversations take place, multiple points of view are considered, and the opinions are not always aligned.

We need data, we need to have a starting point to understand how to tackle the opportunity, possibly using the less resources as possible, because resources are scarce, as usual.

What would be the best position within the journey for that brand-new feature? Would people understand what it is and how it works? What ‘s the minimum that we should build for it to work efficiently, being self-explanatory and give our product that edge that would make it stand from competition?

Testing the idea.

Delivering an efficient, satisfying, and effective product is a complex job, especially because there is a constant risk of investing time and resources in something people do not need nor understand. 

The best way to avoid such a situation is to test as early and as often as possible and two effective methodologies in similar cases are Concept testing and Usability testing. 

They happen in two distinct phases in the product development process, each serving a unique purpose and focusing on different aspects of a product, even if they can be performed multiple times during the ideation and delivery process. 

Concept Testing is typically conducted during the ideation phase, and it’s goal is to evaluate the validity of an hypothesis or concept, usually in the shape of a clickable mock up (but it’s not a conditio sine qua non). 

It aims to gather feedback and insights from potential users to determine whether the concept resonates with them, addresses their needs, and has the potential to succeed in the market, but also to gather information about what are their expectations and ultimately their mental model. 

It has a strong attitudinal component (what people think, and how they interpret the world around them).

On the other hand, Usability Testing is typically performed on a functional prototype (it can be performed on any level of fidelity, from wireframes till fully clickable prototypes, even on paper) or a working product to assess its usability.

It focuses on identifying usability issues (discoverability, learnability etc), the journey’s adherence to user expectations (narrative and storytelling), user interface problems, and user interaction challenges, and it’s mainly behavioural, meaning that the focus is observing what people do.

Concept Testing usually occurs in the ideation and concept development phase before a final prototype or product is built, and its goal is to validate the product concept, gather feedback, and make decisions about whether to proceed with further development.

In a nutshell, it gives the team enough information to design a good solution that meets users’ needs and expectations. It can also help assess market fit in some conditions.

Usability Testing happens in the later stages of product development when a functional version of the iteration is ready (from a wireframe to a coded prototype, virtually you can test everything) for testing, informing the iterative process that should follow. 

By delivering the product in small releases and adopting an iterative approach, we can get feedback very quickly, avoiding situations where making amendments would require a lot of effort or even be impossible.

That's why testing must be an integral part of the iterative process.

Our Recipe.

There are multiple types of concept testing; the one that works best for us is based on a despicable mix of usability testing and interview techniques. 

We mock up a clickable prototype of the full journey, covering multiple scenarios, pretty much mimicking a mature product – depending on a number of variables, it can be from low fidelity to hi-fidelity – then we ask our panel to perform several tasks on it while they think aloud, describing what they are doing, thinking and feeling. 

On top of it, we interact and we create a rapport with the users, we ask what they were expecting, if they’d like to move things around, we even ask them to make suggestions and so on, which it’s the opposite of what you’d like to do when you test the usability of a product. 

Just to clarify, it’s basically usability testing on a concept, just with an expanded conversation (it shares a lot also with a contextual enquiry), and as such, we must be very careful when we create the screening for the panel, and analyse what the users say.

The goal is to bring people into a controlled conversation (avoiding leading questions). We want to put people in a comfortable space where the narrative that makes more sense to them slowly emerges, bringing qualitative feedback related to the overall concept, potential features, and perceived value.

In the end, the prototype can be even scraped entirely, what we are interested in is to get enough data to inform the ideation and delivery of the actual product.

Risks.

We are supposed to challenge our hypothesis, validate the good one and discard the wrong, so it’s important to consider that we have biases and expectations too, and it’s easy to be affected by what we think should happen. 

A common trap we should avoid - among others - is confirmation bias, which causes us to consider only the information that is consistent with what we are already thinking, which will prevent us from having real benefits from the test. 

The ostrich effect is another enemy, also known as the ostrich problem, which is a cognitive bias that describes how people often avoid negative information. 

For example, it’s very easy to find a good reason why the user didn’t get a certain process/journey, while probably we simply got a bad design.

Final recap.

In summary, concept testing helps determine whether a product concept is worth pursuing, while usability testing assesses the usability and user experience of an already developed product or prototype.

Both are crucial steps in the product development process, with concept testing occurring in the early stages and usability testing taking place later to refine the product based on user feedback.