Validate or Die: Using Validation to Build the Right Product

This was a guest post featured on Mind The Product, an international product community. Mind The Product started in 2010 with the very first ProductTank meetup in London and followed by the Mind the Product Conference in 2012 it has now grown to consist of 2,500+ members and sold out events in 12 cities around the world.

As product managers and entrepreneurs, we know the importance of validating an idea before committing to getting it built. However, validation is easier said than done. In fact, it’s possibly one of the hardest things you will ever do in your product life cycle, otherwise products and startups wouldn’t have the abysmal 90%+ failure rate they’ve become famous for.

In this post I will draw on what I’ve learned over the years, through first-hand experiences and other sources and leave you with a validation framework that I hope you’ll be able to use. And, in the remote case that you’ve been living under a rock and the whole lean startup movement somehow got past you, this post will try to bring you up to speed (and possibly, get you slightly ahead of the pack), provided you force yourself and your team to take action!

But first…

Why Validate?

1. When it comes to new products, by definition you’re making assumptions, sometimes a lot of them. You need to validate your key hypotheses and assumptions as early as possible. Some of your assumptions are going to be wrong. If the ones you get wrong are significant, you’re not going to be in business for very long.
2. Validation will help you avoid the most common problem of them all, spending time and money building something nobody wants. If you ever come across a failed product, chances are that it was based on an idea that the founder was enamoured by or found cool and interesting, rather than something that solved a real pain point for real people.
3. Validation also forces you to get in touch with your users, which could save you the pain of building a product that is hard to use or understand, or something that is ahead of it’s time. (Remember Webvan?)
4. If you figure out that you’re headed down the wrong path, it’s also easier (and a lot cheaper) to change course sooner rather than later. And, it’s all the less likely that you’ll have to pivot later.
5. Validation will help you refine your ideas and learn more about your users.
6. And last (but certainly not least), it will help you figure out if people will buy your product before you build it.

Validation will help you answer questions like:

• What is the target market that I want to address? How big is the addressable market size?
• What is the biggest problem these people face? Is it an isolated or recurring problem?
• What kind of product would help relieve that pain?
• Are there real people willing to pay for such a product if it existed?
• Will building it be worth it? Or, is the market big enough?
• And, am I building the right product?

So let’s get to it.

What Should You Validate And HOW?

People don’t want to buy a quarter-inch drill. They want a quarter-inch hole!
-Theodore Levitt

We need to have a clear understanding that customer needs are separate and distinct from the solutions that people buy to address those needs. Keeping this in mind, validation comes in many forms:

• Validating market demand
• Validating the customer problem/pain point
• Validating the product or solution

Validating demand and validating the problem come first. Then you come up with a solution (a prototype or design of your product) to solve the problem that you have identified. Finally, you validate the product you have thought up, separately. Validating the product you have imagined (before building it) is important, because there are a gazillion ways to solve the same problem.

Validating Demand

The goal of validating demand is to get an answer to the question:

What is your total market potential, and how much of that market can you/will you be able to capture?

Here, you want to work out a realistic market size that is feasible, otherwise you risk there not being enough demand for the product because it doesn’t solve a big enough problem for its market or it doesn’t solve the problem in a way that works well for users.

You want to ensure that the market is big enough and that there are enough people willing to pay for your product or service. Ideally, at the end of this exercise you should be able to understand your users’ interests and motivations better, and hone in on a small group of people who feel enough pain to buy your product. Also, if done right, you’ll be able to use what you’ve learned to find similar (or parallel) markets to expand into in the future.

Be sure to hone in on a specific target market where the constituents have similar behavior patterns. It doesn’t matter if it is small, as long as it is sizable enough and there is potential to expand into other niche markets in future. For example: music lovers is too broad;college students that listen to punk rock music is better.

Methodology, tools and techniques for validating demand:

• Industry reports
• Look at traffic of competing websites, using tools like comScore,Compete and Quantcast. If you know that your competitor has 1 million unique visitors per month, at the very least you know that the market potential for your website is 1 million.
• Google Adwords Keyword Planner – use the Google Adwords Keyword Planner to determine how many searches related to your market take place on Google each month
• Look at Google Trends data to look at search query volume
• Look at competitor offerings and try to find (or make an accurate guess of) pricing and revenue numbers
• Note: Be wary of forecasts of market size, many of which use questionable methodologies or flawed assumptions

Validating The Problem

The goal here is to make sure that we find a real problem or customer pain point that enough people face, so that we can later focus on solving that problem.

Methodology, tools and techniques for validating the problem:

As Laura Klein mentions in her book UX for Lean Startups:

Now that you’ve picked your specific market, find five people who are in it. If you can’t do this fairly easily, either you’ve picked too small a market, your market is too hard to reach, or you’re just not trying hard enough.

Don’t just go out and start talking to friends, family and random strangers on the street – that could be a mistake. These people are (likely, but not always) going to be very different from your target market. Go where your REAL target users are, observe what they are doing, and think about how you can make those tasks better and/or easier. Sometimes, your users are out there in the real world, but sometimes they are more easily accessible online (for example, on discussion forums, Facebook groups, and online review sites like Yelp or Epinions, etc. If possible, schedule a Skype call with them.)

Ask them open-ended questions about the tasks they go about doing in the identified problem domain and the pain they are experiencing. Ideally, they will be able to show this to you first hand. This will help you learn how people do things, why them do them a certain way, and also what solutions or workarounds they are currently using to solve their problem.

Look out for patterns – groups of people doing things the same way, or complaining about the same thing and either observe or ask them how they solve those pain points today. Keep in mind, however, that sometimes users don’t know they had a problem until they’re shown the solution (your product). Dropbox is a classic example of that. Just remember that you’re there to observe, listen and learn – not to sell your test subject on your product idea.

This kind of thing can have a big impact on your product implementation, your user interfaces and so on and will help you ensure that your product more naturally fits into the daily lives of your customers. It will help you minimize the amount of training you will have to give your customers once the product gets built because they don’t have to learn new habits to use your product, thereby shortening abandonment rate and increasing the likelihood of the customer sticking it through after the first interaction with the product. It will help you trash bad ideas that you were previously considering very quickly. All this before you start building anything. At the end of this exercise, you should be left with a validated problem in a validated market.

Validating The Product Or Solution

Ideally, building and validating the product should go hand-in-hand and be an iterative process, with some amount of validated learning along the way (or at the very least, with every iteration you make). This is the final step and comes after validating demand and validating the problem. It involves constantly asking yourself the question: ‘Does this product really solve the problem, in the market we have identified?’. Product validation is more complicated and should be done via iteration. Ideally, it looks something like this:

Methodology, tools and techniques for validating the product or solution:

You can use landing page tests to market and sell your product before it even exists. Now, drive relevant, targeted traffic to your landing page using some marketing channel (free or paid) – Google Adwords, Facebook Ads, or anything else you can think of.

If you’re unable to generate enough interest in your landing page tests before you have a product, it either means that your marketing strategy isn’t good enough (in which case A/B testing more landing pages is the way to go), or you’re trying to sell a product that there isn’t enough demand for. This is not to say that the problem you have identified cannot be solved in another way (you have validated the problem already, after all). It just means that you need to create a different solution for the problem. This approach drastically reduces your chances of failure when you eventually hone in on a product worth building and also helps you prioritize your product features based on how users react to your marketing copy. Landing page tests have the added advantage of validating the market at the same time, so you could start here if you have already narrowed down a problem you want to solve.

Once you have validated the idea, you need to validate the product you have imagined (before building it), because there are still several ways to solve the same problem. The best way to do this is by showing potential users a semi-functional (or even easier, a non-functional, but interactive) prototype that looks and feels like the product. This is as opposed to asking them for an opinion on your idea, which bears no correlation with real people using your product).

Then, you (silently) observe how potential users interact with your prototype, looking for common problems, symptoms and patterns, find out if it’s compelling enough to pay for, and course-correct if needed. Just don’t bias your test subjects by treating them as potential customers – now’s not the time to sell them on your product. If they don’t ‘get it’, either because they don’t understand what it does or how to use it, or they’re unable to find all the features, that’s an obvious sign that something needs to change. As you can see, including prototyping in the product validation cycle will both help communicate your ideas and allow you to harness the power of failure to create something truly innovative.

You could also make your users interact with your competitors’ products, to uncover holes in their products that you will smartly fill.

You may have to iterate through a number of prototypes before you end up with a validated prototype at the end of the exercise. The same validation would then have to be done for your designs and, eventually, the end product you are building.

The validation story doesn’t end here, of course – it goes on at every step of the product life cycle. And then again later, after the product is out there, every time you fix a bug, add a new feature or make a product change.

A few resources worth mentioning:

1. fivesecondtest.com – to help you guage a user’s ‘first impression’ of your exer experience or design
2. unbounce.com – to help you create landing pages easily
3. aytm.com (ask your target market) – could be a useful tool to find your target market and send them surveys
4. optimizely.com (Google Analytics also offers A/B testing functionality) – to run A/B tests and measure click-through rates on different page elements
5. usertesting.com – to help you get real user feedback (though, not necessarily from your target demographic)

Bottom Line

If you use validation the right way (at every step of the product development cycle, and are disciplined enough about it), your customers will practically lead you to the goldmine, telling you virtually everything you need to know to make your product a success.

References:

• Wanna Create A Great Product? Fail Early, Fail Fast, Fail Often
• UX for Lean Startups: Faster, Smarter User Experience Research and Design by Laura Klein
• Creating A Killer Product

17. September 2013 by Kunal Punjabi
Categories: Uncategorized | Tags: A/B testing, lean product manager, Lean Startup, Product Management, product planning, Slider, validation | Leave a comment