Tag Archives: alpha testing

The Startup Product Manager’s Dilemma: Getting the Right Product by Ignoring Some of Your Customers

By:  Andrew Johnson, Ph.D.

swiss army knife

Make sure that your product offering has all of the features that it needs to be successful but no more than that.

Figuring out what you first product or service is going to look like, how it will be priced and how you will sell it is at the top of the ‘to-do’ list for the Product Manager of a startup.  As an experienced startup product manager, you have been very careful to balance the enthusiastic advice and suggestions of the founding team with the wealth of feedback that you have gotten from potential customers in the field.  After getting all of this input, you will rarely have an obvious answer to the ‘What is it?, Who wants it? and How much would they be willing to pay for it?’ questions.

Don’t listen (only) to your Sales or Operations colleagues
Everyone on the team wants what is best for the company but you should be wary of your Sales and Operations team members.  From an operations stand point, the fewer the options the better for them.  They are looking to provide the best product at the lowest manufacturing cost with the highest quality.  Every different version of a product or additional set of features introduces complexity into meeting their goals.  On the other hand, your Sales team, as good representatives of your customers and their wants and needs, will want to have as many versions as possible so that they have the ‘perfect’ solution for every one of their customers.  Simply stated, you need to consider both viewpoints as you determine the final feature set or version options of your initial product offering.  The correct answer, of course, is somewhere between 1 and 100 versions or feature sets.

Determining the right number
You may not get this exactly right but you need to be close.  This is why you need to lean toward determining the smallest feature set of your offering that will be compelling to a large enough number of customers.  Another way of looking at this is that you need to determine what product your development team will most easily and quickly be able to produce that is commercially viable.  Running a well-managed alpha evaluation (see ‘Alpha Evaluations: Going from Great Science to Great Products’) will help you to know if you have got the right offering for your launch or if you will need to make changes.  Steve Blank defines this as the High-Fidelity Minimum Viable Product.

“Minimum Viable Product:  (The) simplest minimum viable product (i.e. a website with the core features implemented, a demo version of a physical product)…”   Steve Blank – The Startup Owner’s Manual

Why go small?
Until you have your first sales with paying customers (free or hugely discounted placements don’t count), you will not know if you have got the right product.  The best feedback that you will ever get is from paying customers.  If everybody tells you that you have a great product but nobody will actually buy it, you have the wrong product.  If only a few people will buy your product or after an initial burst of enthusiasm from ‘early adopters’ you find it difficult to grow your sales, you do not have the right product for the customers that you are targeting.  Going small at the beginning will allow you to get valuable feedback from paying customers as early as possible and limits the amount of effort and resources that could be wasted if your first product to market is not quite right.  This allows you and the team to make changes and/or additional features that will allow you to start winning the sales that your great product deserves.  By keeping the initial offering limited, you will still have the resources to make the changes you need to end up with a winning product.

Selected Tips for Getting the Right Product

  • When talking with potential customers, be sure to determine which of their suggestions for your product or service are ‘needs’ and which are ‘wants’.  Your first product should include the smallest configuration or feature set that will satisfy the majority of the consensus ‘needs’ you learned from this group.
  • Use the ‘wants’ and ‘nice to haves’ from your outside feedback to help you to develop a ‘Product Roadmap’ to guide you with future releases of your offering.
  • Make sure that the issue that your product or service aims to fix is compelling enough for people to want to ‘pay to have it go away’.  A product that is just more convenient may not be seen as worth the additional cost by your future customers.  Your discussions with them should focus as much or more on the problem you aim to fix as you do about your proposed ‘fix’.
  • Have an upgrade strategy worked out in advance of your first product launch.  You need to make sure that your earliest customers will have access to the latest versions of your offering that still makes sense for your Revenue Model.  This will help you to deal with any push back from your first customers who may wish to see your product proven by others first.  Being able to offer the chance to be the first one on the block with a viable path for access to future upgrades and releases will significantly help you to deal with this sales issue.

Highly Suggested Reading:

The Startup Owner’s Manual: The Step-by-Step Guide for Building a Great Company’, By Steve Blank and Bob Dorf, K&S Ranch Publishing Division, 2012.

Picture Credit:  © IbanMontero | Dreamstime Stock Photos & Stock Free Images

Alpha Evaluations: Going from Great Science to Great Products

OK sign made by hand

Give your customers what they want with great alpha testing.

By:  Andrew Johnson, Ph.D.

Why do alpha testing?
Alpha evaluations, also known as alpha testing, can be a crucial way to reduce the risk of launching a product that will not be accepted by your customers.  When alpha testing is done well, this can provide crucial insights on how well your final product is going to be valued by customers and what is not.  A good early understanding of these product attributes will save you considerable effort and expense during your product development.   Essentially, this effort will tell you how to best transform your innovative science into products your customers will want (and purchase).

Alpha evaluations are:

        • Run by R&D not Product Management
          • These efforts should provide R&D with the insights they need to make changes to the product performance and/or feature set.  Further R&D may be necessary if the current prototype does not meet the performance expectations of your target customers.  Remember to keep product management in the loop during this effort though.
        • Tests of very early prototypes to gauge customer acceptance of proposed product
          • For software products, consider creating a mockup of what the software will be like using PowerPoint slides.   You walk the evaluator through what each tab and feature might be like very quickly with a minimum of up-front effort.
          •  For reagent products or kits, don’t worry about any fancy labels or packaging.  Prepare some samples and use them with your tester.  It is important not to just send them to the tester and ask for feedback but really to work side by side with them in your lab or theirs to maximize getting the most relevant feedback on your current concept.
          • For instrumentation or equipment, bring ‘bread-board’ version or other mock up prototype to tester’s lab or invite them into yours, for the evaluation.  This does not need to be ‘pretty’ and not all of the functionality needs to be in place.  You want to get feedback on the core function of your instrument or equipment to see how well they accept it.
        • A source of customer insights that will change your final product
          • Unlike beta testing, changes to the product will be made based on alpha evaluator feedback.  Be sure to get feedback on the desired (or expected) performance characteristics of your proposed product (e.g. acceptable linear dynamic range, sensitivity limits, through-put, ease of use etc.)

Important safety tip
Beware the danger of this effort becoming a research project (leading to time and cost overruns not to mention confounding your commercialization plans).  Alpha testing is not feasibility testing or discovery research.  The goal of this effort is to see if the current concept for your product will be valued by potential customers.  At this point, it is assumed that the underlying science works.  Set specific goals for what you hope to learn and stick to that.

When not to do alpha testing
Alpha evaluations may not always be appropriate.  If you are making a better version of an existing product (yours or a competitors) alpha testing is most likely not needed since you already have plenty of information on what customers like and don’t like about the current offering(s).  Use alpha testing when you can’t easily answer “yes” to this this statement.

“The design of my current product offering will meet or exceed the expectations of my customers.”

© Simonkr | Stock Free Images & Dreamstime Stock Photos