to benchmark or not? that’s not quite the question.

After nearly 20 years in the insights game, we have seen our fair share of products, innovations and renovations. Everyone wants an answer to the same old question: ‘Is my product good enough?’ Naturally, that’s the point of consumer research.

How you determine what’s ‘good’ product performance is a little more flexible than one might think. Our clients often ask:

• Do you have a benchmark database?
• What is a ‘good’ score?
• Is our performance strong enough to launch?

Sure, these are great questions, but there’s no right answer. We understand the comfort in knowing your proposition stacks up against a big database of norms. Yet, it’s more important to ask, ‘what are we really comparing against?’ In this article, we’ll discuss whether or not to rely on benchmark data in product performance analysis.

fetch the right yardstick

NPD projects are like snowflakes — no two are ever alike. And we mean, ever! Each project has different specifications and a different product, likely in a category distinct from the last. These unique qualities call for benchmarks that make sense for your project, not what’s been done or what a database might make you think.

Historical benchmarks are a good yardstick, though they aren’t the be-all and end-all when progressing a new or reformulated product. Break new ground and you will have a new product promise potentially in a new category. Which begs the question — what’s your comparison point? Can you find anything close to what you’re trying to achieve for a useful gauge of success?

historical data versus current data

Broad historical data can give us a guide for benchmarking, however, current data is far more useful. Perhaps there isn’t a benchmark you can include in your assessment. That’s when you revisit those fundamental key performance indicators and decide whether you should proceed with the innovation:

• If you’re talking to a broad audience, use the available information to decide whether the opportunity is worth pursuing. Factor in considerations like margin and distribution.
• If you’re talking to a targeted audience (to whom a proposition is specifically aimed), then look at how strong the performance is right now.

Here’s the thing. There’s a significant challenge in innovation, but also in product renovation:

• If you’re breaking out, don’t expect to have an old-fashioned, 65% top-two-box purchase intent (unless you have found the absolute heartland consumer for your intended proposition).
• If you’re renovating, gauging customer expectations in your category is vital. Compare your product to what they say and the core competition. Those are your benchmarks.

Generally speaking, if you have a key competitor, a comparable offering in-market or an existing formulation you want to tweak — include it!

Categories change. Competitors change. Consumer palates change. So we need to keep our methods up to date as well. Don’t get me wrong — expansive databases of concept, pack and product scores are a useful reference point. But don’t make them your ultimate benchmark.

And, if all else fails, lean on your insight and innovation partner to assist you.

subscribe for the latest research and insightful updates on NPD innovation

pointers for product performance analysis 

Benchmarking databases are handy for ‘traffic light’ assessments, a substitute for real-world experience. Remember, though, that each project is different. Numbers (against benchmarks) in a bit of green, amber or red can provide a false sense of certainty that a product will succeed or fail.

Extensive innovation experience and advice beats any benchmark database and the high-level of noise within it. Sometimes, it draws a different conclusion. It’s an insights and innovation partner’s job to cut through the noise of peripheral data and historical ‘expectations’, and work with your team to make an educated decision about progression, optimisation or deletion. Most researchers with a couple of decades of experience can help you work out whether your product is good or bad.

At the end of the day, experience in innovation and new product development wins out. Keep those in-the-know closest to your project for an honest assessment of where you sit.

‘Why?’, you ask? Because consumer expectations are constantly evolving. As is the consumer palate, which influences what is acceptable or unacceptable in a sensory profile. And because a game-changer in your category could reshape consumer expectations of your brand.

over to you 

Product performance analysis is more than determining a ‘score’. Understanding your audience and the gaps to optimise all help diagnose if you’re onto something big. Should you ever find yourself stuck on making sense of things, call on your experts. Heck, call on us to help!

Get in touch here or via hello@playinnovation.com.au or (02) 8097 0200. We can’t wait to hear from you.