How to predict demand for New Products?

Originally published on LinkedIn · October 18, 2017

Research shows that approximately 39% of a retailer’s revenue comes from New Product Introductions (NPIs). That’s an excellent rate, but what makes NPIs so different from other products?

Every season, a retailer introduces dozens of new products. In this post however, we’ll cover only new products that are incrementally new, not new inventions. That includes the following cases:

a)   Newer versions of existing products.

b)   New products in an existing product category.

c)    New products in an entirely new product category.

Each of the above cases comes attached to a different set of challenges and approaches when it comes to predicting demand. Let’s take a closer look at some of the differences among each set:

(a) Newer versions of existing products:

a.    These are relatively straightforward. These products will have a set trend and sales profile, allowing us to leverage the sales history of the existing products.

b.    New products in this group are assumed to be similar to existing products in terms of market space, customer base, geography etc.,

c.    Forecasting for these products can be accomplished with a simple regression analysis using historical sales data.

(b) New products in an existing product category:

a.    These products fall into the same product line or category at large but are new products that were never launched in your brand.

i.    These products may or may not have a truly comparable item in your product catalog.

ii.    Predicting demand for these NPIs is a challenge, as they do not have any historical sales that may help project future sales.

iii.    Before anyone tries to understand what the solution looks like, we should consider what details we know about the NPI that can be useful with the tools and techniques at our disposal.

(c)  New products in an entirely new product category:

a.  This includes launching a new product in a fully new category, something along the lines of expanding your brand.

b.  Market research, competitor analysis and external factors play a big role in predicting this group of NPIs

Just by looking at the above breakdown, it becomes pretty obvious that there is a massive set of differences between the various types of NPIs. You might even have a product launch planned that doesn’t fit neatly into any of those categories. To make things a bit easier, let’s simplify it even more. At the most basic level, we’re dealing with two classes of NPIs: Those with historical sales data and those without. Traditional methods of regression and simple statistical methods like moving average and exponential methods are plenty for NPIs with historical data. But when we’re in a situation without historical sales data, what data can fill in the gaps? You should start with qualitative details like:

a.    Product attributes

b.    Usage attributes

c.    Marketing parameters

i.    Catalog featuring

ii.    Ads on your (or affiliate) websites

iii.    Any personal consumer interviews or feedback collected.

iv.    Social media etc.,

Essentially, your goal is to collect data about any qualifier that defines what your NPI is, what it does, how it is used and what marketing efforts support it. We can set aside price for the time being, as price elasticity is an entirely adjacent use case that goes with it.

This qualitative data becomes our starting point for the next steps. Since every brand is unique and every product includes a distinct sales profile, how can we move forward without an actual sales history? Let’s start by formulating the problem a bit more clearly.

We’re looking at a bunch of distinct and different products, all with their own unique history and traits. But are all the existing products really so unique that we can’t separate them into groups?

Imagine we group all of the products in question into three or four clusters and start to take a closer look at the similarities among them. We have the sales history for those products, as well as their time series. With some digging, it should be possible to fill out the various qualitative attributes as well, giving us their launch time attributes. Armed with this information, we could group our products into sales profile clusters and launch profile clusters to see where each of the products belong. That information in hand, we’re setting the stage to determine how your knowns can be used to apply for your unknowns.

Now what do we do with the NPI? At this point, we’re still not sure how to take advantage of the above concept to project demand for our new NPI.

We can start by grouping the NPI into one of the launch profile clusters and take a closer look at its neighbors based solely on qualitative parameters. At this point, we’ve progressed far enough to have something we can at least touch and feel. With this information in hand, we can investigate and determine what sales profiles the neighboring products fall under. This is where it gets a bit tricky.

Let’s say we are able to precisely identify each of these products’ attributes, as well as their sales groups. There are a variety of creative ways to determine what products should be considered and based on what technical criteria: weighted average, hierarchical groupings by significant features, etc. (This is a rather technical zone, and can be addressed effectively through the use of a purpose-built tool or technique.) At this point, we’ve identified a path and a conceptual understanding of what will be done with your data. But who should handle it, and how?

Someone with a deeper understanding of how NPIs for your business are launched, coupled with a strong knowledge of the conceptual framework laid out above, can connect the dots here and take it to the next level of actual execution. Ideally, they’ll start with machine learning based regressors, clustering algorithms and feature extraction processes that extract related characteristics from each product’s time series to drive the sales cluster, forming the cornerstone of the results that will allow you to predict demand at varying confidence intervals.

The benefits of knowing the accurate predicted demand for NPIs should be obvious. With a more accurate forecast, you can better plan your inventory, avoid stockouts, minimize obsolescence risks and, most importantly, set optimal price points based on predicted demand, not cost-based pricing. This is especially valuable, as we often fail to realize how many sales opportunities we miss out on as a result of failing to capitalize on maximum revenue margins.

The benefits of predicting demand for NPIs are many-fold, as an accurate prediction forms the starting point of an upstream process that drives inventory optimization and revenue maximization.

References: https://ocw.mit.edu/courses/engineering-systems-division/esd-260j-logistics-systems-fall-2006/lecture-notes/lect5.pdf