Appreciating problems.

One of the true qualities of a Product Manager is appreciating the beauty and challenge of a problem. Now and then, a product manager comes across a unique problem that invokes admiration, I dare say.

Smart basket happens to be one of these for our company Bizom. Let me first quickly introduce the Bizom Smart Basket to you. It is a self-learning algorithm that, at the backend, predicts what the retailer is going to order next, and then already adds it to his cart to shorten the cycle of conversion. For every visit your salesman makes to a retailer, the most likely to sell SKUs are already added in the cart in the right quantity.

Now, this truly is an appreciable problem. And at the surface it seemed simple. How do we predict what a retailer is going to order next? Just look at what he ordered last and you know what the demand is. 

In theory, this sounds lovely but then the conclusion follows: does this mean that the retailer will keep ordering the same stuff till perpetuity? Will the value of his order ever increase? How will new products and SKUs ever win over a comfortable shelf space in the outlet and how will consumers discover new products? 

Does this also guarantee that orders will never decrease or stop? Because without growth there's no point. In real life, the salesman does the humongous task of growing business in an outlet; they introduce new SKU lines, create awareness about new schemes and discounts and, through the art of sale nurture every business. That means our machines and algorithms need to replicate all of this. Randomly prediction should throw up new SKUs for testing, old SKUs that were once ordered, and SKUs that due to heavy discounts would have a high chance of being bought.

Imagine getting an algorithm to predict ALL of this at the right time for every outlet in the right quantity. And this is just the surface of the problem.

Eventually, brands would want to grow business from each store, so just like salesmen push themselves and stores to order more, the algorithm should always try to push just a little more and increase the average order value. But how much of a push is okay? There is a thin line between stocked well and overstocking. Overstocking results in a false impression of business, expired products, sales return and most importantly, corrodes your brand's importance at the outlet. 

Halfway through our journey with Smart Basket we came across a different problem. Not all brands are focused on growing sales at an outlet. Wait what?! That's not universally true business sense? And then our customers humbled us with a different nuance of business. "We don't want to keep pushing stores to buy more unless there is a true demand. Otherwise, eventually it hurts our business in the long run. Our game is all about the product shelf life, and identifying the right quantity for an outlet instead of what more can be pushed.", enters the perishable category in the game. This got us thinking of the problem again. That means industries and categories will have different priorities in order to really have visibility of the business on ground and predict .

With this knowledge, we were starting to take one step forward when the looming question had to be addressed. If the underlying principle is growing business and a clear way to do it is to experiment with new SKUs, gently push for more and keep experimenting at every outlet till you see clearly what's worked. How do we measure if our product is successful? There will always be variance. At first, all the brands including us seemed to think variance in order is the only way to estimate if Smart Basket is working. Recommended vs Actuals. But soon, we started seeing gaping holes in this rose-tinted logic. 

The algorithm is literally designed to increase the ordering capacity of an outlet. It is going to step on toes, push more and then self learn it's boundaries. Of course there will be variance. Time matters here. We're looking at sustainably changing ordering patterns and human behavior, not short term gains after all. Hence variance didn't seem to be a good metric on its own. Trend in variance can definitely help us evaluate the algorithms internally, but it still doesn't work for understanding it's value for a customer. Having only 30% variance or 80% variance doesn't mean anything to a brand. Oh, You've 10% variance? Great. But so what? How does that help me? This metric just becomes a number or statistic unless it is in service of something.

And that's when it hit us. Accuracy and variance are internal symptoms but we need to track if we're curing a business problem. Eventually, over the period of time some shift should happen that's desired by the brand. We roughly recognised these in three main buckets:

  1. Avoiding under or overstocking - reduce sales return
  2. Increase the health of the product portfolio from an outlet - pushing/experimenting with newer sku lines. Ergo , average lines sold.
  3. Impacting the ordering capacity- increase in average drop size

When these real metrics move a business outcome, that's a pretty good way to measure, right?

But a core algorithm cannot impact all three outcomes. They need to work differently. So once again, we go back to the drawing board.

Overtime questions such as seasonality effects, current stock at outlets, area demands and even the mere question of if we should give brands a say on what products they want to "push" or let the system decide it based on data. One thing was clear that if we were relying solely on data, YoY patterns needed to be analysed to truly understand an outlet' consumption behaviour at any given point of time. 

Building a product is an iterative process. The problem statement is huge and exciting, but while we can think at scale, one always builds one step at a time. We need to establish proof of concept and always reassess if we're building in the right direction, how much time and effort do we want to invest.

And with this, Smart Basket has evolved. I'm not delving into solutions or doing a sales pitch here. Today is just about admiring and appreciating the problem space of Smart Basket. It has all my appreciation.