Insights

Robust Artificial Intelligence-based Test and Learn Is Retail’s Moneyball

Peter J. Charness VP Retail Strategy

Many baseball games are won through a strategy of solid and consistent base hits. But every now and then, players hit grand slams, immediately adding four runs to their team’s total. Of course, no grand slams occur without walks and base hits loading up the bases.

This is instructive when we at UST think about Artificial Intelligence (AI). While retailers clearly want high-profile, high-value Grand Slam AI use-cases, such as a solution that provides winning, market-by-market assortment advice, they cannot forget the singles and doubles that precede and make possible the Grand Slam.

AI Grand Slams are hard to achieve, require high skills that are not available to many Retailers, and can be risky projects to undertake, with uncertain outcomes and uncertain timelines. The ratio of "outs to wins" is pretty high. So AI Grand Slams should be part of a balanced risk/reward profile.

While Retailers work on those AI Grand Slams, they should also focus on AI “singles,” such as the continuous improvement found through Test-and-Learn methodologies, which enhances organizational learning and reinforces important corporate culture if combined with other capabilities.

There are various use cases for Test and Learn new product introductions, own-brand product development choices, pricing strategies, promotions (in particular personalized ones), website shopping experiences, etc. Let's look at how Retailers can benefit by creating a successful Test-and-Learn AI base hit into a grand slam.

Stepping to the Plate

All Retailers are continuously offered (or create) new products to sell in their stores. Retailers must not only decide whether or not to carry the new product, but they must also decide which products to delist to make room for the new entry, where to place the product in the store, how to price it, and which locations would be able to sell the product at an acceptable rate and margin.

If the merchants can easily load this new product as an experiment into an appropriate Test-and-Learn system, that solution can recommend how to structure the experiment, orchestrate the experiment (issues purchase orders and allocations), keep track of the progress, and notify the buyer of interim and final results and recommendations, all the while continuing to learn and improve the entire Test and Learn lifecycle.

Instead of the merchant guessing at, or returning again and again to the same test markets to try out a new item, an AI model can supply the best "bellwether stores" (or bellwether customers) to provide the most valid testing outcomes for the particular type of product in mind. Unlike the usual attempt to prove out that "in general" a new product will or won't work, the test can show not just the overall viability of the product but also which specific stores (or clusters) are the most likely to have good selling outcomes and which will not.

A properly trained model will be picking the locations (or website, promotional elements) that produce the most reliable predictions in the shortest time period. For short life cycle products, using AI to make an up or down decision quickly is imperative. Further there may be stores that have never carried this type of product before and therefore have no track record to base a prediction upon. Normal process and methods fall victim to that well known Retail idiom - the self-fulfilling prophecy: If I've never sold a product in this location, they will never get one as they have no history of success. AI models and proper testing can overcome this sales and margin limiting paradigm.

Training a model to select stores, test pricing, validate store placements is a relatively straightforward and very achievable use of AI technology. While each Retailer must finetune the model, this is not a high-cost or high-risk endeavour. It’s like an opposite-field, line-drive single.

Turning a Base Hit, Into a Double….

Further, encouraging a higher level of Test and Learn within an organization should also come with systemic support for the merchants who have to now organize, execute, follow up, and evaluate the results of all these tests, and to conduct more tests with greater frequency. Chances are there is no formal system (well, maybe an excel control sheet) within the Retail organization designed to make it easy for the merchants to significantly increase their Test and Learn regimen, and capture the data and criteria that will be important for the AI model training.

If the merchants can easily load their experiment into an appropriate Test-and-Learn system, that solution can recommend how to structure the experiment, orchestrate the experiment (issues purchase orders and allocations), keep track of the progress, and notify the buyer of interim and final results and recommendations, all the while continuing to learn and improve the entire Test-and-Learn lifecycle.

Summary

UST helps Retailers pursue a culture of experimentation with our high-impact, AI-powered, Test-and-Learn solution. Through our Innovation Lab, data scientists, Retail domain experts, and powerful Orchestration Platforms, we help Retailers drive increased sales, margins, and shopper satisfaction.

While we love watching home runs, innovative thinkers have challenged the idea of prioritizing hitting for power in recent years. If you have read or watched the Brad Pitt-starring movie Moneyball, you know a former player named Billy Beane flipped the standard theory about what makes teams great on its head. It’s not who hits the most home runs; it is who gets on base the most often. Under the right circumstances, singles are just as important or even more important than home runs. It’s time to bring that revolutionary thinking to Retail.