During the 2017 solar eclipse, eBay noticed shoppers were leaving its website to visit Amazon. Potential customers were purchasing eclipse-ready sunglasses that the eBay rival was selling for $59.99, as opposed to the $9.95 they were going for on eBay. The sunglasses became the number-one selling item on Amazon; clearly, eBay needed to change its strategy. The retailer and reseller decided to update its pricing model, capitalizing on the increased willingness to pay more for “premium” glasses, despite the fact that the products were identical. That was all made possible thanks to machine learning.
“A manual process simply could not analyze the ridiculous amounts of data necessary, and provide product-specific actionable insights,” explains Yuval Yifrach. Yifrach is the CEO at Market Beyond, a Tel Aviv-based AI company that provides the machine learning analysis eBay used to tweak its pricing. It’s one of many startups harnessing machine learning and big data to predict how consumer behavior, doing for the future what surveillance platforms like Facebook and Google have done for our presents and pasts.
Future behavior is fast-becoming the holy grail of marketing, and artificial intelligence is proving a key weapon in convincing consumers how and where to spend their money. More and more companies use machine learning to accurately predict what will encourage people to part with their money; others use it to steer their product roadmaps entirely.
This new market is impressive, but it also raises serious concerns. These technologies may create a self-fulfilling loop. As human action becomes more predictable, the more consumers are fed with prediction-informed products versus new innovations. The use of machine learning potentially reduces choice and variety.
The unsettling possibilities haven’t stopped startups and established companies from launching tools targeting consumer prediction. One of the biggest names in the market is Xineoh, a Johannesburg-based company that uses machine learning to match potential customers with products so retailers know which items to stock.
“We transform customers’ raw transactional data housed in their respective data lakes into actionable predictions of the customers’ behavior,” says Xineoh CEO Vian Chinner. “This roughly translates into predicting who will buy, or consume what, at what price, at what quantity, when and where they will buy or consume.”
Chinner explains that because of the vast quantities of data involved, machine learning and neural networks are necessary to predict consumer behavior in this way. Any standalone equation built to predict behavior would be much too complex and unwieldy. “On a small amount of data, humans are superior to [machine learning] on inferring patterns, but on a large amount of data, machine learning can much more readily extract meaningful correlations and patterns.”
More importantly, Chinner says that machine learning provides far greater accuracy and insight than previous methods of predicting consumer preferences. He claims that Xineoh’s platform consistently provides clients with superior performance and results compared to its rivals. “We have won every single such ‘bake-off’ scenario and have done so in many cases beating the closest competitor by multiples,” he says.
Xineoh is far from the only AI-driven solution of its kind. The previously mentioned Market Beyond harnesses its proprietary neural network to provide brands and retailers with insights about what products are likely to sell and which aren’t, as well as with data to help retailers prevent customers from shopping elsewhere.
Yifrach adds that Market Beyond’s services have detected similar “deficiencies” in the pricing and marketing of millions of products, in each case identifying a variety of factors that could put off customers–which includes things like product assortment, pricing, search, traffic, and shipping. And in eBay’s case, processing these factors via machine learning and then predicting what consumers will buy enabled it “to recoup 15 percent of the revenues it had previously lost due to unfavorable price comparisons with Amazon,” Yifrach says, which equated to $100 million of additional revenues per month.
…
This market is growing quickly. Companies like Quantcast, Selligent, and Flytxt (and of course Amazon) creating similar AI-based prediction tools. And according to a study published in March by the International Data Corporation, the global retail sector is set to spend around $5.9 billion this year alone on AI-based solutions like automated customer service agents and product recommendations. Meanwhile, overall spending on AI technologies will reach $79.2 billion by 2022 according to IDC, and Gartner predicts that by the same time the overall business value provided by AI will reach $3.9 trillion.
These projections aside, most of the new AI-based prediction tools are still too new to reveal their accuracy. Even so, businesses are bullish about their results. Jersey-based AI firm GNY.io built a blockchain-powered machine learning platform and claims a 70 percent accuracy rate for its predictions, which according to CEO Cosmas Wong are used by a variety of blockchain companies to forecast the behavior of their users.
According to Wong, the GNY.io platform analyzes millions of transactions made by hundreds of thousands of customers. It can use this data to arrive at predictions of which cryptocurrencies the users of a crypto-exchange would be interested in trading.
AI-based analysis is also being used to dream up products that don’t even exist yet. London-based Black Swan Data uses machine learning algorithms and social media data to predict consumer trends, enabling its clients to design products. One client, Pepsico, used Black Swan’s natural language processing technology to analyze real-time internet chatter about beverages while its constantly learning algorithms organized these conversations according to what people want in a soft drink. This included things like “natural ingredients” and “health benefits.”
Thanks to Black Swan’s analysis, Pepsico used macro trends to launch Bubly, a range of zero-calorie, sweetener-free sparkling waters. Bloomberg reported in April that Bubly is competing well against its rivals.
…
As powerful as these systems are, there’s concern that using AI to predict consumer shopping decisions could have some less-than-desirable effects. Most notably, there’s the chance that overuse of this kind of artificial intelligence could result in a narrowing of product choice, as AI-based platforms encourage retailers to focus on products most likely to sell.
As Xineoh’s Vian Chinner explained to VentureBeat last year, Netflix is a good example of how data analysis can lead to reduced choice. “Initially, a large portion of their customers visited with a specific video in mind that they wanted to watch,” he said. “If a streaming provider wants to satisfy this need best, they would need to have a large selection of videos, starting with the most popular items. They realized […] that if they simply held the items that are most relevant to each customer, they needed only a quarter of the expected items in inventory, leading to large savings.”
Narrowing choice is often implicit in AI-based analysis systems. Yuval Yifrach explains that Market Beyond’s technology helps retailers identify competitors’ popular products, which influences its clients’ product stocking choices. “There is no doubt that choice reduction is a great risk posed by AI,” agrees Brandon Purcell, a principal analyst at market research company Forrester. “We already see this today on social networks where people end up in echo chambers that reflect their own views and on e-commerce sites where the products displayed are products our ‘lookalikes’ purchased.”
Still, Purcell notes that even if AI narrows in on a smaller range of products for each customer, differences between customers should prevent markets from becoming too homogeneous. “Different types of people will always have different tastes and desire different products,” he says. “AI can be used to exploit those differences, identifying trends as they arise, and creating the right types of products for different groups. Since AI learns from people’s behavior, product homogenization won’t happen unless human homogenization does first.”
Despite this, the situation is potentially exacerbated by retailers’ desire to use AI to manipulate consumer behavior. Once it’s determined certain products sell better than others, retailers are less inclined to stock alternatives, leaving customers with fewer choices, both in items and in price.
“Choice reduction will have the greatest impact here, because while the number of total choices may not decrease, the number of choices a specific customer is presented with necessarily will,” Purcell predicts. “Retailers may end up marketing the same small group of products to the same small groups of people without identifying new preferences and therefore opportunities.”
On the flip side, Purcell also believes the best AI-based recommendation engines will adopt an “explore and exploit” model, where new as well as tried-and-true suggestions will confront the customer. “This approach uses both classic machine learning as well as reinforcement learning which is a relatively new approach that will ultimately become ubiquitous,” he says, referring to an emerging form of machine learning that trains algorithms by rewarding or punishing them for desired and undesired outputs.
Similarly, Xineoh’s Vian Chinner suggests AI-based platforms can be built to prevent market homogenization. “In theory, this [problem] is solved algorithmically by leaving a certain small amount of predictions to randomness,” he says. “In practice, there are many layers of interference with the algorithm from human intervention to random inputs like supply chain issues, so there are many opportunities for the algorithm to see results beyond the horizon.” There is also the argument that because of trend-prediction companies like Black Swan Data, manufacturers could use AI to help invent new products that inject novelty and diversity into markets.
Even in this best possible outcome scenario, AI still has the potential to reduce the roles chance, creativity, and market forces play. “In a new world of AI-driven personalized marketing, there will most likely be a narrower band of products being presented to consumers,” says Katie King, an AI expert and CEO of AI in Marketing. “But that’s not necessarily a bad thing. It may well appear that AI is restricting consumers’ choice but the reality could well be that via machine learning, the consumer will be presented with fewer but more tailored and relevant choices.”
Ultimately there may be no way around concerns this technology could stifle innovation. AI feeds the market only more of the products the masses want now–at a cost to those they might one day need.
