Amazon’s Unhelpful AI Shopping Assistant

Amazon’s AI shopping assistant is great at answering questions about a specific product but cannot help sort through the endless aisles of its infinite catalog.

Rufus, Amazon’s generative AI-powered shopping assistant, has been available to all U.S. customers since July 12th. Six months ago, in February, Amazon introduced a limited beta. It has been hiring for shopping AI positions and testing AI features for over a year. The assistant is prominently visible in the app, showing Amazon’s aggressive commitment to the new feature.

Amazon’s AI assistant has two modes: broad and contextual. Broad mode is for questions like “What are the best cycling gloves for winter?” Contextual mode is bound to the selected product — it can answer questions like “Do these allow using a smartphone?”

Contextual assistance is good, except for occasional hallucinations, which are present in most AI chatbots. Rufus is trained on product details, customer reviews, and community Q&As. However, it cannot answer price-related questions like “What’s the price?” or more useful questions like “Was it cheaper during Prime Day?”

Amazon’s AI assistant fails to help shoppers find the best product among the millions in the catalog. It transforms broad questions like “What are the best cycling gloves for winter?” into a few links to product searches — the same searches a shopper could have typed themselves. It refuses to make product recommendations, show specific products, or suggest from the thousands of options. It can’t directly answer the question, “What are the cheapest batteries for my TV remote?” It won’t offer to pick the Amazon’s Choice product either.

“Nobody wants to go through 10,000 iPhones. Show me the four best options,” said Devin Wenig, then CEO of eBay, in 2018. At the time, eBay was trying to transition from a catalog of listings (one for each iPhone listed) to a catalog of products (one for each iPhone model). Amazon has the same problem today, made worse by the explosion of private-label brands. It has thousands of lookalike results for, for example, “garlic press.” Rufus cannot help picking the best one. It can answer questions about garlic presses in general and about any garlic press the shopper picks, but not more.

According to an industry anecdote, Amazon found that every 100ms increase in latency costs them 1% in sales. Rufus takes a lot longer than 100ms. “Etsy CEO Josh Silverman previously said that while Etsy measures latency in search by the millisecond, latency in GenAI is measured in seconds, adding that such a length of time is “more than most consumers will bear,” wrote the Wall Street Journal.

Rufus also cannot take action — it cannot add items to the cart or reorder previously bought items (Amazon’s voice assistant Alexa could do that for years). Thus, the shopper has to switch from the AI assistant to the regular app buttons and back multiple times during the flow.

Amazon’s AI shopping assistant isn’t useless, but it isn’t magical, either. It saves from reading product reviews and quickly answers direct questions about the product, but it appears too careful and restricted to recommend products to be helpful. When hiring engineers for the team last year, Amazon described the feature as “a once in a generation transformation for Search, just like the Mosaic browser made the Internet easier to engage with three decades ago.” Not yet. For an artificial intelligence shopping assistant, it is not that intelligent, nor is it an assistant.