Exclusive: Leaked Amazon memos identify critical flaws in the upcoming AI version of Alexa

The problem, known as latency, is a critical shortcoming, employees said in an internal memo from earlier this month obtained by Fortune. If released as is, customers could become frustrated and the product—a particularly critical one to Amazon as it tries to keep up in the crucial battle to launch blockbuster consumer AI products—could end up as a failure, some employees fear. 

“Latency remains a critical issue requiring significant improvements,” before the new version of Alexa could launch, the memo said.

The latency problem—a common challenge when building complex generative AI applications—is just one of several concerns that Amazon employees cited in internal communications over the last few months that were viewed by Fortune. They show the hurdles that Amazon must overcome to ultimately release the updated Alexa, a major priority at the company because it could open a new door to selling subscriptions to access the new voice assistant and may supercharge sales of Amazon’s Echo line of smart devices. 

The updated Alexa is also a key barometer for Amazon’s standing in the race among Big Tech companies to dominate consumer-facing AI and for the financial windfall that’s expected to come with it. Currently, many industry observers consider the company to be trailing its peers like Google parent Alphabet and Microsoft, along with buzzy newcomers such as OpenAI and Perplexity AI in creating breakthrough generative AI applications for consumers. 

The documents obtained by Fortune, totaling more than a dozen pages, date from August to as recently as early November. While Fortune has previously reported on Amazon’s tech and structural struggles with the new Alexa, this is the first time that internal communications have come to light highlighting key details about the project. In addition to the latency concerns, they show that testers have given the technology disappointing satisfaction scores; some older Echo smart speakers may be incompatible with the new Alexa; and reveal the names of potential external partners that will provide their services through Alexa. 

However, the documents are a snapshot in time and, in some cases, reflect the struggles involved in any complicated and innovative tech project. Some of the problems discussed in them may already have been fixed or will be before any future release. 

Amazon spokesperson Lauren Raemhild told Fortune in a statement that the company’s vision for Alexa is to become “the world’s best personal assistant.”

“Generative AI offers a huge opportunity to make Alexa even better for our customers,” she said in the statement, “and we are working hard to enable even more proactive and capable assistance on the over half a billion Alexa-enabled devices already in homes around the world.”

Raemhild added that some of the plans reviewed by Fortune don’t reflect what the upgraded Alexa experience will offer when it eventually launches, but she declined to specify which features she was referring to.

Amazon first previewed a more conversational, gen AI-powered version of Alexa at a press event in September 2023. Amazon executive Rohit Prasad called it a “massive transformation of the assistant we love.”

The upgrade is supposed to restore some of Alexa’s luster, which had been gradually lost since it was initially received with great fanfare after it launched 10 years ago. While the company has said there are still hundreds of millions of active Alexa devices, their users increasingly complain about them. A relatively common critique is that the voice assistant does not understand, or respond to requests as accurately as it once did. The original Alexa wasn’t built for natural conversations involving back-and-forth chatter. Instead, it replies to simple questions or commands by pulling from an evolving set of pre-determined responses.

Most importantly, Alexa is the brains for the Echo voice-controlled smart speakers, which include a smaller version known as the Echo Dot as well as voice-enabled tablet screens, the Echo Show. In all, Amazon and its partners have sold around 500 million Alexa-powered devices, the company has said.

But Amazon was caught flat-footed when OpenAI’s ChatGPT chatbot launched in 2022, sparking the gen AI gold rush. Within hours, Alexa staff members began testing it and found that the software code they asked it to generate for controlling certain Alexa features was “at times better than those from Amazon’s internal systems,” The Wall Street Journal reported.   

After announcing plans for the new Alexa during a splashy event at Amazon’s Seattle headquarters last year, the team working on the chatbot ran into a number of technological and bureaucratic problems, Fortune previously reported. For example, the company lacked access to enough data to feed into the new large language model that is necessary to train Alexa. Meanwhile, employees complained that executives failed to provide enough resources to the project as they instead emphasized work on developing Gen AI capabilities for the company’s cloud computing unit. Some employees also said that when siloed teams within the Alexa division, like Alexa Home or Alexa Music, wanted to fine-tune the new Alexa LLM for their purposes, the overall performance of the in-development voice assistant at times worsened.

The internal documents obtained by Fortune this month reference a succession of at least three different target dates in 2024 to publicly announce the new Alexa, codenamed Banyan. All of them, including the most recent one on November 14, have since passed. 

In recent months, executives from CEO Andy Jassy to new Amazon devices boss Panos Panay have discussed the planned Alexa upgrade in general terms, but they have not commented on when they expect it to premiere. 

Some employees have reportedly been told that their new deadline for a finished AI-powered Alexa has been delayed to 2025, according to a recent Bloomberg report.

The documents show that the reason for the delay is because the upgrade is still a work in progress. As recently as this month, employees reported in a document that the technology was still producing subpar results. Users who had tested the new version of Alexa had given it a customer satisfaction score of 4.57 out of 7, lower than the company’s target of 5.5 out of 7.

In an interview, one current Amazon employee with intimate knowledge of the initiative echoed that the technology still needed work, including fixing the problem with delayed responses. “There are severe concerns over the latency aspect,” said the employee, who requested anonymity because they were not authorized to speak publicly. 

The employee added that they feared executives may feel pressured to release the new version of Alexa soon despite these concerns. If so, it would be a mistake, the employee said, because of internal worries that it still wasn’t ready for primetime.

Another wrinkle outlined by the documents is that the new Alexa won’t be compatible with some older Amazon Echo devices. Nearly 10% of people who are active users of the voice assistant through Amazon Echo devices—or 3.8 million people in total—won’t be able to access the new version through those devices. 

Translation: they’ll have to upgrade by buying a new Echo or lose out.

Amazon’s marketing team plans to target those Amazon Echo owners with “recycle and save” offers intended to persuade them to purchase a new device compatible with the upgraded voice assistant, the memo says. Some Alexa users also recently received a survey, viewed by Fortune, which asked how likely they’d be to purchase a new Echo device if “it was released with the latest AI technology.”

The same internal communication from August also anticipated customer frustration with the new Alexa because it would initially only have a limited selection of Skills (a quasi equivalent to a smartphone app). Just north of 300 Skills will be available at launch, according to the documents, compared to the 100,000 or so available through the legacy Alexa today.

The lack of Skills could make it difficult for Amazon to sell subscriptions to access the new voice assistant, the memo says. That comment seems to confirm multiple news reports that Amazon is considering charging such a fee, though other sources have told Fortune that those plans could change before launch. 

“This presents a high risk for customers to be frustrated with paying for a subscription and having a diminished feature-set with [third parties],” the document reads. 

In her statement, the Amazon spokesperson stressed the complexity and difficulty of building such an ambitious consumer product. She noted that it’s not as simple as layering a new AI model atop the legacy Alexa service. 

Large language models, the backbone of today’s AI chatbots, are great at producing conversational exchanges or text-based content, Raemhild said. But getting them to accurately and reliably perform actions based on customer requests without so-called “hallucinations,” or mistakes, is a tall task.

Raemhild seemed to be echoing part of what CEO Jassy alluded to on the company’s most recent earnings call: that Amazon wants the new Alexa to be known more for the actions it takes on a user’s behalf as much as, or perhaps more than, possessing encyclopedic knowledge. In the AI sector, this type of product would be known as an “agent.”

To that end, some of the documents reviewed by Fortune from August list some of the big-company partners Amazon plans to work with to power various user experiences for the revamped Alexa. The document says the company planned to let users easily order a ride through Uber and make restaurant reservations through OpenTable by making simple verbal requests through Alexa.

The memo also listed Instacart as a launch partner for grocery delivery, GrubHub for food delivery, Ticketmaster for event tickets, Atom Tickets to buy movie tickets, and Thumbtack for home services. One document notes that Amazon estimates adding around 200 such partners to the upgraded version of Alexa within three years following launch.

Business Insider first reported about some of the partnerships.

Some of these companies previously offered their own Skill—or app—through the original Alexa service, but many have since withdrawn them. Plus, the new integrations are intended to work more seamlessly, allowing Alexa users to make purchase or reservation requests via the voice assistant in a much more natural and conversational way. 

Raemhild, the Amazon spokesperson, would not confirm nor deny these partnerships. It’s also possible the company’s plans or partner lineup could change before launch.

Amazon’s top devices executive, Panay, told Fortune last month that the traits of the new Alexa will inspire him to introduce it using terms like “emotion” and “connection.” 

“The products coming are pretty awesome,” he said.

Based on the company’s own internal documents reported here, there is still significant work to do to ensure the new Alexa meets that bar.

Are you a current or former Amazon employee with thoughts on this topic or a tip to share? Contact Jason Del Rey at jason.delrey@fortune.com, jasondelrey@protonmail.com, or through Signal or WhatsApp at 917-655-4267. You can also message him on LinkedIn or at @delrey on X.