Turning to AI for Business Advice Has Mixed Results, Experts Say | PYMNTS.com

Artificial intelligence (AI) business advice could be a double-edged sword, boosting profits for high-performing companies while leading to setbacks for those already struggling.

An experimental AI mentor for Kenyan entrepreneurs yielded divergent results, improving the performance of successful businesses but harming struggling ones. A study found that high-performing companies increased their performance by 15% when following AI-generated advice. Struggling businesses saw a 10% decline in revenues and profits.

“These findings suggest that AI advice may be amplifying existing advantages and disadvantages rather than leveling the playing field,” the researchers noted in their paper. The contrast raises questions about AI’s effectiveness in business consulting.

Tech executives emphasize AI’s strengths but also highlight its limitations. Sidharth Ramsinghaney of Twilio told PYMNTS that AI’s prowess is in “pattern recognition, data analysis and predictive modeling” but noted it may not replace human consultants’ nuanced understanding.

Importance of Context

The researchers developed a GPT-4-powered AI business “mentor” that entrepreneurs could access via WhatsApp. The AI was customized to fit the Kenyan business landscape and programmed to provide multiple pieces of practical advice with detailed implementation steps for each query. Entrepreneurs participating in the study were randomly assigned to receive a standard business guide or interact with the AI mentor.

The entrepreneurs could consult the AI mentor as much as they wished on any business-related questions, using it in various ways. For instance, a restaurant owner asked for help about a possible menu change.

The researchers documented thousands of interactions with the AI mentor. Initially, their analysis showed no average positive impact on business performance for those who used the AI mentor.

However, differences became apparent when they divided the participants based on their initial business performance — above or below the median. Businesses performing above the median experienced a 15% increase in profits or revenue, while those below the median saw an 8% decline in revenues.

Keith Lauver, creator of AI marketing adviser Ella, argued that generalized AI models offer “broad” advice “unlikely to be helpful,” advocating instead for specialized systems.

“AI advisers can be built to act like co-creators, consultants, coaches, or even critics,” Lauver told PYMNTS.

The role of context in AI-driven business advice is a crucial factor. Ilia Badeev, head of data science at Trevolution Group, told PYMNTS, “A human consultant often delves into the essence of the problem, communicates with all stakeholders, adds ‘external’ knowledge, and seeks solutions.”

With AI, the onus is on the business to provide context. “A correctly posed question is half the answer,” Badeev noted, highlighting the need for prompt engineering.

Pavlo Tkhir, CTO at the digital transformation company Euristiq, added that AI excels in personalization and data analysis, stating it “speeds up the data collection and analysis process and practically does the job for you.”

But he told PYMNTS that AI “often struggles to grasp the context around the data.”

Experts agree that AI is particularly useful for peripheral tasks. Badeev suggested, “In small and medium-sized businesses, there are always core functions that, in theory, they should do better than others. But there are also many peripheral tasks. It is precisely these that can be transferred to AI.”

Ramsinghaney said AI could enable small businesses to “compete more effectively by optimizing operations, reducing costs and enhancing customer experiences.” Tkhir agreed, noting that AI-driven analytics can help small businesses make “more informed decisions while keeping it budget-friendly.”

Balancing AI, Human Expertise

Lauver warned that reliance on general AI tools may lead to generic advice. “For companies who want to build exceptional strategies and who need to differentiate from competitors, they will need specialized AI advisers,” he argued.

The key takeaway appears to be the need for a balanced approach. AI provides valuable insights, but it’s most effective when combined with human judgment and specialized to fit specific business needs.

Badeev predicted, “In the future, with the growth of business digitization and the expansion of AI’s contextual capabilities, this difference will level out. I expect AI consultants will soon surpass human consultants in most parameters.”

For all PYMNTS AI coverage, subscribe to the daily AI Newsletter.