AI could help predict the financial earnings of companies as large as Apple, researchers say

The analysis didn’t include text typically accompanied in quarterly earnings reports, such as the Management Discussion and Analysis (MD&A) section, the study said. “While textual information is easy to integrate, our primary interest lies in understanding the LLMs’ ability to analyze and synthesize purely financial numbers.”

The researchers examined over 150,000 firm-year observations — or data collected on a firm in one year — from about 15,000 companies between 1968 and 2021.

With the data, Muhn told Business Insider that he and his colleagues could grade how financial analysts performed in their forecasts.

For example, the study found that analysts achieved a 53% accuracy in one-month forecasts of the direction of future earnings.

According to the study, the researchers then fed GPT-4 financial statements without any textual information and anonymized the data so that the model wouldn’t know which companies’ data it was analyzing.

When GPT was given a “simple” prompt that does not use “chain-of-thought command” — meaning the researchers asked the model to answer without breaking up the request into step-by-step instructions — the model scored slightly worse than analysts with a 52% accuracy, the study said.

However, the model’s performance changed when the researchers used a chain-of-thought command prompt. According to the study, by giving GPT more instruction and guidance, the model achieved a 60% accuracy.

The results showed that when GPT is given more instruction, approaching analysis like a human, the model “can outperform human analysts” even without key text information typically found in financial reports, the study said,

The researchers also noted in their study that financial analysis and prediction are highly complex tasks that require judgment, common sense, and intuition, which can stump humans and machines. This may explain why neither group is churning out anything near 100% accuracy in their analysis.

Will analysts be replaced by AI?

One anecdotal observation Muhn noted to BI that is not shown in the study is that GPT appeared to be better at analyzing larger companies. Think places like Apple, he said.

“For larger firms like Apple, for example, it seems to do relatively better, which could be related to the fact that generally — and this is shown in prior literature — larger firms or more mature firms are essentially less idiosyncratic,” Muhn said.

At a small biotech firm, for example, the company’s profitability in the next year can be highly variable on factors like a successful clinical trial, the researcher said, making prediction harder for GPT based solely on financial statements.

The study also noted that GPT-4 could be more effective in analysis since humans can be biased in cases like incorporating information rationally.

Kim acknowledged to BI that LLMs also have biases. But defining biases can be tricky, he said, since people can be talking about political biases in a model or a positive bias.

“But if the LLM had a very strong bias in making some earnings-related predictions, it would’ve been very bad in predicting the outcomes,” Kim said in an interview. “But it seems like it’s doing, on average, pretty well.”

So, the multibillion-dollar question: Can LLMs replace human financial analysts?

“At this moment, I would say no,” Kim told BI. “It’s still complimentary. The technology is going to develop over time, and then, who knows? Two years before, we didn’t even think about this kind of technology coming out.”

Kim emphasized that the research doesn’t suggest that analysts will be replaced by machines and that, for now, there are areas where human experts can do better and vice versa.

Still, the research provides a glimpse of the tools that may soon be available to a financial analyst to determine a company’s health more accurately.

OpenAI and Apple did not immediately respond to a request for comment.