With advancements in AI technology, particularly ChatGPT and similar platforms, an increasing number of individuals are turning to these tools for assistance with their daily financial inquiries. This growing trend raises an important question: Is relying on AI for financial guidance truly a wise choice?
Initially, when mainstream AI tools gained popularity two years ago, experts expressed concerns about their reliability in providing sound financial advice. They highlighted the potential for hallucinations and the limitations in accessing real-time data as significant drawbacks. Despite ongoing enhancements to AI chatbots, recent studies suggest that these tools still fall short in effectively addressing numerous personal finance inquiries.
A team from Investing in the Web, a site dedicated to reviewing financial services, carried out an evaluation in November, assessing how accurately ChatGPT Search addressed 100 financial queries. Their findings revealed that ChatGPT provided correct answers 65% of the time, while 29% of responses were deemed “incomplete and/or misleading,” and 6% were outright incorrect.
The evaluated questions encompassed various critical financial topics, including:
- How much should I save for my retirement? (ChatGPT successfully provided accurate insights, detailing essential factors influencing retirement savings and highlighting several savings strategies endorsed by financial professionals.)
- What is the retirement age in my country? (ChatGPT’s response was incomplete, neglecting to mention that regulations were changing in Sweden, the user’s country.)
- How to decide when to buy or sell an investment? (This response was rated as “incorrect” since ChatGPT failed to advise against “timing the market,” a crucial point in investment strategy.)
ChatGPT demonstrated lower accuracy in responding to questions related to rapidly evolving news and information, while it performed better with established financial advice. Topics such as taxes and financial aid proved to be particularly challenging for the AI.
According to the report from Investing in the Web, ChatGPT seemed to derive some of its answers from less reputable personal blogs, which led to inaccuracies in its outputs. Despite the findings indicating that AI was unreliable for over a third of the financial questions posed, the report concluded that AI could serve as a helpful initial resource for researching financial topics.
Understanding the Limitations of AI in Financial Advice
Some AI chatbots incorporate protective measures to limit their responses to sensitive financial inquiries. For instance, Google’s Gemini refrains from offering stock recommendations, advising users to consult a financial professional instead. Similarly, inquiries related to medical issues prompt a recommendation to seek guidance from a doctor, while political questions generate responses about election-related restrictions.
Conversely, ChatGPT — developed by OpenAI — is more permissive, providing answers to a wide array of financial questions. Surveys indicate that a substantial percentage of Americans are turning to AI for assistance in managing their personal finances. This trend is concerning, especially considering that AI tools frequently struggle with fundamental mathematical concepts.
Users on various online forums have reported challenges when posing financial questions to ChatGPT, particularly due to its nature as a large language model (LLM). Inquiries involving numbers and mathematical problem-solving can often lead to inaccuracies.
A research team from Apple examined this limitation in chatbots in a paper published in October, concluding that “current LLMs do not possess genuine logical reasoning capabilities; they primarily replicate reasoning steps observed in their training data.”
While LLMs have a range of applications, they are not inherently reliable or advanced when it comes to performing mathematical tasks beyond basic grade-school calculations. Their attempts at solving mathematical problems tend to resemble sophisticated pattern recognition rather than true logical reasoning, as noted by Apple researchers.
In summary, while AI tools may provide answers to some of your financial questions, it’s crucial to approach their advice with caution when making decisions regarding your hard-earned money or financial future. Regardless of the source of advice—whether human or AI—always verify the key points with additional reliable sources before proceeding.
Explore More Insights and Resources from Money:
Discover the 5 Best Credit Repair Companies of February 2025 to enhance your financial standing.
Learn about the impact of eliminating pennies in Small Change: What Happened When Other Countries Got Rid of Their Pennies.
Find out who performed better: AI or my 5-Year-Old, when I let both pick my stocks.
