The Canada Revenue Agency’s AI-powered chatbot, Charlie, is back in the spotlight—and not for the right reasons. A recent Auditor General report has slammed the $18 million tool for its poor performance, revealing that Charlie still delivers incorrect answers to basic tax questions. After five years and millions in taxpayer funding, it seems Charlie has a long way to go.
Launched in 2020 and touted as a cutting-edge support tool for Canadians, Charlie was supposed to help reduce pressure on call centres and make tax information more accessible. But according to the Auditor General, the tool still gets basic answers wrong—and alternatives like ChatGPT or Google’s Gemini do a better job.
Table of Contents
Failure
Charlie was introduced with the promise of simplifying complex tax questions. Instead, it’s become a case study in expensive government tech gone sideways. The Auditor General’s audit found that out of six tax-related questions tested, Charlie only got two right. Other publicly available AI tools managed to answer five correctly.
The verdict? Charlie is underperforming—and taxpayers are footing the bill.
Auditor General Karen Hogan’s report noted that Charlie often provided “brief” answers that lacked context or guidance. That’s not helpful when Canadians are trying to navigate things like deductions, benefits, or tax brackets.
Spending
So, how much has this cost us? A lot more than you’d expect for a tool that’s still failing basic tasks.
Here’s the spending breakdown on Charlie:
| Expense Category | Amount Spent |
|---|---|
| CRA Salaries | $13.67 million |
| IT Consultants | $3.21 million |
| Total Cost | $18+ million |
| Not Included | Employee benefits, travel costs |
Most of that spending happened after Charlie’s full rollout in 2021–2022. And no, this doesn’t include other overhead expenses like benefits or travel. Just the core cost of building and maintaining the bot.
Accuracy
The CRA has long defended Charlie, saying internal tests showed it had a 70% accuracy rate. That sounds decent—until you realize it means almost 1 in 3 answers were wrong. And when it comes to taxes, one wrong answer can lead to penalties, delays, or lost benefits.
To address the issues, the CRA recently upgraded Charlie with a generative AI system, similar to ChatGPT. The updated version is currently in beta and reportedly tested at about 90% accuracy before launch. But here’s the catch: the CRA can’t confirm that rate in real-world use unless it manually reviews every conversation—which it isn’t doing.
Usage
Despite its flaws, Canadians are using Charlie—heavily, in fact. Since launch, the chatbot has handled over 7 million conversations and responded to more than 18 million questions. That’s a huge volume. But the question remains: how many of those interactions led to the right answers?
It’s also worth asking: are Canadians using Charlie because it’s helpful—or because they can’t get through to a human at the CRA?
Comparison
When Liberal MP Anthony Housefather asked why Charlie performed worse than general-use AI tools, Auditor General Hogan didn’t have a clear answer—but the message was clear: private tech companies are delivering better tools, faster and cheaper.
Many Canadians now turn to platforms like ChatGPT or Google’s Gemini to get reliable answers to tax questions faster than Charlie can manage. In a way, Charlie’s $18 million price tag becomes harder to justify when free alternatives do a better job.
And while Charlie may not be as infamous as the $60 million ArriveCAN app, it’s starting to give it a run for its money in terms of public criticism.
Future
The CRA insists it’s working on it. They say Charlie is getting better thanks to GenAI improvements and internal testing. Still, there’s no timeline on when Charlie might actually match or beat the accuracy of its tech competitors.
With over 53,000 CRA employees and pressure to reduce wait times and improve digital tools, the agency is trying to modernize its services—but tools like Charlie may not be helping the cause, at least not yet.
Bottom Line
Charlie was supposed to be a digital game-changer for the CRA. Instead, it’s become a costly symbol of how not to roll out government technology. With basic questions going unanswered or answered incorrectly, many Canadians are turning elsewhere for help—and they’re getting better results.
Until Charlie can prove it’s truly helpful, its $18 million price tag will continue to be a tough sell to taxpayers.
FAQs
How much did the CRA chatbot cost?
Over $18 million since 2018–2019.
Is Charlie the chatbot accurate?
No, it answered only 2 out of 6 test questions correctly.
Has Charlie been upgraded recently?
Yes, it now uses a GenAI system in beta.
What’s Charlie’s real-world accuracy?
Unclear—CRA isn’t reviewing all conversations.
Is Charlie better than ChatGPT?
No, other public AI tools outperformed Charlie.
