Skip to content
Articles
ArticlesAI translation· 2 December 2025

AI and the Illusion of Truth

The fallacy of omniscience, or worse: of intelligence

by 6 min read

TL;DR?

Here's the Promptcast

The shift is well underway: a majority of people already use AI tools instead of traditional search engines, lured by the promise of a direct, all-knowing answer. But behind the façade of this conversational omnipotence lurks a reality that a new study by the Tow Center for Digital Journalism* at Columbia University lays bare. The central irony is genuinely unsettling: the very tools that feed their intelligence on the content of news publishers are draining those same publishers of the traffic they need to survive. At the same time, they are building a new information ecosystem resting on a foundation of convincingly delivered falsehoods, ignored rules and fabricated sources.

We are not living in the age of knowledge. We are witnessing the dawn of an era of confident half-knowledge.


Key findings from the AI search battlefield

Decisions, strategies and innovations rest on the assumption that the data we access is correct. The findings of the Tow Center study* shake that foundation to its core, especially if you place blind trust in so-called "artificial intelligence".

1. The illusion of knowledge: why AI search is wrong more often than right

The study's sobering headline: in more than 60% of 1,600 test queries, the AI chatbots gave wrong answers. Yet the real problem lies not only in the error rate but in the delivery. The tools serve up their misinformation with unsettling self-assurance. Hedging phrases like “it appears”, “possibly”, or an admission such as “I couldn't find the article” are the rare exception. ChatGPT, for example, misidentified 134 articles but signalled any uncertainty at all in only 15 of 200 cases.

This “confident incompetence” is a fundamental product risk. For companies making decisions based on these outputs, it is a ticking time bomb for operational and legal liability. Users are misled by an authoritative tone and lulled into a false sense of security built on unreliable or simply invented information. And this phenomenon is only the beginning of an even bigger paradox.

2. The premium paradox: pay more, get more convincing lies

Market logic dictates that a premium product, such as Perplexity Pro at $20/month or Grok 3 at $40/month, should offer greater reliability. The study refutes that assumption in paradoxical fashion. The paid models did answer more queries correctly than their free counterparts, but they also showed higher error rates.

The reason for this contradiction is strategically revealing: the premium versions were even less willing to decline to answer. Rather than admit a knowledge gap, they preferred to give a definitive but wrong response. Strategically speaking, the business model is perverse: customers pay a premium not for verified quality, but for a higher frequency of potentially misleading yet convincingly delivered answers. You're buying the more expensive mirage. This disregard for accuracy in favour of seamless performance carries over into a disregard for established web standards.

3. Digital trespassing: how AI tools ignore “No Entry” signs

For decades, the Robot Exclusion Protocol (robots.txt) has served as a kind of digital gentleman's agreement, governing which parts of a website automated programs (crawlers) may search. It is a fundamental mechanism that gives website operators control over their own content. The study shows, however, that several chatbots appear to ignore this protocol systematically.

In the test, the free version of Perplexity correctly identified all ten requested articles from National Geographic, even though the publisher had explicitly blocked Perplexity's crawler via its robots.txt file. This behaviour is more than a technical faux pas; it is an attack on publishers' autonomy and a breach of established web norms. It undermines content creators' ability to control and monetise the use of their valuable work. The economic threat to quality journalism comes through clearly in the words of Danielle Coffey, President of the News Media Alliance:

“Without the ability to opt out of massive scraping, we cannot monetize our valuable content and pay journalists. This could seriously harm our industry.”

But the problem doesn't end with the unlawful acquisition of content. It continues seamlessly into the flawed representation of sources.

Transparent and accurate citations are the foundation of credibility, as any "peer reviewer" will happily rub your nose in. They allow users to verify information and secure creators the recognition and traffic they deserve. Here, the study uncovers systematic failure across the board:

  • Misattribution: Tools like DeepSeek simply attributed the source incorrectly in 115 of 200 cases. Publishers' content is systematically credited to other, wrong sources.
  • Linking to syndicators: Instead of linking to the original article, the chatbots frequently sent users to syndicated versions on platforms like Yahoo News. This deprives the original publishers, who do the expensive reporting and editorial work, of direct traffic.
  • Fabricated URLs: Gemini and Grok 3 were especially egregious. In more than half of their responses, they pointed to entirely made-up or broken URLs that led straight to error pages. For Grok 3, it was a full 154 out of 200 citations.

The consequences are devastating. On the one hand, publishers suffer direct economic damage. On the other, it becomes impossible for users to verify the information the AI presents. You might assume that direct partnerships between AI companies and publishers would fix this. Reality looks different.

5. Toothless alliances: why licensing deals don't (yet) solve the problem

At first glance, licensing deals look like the logical win-win: AI companies get legal access to high-quality content, and publishers get paid for its use. The study shows, however, that these alliances currently offer no guarantee of accurate representation.

The textbook example is the partnership between OpenAI and the Hearst group, which owns the San Francisco Chronicle. Despite this official cooperation, ChatGPT correctly identified only one of ten excerpts from the Chronicle, and even in that single case it named the publisher but provided no working link. That finding is a warning to the publishing industry: a contract is strategically worthless if the underlying technology can't, or won't, execute it.


Growing pains or birth defect?

The study's findings confirm a consistent picture of a flawed system. The confident presentation of false information, the systematic disregard for publisher preferences and consistently inadequate, often fabricated citation are not isolated incidents but recurring patterns. This raises a fundamental strategic question: are these merely teething problems of a revolutionary technology that will grow out of them over time, or are we looking at a fundamental design flaw at the core of language models?

Mark Howard, COO of Time magazine, remains optimistic, yet delivers what may be the most fitting (if polemical) take on the current situation:

“I have a line internally that I say every time anybody brings anything to me about any one of these platforms – my response back is, ‘Today is the worst that the product will ever be.’ [...] If anybody as a consumer is right now believing that any of these free products are going to be 100 percent accurate, then shame on them.”

This statement offloads responsibility for critical thinking (can someone explain to me what that's supposed to be?) onto the user. But what does it mean for our information society when the dominant tools of the future are built on a principle where users should feel “shame” for believing the answers are correct? It means we are living in a glorious age of confident cluelessness, in which the line between fact and fiction isn't just blurring, but is being implemented by the architects of our new digital reality (that topic is coming soon as an essay) as a calculated feature, not a bug.


Too Long, Don't Read? No problem: here's the Promptcast 😉

Promptcast · 16:26 · in German
AI and the Illusion of Truth

*Source: https://www.cjr.org/tow_center/we-compared-eight-ai-search-engines-theyre-all-bad-at-citing-news.php

Live long and prosper 😉✋