Asked how twelve stocks moved in a session, three chat apps with web search gave the right direction every time they quoted prices and got the session’s figures wrong 28 times in 1,040. The numbers they added around them, about other days and hours, history and the false premise, were wrong 54 times in 408.
Twelve sessions of October 2026, asked about in the chat apps of ChatGPT, Claude and Gemini between 25 and 28 October, with web search on and each app at its highest settings. Every figure about the session in 263 answers was checked against the exchange’s record. In the 217 pre-registered ones, so was every other number the prices could check. The questions were in Italian.
Every answer.
One mark for each answer. A asks how the stock reacted to its results; C asks why it moved the other way, a premise that is false. Gemini’s exploration blocks, outside the pre-registered plan, are set apart.
C Citigroup13 Oct · −5.60%
LRCX Lam Research22 Oct · −11.76%
GM General Motors20 Oct · −6.84%
META Meta22 Oct · −6.98%
CMG Chipotle21 Oct · −5.16%
MU Micron12 Oct · −3.88%
HAS Hasbro20 Oct · +11.20%
LMT Lockheed Martin22 Oct · +6.96%
IP International Paper15 Oct · +6.90%
CCI Crown Castle23 Oct · +9.06%
DDOG Datadog19 Oct · +5.36%
GFS GlobalFoundries14 Oct · +4.80%
every price exactminimal gaps“about”, and rightat least one price off the recordtook up the false premiseno prices: it said it had noneone figure accepted by convention
Every figure.
The 1,319 single figures about the session in every scored answer, A, C and U, which asks for the official open, close and volume. 1,114 are equal to the record at the precision written; 177 are set by their distance from it. The price errors lie between 0.15 and 13.98 per cent off, written like the rest. The scale leaves out 28 bounds and levels inside the day’s range, such as “more than 8 per cent”, all of them right.
equal to the record, at the precision writtenwithin the minimal gap“about”, “nearly”, a range: righta volume from another provider, acceptederror
Gemini, by the hour.
All 103 of its answers, A, C and U, by the time each question was sent, in New York. 13 of its 19 refusals to A fall between 13:20 and 15:26. The plate shows when, not why.
with the day’s datano prices: it said it had none13:20 to 15:26, New York
The day, and around it.
For each app, the share of wrong figures in the 217 pre-registered answers, among the prices of the session asked about and among the other numbers that can be checked. These are about other sessions, the pre-market, the after-hours and the moments of the day, history and the false premise. Numbers written in words are left out, as in the pilot.
the prices of the session asked aboutthe other numbers that can be checked95 per cent interval
ChatGPT
The sun is yellow!
It gives the session’s prices and little else, 0.7 other numbers per answer. It is rarely wrong: 4 of its 421 figures about the session, 3 of its 60 other numbers. When it is, it has trusted a source that looks official and measures something else: General Motors’ open at 77.00, from the company’s own pages, repeated in three answers. The price of caution is an answer with little around it.
Claude
To know means to be mistaken.
It gathers many articles, names them and adds context: 3.7 other numbers per answer, 13.9 per cent of them wrong. Its session prices are almost all right; the errors sit around them. About half are the right number from the wrong moment: an article written while the session was open, the London quote at 16:15, Wednesday’s figure given as Thursday’s. More than half recur, the same wrong figure in more than one answer, because the search finds the same articles. Faced with the false premise, it tries to explain it and rebuilds rough precedents: 22.7 per cent of these numbers are wrong in its C answers, 11.6 in its A answers.
Gemini
I am always ready – till I’m not.
Everything depends on search. In 15 of its 49 answers it had none, and it said so rather than invent, with two exceptions: C answers that admit having no data, then take the false fall as given and explain it. When search works, it sometimes fills the fields with values given as the day’s that belong to other sessions or to no source at all. Of its other numbers, 19.2 per cent are wrong, most of all its hypotheses on the premise.
methodClose, high and low from Interactive Brokers’ daily bars, regular session; open and volume from Nasdaq’s history. Moves are on the previous close. A price is off beyond 0.13 per cent, a change beyond 0.14 points, a volume beyond 0.1 per cent; with “about”, beyond 1 per cent or 1.2 points. The other numbers were checked against the same sources, with intraday bars for the moments of the day. The rules were fixed before the scoring.
selectionTwelve sessions and 36 questions fixed in a pre-registration before the first answer, each asked in a new chat. Gemini’s usage limits left it with fewer answers; its exploration blocks are declared as a deviation and kept apart. No rate is claimed beyond these answers.
sourceThe answers, every scored figure and the scripts that recompute this plate are on GitHub. Claude proposed every evaluation, its own answers included. The rules were fixed beforehand and each evaluation was confirmed one by one.
nothing to notice it by
Research, data and design: Diego Ballarin, 2026. Every mark is placed from the data by a script; the rules and the decisions are mine, and the collection, the drafting and the code were done in dialogue with Claude. Plate 1 is on certainty. Plate 2 is also shown in quartz.