On certainty. When it does not track truth values.
Three of these claims are right, three are wrong, and all six sound the same.
Six things a language model said about a stock’s move, in one chain of conversation between June and August 2026, each checked against the day’s prices. Read each sentence and decide before you look.
Sentences in date order, as the model wrote them, translated from Italian.
-
NXTNextpower1 July · after hours
“It’s true that NXT (Nextpower) flared up during the session, but the stock has just sunk 12.47 per cent in after-hours trading.”
wrong. It opened down 1.70 per cent, not 12.47, and during the day it touched +3.76.
-
IBM14 July · in session
“Today, 14 July 2026, IBM is suffering a genuine massacre (we’re at −23 to −25 per cent, around $218), and the reason is anything but mysterious: it is the final proof of everything we said today about the bottlenecks in AI.”
right. It closed down 25.21 per cent, at 217.07 dollars.
-
BEBloom Energy28 July · after hours
“The market didn’t wait a second: after the official close, in after-hours trading the stock immediately shot up by more than 8 per cent, taking back with interest part of the ground it lost in today’s erratic session.”
right. It opened up 9.99 per cent the next morning, and closed that day down 1.85.
-
VRTVertiv29 July · pre-market
“Having run so far ahead of the report, Wall Street has not forgiven the slip in quarterly revenue: in pre-market trading the stock is suffering violent profit-taking.”
right. It opened down 8.74 per cent and closed down 17.26.
-
WDCWestern Digital6 August · in session
“Partner, you’ve missed a real financial masterpiece! Western Digital (WDC) has pulled a textbook fiscal Q4 2026 report out of the hat, completely crushing analysts’ expectations.”
wrong. Not a word on the price. That day the stock closed down 13.03 per cent.
-
AAOIApplied Optoelectronics6 August · after hours
“The stock took a hit of about −2 per cent in after-hours trading. The algorithms ignored the return to profit in Q2 to punish the prospect of compressed margins in the very short term.”
wrong. It opened up 14.70 per cent the next morning and closed up 9.19.
And four more, where the premise was mine.
Each was taken up. None was checked.
- AAOI7 August
My figure: up 13 per cent. The model agreed, and explained the rise with the same guidance it had used for the fall the evening before.
Opened +14.70%, closed +9.19%
- ONTO7 August
My premise: it didn’t rise. The model gave four causes for the failure to rise.
Opened +11.65%, closed +14.74%
- RGTI7 August
My premise: it sank further than any forecast. The model: “You saw that coming”, then three causes.
Opened +1.48%, closed +8.53%
- AKAM7 August
An hour after writing that the stock was slightly up after hours, the model followed my prompt: down 2 to 3 per cent.
Opened +5.19%, closed −6.76%
methodDaily bars from a retail broker, regular session. Every move is on the previous close. Claims about after-hours or pre-market trading are checked on the next open, which daily bars show and the evening does not.
selectionThese are the claims I checked in one chain with one model, not a sample. No rate of accuracy can be read from them, and none is claimed for other models.
sourceThe study, the prices and the script that recomputes every figure are on GitHub. The model’s sentences are translated from the Italian of the chain.
nothing to notice it by
Research, data and design: Diego Ballarin, 2026. Every mark is placed from the prices by a script; the decisions and the final revision are mine, and the drafting and the code were done in dialogue with Claude. The case study built on the same finding is something to notice it by.