Bad News: Nvidia “Really Struggles” on Decode for Giant Models. Fast Tokens Fetch “3 to 5x”
Key Takeaways
-
General Compute CTO Jason Goodison says NVIDIA’s chips struggle with decode on very large AI models, and that fast tokens can be sold for 3 to 5x the price.
-
That puts NVIDIA’s pricing power at risk as companies built for speed, like Cerebras, go after the premium, though NVIDIA loses no sales while supply stays tight.
-
NVIDIA already expects its gross margin to fall to 71% to 72% in the January 2027 quarter, and it paid $13 billion in a Groq deal that adds an inference accelerator to its next platform.
-
At 17x forward earnings, its lowest in five years, the stock already prices in more than this risk, and the Nov. 17 report’s margin guidance is the next thing to watch.
NVIDIA (NVDA:NASDAQ) has a weak spot, and one of the people building on its chips just said so out loud.
Jason Goodison, co-founder and CTO of General Compute, went on the MTS podcast the day his company announced a $100 million Series A to build an inference cloud on chips designed for it. His team runs the first step of answering a prompt, called prefill, on NVIDIA’s GPUs, and Goodison says “NVIDIA does a great job on it.”
The second step, decode, writes the answer one token at a time. He didn’t mince words:
“Decode is where, you know, generating one token at a time is where NVIDIA really struggles on the very large models.”
That’s a tough line for NVIDIA shareholders to hear.
General Compute hands decode to partners like Cerebras (CBRS:NASDAQ), which he says splits a big model across a bunch of wafers and generates “up to 2000 tokens per second.”
TIKR tracks 62 analysts on Nvidia. How many say “Buy”?
See the full breakdown of Buy, Hold, and Sell ratings on Nvidia, along with Wall Street’s revenue and earnings estimates, all in one place. More than 500,000 investors use TIKR to research stocks like this one. See Nvidia’s Analyst Ratings
Customers will pay up for speed
Asked when fast AI becomes the norm, Goodison said the premium is already there: “you can charge many multiples on the same token if it’s fast. So if you want to charge 3 to 5x, you’d want to produce a fast token.”
And he doesn’t think the demand is years away: “I think on the order of a few months to six months is going to be when people really start demanding fast tokens.” His engineer friends, he says, are already switching to OpenAI’s Codex Ultrafast.
Goodison pointed to OpenAI’s big deal with Cerebras, which lets it put its frontier models on “this ultra fast experience.” Cerebras values that deal, for 750 megawatts of computing power, at more than $20 billion, according to its first-quarter release.
So the tokens customers will pay the most for are the ones Goodison says NVIDIA’s hardware is worst at making.
NVIDIA still sells everything it can make
None of this costs NVIDIA a sale today. Goodison says so himself: “right now the whole industry is supply constrained. So any inference is good inference.”
NVIDIA’s own guidance backs him up. Management expects revenue to rise about 70% in the year ending January 2028, and calls that a supply-limited figure. Analysts expect revenue to more than quadruple in three years, from $216 billion in the year ended January 2026 to $903 billion in the year ending January 2029:
Nothing Goodison said should move those estimates: fast tokens come on top of inference customers already can’t get enough of.
The fight is over the premium
The risk is to NVIDIA’s pricing power. Its gross margin has topped 70% in each of the last three fiscal years:
And it’s already under pressure: management expects it to bottom at 71% to 72% in the January 2027 quarter as memory costs climb, from 75.0% in the July 2026 quarter.
If the customers willing to pay 3 to 5x for speed take that business to Cerebras and the clouds General Compute is building, NVIDIA keeps the volume and loses the top of its price list.
NVIDIA already paid for a fix
In December 2025, NVIDIA paid $13 billion in its Groq deal, with $4 billion more still owed, its annual report says.
Management counts the Groq inference accelerator in Vera Rubin, its next platform, and puts Vera Rubin’s revenue opportunity at about $40 billion per gigawatt of data center capacity, against about $25 billion for Grace Blackwell.
My read: NVIDIA sees the same decode problem Goodison does, and it’s spending to keep those premium tokens on its own systems.
At 17x, a lot of bad news is in the price
After a rough October, NVIDIA trades at 17x forward earnings, its lowest in five years and less than half its five-year average of 36x:
Bottom line: Goodison’s point is bad news for NVIDIA’s margins a year or two out, as customers who want the fastest tokens buy them from hardware built for speed. But NVIDIA still sells every chip it can make, it’s already buying its way into fast inference with Groq, and at 17x, I think the market is pricing in far more pain than one premium niche can cause.
Of course, if Goodison’s timeline is right, we’ll know soon. Watch whether NVIDIA holds its January-quarter margin guidance when it reports on Nov. 17.
So what is NVIDIA stock actually worth?
TIKR lets you forecast the future price of any stock in less than a minute. Just enter a few assumptions into TIKR’s valuation model and see what NVIDIA could be worth. Start from Wall Street consensus estimates, or adjust the inputs to reflect your own view of the business. It’s free to use.