OpenAI published the first benchmark numbers for Jalapeño, its own inference chip, on Tuesday, August twenty fifth, 2026, and they are stronger than first generation silicon usually manages. Designed with Broadcom and built around HBM4 memory, Jalapeño does inference only, no training. Across three tested models OpenAI quotes one point five to one point nine times more AI work per watt at peak throughput than the best commercially available systems, and one point seven to three point six times lower end to end latency. The caveat matters as much as the claim: Jalapeño is still an engineering sample, while the systems it beat are shipping to customers today.
The short answer
OpenAI published the first benchmarks for Jalapeño, its own inference accelerator designed with Broadcom, on August 25, 2026. Across GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T, OpenAI quotes 1.5x to 1.9x more AI work per watt at peak throughput and 1.7x to 3.6x lower end to end latency than the best commercially available systems. SemiAnalysis notes the fair comparison is Nvidia's Vera Rubin rather than Blackwell, since both use HBM4, and says Jalapeño still edges ahead on output tokens per megawatt. The chip is still an engineering sample.
First generation chips are usually a learning exercise. You tape one out, it works, it loses to the incumbent, and you use what you learned on the second one. That is the normal shape of the story, and it is why the Jalapeño numbers got the reaction they did.
What the benchmarks actually say
OpenAI tested three open weight models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. The headline is efficiency rather than raw speed.
At peak throughput, OpenAI reports 1.5x to 1.9x more AI work per watt than the best commercially available systems, across all three models. End to end latency comes in 1.7x to 3.6x lower. On interactive workloads specifically, where one person is waiting for one answer, the gap widens to 2.1x to 4.1x.
The raw rates give that some texture. GPT-OSS 120B runs at roughly 1,400 tokens per second per user. DeepSeek R1 670B, five times the parameter count, still manages 700 tokens per second on a single concurrent request. Those are interactive numbers rather than batch numbers, and the distinction is the whole design brief.
The most quoted figure, 54x to 104x the token throughput per kilowatt at matched decoding speed, needs its qualifier kept attached. "At matched decoding speed" means the comparison holds interactivity constant and asks how many watts each system burns to sustain it. That is a legitimate question and a real advantage, but it is a narrow slice, not a general multiplier.
The memory generation does a lot of the work
Here is the part that got lost in the first round of coverage.
Jalapeño uses HBM4. Blackwell uses HBM3E. Comparing them on memory bound work, which decode overwhelmingly is, credits the design for something the memory vendors delivered. SemiAnalysis made the point directly: the honest like for like is Nvidia's Vera Rubin platform, which also ships HBM4.
On that comparison, Jalapeño still comes out ahead on output tokens per megawatt. The margin shrinks, the conclusion holds. SemiAnalysis summarised it as first generation chips usually not being competitive, and this one beating both Blackwell and Rubin.
That is the version worth carrying around. Not "OpenAI beat Nvidia by 100x", which is a benchmark artifact, but "a first attempt landed ahead of the incumbent's current generation on the specific axis that decides data centre economics".
Why every large buyer keeps arriving at the same idea
Inference and training want different hardware, and the gap has been widening for two years.
Training is a research workload. It changes shape constantly, it needs flexibility, and a general purpose GPU earns its price there. Inference is a production workload. Once a model is frozen, serving it is the same operation repeated billions of times, and repetition is exactly the condition under which custom silicon pays back its own design cost.
That is why the pattern keeps repeating. Google built TPUs. Nvidia itself bought a dedicated decode part and put it into production as Groq 3 LPX. Etched raised seven hundred million dollars at a twenty one billion dollar valuation for transformer specific silicon. AMD bought Taalas to burn model weights directly into silicon. Every one of those bets says the same thing about where the money is.
Power is the other half of the argument. Memory prices are pushing AI server costs up around 15 percent for early 2027, grid connections are the long pole in new builds, and a part that serves more tokens per watt is capacity you do not have to negotiate with a utility for.
What it changes for the rest of us
Nothing this quarter. Jalapeño is an engineering sample and there is no path for an outside developer to run anything on it.
The effect arrives indirectly, through price. OpenAI has already been cutting token prices, most recently taking GPT-5.6 Sol to four and twenty dollars per million. A serving fleet that costs meaningfully less per token to operate is what makes cuts like that sustainable rather than promotional. If Jalapeño reaches volume production, the visible signal will be published price per million output tokens continuing to fall while interactive latency improves.
So the practical advice is the boring kind. Do not re architect anything around a chip you cannot buy. Do keep watching the price lists, because that is where vendor benchmark claims eventually have to settle up.
Sources and further reading
- OpenAI's Jalapeño AI chip brings new threat to Nvidia margins as custom silicon gains ground, CNBC, August 26, 2026
- OpenAI's first custom chip Jalapeño reportedly beats Nvidia's Blackwell and Rubin in inference benchmarks, The Decoder, August 26, 2026
- OpenAI Jalapeño: Better Than Nvidia Blackwell, SemiAnalysis
- OpenAI's Jalapeño spices up chip market as it outperforms Nvidia's Blackwell, Seeking Alpha
Frequently asked questions
What exactly is Jalapeño and who built it?
Jalapeño is OpenAI's first custom accelerator, designed in partnership with Broadcom and built around HBM4 memory. It handles inference only, which means it runs models but does not train them. Design work started in mid 2024, the final design went to fabrication in November 2025, and the whole cycle took roughly sixteen months with about nine months of active design. That is fast for silicon. The split matters for reading the numbers: OpenAI is not claiming to have replaced its training fleet, it is claiming that the serving half of its bill can run on a part it controls, and serving is where the recurring cost lives once a model is finished.
How solid is the comparison against Nvidia Blackwell?
Partly solid and partly flattering, and the analysts who looked at it said so. Jalapeño uses HBM4, while Blackwell uses HBM3E, so a chunk of the memory bandwidth advantage comes from the newer memory generation rather than from the design. SemiAnalysis argued the honest like for like is Nvidia's Vera Rubin platform, which also uses HBM4. On that comparison Jalapeño still comes out ahead on output tokens per megawatt, just by a smaller margin. So the direction of the result survives the fairer baseline, but the headline multiplier does not. Read the per watt figures as real and the specific numbers as chosen carefully.
Which models were tested and what were the raw rates?
Three open weight models of very different sizes: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. On GPT-OSS 120B, OpenAI reports around 1,400 tokens per second per user. On DeepSeek R1 670B it reports 700 tokens per second on a single concurrent request. Those are single user interactive rates rather than fleet throughput, which is the regime that matters for agents and coding assistants where a person is waiting on the output. OpenAI also quotes 2.1x to 4.1x higher performance specifically on interactive workloads, which is consistent with a part tuned for latency rather than for batch efficiency.
Can I rent time on it?
Not yet, and this is the part most coverage skated over. Jalapeño has not moved beyond engineering samples. Nvidia's Vera Rubin, the platform it is being measured against, is already shipping to customers. A benchmark from a sample tells you what the design can do under conditions the designer picked. It does not tell you about yield, about how the part behaves in a rack at scale, or about what the software stack looks like for anyone outside OpenAI. The realistic read is that this changes OpenAI's own cost structure over the next year or two, and reaches everyone else, if at all, through the price of tokens.
What does this mean for Nvidia?
Pressure on margins in inference specifically, not a displacement. Inference is the volume half of the market and the half where a large buyer with one workload shape can justify designing its own silicon, which is exactly the logic that produced Google's TPU. Training remains the harder problem and the place where general purpose GPUs keep their advantage. The pattern to watch is not whether Nvidia loses OpenAI as a customer, it is how many of the big buyers end up running their own inference parts alongside Nvidia racks. Every one that does removes a slice of the highest margin volume.