OpenAI puts Jalapeño to the test as AI hardware race shifts to efficiency

137

TL;DR:

OpenAI’s Jalapeño chip moves its AI hardware strategy from promise to measured performance: the inference ASIC delivered up to 1.9x more work per watt and 3.6x lower latency in tests, sharpening the race to cut AI serving costs.

Article:

OpenAI has put numbers behind Jalapeño, its first custom AI inference chip developed with Broadcom, reporting 1.5–1.9x more AI work per watt and 1.7–3.6x lower end-to-end latency than comparison systems across GPT‑OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T. The results matter because inference, the computing used when models answer users, is a major cost and power constraint as AI products scale.

Unveiled in June, Jalapeño is scheduled for initial deployment by the end of 2026. OpenAI says the chip is rated at 700 watts, while sustained power stayed at or below 550 watts on the workloads tested.

The strategic prize is efficiency at scale. More work per watt can mean more tokens from fixed data-centre power, lower serving costs and greater leverage with outside chip suppliers. OpenAI and Broadcom previously outlined plans for 10 gigawatts of custom accelerators, with deployment targeted to start in the second half of 2026 and finish by end-2029.

Independent scrutiny is encouraging but incomplete. SemiAnalysis, which observed InferenceX runs with OpenAI engineers, wrote that “Jalapeño beats Blackwell on perf/W across almost all scenarios.” It also cautioned that OpenAI supplied the numbers, it did not run the full InferenceX suite, and it has not yet seen AgentX results.

That caveat matters. Benchmark leadership must still survive manufacturing yield, software maturity, reliability and rack-scale deployment. The next test is whether Jalapeño can retain its efficiency advantage in production while staying flexible across model families.

For users and developers, the upside is faster responses and potentially lower inference costs. The move also follows a broader hyperscaler pattern of custom accelerators, but OpenAI’s bet puts unusual weight on inference economics. For the AI hardware market, Jalapeño raises pressure on price, power efficiency and supplier lock-in.