OpenAI named their first custom silicon “Jalapeño.” That tells you something about how the company is thinking: this is a chip that is supposed to have bite.
On June 24, 2026, OpenAI officially unveiled an inference processor designed in-house and manufactured by Broadcom. Early numbers show “significantly better performance-per-watt than current state-of-the-art alternatives” — the company’s own words, which are carefully chosen and carefully vague. The chip is still being tested. A full production deployment timeline has not been announced.
This article is not a news recap. TechCrunch and The Verge have already covered what OpenAI said. What I want to do here is work through what this chip actually means — for OpenAI’s economics, for the inference market, for Nvidia, and for the broader question of whether vertical integration in AI hardware is a winning strategy or a seductive trap.
The short answer: Jalapeño is strategically sound and operationally necessary. But it is also at minimum three years behind Google, poorly timed against Nvidia’s roadmap, and entering a market where the definition of “inference” is shifting faster than any chip design cycle. Whether it pays off depends almost entirely on how quickly OpenAI can close the gap between “works in testing” and “runs production at scale.”
Why Custom Silicon, and Why Now
Start with the basic economics. OpenAI runs models at a scale that no company has run before. ChatGPT reportedly hit 1 billion monthly active users in May 2026. Codex has 5 million weekly users. Every one of those interactions requires compute. The majority of that compute, at inference time, runs on Nvidia H100 and H200 GPUs.
Nvidia’s GPU pricing is famously aggressive. H100 clusters have been renting at $2–3/hr per GPU in the spot market, and committed capacity on H200s has been quoted even higher. For a company running inference at OpenAI’s scale, GPU cost is not a line item in the budget — it is the budget.
There is a secondary problem beyond raw cost: GPU utilization efficiency. Nvidia GPUs are general-purpose accelerators. They are very good at the matrix multiplications that dominate both training and inference, but they carry a significant amount of silicon — and power budget — that is wasted on inference-specific workloads. HBM bandwidth, for instance, matters enormously during training. During inference on a deployed model with a fixed weight set, the bottleneck shifts. The ratio of memory bandwidth needed to arithmetic operations is different. A chip designed specifically for inference can exploit this and get dramatically better FLOPS-per-watt on the specific task it is being asked to do.
Google understood this in 2015. Their first Tensor Processing Unit went into production that year, designed specifically for inference on their neural networks. By the time they went public with TPU details in 2017, they had already been running custom silicon at datacenter scale for two years. Amazon launched Inferentia in 2018 and Trainium in 2021. Meta has their MTIA (Meta Training and Inference Accelerator), now in its second generation. Microsoft has had internal custom silicon efforts tied to their Maia project.
OpenAI is late. By the standards of how long it takes to design, tape out, validate, and deploy a custom chip at scale, they are at minimum two full product generations behind Google. That is not a trivial gap.
But the question is not whether they were late. The question is whether they are late enough that it does not matter. And the answer here is: no. The inference market is large enough and growing fast enough that a well-executed custom chip in 2026 can still generate real returns, even entering a market where competitors have been running custom silicon for a decade.
What We Know About Jalapeño
OpenAI has disclosed relatively little technical detail. Here is what is confirmed:
- Purpose-built for inference, not training
- Designed specifically around real-time coding model workloads (the announcement emphasized this)
- Manufactured by Broadcom — which means it is almost certainly built on TSMC process nodes, since that is Broadcom’s custom ASIC model
- OpenAI’s own AI models assisted in the chip design process
- Early testing shows “significantly better performance-per-watt than current state-of-the-art alternatives”
- Still in testing; no production deployment timeline announced
What is not confirmed but can be inferred:
The Broadcom manufacturing partnership almost certainly means the chip is built on TSMC N3 or N3E, possibly N2 if the timeline permits. Broadcom has a longstanding relationship with TSMC for custom ASIC production, and their existing AI accelerator work (notably for Google TPUv5 and reportedly for Meta) runs through that pipeline. A chip announced in mid-2026 would have been taped out 18–24 months earlier, which puts initial design work in late 2024, just as the Broadcom partnership was being formally established.
The focus on “coding models” and “real-time” workloads is telling. Coding inference — Codex, the operator API, code completions — has different latency and batch size characteristics than general chat inference. It tends to involve longer context windows, more structured output, and users who are more sensitive to latency than consumers chatting casually. Designing for this workload specifically suggests OpenAI has a very clear operational target: reduce the cost of running Codex at scale while keeping latency competitive.
The “AI-assisted design” claim deserves scrutiny. OpenAI has an incentive to say this. It is a compelling narrative: AI designing the chips that run AI. But custom ASIC design is a mature engineering discipline with well-established toolchains. AI-assisted place-and-route, power optimization, and simulation is real and is being used at most major chip companies. Whether OpenAI’s use of their own models provided material advantage over what EDA software already does is unknowable from the outside. It is plausible. It is also very convenient marketing.
The Vertical Integration Thesis
Greg Brockman articulated the strategic logic clearly: OpenAI has “a deep understanding of the workload.” When you control the model architecture, the training process, the serving infrastructure, and the hardware, you can optimize each layer for the others. That is the vertical integration thesis, and it is the same thesis that justified Google’s TPU investment, Apple’s neural engine, and Amazon’s Trainium.
The thesis is correct. Vertical integration in AI infrastructure is genuinely value-creating when done well. Google has demonstrated this for a decade. The efficiency advantage of running a model on hardware you co-designed with the model in mind is real. You can make architectural decisions in the model — attention head sizes, weight quantization schemes, KV-cache management strategies — that map directly to hardware features. You cannot do this if you are buying commodity GPUs.
OpenAI also stated this directly in their announcement: “OpenAI is not only developing frontier models or building products on top of them; it is designing the infrastructure underneath them: chip architecture, kernels, memory systems, networking, scheduling, deployment systems, and product experience.”
This is a complete description of vertical integration. It is also an enormous operational surface area. Each of those layers requires specialized engineering talent that is genuinely scarce. Chip architecture teams, kernel engineers, networking specialists, hardware-software co-design experts — these are not the same people who train LLMs. They are a different discipline with a different culture and different compensation expectations. Building this from scratch while simultaneously competing at the frontier of AI capabilities is a significant organizational challenge.
The companies that have done this successfully — Google, Apple, Amazon — all had hardware engineering culture already embedded before they extended into AI accelerators. Google had already built custom network switches and storage systems before TPU. Apple had been designing custom SoCs since A4 in 2010. Amazon had years of custom silicon experience in networking (Annapurna) before Inferentia.
OpenAI is building this capability from scratch, at the same time it is scaling a product used by a billion people, pursuing IPO readiness, and trying to stay at the frontier of model capability. That is a lot to do simultaneously.
Comparing the Field: TPU, Trainium, MTIA
Let me put Jalapeño in context by looking at what the competitors have built.
Google TPU v5p / v5e: Currently in production, TPU v5 is optimized for both training and inference. Google Cloud offers TPU v5e specifically for inference workloads at competitive price points. The architecture supports the custom Pathways distributed training system, which is tightly co-designed with the hardware. Google has had 11 years to iterate on this. The hardware-software co-design depth at Google is the benchmark against which everyone else is measured.
AWS Inferentia2 / Trainium2: Amazon’s Inferentia2 is their second-generation inference accelerator. AWS offers it through Inf2 instances on EC2. Trainium2 is for training. Both are manufactured by TSMC at advanced nodes. Amazon has had external customers running workloads on Inferentia for years, which has driven a real-world feedback loop that purely internal chips do not benefit from. The NeuronSDK has matured significantly. The gap between “supported” and “optimized” for common model architectures has narrowed.
Meta MTIA v2: Meta’s second-generation MTIA is deployed at scale for recommendation models internally. It is not available externally. For recommendation inference, Meta claims MTIA is significantly more efficient than GPUs. The recommendation model use case is very different from generative AI inference — the access patterns, memory pressure, and arithmetic intensity are different enough that MTIA v2 tells us relatively little about what Jalapeño can or cannot do.
Nvidia H200 / B200 Blackwell: Nvidia’s Blackwell architecture launched in 2025. The B200 delivers substantially more inference throughput than H100, with improved FP8 performance and higher HBM bandwidth. The NVLink Switch fabric for multi-GPU inference is more capable. Critically, Nvidia’s software ecosystem — CUDA, TensorRT, vLLM support — is deeply embedded in the model serving infrastructure that almost every AI company uses. Displacing CUDA is a decade-long project, and anyone who claims otherwise has not worked in production ML infrastructure.
Where does Jalapeño sit in this landscape? On the technical side: unknown, because no independently verifiable benchmarks exist. On the software side: at the very beginning. On the maturity side: far behind every named competitor.
What Jalapeño has that the others do not: the model architect and the chip designer are at the same company. OpenAI can make co-design decisions at a depth that even Amazon cannot match when running third-party models on Inferentia. This is the theoretical advantage. Whether it translates to real-world performance advantage at deployment scale is the question that cannot be answered from an announcement blog post.
The Inference Economics Argument
Here is the financial logic that makes Jalapeño worth the investment, even starting late.
Inference cost at OpenAI’s scale is enormous. Even conservative estimates suggest OpenAI spends hundreds of millions of dollars per year on inference compute. If Jalapeño delivers 2x better performance-per-watt on inference workloads — plausible for a purpose-built chip versus a general-purpose GPU — and if it reaches production deployment at meaningful scale within 12–18 months, the ROI on chip development amortizes quickly.
The math is straightforward. If inference compute costs $500M/year on Nvidia GPUs, and a custom inference chip reduces that by 40%, that is $200M/year in savings. The development cost for a chip like Jalapeño — design team, EDA tools, tape-out costs, test and validation — is in the range of $300–500M for a first generation. So break-even is roughly 18–30 months post-deployment at that scale. Subsequent generations get cheaper to develop and the savings compound.
This is exactly the math that justified Google’s TPU investment. Google published a paper in 2017 noting that TPU v1 delivered 15–30x better performance-per-watt than GPUs for inference. That number is almost certainly not achievable for Jalapeño against modern Blackwell GPUs — the GPU has improved dramatically since 2015. But even a 2–3x efficiency advantage is economically significant at OpenAI’s scale.
The critical variable is the timeline from “works in testing” to “runs significant fraction of production inference.” This is where most custom chip programs underperform their projections. The gap between a chip that passes functional tests and a chip that runs production workloads reliably, at the utilization rates needed to justify the capital investment, is bridged by software work that is often harder than the hardware design itself. Kernel optimizations, compiler support, model conversion tooling, operational monitoring, failover logic — all of this has to be built and hardened before a custom chip is truly production-ready.
Google had 18–24 months of internal testing before broadly deploying TPU at scale. Amazon Inferentia took over a year from announcement to broad availability on EC2. OpenAI’s timeline from today’s announcement to meaningful production deployment is probably 12–24 months minimum, depending on how much of the software stack is already built.
The Nvidia Dependency Problem
OpenAI’s relationship with Nvidia is complicated. Microsoft, OpenAI’s largest investor, has its own Nvidia relationship for Azure GPU capacity. OpenAI also reportedly has direct Nvidia commitments as part of the Stargate partnership. The company is simultaneously one of Nvidia’s biggest customers and now a potential competitor in the inference accelerator space.
Nvidia has a history of dealing with this kind of situation strategically. When hyperscalers started building custom silicon, Nvidia’s response was to move up the stack — better software tools, better networking (NVLink, Infiniband), better ecosystem lock-in. The CUDA ecosystem is not an accident; it is a deliberate moat that makes it extremely painful to migrate workloads away from Nvidia hardware.
OpenAI’s custom inference chip does not, on its own, break the CUDA moat. Training will continue on Nvidia for the foreseeable future. The most compute-intensive workloads — pre-training frontier models — will remain on H200/B200 clusters because no one builds custom training silicon at the scale needed to compete with Nvidia for that workload, except Google and possibly Microsoft if their Maia project scales.
What Jalapeño could do is reduce OpenAI’s inference dependency on Nvidia. If inference moves to custom silicon and training stays on Nvidia, OpenAI’s GPU procurement shifts from “inference + training” to “training only.” That changes the leverage dynamic in procurement negotiations. It also reduces OpenAI’s exposure to Nvidia supply constraints, which have been a real operational problem during GPU shortage periods.
This is strategically valuable independent of whether Jalapeño achieves every performance claim. Optionality has value. Having a credible internal inference platform gives OpenAI negotiating leverage with Nvidia that it does not currently have.
The “AI-Designed AI Chips” Narrative
I want to spend some time on the claim that OpenAI’s AI models assisted in Jalapeño’s design, because it is going to generate a lot of breathless coverage and most of it will miss the actual significance.
What is probably true: OpenAI used LLM-assisted code generation for RTL (register-transfer level) design, power analysis scripts, verification testbench generation, and documentation. These are real productivity multipliers. Google, NVIDIA, and Intel all have internal programs using ML to assist with specific chip design tasks. The EDA industry (Synopsys, Cadence) has been integrating ML into their toolchains for years.
What is probably overstated: that the AI assistance was uniquely transformative in a way that is not achievable with commercial EDA tools and standard ML-in-design techniques. If OpenAI’s models provided a genuine breakthrough in chip design methodology — something beyond what the commercial tools do — that would be significant enough that it would be published in a paper, not mentioned in a product announcement.
What is actually interesting: that OpenAI is dogfooding their own models on hard technical tasks at the frontier of what those models can do. Using GPT-4 class models on complex hardware engineering is a genuine test of capability. If the assistance was material, it tells us something real about where LLM coding assistance is in 2026 on highly specialized, formally constrained domains. That is a data point worth having.
The “AI designing its own chips” narrative is compelling because it suggests an acceleration flywheel: better AI designs better chips, which runs better AI. This is theoretically possible. In practice, it is constrained by the physics of chip design — you still need to tape out, fabricate, test, and deploy, which takes 18–24 months minimum from design completion regardless of how smart your AI assistant is. The flywheel spins, but not faster than TSMC’s process node schedule.
What Broadcom’s Role Tells Us
The choice of Broadcom as manufacturing and design partner is informative. Broadcom is not a chip designer in the way NVIDIA or AMD is — they are primarily a custom ASIC and networking chip company. Their business model is working with hyperscalers to design purpose-built silicon and then manufacturing it through TSMC.
Broadcom already manufactures Google’s TPU. They reportedly manufacture Meta’s MTIA. They are the dominant provider of custom AI accelerator silicon to hyperscalers. This relationship with OpenAI brings their client count to at least three major AI-focused custom chip programs.
What Broadcom brings: deep TSMC relationship, a large library of IP blocks (SerDes, PCIe, memory controllers, fabric interfaces), experienced custom ASIC design teams, and a supply chain that is already proven at datacenter scale for similar workloads.
What Broadcom does not bring: model-level expertise, inference serving knowledge, or ML-specific compiler toolchains. The software stack is entirely OpenAI’s problem to solve.
The partnership structure likely involves Broadcom doing the physical implementation (layout, place-and-route, timing closure) while OpenAI owns the microarchitecture specification and any custom functional units. This is the standard hyperscaler custom ASIC model. It reduces OpenAI’s need to build a full chip design team while still giving them meaningful control over the architectural decisions that matter for ML performance.
Production Readiness: The Underrated Hard Part
Custom chip announcements have a consistent pattern: the hardware announcement generates significant coverage, followed by a long quiet period, followed by a quiet production deployment, followed by a much smaller “actually running at scale” acknowledgment 18–36 months later. The gap between announcement and scale is where custom chip programs live or die, and it is almost entirely a software and operations story.
OpenAI needs to build or has already been building:
Compiler/kernel support: The inference kernels that run on Jalapeño need to be hand-optimized for the specific memory hierarchy and execution unit configuration of the chip. This is not automatic. Flash Attention, the CUDA-optimized attention kernel that is now standard in production inference, took significant engineering work to implement. Every equivalent optimization needs to be rebuilt for the new hardware. The toolchain for compiling PyTorch models to Jalapeño’s native format needs to be developed, tested, and integrated into OpenAI’s existing deployment pipeline.
Model quantization and precision handling: Production inference at scale uses INT8, FP8, or lower precision to maximize throughput. The quantization strategy needs to be co-designed with the chip’s arithmetic units. Post-training quantization of OpenAI’s models for Jalapeño is a non-trivial engineering project.
Reliability and operational tooling: A Nvidia GPU failure is handled by mature tools with years of operational experience. Jalapeño failures, thermal events, and partial hardware faults need custom monitoring, alerting, and recovery tooling. This gets built during the production ramp period, which is messy.
KV cache management: Modern inference serving for long-context models is dominated by key-value cache memory pressure. How Jalapeño’s memory hierarchy handles KV cache — both the capacity and the bandwidth — will largely determine whether it is competitive for long-context coding workloads. This is architectural and cannot be fixed in software if the hardware design does not support it.
Google had a multi-year lead in solving these problems for TPU. OpenAI is starting from scratch on all of them. The announced 12–18 month testing window before production deployment is probably optimistic for reaching the kind of scale where Jalapeño significantly displaces Nvidia silicon in OpenAI’s inference stack.
The Broader Implications for the AI Infrastructure Market
Jalapeño’s announcement reinforces a trend that has been building for several years: the large hyperscalers and frontier AI labs are all moving toward custom inference silicon. This creates a bifurcated market.
For the small end — startups, mid-size companies, research labs — Nvidia retains total dominance. The CUDA ecosystem, the cloud availability (A100/H100/H200 on every major cloud), and the community around PyTorch and CUDA-optimized kernels means no one at this scale has a reason to consider alternatives. The volume is too low to justify custom silicon development, and the commercial alternatives (Inferentia, Trainium, TPU) require significant engineering investment to use effectively.
For the large end — OpenAI, Google, Meta, Microsoft, Amazon — custom silicon is increasingly a cost of doing business. The volume of inference justifies the capital investment. The model architecture control allows co-design. The competitive pressure on inference pricing makes any efficiency advantage worth pursuing.
This bifurcation is uncomfortable for Nvidia. Their highest-volume inference customers are moving toward custom silicon. Their GPU ASP (average selling price) will increasingly be driven by training workloads, which are less continuous and more project-driven than inference. The secular growth story for Nvidia in AI inference is weakening as the largest customers vertically integrate.
Nvidia’s response — better networking, richer software ecosystem, data center-level integrated solutions — is the right response, but it only helps with customers who are not yet vertically integrated. For OpenAI, Google, Meta, and Amazon, the train has left the station on inference silicon independence.
The company most threatened by this trend, somewhat counterintuitively, is not Nvidia but the emerging commercial AI accelerator vendors: Cerebras, Groq, d-Matrix, and others who have bet on selling custom inference silicon to large AI companies. OpenAI building their own chip removes them from OpenAI’s addressable customer list. If the trend continues and every frontier lab builds internally, the commercial AI accelerator market shrinks to mid-tier customers — a harder segment to profitably serve.
What This Means for OpenAI’s Competitive Position
Jalapeño is not primarily a technology announcement. It is a cost structure announcement. OpenAI is telling investors and partners: our inference cost basis will improve as we ramp custom silicon. This matters for IPO readiness.
OpenAI’s gross margins on inference are, by all external estimates, significantly negative or marginally positive today. The revenue per token from API users and ChatGPT subscriptions has been declining as models become more capable and users push toward longer contexts. The cost per token has been declining too, but more slowly. Custom inference silicon that delivers 2–3x better performance-per-watt directly improves this equation.
For the IPO narrative, being able to say “we are reducing our dependence on Nvidia for inference through our own silicon program” is a meaningful story. It addresses the gross margin concern and demonstrates the kind of vertical integration sophistication that public market investors in infrastructure companies reward.
It also adds to OpenAI’s technical moat in a way that model capability alone does not. Models can be replicated or approximated. Custom silicon and the co-design expertise to use it effectively cannot be easily replicated. It is a different kind of defensibility.
Honest Assessment: Where Jalapeño Lands
Let me give a direct evaluation rather than hedging.
What Jalapeño does well: It proves OpenAI can execute on hardware engineering at the ASIC level, partnering with the right manufacturing partner (Broadcom/TSMC). It is specifically designed for the workload that matters most to OpenAI’s current business (coding inference). It creates optionality in the vendor relationship with Nvidia. The announcement timing is smart — pre-IPO, demonstrates strategic depth.
What Jalapeño does not do: It does not make OpenAI self-sufficient on compute. Training will remain on Nvidia indefinitely. The software stack needed for production deployment does not exist yet in any mature form. The performance claims are unverified and stated in the vaguest possible way (“significantly better performance-per-watt than current state-of-the-art alternatives” — better than what, measured how, at what workload?). It does not close a three-generation gap with Google TPU on inference optimization depth.
What Jalapeño might become: If the production ramp goes well, if the kernel and compiler stack matures over 18–24 months, if OpenAI builds the operational expertise to run a heterogeneous inference fleet with both Jalapeño and Nvidia hardware, then the second and third generations of this chip will be genuinely competitive. Custom silicon programs always look uncertain in generation one. The value compounds in generation two and three when the co-design feedback loop has had time to work.
The right comparison is not “Jalapeño vs. H200 today.” It is “Jalapeño v3 in 2029 vs. whatever OpenAI’s inference cost structure would be if it had never invested in custom silicon.” On that timescale, the investment almost certainly pays off — if OpenAI exists and operates at scale in 2029, which is the real uncertainty here, not the chip design.
The Name
One last note. Naming a chip Jalapeño is either very confident or very clever marketing. Jalapeños are not the hottest chili — not by far. They sit in the middle of the Scoville scale, well below habanero, ghost pepper, Carolina Reaper. They are, however, the most commonly used hot pepper in the world. Ubiquitous, reliable, versatile.
If OpenAI’s chip naming philosophy follows the pepper scale, this is a chip designed to be everywhere, not to top the performance charts. Purpose-built for the most common inference workload, optimized for cost and throughput rather than peak performance, deployed at a scale that makes it the default rather than the exception.
That is actually a coherent strategy. The best inference chip is not the one with the highest peak FLOPS. It is the one running the most tokens per dollar at production scale. By that measure, Jalapeño has the right ambition — if OpenAI can close the execution gap between today’s announcement and tomorrow’s production deployment.
The peppers are in. The kitchen is still being built.
Discussion