Why Building an Nvidia-Killer Chip Is Harder Than It Looks
Sign up for ARPU: Stay ahead of the curve on tech business trends.
Broadcom, a key enabler for tech giants looking to design their own custom AI chips, has seen its market value soar past $1 trillion, fueled by the AI boom. By helping companies like Google and Meta Platforms create specialized silicon, Broadcom has carved out a lucrative niche, with its AI-related revenue projected to hit $5.1 billion in a single quarter. This "custom silicon" trend is often seen as a direct threat to Nvidia's dominance. However, a closer look at the challenges facing even a successful player like Broadcom reveals precisely why Nvidia's position is so difficult to assail.
Why are tech giants trying to build their own chips?
The primary motivation for tech giants to pursue custom chips is to create alternatives to Nvidia’s popular and expensive AI processors. By designing their own silicon, companies like Google and Meta hope to reduce their dependence on a single supplier and create chips that are highly optimized for their specific AI workloads, which could lower operational costs over the long run. The market for these custom-designed chips is growing, with some estimates projecting it will reach around $30 billion by 2027. This has created a significant opportunity for companies like Broadcom, which partner with these tech giants to help manufacture their designs.
What are the performance and cost hurdles of custom chips?
Despite the potential benefits, the road to custom silicon is fraught with obstacles. These projects require massive upfront investments, often in the hundreds of millions of dollars, and frequently fall short of performance goals. According to a recent report from TD Cowen, Google’s latest custom AI chips are only about half as powerful as their Nvidia counterparts, placing them just at the threshold where producing them makes financial sense. This performance gap means that even the companies investing heavily in their own chips are hedging their bets. Morgan Stanley analysts estimate that Google, while increasing spending on its custom chips by 10% to 20% this year, will simultaneously triple or quadruple its spending with Nvidia.
How does Nvidia's software create a "moat"?
One of the biggest hidden costs of custom silicon is software development. While Nvidia's hardware is expensive, it comes with CUDA, a mature, proprietary software platform that has been the de facto standard for GPU computing for over a decade. The vast ecosystem of libraries, tools, and developers experienced with CUDA allows companies to deploy AI workloads relatively easily. In contrast, custom chips require bespoke software to tell them what to do, a significant and costly "chore" that companies must undertake themselves. This software advantage is a powerful lock-in for Nvidia, as rewriting codebases and training developers for new, unproven hardware is a major deterrent for many potential customers.
What about the rest of the hardware system?
Nvidia's advantage extends beyond the chip and its software; it offers a "full-stack AI solution," including optimized server and networking designs. As noted in the case of Broadcom’s business, computing systems packed with custom chips often must use more expensive optical networking technology. This contrasts with Nvidia's integrated systems, which are designed to work together seamlessly. This system-level optimization, including high-speed NVLink interconnects that link GPUs together, minimizes performance bottlenecks and can reduce overall system cost and complexity—a feat difficult to replicate for companies assembling systems with disparate custom components.
What other risks do custom chip makers and their enablers face?
The custom chip market, while growing, is a competitive niche. Broadcom faces rivals like Marvell and Taiwan’s MediaTek, who are also vying for business from the same small pool of well-heeled tech giants. Furthermore, this business is exposed to significant geopolitical risk. Analysts believe one of Broadcom’s biggest clients is ByteDance, the Chinese owner of TikTok. This relationship makes the business vulnerable to US export controls, a risk that Broadcom's CEO, Hock Tan, acknowledged by stating, “Nobody can give anybody comfort in this environment.” This contrasts with Nvidia, which, despite being a primary target of US restrictions, serves a much broader global market that extends well beyond a few large customers in geopolitically sensitive positions. Ultimately, the high costs, performance risks, and software complexity mean custom chips are likely to remain the domain of only the wealthiest tech companies, leaving Nvidia to address a much larger total market.
Reference Shelf: