NVIDIA thinks the CPU industry chased too many cores, and AI agents are bringing single-core speed back

For years, the data-center CPU race has revolved around a fairly straightforward question: how many cores can fit into a server, and how much work can they chew through at once? NVIDIA now argues that this pursuit of scale came with a trade-off that matters more as AI systems become increasingly autonomous.

Did the CPU industry get too obsessed with core counts?

In an interview with Wccftech, the company’s Vera team said the industry’s shift toward chiplet-based designs and higher core counts came at the expense of single-threaded performance and instructions per cycle (IPC). With AI agents demanding faster execution between successive reasoning steps, NVIDIA believes the CPU needs a rethink.

As cloud data centers grew, chipmakers focused on squeezing more cores into servers to handle more workloads at once. Chiplet designs helped make this possible by splitting a processor across multiple smaller dies instead of building everything on a single, larger die. But NVIDIA’s Vera team argues that this approach comes with trade-offs. Moving data between dies can add communication delays and complicate memory access, potentially limiting how much single-threaded performance each core can deliver. That doesn’t make chiplets a bad design choice, but NVIDIA believes the industry’s push for more cores has come at the expense of faster individual cores.

“…what suffered over that time in terms of CPU design was really single-threaded perf and IPC because when you fill, when you use chiplets, you just, you literally don’t have the space on the, on the chip to design for per core perf.”

Agentic AI is changing that equation. Unlike a chatbot that simply generates a response, an AI agent may need to execute code, call external tools, retrieve information, analyze the results, and then decide what to do next. While some tasks can run simultaneously, others must finish before the next step can begin. That’s where single-threaded performance, or how quickly a CPU can handle work on one thread, becomes important. NVIDIA argues that faster execution and lower latency can help agents move through these steps more quickly.

There’s also a knock-on effect for GPUs. If a CPU takes too long to prepare the next task or process an agent’s output, the GPU may sit idle while waiting for instructions, wasting valuable computing capacity. Simply adding more CPU cores won’t necessarily solve that problem if individual tasks remain slow. NVIDIA argues that agentic AI needs a better balance between core counts and per-core speed, so the CPU can keep pace with the GPU rather than becoming the bottleneck

How does NVIDIA Vera plan to bring single-core speed back?

Vera aims to combine faster individual cores with the throughput modern data centers need. It packs 88 custom Olympus CPU cores onto a monolithic compute die, alongside a custom LPDDR5X memory subsystem. NVIDIA says its second-generation Scalable Coherency Fabric delivers up to 3.4 TB/s of bisection bandwidth, three times that of leading x86 chips. Keeping the cores on a single compute die also avoids communication across separate chiplets. These are NVIDIA’s claims, but the strategy is clear: prioritize fast, efficient communication without giving up the benefits of high core counts.

Vera also signals a shift in how NVIDIA sees CPUs fitting into AI infrastructure. While its Grace CPU primarily serves as a host for GPU-powered systems, NVIDIA says agentic AI is creating demand for standalone CPU racks to handle tool calls, code execution, and reinforcement learning. That doesn’t mean GPUs are becoming less important or that chiplets are going away. NVIDIA itself says it remains open to chiplet designs when they make sense.

The bigger point is that AI is changing what a good CPU looks like. As agents move beyond answering questions to executing multi-step tasks, raw core counts alone may no longer tell the whole story. The next battle isn’t just about cramming more cores into a chip. It’s about making each core fast enough to keep the AI factory running, rather than leaving its most expensive machinery waiting around.

RELATED COVERAGE: