The July 7 NVIDIA article, “AI Innovators Adopt NVIDIA Vera — Why Max Single-Threaded CPU at Scale Matters,” introduces a useful concept NVIDIA calls “max single-threaded CPU at scale.” The phrase is partly NVIDIA marketing terminology, but it describes a legitimate architectural problem. 

Read the original NVIDIA article⁠

Traditional server CPU development has increasingly emphasized large numbers of cores and overall throughput. That makes economic sense for cloud computing: if a processor can support hundreds of independent virtual machines, containers or services, a cloud provider can sell more computing capacity from each server.

Agentic AI creates a somewhat different optimization problem.

An AI agent repeatedly performs something resembling:

GPU/model reasoning → CPU action → result → GPU/model reasoning → CPU action → result.

For example, an AI coding agent might ask the model what to do, execute Python code on the CPU, compile a program, run tests, inspect the results and then send those results back to the model.

The crucial point is that many of these operations are sequentially dependent. Step four cannot begin until step three finishes.

Adding another hundred CPU cores does not necessarily make step three finish sooner.

Making the individual CPU thread executing step three faster does.

That is the foundation of NVIDIA’s Vera strategy. 

Why this matters more than it initially appears

There is a classic computer-science principle called Amdahl’s Law. In simple English, it says that making one part of a computer system enormously faster eventually accomplishes little if another part remains slow.

Modern AI systems illustrate this perfectly.

GPUs have become extraordinarily fast at matrix mathematics. But an agent doesn’t spend its entire life performing matrix multiplication.

It may spend considerable time waiting for:
Python interpreters,database queries, retrieval systems, web services, code compilation, sandbox execution, data preparation, compression, encryption, operating-system functions, and other CPU-controlled activities.

NVIDIA therefore argues that an expensive Rubin GPU sitting idle while waiting for a CPU represents wasted AI-factory capacity. This is technically reasonable. NVIDIA explicitly describes CPU-side serial work as an increasingly important limitation on end-to-end AI performance. 

What NVIDIA actually built

Vera is substantially different from NVIDIA’s previous Grace CPU.

Grace uses Arm-designed Neoverse cores. Vera introduces Olympus, NVIDIA’s first fully custom data-center CPU core.

Vera contains 88 Olympus cores and 176 hardware threads. It implements the Arm v9.2 instruction architecture. 

The architecture is particularly interesting because NVIDIA did not simply pursue very high clock frequency.

Instead, it concentrated heavily on IPC—Instructions Per Cycle.

IPC means approximately how much useful work a CPU core can accomplish during each clock cycle.

NVIDIA says Olympus provides roughly 50% higher IPC than Grace. 

Several architectural features contribute to that.

1. A very wide instruction front end

Olympus has a 10-wide decode engine.

The CPU attempts to find and prepare many instructions simultaneously rather than processing instructions strictly one after another.

This increases the amount of potential work available to the execution engine. 

2. Neural branch prediction

Modern software constantly encounters decisions:

if this happens, execute A; otherwise execute B.

The CPU attempts to predict which path will occur before the program actually reaches the decision.

A wrong prediction wastes cycles because incorrectly anticipated instructions must be discarded.

Olympus incorporates what NVIDIA describes as a neural branch predictor, capable of handling as many as two taken branches per cycle. This is especially relevant to interpreters, databases, graph processing and agent software because these workloads contain considerably more irregular decision-making than the relatively predictable mathematics performed by GPUs. 

3. Deep out-of-order execution

Programs appear sequential to the programmer, but modern processors rearrange instructions internally whenever doing so allows useful work to continue while another instruction waits.

Olympus uses a large reorder capability, extensive physical registers and mechanisms including memory renaming and value prediction.

The objective is straightforward:

don’t allow one slow instruction to stall the entire CPU core. 

4. Specialized memory prediction and prefetching

AI-agent software frequently manipulates irregular data structures—objects, graphs, retrieval indexes, databases and pointers.

These are difficult workloads for conventional memory prefetchers because the next memory address isn’t necessarily predictable.

Olympus therefore includes multiple hardware prefetch mechanisms, including what NVIDIA calls a graph prefetcher, designed to anticipate these irregular accesses and move information closer to the processor before it is requested. 

Vera’s enormous memory bandwidth may be just as important as Olympus

One of Vera’s most impressive specifications isn’t the core count.

It is memory bandwidth.

Vera provides as much as 1.2 terabytes per second of LPDDR5X memory bandwidth, with as much as 1.5 TB of memory capacity. NVIDIA says this amounts to approximately 14 GB/s of memory bandwidth per CPU core. 

That matters because a powerful processor accomplishes nothing while waiting for data.

NVIDIA is essentially trying to prevent a situation where it builds 88 extremely fast cores and then forces them to fight each other for memory bandwidth.

The memory uses SOCAMM, NVIDIA’s modular implementation of LPDDR memory. Unlike soldered LPDDR commonly found in laptops and phones, the modules are replaceable and designed for data-center serviceability. 

NVIDIA also claims the memory consumes less than 40 watts while delivering that 1.2 TB/s bandwidth. 

The monolithic architecture is another important decision

AMD and increasingly Intel have embraced chiplets—smaller semiconductor dies connected together to construct a larger processor.

Chiplets provide major manufacturing and economic advantages.

NVIDIA deliberately chose a different approach for Vera’s compute fabric.

The 88 Olympus cores and shared cache operate through a largely monolithic compute die, connected by NVIDIA’s second-generation Scalable Coherency Fabric, or SCF.

SCF provides approximately 3.4 TB/s of bisection bandwidth and connects the cores, shared cache, memory controllers, I/O and NVLink interfaces. Vera has a 164 MB unified L3 cache. 

NVIDIA argues that this eliminates what it calls the “chiplet tax”—latency and bandwidth variability resulting from data having to move between different processor dies. 

There is legitimate engineering behind that argument, but NVIDIA overstates it somewhat.

Chiplets are not inherently inferior. AMD has demonstrated extraordinarily successful high-performance chiplet CPUs. Chiplets improve manufacturing yield, scalability and cost.

NVIDIA has simply decided that predictable memory latency and extremely high bandwidth across all cores are valuable enough for this particular workload to justify a different design tradeoff.

That is a more technically neutral way of describing the decision.

NVIDIA Spatial Multithreading

Vera also introduces an interesting variation on conventional simultaneous multithreading.

Traditional SMT—Intel’s familiar Hyper-Threading is one example—allows two software threads to share one physical CPU core.

The problem is resource contention.

Both threads may compete for caches, execution units, instruction bandwidth and other resources.

NVIDIA calls its approach Spatial Multithreading.

Olympus’s unusually wide architecture allows resources to be partitioned more deliberately between two threads. The processor can emphasize maximum performance for one thread or divide resources between two execution contexts.

Consequently:

88 physical cores → 176 hardware threads.

NVIDIA’s objective is not merely increased utilization but more predictable performance when thousands of independent agent environments are executing simultaneously. 

CPU and GPU become parts of one architecture

Vera becomes even more interesting when paired with Rubin GPUs.

The connection uses second-generation NVLink-C2C, providing as much as 1.8 TB/s of coherent CPU-GPU bandwidth. 

“Coherent” is important.

It means CPU and GPU memory systems can maintain a consistent view of data rather than requiring programmers continually to copy information manually between isolated memory spaces.

The resulting architecture increasingly resembles:

CPU + GPU + high-bandwidth memory + networking + storage + software = one large AI computer.

That is the larger NVIDIA strategy.

The unit of competition is gradually shifting from the individual GPU to the rack and ultimately the entire data center.

Do the performance claims hold up?

There is encouraging independent evidence, but this requires an important qualification.

Phoronix received early access to Vera and benchmarked an 88-core/176-thread Vera system against contemporary AMD EPYC and Intel Xeon processors.

Across the permitted test suite, Vera produced a geometric-mean result approximately 10% ahead of AMD’s high-frequency EPYC 9575F and about 55% ahead of a single Intel Xeon 6980P, while producing approximately 1.63× the performance of NVIDIA Grace. 

Phoronix described Vera as the strongest Arm server CPU competitor to x86 it had encountered. 

It also produced particularly strong results in workloads such as ClickHouse database processing and some compression and Java workloads. 

However, there is an important limitation:

NVIDIA restricted which workloads could be tested during this initial evaluation.

Therefore these results should not yet be interpreted as proving that Vera is universally faster than AMD EPYC or Intel Xeon.

They demonstrate something narrower but still significant:

Vera appears extremely competitive on the workloads NVIDIA specifically designed it to execute.

That is actually more relevant to the agentic-AI argument than winning every conventional server benchmark.

NVIDIA’s partner results

NVIDIA reports several additional real-world tests.

Perplexity reportedly tested a coding-agent workload involving cloning software repositories and executing test suites. Vera completed the workflow approximately 1.5× faster than x86, while concurrent sandbox startup was as much as 1.9× faster.

Starburst reportedly measured approximately 3× faster large-scale SQL analytics, while Redpanda reported as much as 6× lower latency for real-time streaming workloads, compared with leading x86 server CPUs. 

These are interesting results, but they remain vendor/partner claims rather than a comprehensive neutral benchmark suite.

The most important implication

[Likely] Vera represents something larger than NVIDIA entering the CPU market.

It illustrates where AI computer architecture is heading.

The first stage of the AI infrastructure race concentrated overwhelmingly on:

Who has the fastest GPU?

The next stage increasingly becomes:

Who can build the fastest complete AI computing system?

That means CPU performance, GPU performance, memory bandwidth, interconnect bandwidth, networking, storage and software orchestration all matter simultaneously.

Vera is therefore not really an attempt to replace Rubin GPUs.

Its purpose is to keep Rubin GPUs working.

And that becomes increasingly important with agentic AI because an agent repeatedly crosses the boundary between neural-network computation and conventional software execution.

Evaluation

[Likely] I would rate Vera as a potentially very important CPU architecture, rather than simply another server processor.

NVIDIA identified a genuine architectural change: AI is moving from relatively simple model inference toward systems in which models continually interact with conventional computing environments.

That changes the optimum CPU.

The combination of 88 high-IPC Olympus cores, 1.2 TB/s memory bandwidth, 3.4 TB/s internal fabric bandwidth, a 164 MB shared L3 cache, Spatial Multithreading and 1.8 TB/s coherent CPU-GPU connectivity is unusually well aligned with that workload. 

The principal uncertainty is benchmarking. NVIDIA’s architectural explanation is convincing, and the early independent Phoronix results are impressive, but unrestricted testing across broader workloads—and especially comparisons with the next generations of AMD EPYC and Intel Xeon—will be necessary before claims of general CPU leadership can be established.

The larger conclusion is more significant than who wins an individual benchmark:

Vera demonstrates that CPUs are becoming more important, not less important, as AI advances.

GPUs perform the neural computation, but increasingly capable AI agents must operate conventional computers, execute software and manipulate enormous amounts of data. That creates a new bottleneck between successive model calls. NVIDIA is betting that removing that bottleneck requires a CPU optimized not simply for maximum core count, but for very fast individual threads, sustained simultaneously across a heavily loaded data center.

That is the real significance of Vera—and the reasoning behind it is technically compelling.