Hot Chips 2026: Arm AGI CPU and Scaling Compute for the Agentic AI Er

At Hot Chips 2026, Arm presented its AGI CPU, a disaggregated, chiplet-based server SoC designed to deliver scalable coherency and memory bandwidth as systems enter what Arm describes as the “terabyte/s era.”

The presentation brings together several trends reshaping CPU architecture: agentic AI, chiplet-based system design, increasing memory demands and closer integration between CPUs and accelerators.

Those trends raise an interesting question for us: as server architectures become more modular and workloads become more parallel, what additional capabilities can we add alongside the CPU?

Building CPUs for agentic AI

Arm describes AGI CPU as purpose-built for agentic AI, with an architecture focused on memory and I/O bandwidth per core and increased rack-level compute density.

The Q&A made the architectural reasoning behind this particularly interesting.

When asked what Arm had done specifically for agentic workloads, the presenter highlighted optimized memory latency, improvements to extract instruction-level parallelism, and high I/O bandwidth connecting the CPU to accelerators.

Those priorities reflect how AI systems are evolving. CPU performance increasingly depends not only on what happens inside an individual core, but also on how efficiently data can move through the broader system.

Arm's architecture reflects this directly. Its coherent mesh connects CPUs, memory, I/O and accelerators, while supporting coherent multi-chiplet scaling.

Industry analysis around Hot Chips has pointed in the same direction. Computerworld highlighted analyst observations that system architectures are changing as designers address data-movement bottlenecks and explore ways to bring computation closer to memory.

Building CPUs for agentic AI

Arm describes AGI CPU as purpose-built for agentic AI, with an architecture focused on memory and I/O bandwidth per core and increased rack-level compute density.

The Q&A made the architectural reasoning behind this particularly interesting.

When asked what Arm had done specifically for agentic workloads, the presenter highlighted optimized memory latency, improvements to extract instruction-level parallelism, and high I/O bandwidth connecting the CPU to accelerators.

Those priorities reflect how AI systems are evolving. CPU performance increasingly depends not only on what happens inside an individual core, but also on how efficiently data can move through the broader system.

Arm's architecture reflects this directly. Its coherent mesh connects CPUs, memory, I/O and accelerators, while supporting coherent multi-chiplet scaling.

Industry analysis around Hot Chips has pointed in the same direction. Computerworld highlighted analyst observations that system architectures are changing as designers address data-movement bottlenecks and explore ways to bring computation closer to memory.

Chiplets create new architectural possibilities

Arm's AGI CPU also demonstrates how chiplets are changing the way processors can be constructed. Its design supports coherent multi-chiplet scaling and uses UCIe for chiplet-to-chiplet communication.

Flow PPU is similarly designed with integration flexibility in mind. It can be implemented alongside a CPU as on-die IP or as a chiplet, and its instruction-set-independent architecture can complement CPU ecosystems including Arm.

That doesn't mean replacing the CPU or the accelerators already surrounding it. It means adding another complementary capability to the CPU-powered system.

CPU + the right architecture for the workload

Arm's presentation shows a server CPU evolving at multiple levels simultaneously: stronger CPU execution, greater memory bandwidth, lower memory latency, chiplet-based scaling and high-bandwidth connectivity to accelerators.

We approach another dimension of that evolution: extending CPU performance with closely coupled parallel acceleration for suitable general-purpose workloads.

As agentic AI and cloud workloads grow more complex, the future may not be about asking one processor architecture to handle everything.

It may instead be about building systems where CPU cores, memory systems and complementary acceleration architectures work together, each optimized for the work it does best.

Explore Flow's architecture and latest performance results. →