Qualcomm Cloud AI 100 Ultra Inference Card for Robot Fleet Backend Services
The on-site inference tier that makes large language models fast enough for robot fleets.
A robot fleet running fifty or a hundred units needs a place to run inference that isn't the robot itself and isn't a cloud region three hundred miles away. Local, on-robot compute handles perception and reflexes in 15 to 45 milliseconds end to end. Routing that same request to the cloud requires counting transmission, queuing, and the round trip back, which totals 800 to 2,400 milliseconds: unusable for anything a robot needs to act on now. Qualcomm's Cloud AI 100 Ultra sits in the gap between those two, a facility-layer card that holds a model too large for the robot's own chip but too time-sensitive to hand off to a data center somewhere else.
That gap is where a fleet's shared intelligence actually lives: multi-billion-parameter vision-language-action models deciding what a robot arm does next, world-model rollouts simulating outcomes before a robot commits to a move, and semantic maps of the facility that every robot on the floor reads from and writes to. Most people picture a robot deployment as two tiers, the device and the cloud, done. That picture is wrong, and it's the reason so many fleet projects bolt on a middle tier after the fact instead of designing for it from day one. The on-site cluster is what nobody plans for early enough, and it ends up deciding whether the fleet works at scale.
What the AI 100 Ultra is and how its architecture enables this tier
The AI 100 Ultra is the newest card in Qualcomm's Cloud AI inference lineup, built specifically for generative AI and large language models. Qualcomm announced it in November 2023 and has since moved it into production deployments rather than leaving it as a paper spec.
What sets it apart from a single-chip accelerator card is the decision to put four AI 100 system-on-chips on one board, tied together through a PCIe switch. That switch talks to the host server over PCIe Gen4 x16, and to each of the four SoCs individually over PCIe Gen4 x8. Each SoC carries 16 seventh-generation AI cores and 144 MB of on-chip memory, with three network-on-chip fabrics linking everything internally at 186 GB/s, fast enough to keep the cores in sync and multicast data across them without waiting on the slower path out to DRAM.
Inside each AI core, the work splits three ways. A tensor unit handles the matrix multiplication that transformer attention layers are built from. A vector unit handles a broad instruction set covering general AI math and image processing. A scalar unit, a 4-way VLIW design with six hardware threads, picks up the control logic and branching that the other two units aren't suited for. Splitting the work this way, instead of forcing one generic core to do everything, is what lets the chip hit high throughput at a power draw low enough to matter for a facility cluster running all day, every day.
The specification profile that makes 100-billion-parameter inference practical on 150 watts
Start with the number that decides whether this card belongs in a server room. A study run through UC San Diego and the National Research Platform found that some 70-billion-parameter models needed only one Qualcomm QAic card, drawing 148 watts, to do what took eight rival GPUs from another vendor pulling 2,983 watts combined. That's a 20-times reduction in power for the same model. For smaller models, the same study measured a single card at 36 watts against a four-GPU A100 setup at 1,246 watts, a 35-times gap. Numbers like that change what a facility can run continuously on a fixed power and cooling budget, instead of the kind of budget a purpose-built data center gets.
The card's headline throughput figure is 870 TOPs at INT8 precision, enough raw capacity to serve several concurrent inference requests from a fleet without stalling to batch them together. But throughput isn't the bottleneck that matters most for this kind of large language model. Autoregressive decoding, generating one token, feeding it back in, generating the next, lives or dies on memory bandwidth, and the Ultra carries 128 GB of LPDDR4x memory at 548 GB/s. That's the number that keeps token generation fast enough that a robot waiting on a coordination message doesn't notice the wait. On top of that, 576 MB of on-die SRAM acts as a scratch-pad holding the most frequently used weights and fragments of the key-value cache, so the chip skips the trip to slower DRAM on every step of generation.
All of that runs inside a 150-watt thermal envelope, a figure the independent study confirmed in practice at 148 watts for 70-billion-parameter models. A single card handles a 100-billion-parameter model. Pairing two cards roughly doubles the parameter count a single card manages, and Qualcomm's AI Stack and Cloud AI SDK support spreading a model across more cards beyond that. Running a rack of these continuously against a comparable GPU cluster makes the power gap stop being a rounding error: a cooling system sized for the Qualcomm cluster stays adequate, where one sized for the GPU cluster falls short.
How the Software Stack Bridges the Card to a Running Fleet Inference Service
Hardware specs mean little if the software layer forces a custom integration project every time a fleet operator wants to swap models or add a card. Qualcomm's answer is the AI Inference Suite, a set of software and services covering both on-premises and cloud deployments, built around OpenAI-compatible APIs alongside user management, chat, image generation, multi-modal support, and retrieval-augmented generation. The API compatibility is the detail that matters day to day: fleet orchestration software already written to talk to an API endpoint in that common style can point at this card's backend with no rewrite.
The Cloud AI SDK sits beneath that layer, built around a compile-once, run-everywhere model. A model gets compiled once into a deployable package, and after that it loads on any card running the same SDK version. The SDK handles the ONNX export, the compilation step, and the deployment itself, so scaling from a small pilot cluster to a full production fleet doesn't mean recompiling every model from scratch.
Kubernetes integration is documented directly by Qualcomm rather than left to third parties. AI applications get containerized with Docker and deployed through Kubernetes, and the Cloud AI 100 Kubernetes device plugin is available as part of the SDK tooling. Each physical accelerator shows up in the cluster as one allocatable resource under the type qualcomm.com/qaic. In the National Research Platform's production cluster, 32 physical devices show up as 32 separate allocatable resources, the exact granularity a fleet backend team writes its resource requests against when deciding how many pods run per node. Once a compiled model clears whatever latency or accuracy bar the use case demands, it gets served through an inference server such as Triton, with Kubernetes handling the orchestration around it.
Qualcomm's public SDK repository has recently added a Kubernetes tutorial, an Efficient Transformers tutorial, and worked examples for DETR ResNet-50 and YOLOv8. Those two vision models are the exact model families a fleet operator uses for camera-based perception, which makes the SDK directly relevant to anyone building the vision side of a facility inference service, not only t... They're the exact model families a fleet operator uses for camera-based perception, which makes the SDK directly relevant to anyone building the vision side of a facility inference service, not only the language side.
The Dragonwing AI On-Prem Appliance as the packaged facility-layer deployment unit
An operator who doesn't want to build a server from a parts list can buy the Dragonwing AI On-Prem Appliance instead: a compact host computer with Cloud AI 100 accelerator cards already installed, sold through approved hardware partners Aetina, Advantech, and Lanner.
Each appliance holds up to two accelerator cards, and the option list includes Cloud AI 100 Ultra, Cloud AI 80 Ultra, or Cloud AI 100 Pro. Configured with two Ultra cards, a single appliance delivers substantially more throughput than a single card's 870 TOPS. The box ships running Linux with the Cloud AI SDK and the AI Inference Suite SDK already installed, so there's no separate software procurement step once the hardware arrives. The Cloud AI SDK handles deploying and optimizing the models, and the AI Inference Suite APIs handle controlling and customizing how inference runs.
Scaling beyond one box doesn't mean redesigning the deployment. Qualcomm's architecture supports running multiple clusters of these appliances side by side for both classic and generative AI workloads on-premises, so a fleet operator adds appliances as the robot count grows instead of re-architecting the inference service every time headcount doubles.
The AI 100 Ultra's Place in Qualcomm's Broader Robotics Hardware Strategy
At CES 2026, Qualcomm laid out a next-generation robotics architecture meant to tie hardware, software, and AI inference capabilities together into one stack, and introduced the Dragonwing IQ10 Series alongside it: a robotics processor aimed at industrial autonomous mobile robots and advanced humanoids.
The IQ10 sits on the robot itself. The AI 100 Ultra, on a bare card or packaged into the On-Prem Appliance, sits at the facility layer, one step back from the robot. The two are meant to interoperate through the same Qualcomm AI Stack: both the facility-layer backend and the robot's own inference hardware sit within the same Qualcomm AI Stack, enabling a more unified development workflow across the deployment.
That software continuity is the real argument for standardizing on Qualcomm across a fleet's full stack rather than mixing vendors tier by tier. There's a practical on-ramp built in too. Cloud AI 100 cards are available through AWS and Cirrascale Cloud Services, so an operator can build and test an inference pipeline in the cloud first, lean on cloud capacity while a fleet expands faster than on-prem hardware can be racked, and move the workload onto facility-layer appliances once the deployment stabilizes.
What Operators Should Evaluate Before Deploying the AI 100 Ultra at the Facility Layer
Model size comes first, and it isn't a detail to leave until the appliance is already on order. A single card handles a substantial parameter count, two cards handle noticeably more, and past that, more cards are needed. Map the actual size of the vision-language-action or world model in question to a card count before specifying an appliance configuration. Guess here and the fleet ends up over-buying compute it never uses or under-provisioning a cluster that chokes the moment a bigger model ships.
Software architecture compatibility is a real constraint. The Apps SDK runs on x86-64 only, while the Platform SDK covers both x86-64 and ARM64. Check which architecture the facility server actually uses before locking in an appliance vendor among Aetina, Advantech, and Lanner. A mismatch here doesn't get fixed after the hardware arrives; it gets fixed by buying the hardware again.
Kubernetes resource planning follows directly from the hardware layout. Each physical card exposes as one qualcomm.com/qaic resource, so a two-card appliance gives the cluster exactly two allocatable resources to schedule against. Fleet backend architects need to plan pod resource requests and replica counts around that specific number, not around some assumed abstraction of accelerator-equivalent capacity.
Benchmark data deserves a skeptical read. The MLPerf v4.0 results for this hardware sit in Closed Preview mode, which limits how directly they compare against other public results. Operators evaluating real-world throughput are better served requesting current benchmark data straight from Qualcomm, or leaning on the UC San Diego and National Research Platform study, which tested twelve open-source LLMs ranging from 124 million to 70 billion parameters using the vLLM framework. That study is independent, public, and specific enough to actually build a deployment decision around, which is more than can be said for a vendor's own preview numbers.
