One rack can contain several different networks
A rack-scale AI system moves different kinds of information for different reasons. GPUs may exchange model-operation data within a configured scale-up domain. Nodes may coordinate one distributed replica through a scale-out fabric. Storage and host services move weights, input data and operating records. An application router selects independent replicas. A useful drawing labels those paths separately, even when a physical design shares some infrastructure.
NVIDIA describes NVLink and NVLink Switch as a scale-up GPU-to-GPU fabric. That manufacturer scope is not the public API network or the facility's complete communications architecture. Its current product page also marks specifications preliminary. Aguasto's proposed topology brief consequently names the generation, qualified system and communication domain without turning rack aggregate bandwidth into per-request throughput. First identify what crosses each boundary; only then ask which topology and transport support the workload.
NVIDIA NVLink and NVLink Switch — official product reference (external source)Model parallelism creates communication obligations
A collective operation coordinates participating ranks rather than simply forwarding an application request. NVIDIA's NCCL reference defines AllReduce as reducing data and returning the result to every rank; AllGather assembles contributions in rank order. The runtime's rank-to-device mapping is therefore part of the execution record. A collection of reachable GPUs is not enough to establish correct or efficient distributed model execution.
Record the parallel strategy and the operations the selected runtime actually performs. Separate tensor, pipeline and expert placement from independent data replicas. vLLM's guide describes these single-replica GPU/node boundaries and requires consistent execution environments across nodes. Preserve where ranks are placed, which paths connect them and which operations cross hosts. A topology selected for a densely communicating replica can differ from one designed for many independent requests. The requirement should come from the model/runtime combination, rather than an interchangeable label such as 'AI network'.
Collective Operations — NVIDIA NCCL documentation (external source) · Parallelism and Scaling — vLLM official documentation (external source)Topology and placement matter inside the system
Transport depends on the actual device relationships. NCCL's GPU troubleshooting guidance describes peer access over supported NVLink or PCIe paths and warns that drivers, virtualization, containers and system configuration can affect it. AMD's RCCL documentation separately gives a fully connected eight-MI300X example with dedicated Infinity Fabric links. These are configuration-specific references; neither establishes the topology of every server or rack carrying the same vendor name.
Ask for the system topology and verify the subset assigned to one workload. Preserve GPU-to-GPU and GPU-to-NIC locality, host affinity and the selected partition mode. A partial allocation can have a different communication pattern from the full system, so aggregate manufacturer figures should not be apportioned as a guaranteed service rate. Keep support and measured behavior distinct. The infrastructure diagram should describe accepted interfaces and rank placement, while leaving exact cable routes or OEM geometry to the qualified system documentation.
GPU troubleshooting — NVIDIA NCCL documentation (external source) · RCCL usage tips — AMD RCCL official documentation (external source)A scale-out link needs an accepted end-to-end path
NVIDIA's GPU Operator documentation distinguishes GPUDirect RDMA, which connects GPU memory with a peer device, from GPUDirect Storage, which addresses storage-to-GPU movement. Kernel, driver and implementation prerequisites still apply. Purchasing a fast network adapter does not establish that a containerized inference worker uses the intended GPU-memory path, nor that the storage path has identical requirements.
Treat the path as a chain of accountable components: GPU and host topology, adapter, link, switch configuration, fabric management and runtime transport. Record which interfaces the collective library selects and which permissions or resources the supported environment needs. Keep a private execution network distinct from public ingress; the vLLM guide explicitly calls for isolating its documented internal cluster traffic. This guide is a qualification framework, not an instruction to copy privileges, firewall rules or deployment examples into an operating cluster.
Parallelism and Scaling — vLLM official documentation (external source) · GPUDirect RDMA and GPUDirect Storage — NVIDIA GPU Operator documentation (external source)KV transfer is another boundary, not a free arrow
Prefill/decode disaggregation adds movement of retained request state between worker pools. Dynamo's architecture describes prefill, KV transfer and decode as separate stages, with NIXL and backend-specific metadata coordinating transfer. SGLang's current guide separately documents allocation and handshake restrictions. A model-serving protocol and its cache layout therefore belong in the network evaluation alongside the physical link.
For a proposed study, label where KV is produced, reserved, transferred and released. Record source/destination versions, state compatibility, admission and cancellation behavior. Measure the actual transferred work and waiting time instead of treating a drawn arrow as zero cost. Maintain an aggregated baseline and inspect whether the split improves the required latency boundary without creating another bottleneck. The existing disaggregation and batching guides explain phase policy; this networking guide identifies the transport, placement and ownership evidence those policies require.
Disaggregated Serving — NVIDIA Dynamo architecture documentation (external source) · PD Disaggregation — SGLang official documentation (external source)Validate the fabric in layers, then test the application
NCCL's networking guidance separates interface selection, link state/rate, bandwidth checks, latency checks and protocol diagnostics. InfiniBand and RoCE have different management and diagnostic conditions. A healthy port or favorable point-to-point test is useful evidence at that boundary, but it does not establish collective behavior for every message shape or a client-observed inference result.
A proposed acceptance record progresses from topology and link health to the relevant transport and collective operations, then the named model workload. Preserve device mappings, message shapes, concurrent traffic, errors and latency distributions. Repeat under the actual allocation and stated failure conditions. Correlate request delays with fabric observations rather than attributing every slow token to the GPU. Keep timestamps and raw records so a reviewer can distinguish congestion, a degraded path, cache admission and model execution. No such test was executed by Aguasto for this article.
Networking Troubleshooting — NVIDIA NCCL documentation (external source)Keep distributed compute separate from geographic routing
Independent facilities can host eligible model replicas, with application routing selecting a permitted destination. That does not make their GPUs one geographically synchronized low-latency model fabric. Define which state remains local to a replica and which shared services the route requires. Adding a region also creates questions about compatible versions, data location, spare capacity and shared dependencies; it does not answer them automatically.
For procurement and operating handover, assign ownership for adapters, switches, fabric configuration, runtime transport, monitoring, incident recovery and topology changes. Preserve manufacturer support boundaries and software provenance for each accepted system. Compare useful completed workloads under a defined service objective, not unlike bandwidth totals. Aguasto's proposed distributed compute thesis is an operating intent; this research establishes no installed fabric, qualified OEM configuration, inter-region performance, failover result or service-level commitment. A credible network decision ends with reviewable evidence and named responsibilities.
Parallelism and Scaling — vLLM official documentation (external source) · Disaggregated Serving — NVIDIA Dynamo architecture documentation (external source)Sources & verification
The register preserves each source's publication and verification date. Manufacturer claims and documentation are attributed; Aguasto has not run the cited products in production.
- NVIDIA NVLink and NVLink Switch — official product reference ↗
Manufacturer describes a scale-up GPU-to-GPU fabric; Specifications are marked preliminary; generation/system boundaries apply. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Collective Operations — NVIDIA NCCL documentation ↗
AllReduce, AllGather and ReduceScatter semantics; Rank order and device mapping matter. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Parallelism and Scaling — vLLM official documentation ↗
Single-replica GPU and node boundaries; Tensor/pipeline parallelism, execution environment and communication dependencies; Documented internal-network isolation requirement. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- GPU troubleshooting — NVIDIA NCCL documentation ↗
GPU-to-GPU and GPU-to-NIC access depend on topology and software; PCIe peer access, driver and virtual/container configuration limitations. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- RCCL usage tips — AMD RCCL official documentation ↗
Eight-MI300X example has dedicated pairwise Infinity Fabric links; Peer access, topology, partition and channel choices are configuration-specific. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- GPUDirect RDMA and GPUDirect Storage — NVIDIA GPU Operator documentation ↗
RDMA GPU-to-peer-device path differs from direct storage path; Kernel/driver and DMA-BUF or nvidia-peermem prerequisites. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Disaggregated Serving — NVIDIA Dynamo architecture documentation ↗
Prefill, KV transfer and decode are distinct phases; NIXL transfer and backend-specific metadata; Pool allocation and communication costs remain workload-dependent. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- PD Disaggregation — SGLang official documentation ↗
Cache reservation, transfer handshake and backend/layout restrictions; Policy support must be checked per communicating-worker revision. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Networking Troubleshooting — NVIDIA NCCL documentation ↗
Interface selection and low-level fabric checks; Link state/rate, bandwidth and latency diagnose different failures; InfiniBand and RoCE diagnostics are distinct. Publication: date not stated in the reviewed source. Verified: 2026-10-13.