Aguasto Research · Revision 1Editorial record: Review and correction policy ↗
What system boundary does this workload actually need?
A published accelerator specification describes a device under stated conditions. A rack reference combines devices, local interconnects and infrastructure interfaces. Neither establishes that a particular checkpoint fits, that a serving stack supports every optimization, or that a customer can purchase commissioned capacity. Start with a workload and preserve those distinctions through the shortlist.
Write down the exact checkpoint or application, quality floor, precision, sequence lengths, concurrency and intended operating period. Then separate memory fit from measured serving performance. A seemingly attractive peak figure is not a latency result. An unfamiliar field should remain unknown until the cited manufacturer or the proposed system integrator resolves its actual scope.
Follow the evidence.
Separate the device from the system
Compare accelerators with accelerators and rack platforms with rack platforms. Check whether memory and bandwidth values describe one device or an aggregate. Cooling type, server power and local fabric topology belong to the same proposed configuration; they cannot safely be assembled from unrelated product headlines.
GPU Library ↗
Exact device and rack references with units and provenance.Hardware comparison ↗
Category-safe comparison without an invented performance ranking.Match software and memory assumptions
The checkpoint identifier, license, runtime version and selected precision constrain a usable system. Weight memory is only one part of allocation. Cache growth, runtime overhead and workspace can change the number of devices required; actual concurrency and latency targets still need measurement.
Weights and runtime memory ↗
Understand the difference between a lower-bound estimate and a deployment.Inference Stack ↗
Inspect backend-specific operating assumptions.Define evidence before a commercial commitment
A useful request names the workload and the evidence that would establish acceptance. Keep configuration, operating period, tenancy, support, data location and service responsibilities explicit. A capacity-interest inquiry starts qualification; it does not reserve equipment or demonstrate that Aguasto has an operating cluster.
Selecting an enterprise GPU system ↗
Turn the shortlist into an acceptance record.Networking boundaries ↗
Separate local scale-up, cluster fabric and independent replicas.
Cornerstone resources
Selecting GPU Systems for Enterprise Inference
Turn a workload brief into a qualified system and acceptance record, separating checkpoint support, memory, topology, facility interfaces and operating evidence from accelerator headline specifications.
Networking Rack-Scale AI Systems: Define the Communication Boundary
Explain GPU scale-up, inter-node collectives, KV transfer, host/storage traffic and independent-replica routing as distinct qualification paths rather than one aggregate bandwidth number.
Model weights are only the beginning of the memory budget
Help engineers separate a transparent weight-storage estimate from the workload-dependent memory required to serve a model.
Keep these boundaries close.
Sources & verification
The register preserves each source's publication and verification date. Manufacturer claims and documentation are attributed; Aguasto has not run the cited products in production.
- Metrics — NVIDIA NIM Benchmarking LLM 2.0.0 ↗
Client-observed TTFT, end-to-end latency and output-throughput boundaries; Tools require aligned definitions; empty and one-token outputs need correct treatment. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Supported Models — vLLM official documentation ↗
Architecture and feature support must be checked for the intended model path; Native and Transformers model implementations are distinct documented paths. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Supported Hardware — NVIDIA TensorRT-LLM ↗
Documented NVIDIA hardware families; Hardware support is distinct from a model/precision qualification. Last modified: 2026-10-09. Verified: 2026-10-13.
- ROCm 10.1.0 compatibility matrix — AMD ↗
Firmware, driver and user-space version alignment; Configuration-specific operating-system and GPU requirements. Publication: 2026-08-26. Verified: 2026-10-13.
- Optimization and Tuning — vLLM V1 documentation ↗
KV-cache pressure and preemption/recomputation; Chunked-prefill token budget and decode-priority policy. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Parallelism and Scaling — vLLM official documentation ↗
Single-replica GPU and node boundaries; Tensor/pipeline parallelism, execution environment and communication dependencies; Documented internal-network isolation requirement. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Data Centers and Telecommunication Facilities — ASHRAE 2023 Handbook, chapter 20 ↗
Air/liquid cooling and facility heat-rejection interfaces; Thermal envelopes and site-specific operating conditions. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- DCGM Diagnostics — NVIDIA DCGM documentation ↗
Deployment, software, topology, power/thermal and memory diagnostic scope; Diagnostics are not comprehensive hardware diagnosis, repair or RMA authorization. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- NVIDIA NVLink and NVLink Switch — official product reference ↗
Manufacturer describes a scale-up GPU-to-GPU fabric; Specifications are marked preliminary; generation/system boundaries apply. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Collective Operations — NVIDIA NCCL documentation ↗
AllReduce, AllGather and ReduceScatter semantics; Rank order and device mapping matter. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- GPU troubleshooting — NVIDIA NCCL documentation ↗
GPU-to-GPU and GPU-to-NIC access depend on topology and software; PCIe peer access, driver and virtual/container configuration limitations. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- RCCL usage tips — AMD RCCL official documentation ↗
Eight-MI300X example has dedicated pairwise Infinity Fabric links; Peer access, topology, partition and channel choices are configuration-specific. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- GPUDirect RDMA and GPUDirect Storage — NVIDIA GPU Operator documentation ↗
RDMA GPU-to-peer-device path differs from direct storage path; Kernel/driver and DMA-BUF or nvidia-peermem prerequisites. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Disaggregated Serving — NVIDIA Dynamo architecture documentation ↗
Prefill, KV transfer and decode are distinct phases; NIXL transfer and backend-specific metadata; Pool allocation and communication costs remain workload-dependent. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- PD Disaggregation — SGLang official documentation ↗
Cache reservation, transfer handshake and backend/layout restrictions; Policy support must be checked per communicating-worker revision. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Networking Troubleshooting — NVIDIA NCCL documentation ↗
Interface selection and low-level fabric checks; Link state/rate, bandwidth and latency diagnose different failures; InfiniBand and RoCE diagnostics are distinct. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Optimizing LLMs for Speed and Memory — Hugging Face ↗
Parameter storage at float32 and BF16/FP16; Weight storage differs from long-context inference memory. Publication: date not stated in the reviewed source. Verified: 2026-10-12.
- Cache strategies — Hugging Face Transformers ↗
KV cache grows during generation; Static cache allocation, offloading and cache quantization trade-offs; Model-specific cache limitations. Publication: date not stated in the reviewed source. Verified: 2026-10-12.
- Bitsandbytes — Hugging Face Transformers ↗
Quantized linear modules and mixed compute/storage precision; Nonquantized modules and offloading constraints. Publication: date not stated in the reviewed source. Verified: 2026-10-12.
- CUDA semantics: Memory management — PyTorch 2.14 ↗
Allocated tensor memory versus allocator-reserved memory; Graph-private memory pools and caching allocator. Publication: date not stated in the reviewed source. Verified: 2026-10-12.