Start with the work the buyer must complete
An enterprise does not purchase a useful inference service by selecting the largest accelerator specification. It needs a defined application to produce acceptable answers under a realistic demand pattern. Begin with the model task, exact checkpoint, permitted data locations, input modalities, expected prompt and output lengths, and quality criteria. Then record when a response becomes useful: the first answer token, a completed structured result, or a full document. These are different acceptance events.
NVIDIA's benchmarking reference distinguishes client-observed first-token time, completed-response latency and total output throughput. Aguasto's proposed selection method uses those distinctions to frame a workload brief before hardware evaluation. State offered demand and latency objectives together; an unloaded demonstration cannot establish sustained capacity. Include burst arrivals, representative long requests and failure treatment. The result is a testable requirement, rather than an unsupported promise that one hardware generation is best for every enterprise.
Metrics — NVIDIA NIM Benchmarking LLM 2.0.0 (external source)Qualify the complete software and hardware combination
A model-family name and a GPU-family name are insufficient configuration identities. vLLM documents supported model architectures and implementation paths; TensorRT-LLM lists supported NVIDIA hardware separately. AMD's ROCm matrix describes coordinated firmware, driver and user-space requirements. These references answer different questions. A listed accelerator does not establish that the intended checkpoint, quantized artifact, attention mode and serving features all work together on that accelerator.
Create one candidate record per actual combination. Include checkpoint and tokenizer revisions, license/access conditions, accelerator variant, server configuration, runtime release, container digest, driver, kernel and selected features. Mark each compatibility assertion as documented, tested or unresolved. Preserve a rejected path and its reason instead of quietly changing the model until an example starts. Procurement should receive the qualified configuration identity, not a generic request for a number of GPUs.
Supported Models — vLLM official documentation (external source) · Supported Hardware — NVIDIA TensorRT-LLM (external source) · ROCm 10.1.0 compatibility matrix — AMD (external source)Treat memory fit as a working envelope
Fitting weights is a prerequisite, not the capacity result. vLLM's optimization guide explains that insufficient KV-cache space can cause preemption and recomputation, affecting latency. Runtime state therefore belongs in the selection alongside weights, execution buffers and the intended workload. The existing weight-memory and KV-cache guides explain the component arithmetic; the buyer decision is whether a configured system retains enough usable state under accepted demand.
Request evidence at short, representative and permitted maximum contexts, with the intended concurrency and output-length mix. Record allocation, observed pressure and whether requests wait, fail or recompute. Distinguish an engine estimate of cache capacity from an executed service test. Do not divide aggregate rack memory by model size and call the quotient customer capacity: placement, parallelism, active state and supported execution limits still matter. Leave unverified capacity as unknown until the actual configuration is measured.
Optimization and Tuning — vLLM V1 documentation (external source)Choose the replica boundary before the cluster size
vLLM's current scaling guide distinguishes a single GPU, a multi-GPU node and a multi-node model replica. Sharding one replica and adding independent replicas solve different problems. A larger sharded replica may be needed for a supported memory or execution path; independent replicas may instead serve additional requests without synchronizing every model step. Neither strategy is established by a count of installed devices alone.
For each candidate, draw where one request's state lives and which links it crosses. Name tensor, pipeline or expert parallelism only when the selected model and engine support that configuration. Record the local interconnect, host resources and inter-node transport. Ask how a worker loss affects in-flight work and what capacity remains for admission. Compare the smallest supported baseline with justified alternatives, retaining communication overhead and operating complexity in the decision. More nodes are an option to evaluate, not automatic evidence of greater useful throughput.
Parallelism and Scaling — vLLM official documentation (external source)Match the qualified system to physical interfaces
ASHRAE separates equipment cooling conditions from the wider facility thermal design. A liquid-cooled system still needs an accepted heat path, and residual air loads remain part of the configuration. The system decision consequently includes electrical input, protection, coolant and heat-rejection interfaces, receiving access and maintenance scope. Device power ratings cannot stand in for an accepted rack and facility operating record.
Ask the manufacturer and qualified integrator for the exact supported bill of materials and interface envelopes. Establish who owns engineering, installation, factory tests, site commissioning and subsequent maintenance. Evaluate the intended site climate and operating modes, including partial load and unavailable equipment. A compelling accelerator reference does not create utility-authorized service or permit an unqualified cooling arrangement. The buyer worksheet should identify these dependencies before a hardware commitment, with unknown interfaces shown as unresolved rather than inferred from marketing material.
Data Centers and Telecommunication Facilities — ASHRAE 2023 Handbook, chapter 20 (external source)Separate diagnostics from workload acceptance
NVIDIA DCGM describes diagnostics for deployment, software, topology, memory and power/thermal issues, while explicitly excluding comprehensive hardware diagnosis, repair and RMA authorization. A diagnostic pass and a model-serving acceptance result are therefore different records. Preserve both without presenting one as a substitute for the other. Supplier-specific tools also need their own supported versions and interpretation.
A proposed acceptance sequence first establishes configuration and basic system health, then runs the named workload under controlled conditions. Measure request-level quality, first-token and completion latency, errors, output throughput and resource pressure over the same interval. Preserve warmup treatment, cache state and offered load. Investigate degraded links or throttling before attributing a result to model architecture. Publish only observed conclusions with raw records; this article describes how to require evidence and contains no executed Aguasto GPU benchmark.
Metrics — NVIDIA NIM Benchmarking LLM 2.0.0 (external source) · DCGM Diagnostics — NVIDIA DCGM documentation (external source)Make a decision that procurement and operations can use
The output should be a dated configuration decision, its acceptance conditions and the reasons alternatives were retained or rejected. Attach a responsibility matrix covering supplied hardware, licensed weights, runtime maintenance, network operations, incident handling, replacement and change control. Separate supplier availability and contractual commitments from documented technical support. A launch announcement or supported-device list does not establish inventory allocated to this buyer.
Connect the accepted workload envelope to the dedicated/shared economics method using verified commercial inputs. Keep reserve capacity, operating effort and failed work visible without inventing a price or universal break-even point. Specify what would trigger reevaluation: a changed checkpoint, feature, driver, demand distribution or site interface. Aguasto is developing its compute platform; the proposed method does not represent an installed system, a reservation, a tested deployment or an available service. It gives enterprise buyers and prospective OEMs a concrete basis for a qualified discussion.
Supported Hardware — NVIDIA TensorRT-LLM (external source) · ROCm 10.1.0 compatibility matrix — AMD (external source)Sources & verification
The register preserves each source's publication and verification date. Manufacturer claims and documentation are attributed; Aguasto has not run the cited products in production.
- Metrics — NVIDIA NIM Benchmarking LLM 2.0.0 ↗
Client-observed TTFT, end-to-end latency and output-throughput boundaries; Tools require aligned definitions; empty and one-token outputs need correct treatment. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Supported Models — vLLM official documentation ↗
Architecture and feature support must be checked for the intended model path; Native and Transformers model implementations are distinct documented paths. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Supported Hardware — NVIDIA TensorRT-LLM ↗
Documented NVIDIA hardware families; Hardware support is distinct from a model/precision qualification. Last modified: 2026-10-09. Verified: 2026-10-13.
- ROCm 10.1.0 compatibility matrix — AMD ↗
Firmware, driver and user-space version alignment; Configuration-specific operating-system and GPU requirements. Publication: 2026-08-26. Verified: 2026-10-13.
- Optimization and Tuning — vLLM V1 documentation ↗
KV-cache pressure and preemption/recomputation; Chunked-prefill token budget and decode-priority policy. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Parallelism and Scaling — vLLM official documentation ↗
Single-replica GPU and node boundaries; Tensor/pipeline parallelism, execution environment and communication dependencies; Documented internal-network isolation requirement. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- Data Centers and Telecommunication Facilities — ASHRAE 2023 Handbook, chapter 20 ↗
Air/liquid cooling and facility heat-rejection interfaces; Thermal envelopes and site-specific operating conditions. Publication: date not stated in the reviewed source. Verified: 2026-10-13.
- DCGM Diagnostics — NVIDIA DCGM documentation ↗
Deployment, software, topology, power/thermal and memory diagnostic scope; Diagnostics are not comprehensive hardware diagnosis, repair or RMA authorization. Publication: date not stated in the reviewed source. Verified: 2026-10-13.