Aguasto Research · Revision 1Editorial record: Review and correction policy ↗
Accelerator
A compute device designed to execute supported workloads alongside a host system. In this library, an individual accelerator is a different comparison category from a complete rack platform.
Practical boundary. Read the exact variant, memory, power and software boundary. A device specification is not an available cluster or measured model service.
Read the sourced explanation ↗Metrics — NVIDIA NIM Benchmarking LLM 2.0.0 (external source) · Supported Models — vLLM official documentation (external source) · Supported Hardware — NVIDIA TensorRT-LLM (external source) · ROCm 10.1.0 compatibility matrix — AMD (external source) · Optimization and Tuning — vLLM V1 documentation (external source) · Parallelism and Scaling — vLLM official documentation (external source) · Data Centers and Telecommunication Facilities — ASHRAE 2023 Handbook, chapter 20 (external source) · DCGM Diagnostics — NVIDIA DCGM documentation (external source)Rack-scale system
A platform that combines multiple compute components, local interconnects and physical interfaces within a manufacturer-defined system boundary. Aggregated rack memory does not create one accelerator with a single undivided memory space.
Practical boundary. Preserve aggregate and per-device units. OEM implementation and accepted facility interfaces still matter.
Read the sourced explanation ↗NVIDIA NVLink and NVLink Switch — official product reference (external source) · Collective Operations — NVIDIA NCCL documentation (external source) · Parallelism and Scaling — vLLM official documentation (external source) · GPU troubleshooting — NVIDIA NCCL documentation (external source) · RCCL usage tips — AMD RCCL official documentation (external source) · GPUDirect RDMA and GPUDirect Storage — NVIDIA GPU Operator documentation (external source) · Disaggregated Serving — NVIDIA Dynamo architecture documentation (external source) · PD Disaggregation — SGLang official documentation (external source) · Networking Troubleshooting — NVIDIA NCCL documentation (external source)Scale-up fabric
In these guides, the local high-bandwidth interconnect within a tightly coupled GPU domain. It is evaluated separately from networking between servers and from application routing to independent replicas.
Practical boundary. A vendor bandwidth figure is not measured model throughput, and cannot be transferred to an unrelated topology.
Read the sourced explanation ↗NVIDIA NVLink and NVLink Switch — official product reference (external source) · Collective Operations — NVIDIA NCCL documentation (external source) · Parallelism and Scaling — vLLM official documentation (external source) · GPU troubleshooting — NVIDIA NCCL documentation (external source) · RCCL usage tips — AMD RCCL official documentation (external source) · GPUDirect RDMA and GPUDirect Storage — NVIDIA GPU Operator documentation (external source) · Disaggregated Serving — NVIDIA Dynamo architecture documentation (external source) · PD Disaggregation — SGLang official documentation (external source) · Networking Troubleshooting — NVIDIA NCCL documentation (external source)Scale-out fabric
Networking that joins compute nodes for a defined distributed workload. Its topology, software communication pattern and accepted latency or bandwidth requirements belong to the proposed cluster.
Practical boundary. A cross-region request-routing layer is a different boundary from a synchronized training or inference collective.
Read the sourced explanation ↗NVIDIA NVLink and NVLink Switch — official product reference (external source) · Collective Operations — NVIDIA NCCL documentation (external source) · Parallelism and Scaling — vLLM official documentation (external source) · GPU troubleshooting — NVIDIA NCCL documentation (external source) · RCCL usage tips — AMD RCCL official documentation (external source) · GPUDirect RDMA and GPUDirect Storage — NVIDIA GPU Operator documentation (external source) · Disaggregated Serving — NVIDIA Dynamo architecture documentation (external source) · PD Disaggregation — SGLang official documentation (external source) · Networking Troubleshooting — NVIDIA NCCL documentation (external source)Model checkpoint
A specific published set of model weights and associated configuration. Its exact publisher identifier, revision, model card and license are inputs to a reproducible deployment evaluation.
Practical boundary. An open-weight reference does not automatically grant unrestricted rights, demonstrate task quality or establish runtime support.
Read the sourced explanation ↗Optimizing LLMs for Speed and Memory — Hugging Face (external source) · Cache strategies — Hugging Face Transformers (external source) · Bitsandbytes — Hugging Face Transformers (external source) · CUDA semantics: Memory management — PyTorch 2.14 (external source)Theoretical weight memory
A lower-bound payload calculation using parameter count and assumed bits per weight. The tools report decimal gigabytes and binary gibibytes separately.
Practical boundary. This estimate excludes cache, activations, workspace, metadata and runtime overhead. It is not a guarantee that a model fits on a device.
Read the sourced explanation ↗Optimizing LLMs for Speed and Memory — Hugging Face (external source) · Cache strategies — Hugging Face Transformers (external source) · Bitsandbytes — Hugging Face Transformers (external source) · CUDA semantics: Memory management — PyTorch 2.14 (external source)KV cache
Stored attention keys and values used during autoregressive generation. Its memory requirements and possible reuse depend on model architecture, request lengths, precision and serving policy.
Practical boundary. Prefix reuse needs exact compatible state and an approved isolation boundary. Approximate text similarity is not a sufficient cache identity.
Read the sourced explanation ↗How caching works — Hugging Face Transformers (external source) · KV Cache System — NVIDIA TensorRT-LLM (external source) · Efficient Memory Management for Large Language Model Serving with PagedAttention (external source) · Paged Attention, IFB, and Request Scheduling — NVIDIA TensorRT-LLM (external source) · Automatic Prefix Caching — vLLM (external source) · Security — vLLM 0.31.0 (external source) · Optimization and Tuning — vLLM (external source)Prefill
The model-serving work that processes input tokens and establishes the state needed to begin generation. It is one phase of a request rather than a complete end-to-end latency measurement.
Practical boundary. Prompt lengths, batching and cache conditions must accompany any reported prefill result.
Read the sourced explanation ↗Disaggregated Serving — NVIDIA Dynamo architecture (external source) · Disaggregated Serving — NVIDIA Dynamo development guide (external source) · Disaggregated Prefilling (experimental) — vLLM (external source) · Automatic Prefix Caching — vLLM (external source)Decode
The generation phase in which a serving system produces subsequent output tokens. Scheduling can interleave work from requests with different lengths and states.
Practical boundary. Decode throughput alone does not show time to first token, end-to-end latency, failures or output quality.
Read the sourced explanation ↗Orca: A Distributed Serving System for Transformer-Based Generative Models — USENIX (external source) · Continuous batching — Hugging Face Transformers 5.11.0 (external source) · Paged Attention, IFB, and Request Scheduling — NVIDIA TensorRT-LLM (external source) · Optimization and Tuning — vLLM (external source) · Metrics — NVIDIA NIM Benchmarking (external source)Time to first token
Elapsed time to the first output token at a declared measurement boundary. A test should state what starts the clock and whether queueing, network transport, prompt processing and client behavior are included.
Practical boundary. Report distributions and workload conditions. A single best-case response is not a service guarantee.
Read the sourced explanation ↗Metrics — NVIDIA NIM Benchmarking 2.0.0 (external source) · Parameters and Best Practices — NVIDIA NIM Benchmarking (external source) · Revisiting Service Level Objectives and System Level Metrics in Large Language Model Serving (external source) · MLPerf Inference: Datacenter — MLCommons (external source) · MLPerf Results Messaging Guidelines — MLCommons (external source)Inference replica
A separately running instance of an accepted model-serving configuration that may receive eligible requests. Replica routing needs health, version, capacity and data-policy information.
Practical boundary. Multiple replicas or sites do not by themselves prove continuity or eliminate shared failure domains.
Read the sourced explanation ↗Add End-to-End Fault Tolerance — Ray Serve (external source) · Disruptions — Kubernetes official documentation (external source) · Inside Fireworks Multi-region Deployments: One Deployment, Global Scale (external source)Modular facility
A facility approach divided into bounded, repeatable assemblies or units. Prefabrication describes selected assembly or integration performed away from the final site; a modular facility may combine factory and site work.
Practical boundary. A container is not a permit exemption, a completed campus or a universal deployment timetable. Integration, approvals, electricity and operating requirements remain project-specific.
Read the sourced explanation ↗Types of Prefabricated Modular Data Centers, revision2 — Schneider Electric (external source) · Prefabricated modular data centers: terminology — Schneider Electric (external source) · DGX SuperPOD Architecture for DGXGB200 — NVIDIA (external source) · Cooling Environments — Open Compute Project (external source) · Liquid Cooling Integration and Logistics White Paper, revision1.0 — Open Compute Project (external source) · Prefabricated Data Center brochure — Vertiv (external source) · Network edge enabled by prefabricated modular data centers — Schneider Electric (external source) · Benefits and Drawbacks of Prefabricated Modules, revision4 — Schneider Electric (external source)Integrated commissioning
A project-specific process for evaluating installed systems together against agreed requirements. Factory checks, site integration and compute-workload acceptance are related but distinct evidence boundaries.
Practical boundary. Neither installed equipment nor a factory acceptance record alone establishes an available commercial compute service.
Read the sourced explanation ↗Commissioning your data center for greater availability and safety — Vertiv (external source) · Prefabricated Data Center brochure — Vertiv (external source) · Liquid Cooling Integration and Logistics White Paper, revision1.0 — Open Compute Project (external source) · Network edge enabled by prefabricated modular data centers — Schneider Electric (external source) · Startup and Site Acceptance Test — Vertiv (external source)Coolant distribution unit
An interface used in liquid-cooling architectures to manage the technology cooling loop and its heat exchange with the facility side. Exact fluid, control and maintenance requirements depend on the selected system.
Practical boundary. Do not assume the facility and technology fluids mix or that liquid cooling removes all room-air heat.
Read the sourced explanation ↗Liquid Cooling Integration and Logistics White Paper, revision1.0 — Open Compute Project (external source) · DGX SuperPOD Architecture for DGXGB200 — NVIDIA (external source) · Cooling Environments — Open Compute Project (external source)Power usage effectiveness (PUE)
A ratio of total data-center energy to IT-equipment energy under the stated reporting boundary and period. It describes facility overhead relative to IT energy.
Practical boundary. PUE is not an application-efficiency, carbon, water or model-quality result. Comparisons need compatible measurement conditions.
Read the sourced explanation ↗Best Practices Guide for Energy-Efficient Data Center Design — DOE FEMP, July 2024 (external source) · Quantifying the impact on PUE and energy consumption when introducing liquid cooling — Vertiv/NVIDIA study summary (external source) · Recommendations for Measuring and Reporting Overall Data Center Efficiency, version 2 — DOE-hosted industry task force (external source) · HPC Data Center Power Usage Effectiveness — National Laboratory of the Rockies (external source) · Cooling Water Efficiency Opportunities for Federal Data Centers — DOE FEMP (external source) · Device Queries: Total Energy Consumption and Power Usage — NVIDIA NVML API Reference (external source) · Data Center Metering and Resource Guide — DOE Better Buildings (external source) · United States Data Center Energy Usage Report: 2025 Update — LBNL, published June 2026 (external source)Water consumption and withdrawal
Withdrawal concerns water taken from a source; consumption concerns the portion not returned to the relevant water system under the declared accounting method. Location, period and system boundary change the interpretation.
Practical boundary. Report local and electricity-related effects separately. A dry-cooling label is not proof of zero total water impact.
Read the sourced explanation ↗2024 United States Data Center Energy Usage Report — LBNL (external source) · Best Management Practice #14: Alternative Water Sources — DOE FEMP (external source) · Alternative Water Projects: Implementation Steps — DOE FEMP (external source) · Cooling Water Efficiency Opportunities for Federal Data Centers — DOE FEMP (external source) · Facility Water Cycle and Emissions Crosswalk — DOE FEMP (external source) · Best Management Practice #10: Cooling Tower Management — DOE FEMP (external source)Quality-qualified output
Aguasto’s analytical term for completed output that meets a declared application-quality and service test. It makes the acceptance criterion explicit before calculating useful throughput or cost.
Practical boundary. This is an evaluation boundary, not an Aguasto benchmark result or universal standard unit.
Read the sourced explanation ↗Metrics — NVIDIA NIM Benchmarking 2.0.0 (external source) · Parameters and Best Practices — NVIDIA NIM Benchmarking (external source) · Revisiting Service Level Objectives and System Level Metrics in Large Language Model Serving (external source) · MLPerf Inference: Datacenter — MLCommons (external source) · MLPerf Results Messaging Guidelines — MLCommons (external source)Ready for service (RFS)
An operator-reported milestone whose scope depends on the particular disclosure and acceptance arrangement. Research records the named system, date, unit and attributed status.
Practical boundary. Do not silently equate a reported RFS phase with an entire announced campus, customer billing or independent site verification.
Read the sourced explanation ↗Applied Digital completes Phase II RFS at the first 100 MW Polaris Forge 1 building (external source) · Applied Digital brings an additional 75 MW online at Polaris Forge 1 (external source) · Applied Digital fiscal first quarter 2027 results (external source)Utilization
The fraction of a stated capacity used during a stated period under a declared measurement method. Device activity, allocated GPU hours and successful application output describe different denominators.
Practical boundary. An illustrative utilization assumption is not observed customer demand, billed revenue or guaranteed economics.
Read the sourced explanation ↗Dedicated Model Inference — Together AI (external source) · Dedicated vs Serverless models — Together AI Knowledge Base (external source) · Autoscaling — Baseten official documentation (external source)Water usage effectiveness (WUE)
A water-intensity ratio whose numerator, energy denominator and reporting period must be declared. Site WUE commonly reports site water usage in liters per kilowatt-hour of IT energy; a source-based boundary additionally considers water associated with energy production.
Practical boundary. Keep water volume, sources, consumption and peak withdrawal visible alongside the ratio. Changing IT energy can change WUE without reducing water volume or local impact.
Read the sourced explanation ↗Best Practices Guide for Energy-Efficient Data Center Design — DOE FEMP, July 2024 (external source) · Quantifying the impact on PUE and energy consumption when introducing liquid cooling — Vertiv/NVIDIA study summary (external source) · Recommendations for Measuring and Reporting Overall Data Center Efficiency, version 2 — DOE-hosted industry task force (external source) · HPC Data Center Power Usage Effectiveness — National Laboratory of the Rockies (external source) · Cooling Water Efficiency Opportunities for Federal Data Centers — DOE FEMP (external source) · Device Queries: Total Energy Consumption and Power Usage — NVIDIA NVML API Reference (external source) · Data Center Metering and Resource Guide — DOE Better Buildings (external source) · United States Data Center Energy Usage Report: 2025 Update — LBNL, published June 2026 (external source)Inter-token latency (ITL)
Timing between generated output tokens at a declared observation boundary. AIPerf’s mean generation-time expression subtracts TTFT from complete-request latency and divides by output-token count minus one. This average does not describe every individual gap or reveal the largest streaming stall.
Practical boundary. A one-output-token response has no subsequent-token gap. Name the client, streaming behavior and token counting; retain distributions when evaluating responsiveness.
Read the sourced explanation ↗Metrics — NVIDIA NIM Benchmarking 2.0.0 (external source) · Parameters and Best Practices — NVIDIA NIM Benchmarking (external source) · Revisiting Service Level Objectives and System Level Metrics in Large Language Model Serving (external source) · MLPerf Inference: Datacenter — MLCommons (external source) · MLPerf Results Messaging Guidelines — MLCommons (external source)Latency-qualified goodput
Completed useful work meeting an explicitly chosen acceptance objective per unit of elapsed time. Aguasto’s proposed request-level method counts successful requests meeting declared first-token and completion limits; it is a stated evaluation convention, distinct from the cited research paper’s smooth-goodput framework.
Practical boundary. Report rejected, failed, timed-out and cancelled work. The objective is a test criterion, not an available Aguasto service level or universal standard.
Read the sourced explanation ↗Metrics — NVIDIA NIM Benchmarking 2.0.0 (external source) · Parameters and Best Practices — NVIDIA NIM Benchmarking (external source) · Revisiting Service Level Objectives and System Level Metrics in Large Language Model Serving (external source) · MLPerf Inference: Datacenter — MLCommons (external source) · MLPerf Results Messaging Guidelines — MLCommons (external source)Critical IT load
An IT-equipment capacity boundary defined by the particular facility, contract or operator disclosure. It differs from secured grid capacity, a generator’s design rating, total facility input and model-serving output.
Practical boundary. Retain exact scope, status and date. A reported RFS IT phase does not independently establish customer billing, utilization or token throughput; overlapping buildout and operating scopes are not additive.
Read the sourced explanation ↗Applied Digital completes Phase II RFS at the first 100 MW Polaris Forge 1 building (external source) · Applied Digital brings an additional 75 MW online at Polaris Forge 1 (external source) · Applied Digital fiscal first quarter 2027 results (external source)Continuous batching
A serving approach in which active request membership can change at eligible generation iterations rather than waiting for a complete fixed group to finish. A newly arrived request still needs prompt processing, admissible state and a scheduler decision.
Practical boundary. Request count, token work per iteration and retained cache are separate quantities. This is not a universal throughput multiplier; workload, latency, failures and policies belong in the evaluation.
Read the sourced explanation ↗Orca: A Distributed Serving System for Transformer-Based Generative Models — USENIX (external source) · Continuous batching — Hugging Face Transformers 5.11.0 (external source) · Paged Attention, IFB, and Request Scheduling — NVIDIA TensorRT-LLM (external source) · Optimization and Tuning — vLLM (external source) · Metrics — NVIDIA NIM Benchmarking (external source)Prefabrication
Selected assembly, integration or testing carried out away from the final installation site. It describes where work occurs; modularity describes how a system is divided into units. Electrical rooms, cooling assemblies and IT enclosures can have different supply and acceptance boundaries.
Practical boundary. Factory preparation can overlap suitable site work when interfaces permit. It does not supply utility rights, local approvals, installed-system acceptance or a universal deployment duration.
Read the sourced explanation ↗Types of Prefabricated Modular Data Centers, revision2 — Schneider Electric (external source) · Prefabricated modular data centers: terminology — Schneider Electric (external source) · DGX SuperPOD Architecture for DGXGB200 — NVIDIA (external source) · Cooling Environments — Open Compute Project (external source) · Liquid Cooling Integration and Logistics White Paper, revision1.0 — Open Compute Project (external source) · Prefabricated Data Center brochure — Vertiv (external source) · Network edge enabled by prefabricated modular data centers — Schneider Electric (external source) · Benefits and Drawbacks of Prefabricated Modules, revision4 — Schneider Electric (external source)