Calculate the storage floor
For a preliminary weight-storage estimate, multiply parameter count by the storage bits per parameter and divide by eight. Expressed in decimal gigabytes, weight GB = parameters in billions × bits ÷ 8. A hypothetical 70-billion-parameter model at 16 bits therefore has a 140 GB storage floor, or about 130.4 GiB after converting bytes with GB × 10^9 ÷ 2^30. This is arithmetic, not a measured deployment. Hugging Face explains the underlying parameter-storage relationship; its older hardware examples are not current accelerator rankings.
Use the documented checkpoint parameter count and storage format. Keep GB and GiB visible rather than silently treating them as equal. A result labeled estimated weight storage is useful; a result labeled required GPU memory would overstate what this calculation establishes.
Optimizing LLMs for Speed and Memory — Hugging Face (external source)Budget for the changing cache
During autoregressive generation, attention models can retain keys and values from earlier tokens. Hugging Face documents dynamic, static, offloaded and quantized cache strategies, with different memory and latency trade-offs. Static allocation may reserve space for a maximum sequence; offloading moves cache state between devices; cache quantization is a separate choice from weight quantization. Some architectures use model-specific state rather than the same attention-cache layout.
Build the runtime budget around the intended prompt and output lengths, concurrent sequences and cache representation. Do not infer cache size from total parameters alone. A long-context service and a short-answer service using the same weights can require different allocation policies.
Cache strategies — Hugging Face Transformers (external source)Separate storage precision from execution
A four-bit checkpoint does not mean every tensor or operation uses four bits. Hugging Face's bitsandbytes documentation describes quantized linear layers, other modules retaining another dtype, and separate compute-dtype settings. Quantization metadata, scales and unconverted tensors make the ideal parameter-times-bits calculation a floor rather than an exact file or runtime size.
Record the checkpoint format, method, group or block settings when available, and the serving engine's support for that specific combination. Evaluate output quality on the actual task before changing precision for a capacity target. The memory estimator should never promise identical answers or a speed improvement from a smaller storage representation.
Bitsandbytes — Hugging Face Transformers (external source)Measure allocation at the right boundary
PyTorch distinguishes memory occupied by live tensors from memory reserved by its caching allocator. Unused reserved blocks may still appear occupied in device monitoring. CUDA graphs can retain private pools across replays. Those mechanisms explain why a simple checkpoint calculation and an observed device-memory value need not match.
For a later executed evaluation, capture idle loaded-model memory, steady-state serving memory and peak memory during a controlled burst. State whether a chart uses allocated tensors, allocator reservation or total device usage. Include the host-memory and transfer implications if any state is offloaded. Aguasto has not executed this GPU evaluation.
CUDA semantics: Memory management — PyTorch 2.14 (external source)Treat multiple devices as a topology decision
After estimating the storage floor, evaluate how the chosen engine places weights and runtime state across devices. A planning worksheet should list per-device allocation, replication or sharding policy, interconnect, supported kernels and expected communication boundaries. Adding device capacities together is only an initial capacity calculation; it does not verify that a particular distributed layout works.
Use a comparison in stages: documented model format, supported engine path, memory estimate, representative workload test and then an operational acceptance decision. Keep any hypothetical fit labeled preliminary. Reject the shortcut from aggregate memory to a claim that the model is available through Aguasto.
A reproducible sizing record
Save a sizing record containing the official model identifier and revision, total parameter count, storage precision, engine version, prompt/output limits, concurrency target, cache settings, parallelism and hardware category. Attach measured allocations only after a real run and keep them distinct from calculator output.
The immediate decision is whether a candidate configuration deserves testing, not whether it is production ready. Publish unknowns explicitly: unverified quantization support, unmeasured cache growth, missing interconnect evidence or unknown task-quality impact. This makes the same model reference useful for both a hardware buyer and an engineer without turning an estimate into a capacity commitment.
Sources & verification
Reference revision 1 · Reviewed . Manufacturer claims and documentation are attributed; Aguasto has not run the cited products in production.
- Optimizing LLMs for Speed and Memory — Hugging Face ↗
Parameter storage at float32 and BF16/FP16; Weight storage differs from long-context inference memory. Publication: date not stated in the reviewed source. Verified: 2026-10-12.
- Cache strategies — Hugging Face Transformers ↗
KV cache grows during generation; Static cache allocation, offloading and cache quantization trade-offs; Model-specific cache limitations. Publication: date not stated in the reviewed source. Verified: 2026-10-12.
- Bitsandbytes — Hugging Face Transformers ↗
Quantized linear modules and mixed compute/storage precision; Nonquantized modules and offloading constraints. Publication: date not stated in the reviewed source. Verified: 2026-10-12.
- CUDA semantics: Memory management — PyTorch 2.14 ↗
Allocated tensor memory versus allocator-reserved memory; Graph-private memory pools and caching allocator. Publication: date not stated in the reviewed source. Verified: 2026-10-12.