--- title: "Inference Stack Library | Aguasto" description: "Understand the software between GPUs and model APIs: serving engines, distributed inference, runtime ecosystems and GPU orchestration." canonical: "https://www.aguasto.com/inference-stack" last-updated: "2026-10-12" --- # Inference Stack Library | Aguasto Canonical HTML: [Inference Stack](https://www.aguasto.com/inference-stack) > Understand the software between GPUs and model APIs: serving engines, distributed inference, runtime ecosystems and GPU orchestration. [Resources](https://www.aguasto.com/resources) [GPU Library](https://www.aguasto.com/gpus) [Model Index](https://www.aguasto.com/models) [Inference Stack](https://www.aguasto.com/inference-stack) [Learn](https://www.aguasto.com/learn) [Insights](https://www.aguasto.com/insights) [Research](https://www.aguasto.com/research) [Engineering tools](https://www.aguasto.com/tools) Aguasto / inference-stack ## Between hardware and answers. Understand the serving engines, runtime ecosystems and distributed systems that connect GPU infrastructure to useful inference. Compatibility and operating trade-offs matter as much as feature lists. Original Aguasto conceptual illustration. Not manufacturer geometry, an operating Aguasto asset, or a measured benchmark. Aguasto Research documents third-party hardware, models and software. References do not represent Aguasto inventory, evaluated deployments or available inference services. Search technologiesCategoryAllInference engineDistributed inference coordinationModel serving serverGPU compute software platformApplication serving and scalingGPU resource orchestrationCharacteristicAllAMDAWS (Inferentia backend)Apple (community Metal plugin)GoogleGoogle (TPU)HuaweiIntelMoore ThreadsNVIDIAClear filters 8 of 8 references Inference engine ### vLLM Serve supported language and multimodal models using continuous batching, attention-cache management and distributed inference. Official documentation · backend-specific assumptions [vLLM](https://www.aguasto.com/inference-stack/vllm) Inference engine ### SGLang Serve supported language and multimodal models with prefix caching and multi-GPU inference. Official documentation · backend-specific assumptions [SGLang](https://www.aguasto.com/inference-stack/sglang) Inference engine ### TensorRT-LLM Optimize and serve supported generative models using NVIDIA GPU kernels, cache management and parallel inference. Official documentation · backend-specific assumptions [TensorRT-LLM](https://www.aguasto.com/inference-stack/tensorrt-llm) Distributed inference coordination ### NVIDIA Dynamo Coordinate inference engines through routing, separate prefill and decode workers, cache management and scaling. Official documentation · backend-specific assumptions [NVIDIA Dynamo](https://www.aguasto.com/inference-stack/dynamo) Model serving server ### NVIDIA Triton Inference Server Expose model inference through supported backends, scheduling, batching and model pipelines. Official documentation · backend-specific assumptions [NVIDIA Triton Inference Server](https://www.aguasto.com/inference-stack/triton) GPU compute software platform ### AMD ROCm Provide GPU runtimes, compilers, math and communication libraries for supported AMD workloads. Official documentation · backend-specific assumptions [AMD ROCm](https://www.aguasto.com/inference-stack/rocm) Application serving and scaling ### Ray Serve Build Python inference applications that compose models and business logic and scale replicas across Ray resources. Official documentation · backend-specific assumptions [Ray Serve](https://www.aguasto.com/inference-stack/ray-serve) GPU resource orchestration ### Kubernetes GPU orchestration — NVIDIA device plugin Expose NVIDIA GPU devices as Kubernetes resources and allocate them to container workloads. Official documentation · backend-specific assumptions [Kubernetes GPU orchestration — NVIDIA device plugin](https://www.aguasto.com/inference-stack/kubernetes) ## Company navigation - [Home](https://www.aguasto.com/) - [Infrastructure](https://www.aguasto.com/infrastructure) - [GPU Cloud](https://www.aguasto.com/gpu-cloud) - [AI Inference](https://www.aguasto.com/ai-inference) - [Development & Investment](https://www.aguasto.com/development) - [Company](https://www.aguasto.com/company) - [Contact](https://www.aguasto.com/contact) - [Privacy](https://www.aguasto.com/privacy) - [Resources](https://www.aguasto.com/resources) - [GPU Library](https://www.aguasto.com/gpus) - [Model Index](https://www.aguasto.com/models) - [Inference Stack](https://www.aguasto.com/inference-stack) - [Learn](https://www.aguasto.com/learn) - [Insights](https://www.aguasto.com/insights) - [Research](https://www.aguasto.com/research) - [Industry cases](https://www.aguasto.com/research/cases) - [Why Modular](https://www.aguasto.com/why-modular) - [Partners](https://www.aguasto.com/partners) - [GPU comparison](https://www.aguasto.com/gpus/compare) - [Model comparison](https://www.aguasto.com/models/compare) - [Engineering tools](https://www.aguasto.com/tools) - [Development friction](https://www.aguasto.com/research/hyperscale-development-friction) - [NVIDIA H100 SXM](https://www.aguasto.com/gpus/h100) - [NVIDIA H200 SXM](https://www.aguasto.com/gpus/h200) - [NVIDIA B200 SXM](https://www.aguasto.com/gpus/b200) - [NVIDIA B300](https://www.aguasto.com/gpus/b300) - [NVIDIA GB200 NVL72](https://www.aguasto.com/gpus/gb200-nvl72) - [NVIDIA GB300 NVL72](https://www.aguasto.com/gpus/gb300-nvl72) - [NVIDIA Vera Rubin NVL72](https://www.aguasto.com/gpus/vera-rubin-nvl72) - [AMD Instinct MI300X](https://www.aguasto.com/gpus/mi300x) - [AMD Instinct MI325X](https://www.aguasto.com/gpus/mi325x) - [AMD Instinct MI355X](https://www.aguasto.com/gpus/mi355x) - [AMD Instinct MI455X](https://www.aguasto.com/gpus/mi455x) - [AMD Helios](https://www.aguasto.com/gpus/helios) - [GLM-4.5](https://www.aguasto.com/models/glm-4-5) - [Qwen3-235B-A22B](https://www.aguasto.com/models/qwen3-235b-a22b) - [DeepSeek-R1](https://www.aguasto.com/models/deepseek-r1) - [Llama 3.3 70B Instruct](https://www.aguasto.com/models/llama-3-3-70b-instruct) - [Gemma 3 27B IT](https://www.aguasto.com/models/gemma-3-27b-it) - [gpt-oss-120b](https://www.aguasto.com/models/gpt-oss-120b) - [Kimi-K2-Instruct](https://www.aguasto.com/models/kimi-k2-instruct) - [vLLM](https://www.aguasto.com/inference-stack/vllm) - [SGLang](https://www.aguasto.com/inference-stack/sglang) - [TensorRT-LLM](https://www.aguasto.com/inference-stack/tensorrt-llm) - [NVIDIA Dynamo](https://www.aguasto.com/inference-stack/dynamo) - [NVIDIA Triton Inference Server](https://www.aguasto.com/inference-stack/triton) - [AMD ROCm](https://www.aguasto.com/inference-stack/rocm) - [Ray Serve](https://www.aguasto.com/inference-stack/ray-serve) - [Kubernetes GPU orchestration — NVIDIA device plugin](https://www.aguasto.com/inference-stack/kubernetes) - [Model weights are only the beginning of the memory budget](https://www.aguasto.com/learn/model-weights-and-runtime-memory) - [Why tokens per second is not enough to benchmark inference](https://www.aguasto.com/research/inference-benchmark-methodology) - [Prefill, decode and the cost of separating them](https://www.aguasto.com/learn/prefill-decode-and-disaggregated-serving) - [Choosing between vLLM, SGLang and TensorRT-LLM](https://www.aguasto.com/learn/choosing-an-inference-serving-framework) - [From Power to Inference—Building a Distributed AI Compute Network](https://www.aguasto.com/learn/from-power-to-inference) - [Dedicated compute and shared inference: compare the whole workload](https://www.aguasto.com/insights/dedicated-and-shared-inference-economics) - [Distributed inference reliability begins with the failure boundary](https://www.aguasto.com/learn/distributed-inference-reliability) - [Core Scientific: reading contracted, energized and billed capacity](https://www.aguasto.com/research/cases/core-scientific-capacity-disclosures) - [Why Modular AI Infrastructure Matters Now](https://www.aguasto.com/research/why-modular-ai-infrastructure-matters-now) - [Official LinkedIn](https://www.linkedin.com/company/aguasto) This is a generated representation of the public HTML page. Diagrams are conceptual. GPU and inference services are planned and are not currently available to book or call. The contact page describes the available inquiry flow.