---
title: "Ray Serve | Aguasto"
description: "Technical reference for Ray Serve: Build Python inference applications that compose models and business logic and scale replicas across Ray resources. Hardware assumptions and operating trade-offs are documented."
canonical: "https://www.aguasto.com/inference-stack/ray-serve"
last-updated: "2026-10-12"
---

# Ray Serve | Aguasto

Canonical HTML: [Ray Serve](https://www.aguasto.com/inference-stack/ray-serve)

> Technical reference for Ray Serve: Build Python inference applications that compose models and business logic and scale replicas across Ray resources. Hardware assumptions and operating trade-offs are documented.

[Aguasto](https://www.aguasto.com/) [Inference Stack](https://www.aguasto.com/inference-stack)  Ray Serve

[Resources](https://www.aguasto.com/resources) [GPU Library](https://www.aguasto.com/gpus) [Model Index](https://www.aguasto.com/models) [Inference Stack](https://www.aguasto.com/inference-stack) [Learn](https://www.aguasto.com/learn) [Insights](https://www.aguasto.com/insights) [Research](https://www.aguasto.com/research) [Engineering tools](https://www.aguasto.com/tools)

Application serving and scaling

## Ray Serve

Build Python inference applications that compose models and business logic and scale replicas across Ray resources.

Original Aguasto conceptual illustration. Not manufacturer geometry, an operating Aguasto asset, or a measured benchmark.

Aguasto Research documents third-party hardware, models and software. References do not represent Aguasto inventory, evaluated deployments or available inference services.

### When it is relevant

- Compose multiple models or preprocessing steps into an application

- Manage serving replicas, batching and streaming around a model engine

[Ray Serve official overview (external source)](https://docs.ray.io/en/latest/serve/index.html)   · [Ray accelerator support (external source)](https://docs.ray.io/en/latest/ray-core/scheduling/accelerators.html)   · [Ray development accelerator support matrix (external source)](https://docs.ray.io/en/master/ray-core/scheduling/accelerators.html)

### Hardware and software assumptions

Documented ecosystem: NVIDIA, AMD, Intel, Google (TPU). Platform support is not a guarantee that every checkpoint, optimization or feature works on every device.

- Provision and operate the Ray cluster and its model dependencies

- Use a compatible inference implementation for each target accelerator

- Aguasto has not deployed or measured Ray Serve

[Ray Serve official overview (external source)](https://docs.ray.io/en/latest/serve/index.html)   · [Ray accelerator support (external source)](https://docs.ray.io/en/latest/ray-core/scheduling/accelerators.html)   · [Ray development accelerator support matrix (external source)](https://docs.ray.io/en/master/ray-core/scheduling/accelerators.html)

### Operating trade-offs

- It adds a distributed application layer rather than performing model-specific kernel optimization

- Fractional GPU allocation needs application memory limits; scheduling fractions do not guarantee physical memory isolation

- Development documentation marks AMD and Intel support as community experimental; verify the deployed release separately

[Ray Serve official overview (external source)](https://docs.ray.io/en/latest/serve/index.html)   · [Ray accelerator support (external source)](https://docs.ray.io/en/latest/ray-core/scheduling/accelerators.html)   · [Ray development accelerator support matrix (external source)](https://docs.ray.io/en/master/ray-core/scheduling/accelerators.html)

### Continue the research

[Choosing an inference framework](https://www.aguasto.com/learn/choosing-an-inference-serving-framework) [Prefill, decoding and disaggregation](https://www.aguasto.com/learn/prefill-decode-and-disaggregated-serving) [Benchmark methodology](https://www.aguasto.com/research/inference-benchmark-methodology) [Model references](https://www.aguasto.com/models) [Infrastructure collaboration](https://www.aguasto.com/partners)

### Sources & verification

Reference revision 1 · Reviewed October 12, 2026. Manufacturer claims and documentation are attributed; Aguasto has not run the cited products in production.

- [Ray Serve official overview ↗](https://docs.ray.io/en/latest/serve/index.html)

Python model composition; streaming, batching, serving and replica scaling; does not itself perform model-specific optimization. Publication: date not stated in the reviewed source. Verified: 2026-10-12.

- [Ray accelerator support ↗](https://docs.ray.io/en/latest/ray-core/scheduling/accelerators.html)

NVIDIA / AMD / Intel and TPU resources; fractional accelerator scheduling; application responsibility for memory use. Publication: date not stated in the reviewed source. Verified: 2026-10-12.

- [Ray development accelerator support matrix ↗](https://docs.ray.io/en/master/ray-core/scheduling/accelerators.html)

Development documentation distinguishes fully tested NVIDIA/TPU from community experimental AMD/Intel support; check the deployed release separately. Publication: date not stated in the reviewed source. Verified: 2026-10-12.

### Reference journey

[Browse the stack ↗](https://www.aguasto.com/inference-stack) [Model Index ↗](https://www.aguasto.com/models) [GPU Library ↗](https://www.aguasto.com/gpus) [Sources & verification ↓](https://www.aguasto.com/inference-stack/ray-serve#sources)

Research reference · No available Aguasto service

## Company navigation

- [Home](https://www.aguasto.com/)
- [Infrastructure](https://www.aguasto.com/infrastructure)
- [GPU Cloud](https://www.aguasto.com/gpu-cloud)
- [AI Inference](https://www.aguasto.com/ai-inference)
- [Development & Investment](https://www.aguasto.com/development)
- [Company](https://www.aguasto.com/company)
- [Contact](https://www.aguasto.com/contact)
- [Privacy](https://www.aguasto.com/privacy)
- [Resources](https://www.aguasto.com/resources)
- [GPU Library](https://www.aguasto.com/gpus)
- [Model Index](https://www.aguasto.com/models)
- [Inference Stack](https://www.aguasto.com/inference-stack)
- [Learn](https://www.aguasto.com/learn)
- [Insights](https://www.aguasto.com/insights)
- [Research](https://www.aguasto.com/research)
- [Industry cases](https://www.aguasto.com/research/cases)
- [Why Modular](https://www.aguasto.com/why-modular)
- [Partners](https://www.aguasto.com/partners)
- [GPU comparison](https://www.aguasto.com/gpus/compare)
- [Model comparison](https://www.aguasto.com/models/compare)
- [Engineering tools](https://www.aguasto.com/tools)
- [Development friction](https://www.aguasto.com/research/hyperscale-development-friction)
- [NVIDIA H100 SXM](https://www.aguasto.com/gpus/h100)
- [NVIDIA H200 SXM](https://www.aguasto.com/gpus/h200)
- [NVIDIA B200 SXM](https://www.aguasto.com/gpus/b200)
- [NVIDIA B300](https://www.aguasto.com/gpus/b300)
- [NVIDIA GB200 NVL72](https://www.aguasto.com/gpus/gb200-nvl72)
- [NVIDIA GB300 NVL72](https://www.aguasto.com/gpus/gb300-nvl72)
- [NVIDIA Vera Rubin NVL72](https://www.aguasto.com/gpus/vera-rubin-nvl72)
- [AMD Instinct MI300X](https://www.aguasto.com/gpus/mi300x)
- [AMD Instinct MI325X](https://www.aguasto.com/gpus/mi325x)
- [AMD Instinct MI355X](https://www.aguasto.com/gpus/mi355x)
- [AMD Instinct MI455X](https://www.aguasto.com/gpus/mi455x)
- [AMD Helios](https://www.aguasto.com/gpus/helios)
- [GLM-4.5](https://www.aguasto.com/models/glm-4-5)
- [Qwen3-235B-A22B](https://www.aguasto.com/models/qwen3-235b-a22b)
- [DeepSeek-R1](https://www.aguasto.com/models/deepseek-r1)
- [Llama 3.3 70B Instruct](https://www.aguasto.com/models/llama-3-3-70b-instruct)
- [Gemma 3 27B IT](https://www.aguasto.com/models/gemma-3-27b-it)
- [gpt-oss-120b](https://www.aguasto.com/models/gpt-oss-120b)
- [Kimi-K2-Instruct](https://www.aguasto.com/models/kimi-k2-instruct)
- [vLLM](https://www.aguasto.com/inference-stack/vllm)
- [SGLang](https://www.aguasto.com/inference-stack/sglang)
- [TensorRT-LLM](https://www.aguasto.com/inference-stack/tensorrt-llm)
- [NVIDIA Dynamo](https://www.aguasto.com/inference-stack/dynamo)
- [NVIDIA Triton Inference Server](https://www.aguasto.com/inference-stack/triton)
- [AMD ROCm](https://www.aguasto.com/inference-stack/rocm)
- [Ray Serve](https://www.aguasto.com/inference-stack/ray-serve)
- [Kubernetes GPU orchestration — NVIDIA device plugin](https://www.aguasto.com/inference-stack/kubernetes)
- [Model weights are only the beginning of the memory budget](https://www.aguasto.com/learn/model-weights-and-runtime-memory)
- [Why tokens per second is not enough to benchmark inference](https://www.aguasto.com/research/inference-benchmark-methodology)
- [Prefill, decode and the cost of separating them](https://www.aguasto.com/learn/prefill-decode-and-disaggregated-serving)
- [Choosing between vLLM, SGLang and TensorRT-LLM](https://www.aguasto.com/learn/choosing-an-inference-serving-framework)
- [From Power to Inference—Building a Distributed AI Compute Network](https://www.aguasto.com/learn/from-power-to-inference)
- [Dedicated compute and shared inference: compare the whole workload](https://www.aguasto.com/insights/dedicated-and-shared-inference-economics)
- [Distributed inference reliability begins with the failure boundary](https://www.aguasto.com/learn/distributed-inference-reliability)
- [Core Scientific: reading contracted, energized and billed capacity](https://www.aguasto.com/research/cases/core-scientific-capacity-disclosures)
- [Why Modular AI Infrastructure Matters Now](https://www.aguasto.com/research/why-modular-ai-infrastructure-matters-now)
- [Official LinkedIn](https://www.linkedin.com/company/aguasto)

This is a generated representation of the public HTML page. Diagrams are conceptual. GPU and inference services are planned and are not currently available to book or call. The contact page describes the available inquiry flow.