RITS AI Stack Overview
Contents
RITS AI Stack Overview#
Last updated: 2026-08-27
1 · Quick Facts#
Metric |
Value |
|---|---|
Total GPUs |
9 |
Machines |
3 (Workstation × 1, Alienware × 2) |
Total GPU Memory |
~243 GiB aggregate VRAM |
GPU Models Represented |
6 (RTX PRO 5000 Blackwell, 5090, 4090 D, 3090, 3060 Ti, 3060) |
Architectures |
sm_120 × 3, sm_89 × 2, sm_86 × 4 — no Turing remains in the fleet |
Core Services Hosted |
Ask System (LLM/TTS/STT/API gateway), embeddings & reranking, Docling, Whisper STT, n8n automation, 3D & audio generation, Open edX CI |
AI Models |
Qwen3-30B-A3B-AWQ, bge-m3, Qwen3-Embedding-0.6B, Qwen3-Reranker-0.6B, Qwen3-0.6B-FP8, whisper-large-v3, Kokoro-82M, Z-Image-Turbo |
2 · Hardware Detail#
Machine |
Hostname |
CPU / RAM / OS |
GPU Configuration |
Notes |
|---|---|---|---|---|
Workstation |
rits-ai-workstation .shanghai.nyu.edu |
AMD Ryzen 9 7950X (16C/32T) 93 GiB RAM Ubuntu 22.04 LTS |
RTX PRO 5000 Blackwell × 2 (internal, 48 GB, sm_120) RTX 4090 D × 2 (eGPU, 24 GB, sm_89) RTX 5090 × 1 (eGPU, 32 GB, sm_120) |
5-GPU flagship compute node. ~175 GiB VRAM. Internal cards run PCIe Gen 5 ×8; eGPUs link at ×4. |
Alienware nyush2920wk |
nyush2920wk-ubuntu |
Intel Core i7-11700F (8C/16T) 62 GiB RAM Ubuntu 22.04 LTS |
RTX 3090 × 1 (24 GB, sm_86) RTX 3060 Ti × 1 (8 GB, sm_86) |
Mid-range support node. ~32 GiB VRAM. |
BlackAlienware nyush2290wk |
nyush2290wk-ubuntu |
Intel Core i9-9900K (8C/16T) 46 GiB RAM Ubuntu 22.04 LTS |
RTX 3090 × 1 (24 GB, sm_86) RTX 3060 × 1 (12 GB, sm_86) |
Uniformly Ampere after the 2026-08-26 upgrade. ~36 GiB VRAM. Also hosts the Open edX CI runners. |
All three machines run NVIDIA driver 580.173.02 and orchestrate their
workloads with Docker + Portainer, fronted by Caddy and Cloudflare Tunnel.
All three report CUDA 13.0. Compute capabilities were measured with
nvidia-smi --query-gpu=compute_cap — the Workstation and Alienware on
2026-08-26, BlackAlienware on 2026-08-27 after its GPU upgrade.
Build targets differ per host and do not interchange. The Workstation spans two architectures (three sm_120 cards, two sm_89), so an image built for one card there will not necessarily run on another in the same box; sm_120 additionally needs CUDA 12.8 or newer. Both Alienwares are uniformly sm_86, so a single Ampere build covers either host end to end. FP8 weights run on every Workstation card (sm_89+) but not on the Ampere hosts; bf16 is now available fleet-wide.
Only total VRAM is recorded here. Free VRAM moves too fast to document — a single model load shifted one card from ~15 GiB free to ~4 GiB within the same day — so measure it at the point of use rather than citing a figure from this page.
Important
BlackAlienware was upgraded on 2026-08-26 and is no longer Turing.
It previously held an RTX 2080 Ti + RTX 2060 SUPER (both sm_75); an
nvidia-smi reading taken on the morning of 2026-08-26 still showed
that configuration, because the swap had not yet happened. The row
above was re-measured on 2026-08-27 after the owner’s burn-in.
Consequences: sm_75 build variants for this host are now obsolete,
and any image pinned to TORCH_ARCH=7.5 for it — including
worldclaw’s Hunyuan3D default — needs rebuilding at 8.6. The
11 GB ceiling that kept the Hunyuan3D paint stage off this box is also
gone; paint’s ~12 GiB peak now fits its RTX 3090.
Two project-built extensions cannot run on the new cards. Audited
with cuobjdump on 2026-08-27. A single sm_75 cubin with no PTX
means there is no JIT fallback: the module imports cleanly and then
fails at first kernel launch.
worldclaw-sam3d _C...so sm_75 only, no PTX BROKEN
worldclaw-hunyuan3d custom_rasterizer_kernel.so sm_75 only, no PTX BROKEN
worldclaw-hunyuan3d mesh_inpaint_processor.so no device code unaffected
worldclaw-sam3 no project-built CUDA ext. vendor libs only likely fine
sam3 differs in kind, not degree: a sweep of all 907 shared
objects in that image found 96 carrying device code and every one
vendor-supplied (CUDA toolkit, driver libs, NCCL, torch,
torchvision) — no project-built extension exists in it at all. It is
still marked likely fine rather than confirmed, because nothing has
actually executed on the new card.
sam3d is the significant one: that is its own core extension, not
an optional dependency, so the service is running right now on a card
it cannot execute on. Stock PyTorch (sm_50–sm_90) and the bundled
NVIDIA libraries (cuBLAS, cuDNN, NPP, cuSOLVER, nvJPEG) are all fat
binaries including sm_86 — the breakage is confined to the two
project-built extensions. Rebuild both at 8.6; sam3 likely
needs no rebuild. Retire the sm_75 images only after that.
The rebuild is required for either Ampere host — the sm_75
cubin is a property of the image, not of the card it failed on.
GPU indices renumbered. Both cards changed, so every
device_ids, CUDA_VISIBLE_DEVICES or GPU_ID pin on this host
must be re-verified. Prefer UUIDs, which survive renumbering:
GPU-6aadc5dc-7349-fd4f-3d70-59f3463b7319 RTX 3090 (24 GB)
GPU-e35c27f7-7b5a-e36e-46ae-23224d506526 RTX 3060 (12 GB)
Note
Pin by UUID, not by index or alias. There is now an RTX 3090 in each Alienware, so “the 3090 box” is ambiguous, and BlackAlienware’s indices renumbered when its cards changed. UUIDs survive both:
Alienware nyush2920wk-ubuntu 10.208.4.12
GPU-393c8b1e-f02a-05aa-6e3f-5d104d18c0c2 RTX 3090 (24 GB)
GPU-8eeefdc8-489d-ca3e-2300-135d4648b828 RTX 3060 Ti (8 GB)
BlackAlienware nyush2290wk-ubuntu
GPU-6aadc5dc-7349-fd4f-3d70-59f3463b7319 RTX 3090 (24 GB)
GPU-e35c27f7-7b5a-e36e-46ae-23224d506526 RTX 3060 (12 GB)
Hostnames and the Alienware IP above are self-reported by each host, not taken from a local ssh config.
Note
GPU descriptions in this fleet’s own build files are not reliable:
worldclaw’s Dockerfile and compose both describe Workstation GPU 4
as an “RTX 3090, 24GB, sm_86”, but nvidia-smi reports an RTX 5090
with 32,607 MiB (Blackwell, sm_120). Verify against the host before
relying on any such comment.
Every image on this fleet has an architecture target and none of them
are labelled in the compose file. Three known instances, in both
directions: worldclaw’s sm_75 builds now pointed at sm_86
hardware; its sm_86 paint image pointed at an sm_120 card; and
create-video-api, whose Dockerfile pins
TORCH_CUDA_ARCH_LIST="12.0" on a CUDA 12.8 base to compile
SageAttention for the RTX 5090 — making that 24 GB image
Blackwell-only and dead on any Ampere host.
The same caution applies to measurements, including the ones on this page. A reading is only as current as its clock: the BlackAlienware figures above were contradicted within hours by a physical GPU swap, and the discrepancy was settled by the machine’s boot time rather than by re-arguing the readings. On a fleet being actively re-hardwared, pair any GPU fact with the date it was taken.
Container logs carry the same trap. On BlackAlienware the most
alarming lines — sam3d reporting cuda:0 = RTX 2080 Ti and
hunyuan3d raising CUDA-capable device(s) is/are busy or
unavailable — are dated 2026-08-19 and 2026-08-20, a week before the
GPU swap. Read them without --timestamps and you would conclude
the services are currently failing on hardware they no longer have.
Conversely, since the swap each service has logged only its startup
banner and holds 0 MiB of VRAM: nothing has been attempted, so nothing
has errored. “Container is Up” has now been a worthless signal three
separate times in this project.
A negative result needs a positive control. The audit above initially
reported no CUDA extensions in sam3 — from a sweep whose filename
pattern could not have matched the very file it was looking for. The
fix was to confirm the sweep could see a known-positive
(libtorch_cuda.so) before trusting it to report an absence. Any
“we found nothing” on this fleet should say what it successfully
found first.
That applies to this page too. It was frozen for thirteen months — the docs workflow last built successfully on 2025-07-11, and stopped only because a dependency’s build requirements broke under a newer runner image. Nothing in the repository changed and nothing announced the breakage: the site kept serving 200s and looked authoritative while describing hardware that had since been replaced twice. A build that has not run in thirteen months is not a passing build, it is an unobserved one. Check that the page changed, not merely that the workflow went green.
3 · Software & Service Map#
Service Group |
Public Endpoint(s) |
Primary Components |
Host Machine |
|---|---|---|---|
Ask System |
https://ask-llm.ritsdev.top https://ask-llm-key.ritsdev.top https://ask-angie.ritsdev.top https://ask-roary.ritsdev.top |
LiteLLM API gateway (ask-llm) LLM (vLLM, stable + canary) TTS, STT API key self-service (gateway also exposes native video-generation routes) |
Workstation (LiteLLM gateway, LLM/TTS/STT) IT server (ask-angie and ask-roary front-ends) |
Retrieval & Document AI |
https://embedder.workstation.ritsdev.top https://reranker.workstation.ritsdev.top https://docling.workstation.ritsdev.top https://unstructured.ritsdev.top |
bge-m3 / Qwen3 embeddings Qwen3 reranker Docling document parsing Unstructured API |
Workstation (embeddings, reranker, Docling) Alienware nyush2920wk (Unstructured) |
Speech |
https://stt.workstation.ritsdev.top https://whisper.ritsdev.top |
Speaches (Whisper) STT Kokoro TTS Whisper WebUI |
Workstation (STT/TTS) Alienware nyush2920wk (Whisper WebUI) |
Automation |
https://n8n.workstation.ritsdev.top https://n8n-data.workstation.ritsdev.top |
n8n workflow automation Agent Mail, memory service SMTP relay |
Workstation |
3D & Audio Generation |
https://musicgen.ritsdev.top https://create-music.ritsdev.top https://moving-images.ritsdev.top |
Hunyuan3D, SAM 3 / SAM 3D (SAM 3D needs an sm_86 rebuild — see section 2) ACE-Step, AudioCraft MusicGen |
BlackAlienware nyush2290wk |
Developer Services |
https://jetbrains.ritsdev.top https://portainer.workstation.ritsdev.top |
Open edX CI runners JetBrains license server Portainer (per machine) |
BlackAlienware nyush2290wk (runners, licence server) All three (Portainer) |
Endpoint status as of 2026-08-27: the Create System is offline and
scheduled for restoration — the owner has confirmed it is not being
retired. Its create-image-api and create-video-api origins
were removed from the Alienware and both hostnames now return 502, as
does preview.ritsdev.top. LiteLLM’s create-image route is broken
as a result, as are its Nanbeige4.2-3B-AWQ and InternVL-30B-A3B-AWQ
routes.
The stack itself was not deleted: /home/uetuluk/create-image-api on
the Workstation still holds the compose file, .env, both Dockerfiles
and ~19 GB of weights. Because those weights are uint4 SDNQ and so
architecture-agnostic, any of the four Ampere cards in the fleet can host
it with no rebuild needed for architecture reasons. The create-image
route in litellm/config.yaml should therefore be repaired rather than
removed.
Caution
Two copies of the image weights exist. The live set is the
./models bind mount (~19 GB) that compose.yml actually uses.
A second, unattached Docker volume create-image-api_models holds
27.95 GB, left from an older config — compose.yml still declares an
unused named volume models:, which is how the orphan arose.
Decide which set is canonical before restoring the service, and do
not reclaim either as “unused disk” without checking which one the
compose file points at.
Roughly 40 further Workstation vhosts (comfy, invokeai, cogvideo, janus, omnigen, live-diffusion, judge0, memgpt and others) point at containers that no longer exist and should be treated as retired.
4 · Topology Snapshot#
flowchart TD
subgraph Workstation["Workstation — rits-ai-workstation"]
WP1["RTX PRO 5000 (internal)"]
WP2["RTX PRO 5000 (internal)"]
W4090a["4090 D-1 (eGPU)"]
W4090b["4090 D-2 (eGPU)"]
W5090["5090 (eGPU)"]
end
subgraph nyush2920wk["Alienware — nyush2920wk"]
B3090["3090 (24 GB)"]
B3060["3060 Ti (8 GB)"]
end
subgraph nyush2290wk["BlackAlienware — nyush2290wk"]
C3090["3090 (24 GB)"]
C3060["3060 (12 GB)"]
end
AskAPI["Ask API / LiteLLM gateway"] -- routes to --> AskLLM["Ask LLM / TTS / STT"]
AskLLM -- runs on --> Workstation
Retrieval["Embeddings / Reranker / Docling"] -- runs on --> Workstation
Automation["n8n / Agent Mail"] -- runs on --> Workstation
CreateImage["Create Image API"] -- runs on --> nyush2920wk
Whisper["Whisper WebUI"] -- runs on --> nyush2920wk
Media["3D / Audio Generation"] -- runs on --> nyush2290wk
CI["Open edX CI Runners"] -- runs on --> nyush2290wk
5 · AI Models & Specifications#
Service |
Model |
Source |
Specifications |
Task |
|---|---|---|---|---|
LLM |
Qwen3-30B-A3B-AWQ |
https://www.modelscope.cn/models/swift/ Qwen3-30B-A3B-AWQ |
30B-parameter MoE model with AWQ quantization and FP8 KV cache; served by vLLM with tool calling and reasoning parsers enabled |
language processing |
STT |
Whisper Large V3 |
https://huggingface.co/openai/ whisper-large-v3 |
OpenAI’s largest Whisper model for high-accuracy speech recognition; served through Speaches |
speech recognition |
TTS |
Kokoro-82M |
82M-parameter text-to-speech model optimized for natural voice synthesis |
speech synthesis |
|
Embedding |
bge-m3 |
Multilingual, multi-granularity embedding model served via Infinity |
text embedding |
|
Embedding |
Qwen3-Embedding-0.6B |
https://huggingface.co/Qwen/ Qwen3-Embedding-0.6B |
Lightweight 0.6B embedding model served by vLLM alongside bge-m3 |
text embedding |
Reranking |
Qwen3-Reranker-0.6B |
https://huggingface.co/Qwen/ Qwen3-Reranker-0.6B |
0.6B cross-encoder scoring model with a 32k context window |
search reranking |
Entity Extraction |
Qwen3-0.6B-FP8 |
FP8-quantized 0.6B model with a 4k context window for structured extraction |
entity extraction |
|
Image Generation (offline) |
Z-Image-Turbo (SDNQ uint4) |
https://huggingface.co/Disty0/ Z-Image-Turbo-SDNQ-uint4-svd-r32 |
uint4-quantized turbo diffusion model (13 GB), paired with a dc-ae-f32c32-sana autoencoder and gemma-2-2b-it text encoder. Weights are arch-agnostic; service is down. |
image generation |