What you can run
Private AI is essentially "we run your model on dedicated GPUs we operate." We support:
- Open-weights LLMs — Llama, Mistral, Qwen, DeepSeek, Gemma, Phi, plus fine-tunes.
- Image models — Stable Diffusion (1.5, XL, 3), Flux, custom checkpoints.
- Speech — Whisper variants, Parakeet, custom STT/TTS.
- Embedding — BGE, GTE, E5, Jina, Nomic, custom.
- Custom fine-tunes — LoRA adapters or full fine-tunes you trained yourself.
Runtimes supported
- vLLM — high-throughput LLM serving.
- TGI (Text Generation Inference).
- Ollama — smaller models and prototyping.
- TensorRT-LLM — latency-critical on H100/H200.
- ComfyUI / Diffusers for image gen.
- Custom PyTorch / Triton servers for the rest.
Need something else? Talk to us — usually game.
Bring your weights
Ship via:
- HTTPS upload to a private endpoint we provision per customer.
- S3-compatible bucket (we pull using credentials you provide).
- Git LFS for smaller checkpoints.
- Physical disk for very large models (we accept couriered NVMe).
Weights are encrypted at rest, accessible only by your dedicated inference processes. We never train on, mirror, or share your weights.
Quantization options
FP16, BF16, INT8, FP8, AWQ, GPTQ, GGUF. Picking the right quantization is part of onboarding — it affects performance and quality.
Fine-tuning on our hardware
Use your Private AI GPU time for training, not just inference. Options:
- Managed pipeline — upload data, we run the fine-tune, we hand back the weights.
- Raw GPU access — your own training scripts via SSH or JupyterHub.
Per-GPU-hour during training, on top of your base inference subscription.
What we will not host
- Models under licenses we cannot legally serve.
- Models requiring an active API key from another vendor.
- Anything we judge creates undue legal risk for you or us (rare).