Hardware isolation
Private AI is "private" at the hardware layer:
- Dedicated GPU(s) — your inference runs on physical GPUs assigned only to you. No fractional GPU sharing, no MIG-slicing across tenants. When your subscription is active, your GPU is your GPU.
- Dedicated host for higher-tier subscriptions, optional on lower tiers.
- Separate network namespace — your inference endpoint is on a network not visible to other customers.
This is meaningfully different from "private endpoint" services from large clouds where your model runs on a shared GPU pool — those still have hypervisor-level isolation, but performance and side-channel risk profiles are different.
Logical isolation
- Per-tenant logs — request logs, model load events, errors. Logs of one customer are never visible to another, including to our own support engineers who only see logs for the customer they are debugging.
- Per-tenant metrics — GPU utilization, request latency, token throughput.
- Per-tenant API keys — you control your own key rotation.
What we see (and do not see)
We see:
- Operational metadata: GPU utilization, request rate, latencies, model versions.
- Errors and crashes (for runtime debugging).
- Resource billing data.
We do not see (unless you explicitly enable it):
- The text of prompts.
- Model outputs.
- Any uploaded files / data sent through the API.
For services that require visibility for debugging (rare), we ask permission per session and the access is logged.
At rest
- Model weights — encrypted with a per-tenant key.
- Cached data — separate disk allocations per tenant.
- Backups — per-tenant, isolated.
In transit
- Our internal traffic between gateway and inference GPU is on a per-tenant VLAN.
- External traffic uses TLS 1.3.
Audit
Independent security review available for enterprise customers (ask during procurement). We do not publish a public SOC 2 / ISO 27001 report — see the data-residency-and-gdpr article for why.