The data scientist launches a notebook on an A100 "to test". Forgot shutdown over the weekend: 720 h × €3.50 — €2,520 of GPU that only loaded pandas. Management talks "cloud too expensive"; nobody read the hourly grid without auto-stop.
Renting a GPU in cloud (OVHcloud, Scaleway, AWS, Lambda Labs, etc.) is now accessible. The absurd bill arrives when model, duration, egress and idle are not framed — see also hosting an LLM for upstream scoping.
Contract questions before the first click
| Question | What it avoids |
|---|---|
| Hourly / monthly / spot billing? | GPU on 24/7 at hourly rate |
| Dedicated physical VRAM? | Mysterious OOM on "shared" GPU |
| Region and egress to S3? | End-of-month transfer surprise |
| CUDA image / drivers included? | Days lost installing stack |
| Spot interruption: SLA? | Training job lost at 90% |
| Quota and account approval? | Prod blocked for GPU quota |
Size the card to the workload
7B–13B quantised inference — L4, T4, or A10 24 GB often enough.
7B fine-tuning — 24–48 GB VRAM, modest batch size.
70B+ inference — multi-GPU or distilled model; do not rent A100 alone by reflex.
Exploratory Jupyter — CPU plus small spot GPU; not A100 by default — see team Jupyter.
Measure real GPU utilisation (nvidia-smi, DCGM) before upgrading. 15% average utilisation = oversizing or bad batching.
Operational cost hygiene
Enable auto-shutdown after idle (scripts, cloud tags). Set budget alerts at 50% and 80% of monthly cap. Save checkpoints to regional object storage — not ephemeral disk alone. Keep single region — data, GPU and buckets together — see choosing cloud region. Run weekly review of orphan instances after POC.
A reasonable GPU bill is a bill switched off when the job is done.
The climax: hourly price flatters, full month accuses
Compare European GPU offers with documented egress in the directory and compare tool.
Decide and move forward without blind spots
Estimate real GPU hours per month — not the theoretical peak from the first test. Choose spot vs reserved by interruption tolerance and service criticality. Enable budget alerts and auto-stop before first launch, not after the first bad surprise. Review VRAM after one week of real metrics. Document who shuts down what on the team to avoid orphan instances.
Frequently asked questions
Hourly or monthly?
Hourly or spot suits proofs of concept and one-off training runs. Monthly reservation makes sense for 24/7 inference — otherwise hourly cost of an idle GPU overnight exceeds the planned budget.
T4, L4 or A100?
T4 and L4 target economical inference with moderate power draw. A100 and H100 serve training and large models, with forty to eighty gigabytes of VRAM. Pick the card by model size and expected throughput, not by prestige on the invoice.
Shared GPUs?
For light inference or development, sharing can suffice. For guaranteed latency or large batch workloads, require a documented dedicated GPU — opaque sharing complicates debugging and skews metrics.
Hidden costs?
Watch egress, oversized NVMe disk, public IP, automatic snapshots, and spot interruptions without saved checkpoints. CUDA drivers or technical support may also appear as separate line items depending on the provider.
Before renting a GPU, calculate: how many hours on per week if nobody shuts down? If the answer surprises you, fix the process — not just the card.