Skip to content
English

Switch theme

Discover▾

Not sure where to start? This week’s hottest software, models and stacks in three minutes.

AI Apps▾

AI software people use: from assistants like ChatGPT to tools for coding, video and workflows, picked by purpose.

For Agents▾

Software for AI: tools that let your agent (Claude Code, Codex, Cursor…) browse, read docs, run code and remember.

Models▾

Choose models, estimate cost, self-host: prices, rankings, benchmarks, free tiers and compute.

Solutions▾

A goal or an error message? Ready-made stacks, steps and fixes by scenario.

Learn▾

A systematic course from zero to the frontier, with an algorithm library, must-read papers and tutorials.

Frontier▾

Daily AI news, weekly highlights, the latest papers and how AI is changing every discipline.

Step 6 · Operations

Keep it running

Hardware is the easy part; most cost and downtime comes from operations.

Routine

Acceptance testBefore signing off a delivery: run DCGM diagnostics on every GPU and nccl-tests across nodes; compare bus bandwidth with the fabric’s rated speed.docs.nvidia.comgithub.com
MonitoringDCGM for health, temperature, ECC errors and utilisation; on Kubernetes the GPU Operator deploys the DCGM exporter for Prometheus.docs.nvidia.comdocs.nvidia.com
Version pinningPin driver, CUDA and engine versions per cluster; upgrade in a canary node first.docs.nvidia.comdocs.nvidia.com
Capacity reviewTrack tokens/s, queue time and utilisation monthly; if utilisation stays low, go back to step 1.

Common pitfalls

NVLink missing on an HGX boxFabric Manager not installed or its version does not match the driver.docs.nvidia.com
“Out of memory” although the model fitsKV cache × concurrency was forgotten; vLLM also pre-allocates memory via gpu-memory-utilization.docs.vllm.ai
Server reachable from the internetOllama binds 127.0.0.1 by default; changing OLLAMA_HOST exposes it – put an authenticated gateway in front.docs.ollama.com
Breakers trip / GPUs throttleMulti-GPU workstations on one 10 A circuit; inlet air above the rated range (10–30 °C for DGX B300).docs.nvidia.com
Slow multi-node trainingRails cabled wrong or one plane down – nccl-tests shows it immediately.github.comdocs.nvidia.com
Quality drop after quantisationEvaluate the quantised model on your own tasks before switching traffic.
Grey-market cardsNo warranty or RMA, and possible export-control exposure for data-centre GPUs.federalregister.govbis.gov
Hardware bought for a peakSelf-host the base load; burst peaks to an API or rented GPUs.

Problem: observability for self-hosted LLMs → · Problem: what model fits my VRAM → · Benchmarks →

Data updated: Compute hardware