NVIDIA Releases Open-Source Tool for Diagnosing AI Workload Failures

It includes a REST API service and a Python orchestrator supporting major schedulers and telemetry sources. Used in production across AI infrastructure, it analyzes millions of workloads weekly and identifies hardware failure patterns on specific host and switch ports. The diagnosis pipeline includes scope definition, observation collection, event correlation via deterministic rules, and structured output with root cause, affected nodes, and symptoms.

Summarised from the original article by an open-source model, then checked automatically: every number and English name must appear in the source. The original article is authoritative.