Writing
Why did CPUs never have this problem?
What I learned about GPU utilization, and where I would like to be corrected. "Utilization" turns out to be three different numbers; a public production trace of 143 GPU workers where two-thirds of one-minute samples record a duty cycle of exactly zero; why the operating system that saved CPUs from this has no GPU equivalent; the "there are no idle GPUs" objection and why it changes the question rather than the measurement; and six questions only people who run real clusters can answer. With a glossary that spells out MFU, HFU, SM occupancy, and throttle reasons.
Read on MediumComing next
The pieces below are in progress and will be published here once the underlying numbers are sourced and checked:
- Throttle-to-dollars. What a clock-throttle bitmask is worth in GPU-hours, worked from a real fleet's cost basis.
- Reading
DCGM_FI_DEV_CLOCK_THROTTLE_REASONS. The bitmask, bit by bit, and which reasons actually cost throughput. - The AMD gap. What
amd-smiexposes, what it does not, and why MI300X-class fleets are the least-studied hardware in the market.
If you run GPU infrastructure and any of this described your fleet a little too accurately, send 24 hours of metrics and get a written analysis back. No agent, no access, no NDA.