19

I am trying to understand why GPUs consume so much power. For example, the max power consumption of the P100 GPU is 250W.

As I understand power is measures in watts as current x voltage. Given a fixed power source (i.e. voltage), I would assume that GPUs draw a lot of current. If I understand this correctly, why does a GPU draw so much current?

I think GPUs mostly consist of transistors? So why does the configuration of transistors in a GPU lead to more power being consumed than in a CPU?

Thanks!

Luke Taylor
  • 375
  • 1
  • 2
  • 6
  • 18
    TU106 has 10 billion transistors. If each transistor needs 10 nanoamps, that's 100A. – J... Jul 23 '20 at 00:54
  • 11
    A GPU is a lot bigger than a CPU, that's the first clue. – Mast Jul 23 '20 at 06:57
  • 4
    Because they do a lot in a short time? A modern GPU is most likely the most powerful special purpose computer available to mortal men these days. – Thorbjørn Ravn Andersen Jul 23 '20 at 08:33
  • 6
    Essentially the reasons for GPUs power consumption are [the same as for CPUs](https://electronics.stackexchange.com/questions/258724/why-do-cpus-need-so-much-current). Top GPUs are bigger than CPUs you compare them with, and thus need more power. – Dmitry Grigoryev Jul 23 '20 at 09:09
  • @ThorbjørnRavnAndersen Colossus CG2 chips are sold to mortals too. – Dmitry Grigoryev Jul 23 '20 at 09:13
  • @DmitryGrigoryev I would venture that more mortalt buy graphics cards than that one. – Thorbjørn Ravn Andersen Jul 23 '20 at 09:17
  • 3
    Note that high-end CPUs are just as power-hungry as high-end GPUs. Current Ryzen Threadripper CPUs are [rated at around 280 W](https://en.wikipedia.org/wiki/List_of_AMD_Ryzen_microprocessors#Zen_2_based). – TooTea Jul 23 '20 at 17:02
  • The premise of the question is unclear. You're asking GPUs consume so much power _compared to what_? An average CPU on an average workload? A CPU performing the same rate of calculations as a GPU? Something else? – Nayuki Jul 25 '20 at 03:15
  • 1
    Unfortunately, I can't add an answer to this due to it being protected, but the short answer is: the better question is why they use so _little_ power. GPUs are easily the most power-efficient mass-market computation device in terms of power per FLOPS. A high-end CPU has a few dozen cores. A high-end GPU has _several thousand_. And the answer to _that_ question is "because their cores are very simple relative to CPU cores and designed to be very good at doing the same floating-point math on large data sets and not very good at anything else." You pretty much need an FPGA or ASIC to improve. – reirab Jul 25 '20 at 06:20
  • (If this gets unprotected, feel free to ping me and I'll try to write up a proper answer not in a comment.) – reirab Jul 25 '20 at 06:32

5 Answers5

39

A GPU is basically a LOT of simplified CPUs in parallel. Each of them is not as capable and flexible as a real CPU, but there are thousands of them to give this massive parallel computing performance.

But this also means that it takes many billions of transistors to build a modern GPU. And for logic chips we use FETs, so with every clock cycle all the billions of gate capacitances have to be charged and discharged. This is where the large amount of power is going.

jusaca
  • 8,749
  • 4
  • 36
  • 58
  • Depending on what computations are running, not every transistor switches every cycle. (e.g. some execution unit is probably idle if another one is busy, but usually one or the other is switching so the average fraction switching is likely similar). And there's also leakage current (static power) from that many transistors, including ones that rarely switch (e.g. in cache arrays). Only low-power idle states that fully cut power to parts of the chip can avoid that, and it takes time to wake up from such states so they can't be used too aggressively. Clock-gating helps for short idles... – Peter Cordes Jul 23 '20 at 04:08
  • 4
    But yes, +1, this is a useful simplification of the details. Giant chip runs hot because even at a lower frequency than CPUs to maximize computation per energy (instead of single-threaded CPU performance), it still takes some minimum, and summed up that's a lot. – Peter Cordes Jul 23 '20 at 04:10
  • @PeterCordes are we actually anywhere close to the theoretical limit yet? – John Dvorak Jul 23 '20 at 17:23
  • 1
    @John: Of energy/computation? Hell no, *many* orders of magnitude. But for any given silicon process and chip design, there's some voltage and frequency combination that minimizes energy / computation, rather than minimizing time / computation. Below that point you're wasting more energy on leakage current than you save on dynamic power, perhaps because you can't lower voltage any more without errors. GPUs aim for closer to that operating point and just spend more transistors, CPUs aim for higher frequencies because they need to deal with problems that don't parallelize nearly perfectly. – Peter Cordes Jul 23 '20 at 17:43
  • @JohnDvorak Regarding the theoretical limit the term to check is "Landauer's principle". And as Peter said, we have a long way to go before we approach that. – jusaca Jul 23 '20 at 18:59
8

This question asks why a faster clock in a given CPU requires more power, and has a number of really good answers.

Take those answers, and add to them the fact that the power consumption is a consequence of not just how fast the logic involved is switching, but how much of it there is -- and a GPU has lots and lots of logic, all of it switching madly when it's hard at work.

TimWescott
  • 44,867
  • 1
  • 41
  • 104
7

GPUs consume a lot of power because they have a large number of transistors switching at high frequency.

Relative to a high end GPU, CPUs usually have many fewer transistors switching at any time and so do not require as much power. This is not always true, you can get low power GPUs and enormous server CPUs with very high power requirements.

user1850479
  • 14,842
  • 1
  • 21
  • 43
7

A GPU is a many, smaller processors operating simultaneously in parallel.

Each processor contains a lot of transistors. Many processors in parallel means even more transistors.

Every time each transistor switches, it must charge or discharge a parasitic capacitance inside the transistor. So the more often a transistor switches, the more often this capacitance is charged/discharged which means the more power is consumed. GPUs run fast so they switch very often.

So the fact that GPUs contain many transistors switching at the same time (because it's not just one processor operating at any one instant, its many processors operating at any one instant) and switching very often means very high currents to charge and discharge all those parasitic capacitors simultaneously.

A normal processor is more complex and has more transistors than than any single processor inside a GPU since it is has higher functionality, but it is also doing only one or two things at a time so only a small fraction of the transistors are switching at any point in time.

DKNguyen
  • 54,733
  • 4
  • 67
  • 153
6

Note: This is intended as a short comment on the answer posted by @user1850479 (Why does GPU consume so much power?); the only reason it exists as an "answer" is that posting a comment requires 50 reputation, which I do not have. If anyone wants to help out by copying its contents to a (true) comment in the appropriate place, it's perfectly fine by me and this post can thus be deleted. No attribution nor acknowledgement is necessary. License: CC0 Thanks. :)

It's also worth noting that CPUs executing vector instructions (SSE, AVX variants) can draw more power than they otherwise would just executing standard instructions. This can be observed by running 3D programs and forcing them to use software rasterizers -- in essence, forcing the CPU to take on a GPU-like workload -- such as WARP or SwiftShader, which usually make heavy use of vector instructions. Intel's processors even have AVX-specific power throttling functionality that kicks in to keep the processor within its rated TDP.