• sushibowl@feddit.nl
    link
    fedilink
    arrow-up
    0
    ·
    5 months ago

    If you amortize training costs over all inference uses, I don’t think 1000MW is too crazy. For a model like GPT3 there’s likely millions of inference calls to split that cost between.

    • AliasAKA@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      5 months ago

      Sure, and I think that these may even be useful and it warrants the cost. But it is to just say that this still isn’t simply running a couple light bulbs or something. This is a major draw on the grid (but likely still pales in comparison to crypto farms).

      Note that most people would be better off using a model that’s trained for a specific task. For example, training image recognition uses vastly less energy because the models are vastly smaller, but they’re exceedingly excellent at image recognition.