If LLMs can’t be made more power efficient, then the technology isn’t ready yet for general use. There may have to be fundamental engineering breakthroughs before what these systems do can be more than a very expensive and inefficient worldwide proof of concept.
Of course, the military and the surveillance industry will still want to keep it, even if no one else can afford it, and we’ll be the ones paying for it.
Well that is an alternative. But I bet each query to a local 27b model takes less energy anyway than the same query to Claude Opus or Fable in some data centre. So it is also a case of improving energy efficiency by running smaller models.
If LLMs can’t be made more power efficient, then the technology isn’t ready yet for general use. There may have to be fundamental engineering breakthroughs before what these systems do can be more than a very expensive and inefficient worldwide proof of concept.
Of course, the military and the surveillance industry will still want to keep it, even if no one else can afford it, and we’ll be the ones paying for it.
someone else replied with what i said
Hm, I don’t see the harm in powering local models with solar.
Well that is an alternative. But I bet each query to a local 27b model takes less energy anyway than the same query to Claude Opus or Fable in some data centre. So it is also a case of improving energy efficiency by running smaller models.