SnausagesinaBlanket@lemmy.world to Ask Lemmy@lemmy.world · 2 days agoIs there a currently an accurate way to say how much power per prompt LLMs use?message-squaremessage-square25linkfedilinkarrow-up145arrow-down15
arrow-up140arrow-down1message-squareIs there a currently an accurate way to say how much power per prompt LLMs use?SnausagesinaBlanket@lemmy.world to Ask Lemmy@lemmy.world · 2 days agomessage-square25linkfedilink
minus-squareSnausagesinaBlanket@lemmy.worldOPlinkfedilinkarrow-up2·13 hours agoHow does a wattage meter on my computer measure power used by an LLM server someplace else per prompt?
minus-squarechicken@lemmy.dbzer0.comlinkfedilinkarrow-up2arrow-down1·13 hours agoI have a LLM server on my computer, so I can tell how much electricity it is using this way. It’s not somewhere else is how
minus-squareSnausagesinaBlanket@lemmy.worldOPlinkfedilinkarrow-up2arrow-down5·13 hours agoCool but that is not the question I asked is it then mate.
minus-squarechicken@lemmy.dbzer0.comlinkfedilinkarrow-up4·13 hours agoYou asked if there’s a way to tell how much power LLMs use, you didn’t specify LLMs on a server you don’t have physical access to.
minus-squareSnausagesinaBlanket@lemmy.worldOPlinkfedilinkarrow-up1arrow-down4·11 hours agoIt says LLMs meaning in general so you can take it any way you like.
How does a wattage meter on my computer measure power used by an LLM server someplace else per prompt?
I have a LLM server on my computer, so I can tell how much electricity it is using this way. It’s not somewhere else is how
Cool but that is not the question I asked is it then mate.
You asked if there’s a way to tell how much power LLMs use, you didn’t specify LLMs on a server you don’t have physical access to.
It says LLMs meaning in general so you can take it any way you like.