To add on, when using GitHub Copilot you can see a distinction between AI “Credits” and Tokens (in the context window for that conversation). One reason is that each Model can architecturally have different tokenizers, so one what one Model considers as one Token is different from another. To normalize the overall compute costs across different Models, the AI Credit unit is used as an abstraction.

deleted by creator

If I may ask, which Quantisation are you running it on? I’m on an Apple setup with oMLX and am still experimenting with finding the right balance

Good.

Growth should be steady and sustainable, not frontloaded

Finally, a true RNG!

O