In theory the “reasoning” part doesn’t need a knowledge update.
Cost/benefit maybe the knowledge update only needs something every 5 years.
Basically I can see a world where inference is provided and consumed at cost. I can’t see one where customers are willing to cough up billions for a slight increase in the knowledge base.
I’m comparing the whole of human knowledge to 5 years of additional facts. Most useful things are known. An llm doesn’t need to know the current US president to invent a biscuit recipe.
That is if you include training. Look at deepseek for an idea of inference costs. At that level AI coding is sustainable (but not worth $2tn).
Why wouldn’t you include training? Training is vital to their product.
Is it? It’s vital to the tech giants marketing and trillion dollar valuation. But I don’t think it’s vital to most of their clients anymore.
If you want it’s “knowledge” to be up to date, the training never ends.
In theory the “reasoning” part doesn’t need a knowledge update.
Cost/benefit maybe the knowledge update only needs something every 5 years.
Basically I can see a world where inference is provided and consumed at cost. I can’t see one where customers are willing to cough up billions for a slight increase in the knowledge base.
5 years ago we were mid-pandemic. That’s a big knowledge gap.
I’m comparing the whole of human knowledge to 5 years of additional facts. Most useful things are known. An llm doesn’t need to know the current US president to invent a biscuit recipe.