• 0 Posts
  • 277 Comments
Joined 3 years ago
cake
Cake day: June 30th, 2023

help-circle










  • AI/llm is just a statistical word predictor.

    You could perhaps make the argument that it’s a statistical token predictor, but that is about as useful as boiling down various weather services as “just statistics” or the economy as “just statistics”.

    Making a language model that speaks a language is not that hard, but the world of science underlying how this is done is anything but simple. Saying it’s just statistics is ridiculously reductive like saying your response to this comment is just chemistry. Context driven tokenization, byte level byte pair encoding, RoBERTa, fine tuning methods, direct preference optimization, dataset curation and management, and curriculum learning for targeted performance and memory are things that are being developed and refined very fast (like weekly or monthly breakthroughs sometimes) and with pretty staggering performance increases. It still is not for everyone because power tools injure, but instead of saying “just a statistics engine” say what you really mean “I don’t understand it, but I believe XXX is a bad use case for LLMs.”

    To a lesser extent that any company is “programming AI”. Not in the way that you mean it; curriculum learning, guard rails, and fine running are all extremely indirect. Nobody had their hands specifically in a model’s parameter space directly.