I know AI/LLM hate is strong here, so this is going to get some blow back. But there’s a lot of Linux folk on here, so let me frame it this way…
My understand of the Linux/unix design philosophy is building small, efficient programs that do a limited set of tasks very well and that can be strung together with other programs that do other tasks very well. This is in opposition to the " be everything" program concept of Windows and Microsoft Office Suite. At least this is how would describe the difference to non technical friends: Nothing you think of as your OS in windows is actually what Linux is replacing. You’re getting the Linux kernel packaged up in a distro that combines a bunch of smaller pieces (file explorer, window manager, etc) that you can still customize from there.
When I look at the approach to AI, I see the same thing. I’ve dabbled enough in ML/LLMs to know that LLMs are effectively very fancy next word predictors or for the case of image/video GenAI, next pixel predictors. As others have said countless times, there’s no consciousness or understanding of the context, but you can ask it things in natural language and it will try to produce whatever you asked for in the same app regardless of context.
From a science project standpoint, this is cool, but it doesn’t seem scalable or consistently reproducable and the energy use and easily found blunders seem to support that thought.
So, my question is why is no one building AI with a Linux philosophy? Small purpose built ML models with a language processing/triage model on top? Oh this person has a question about history, send them to the history module. This person wants to edit a photo, send them to the photo editing module. Then let those modules dig deeper from there. That’s how we do customer service with real people after all. With this way we could refine each specialization individually instead of having a giant model that consumes tons of resources and is error prone.
Accessibility to profit.
ML is more difficult, even if not impossible, to package in a way that is easily consumable by those with the capacity to pay for services.
I definitely see that. Like android or iOS, getting people in your walled garden for one thing creates dependency and the payees are “saving” by only having to pay for 1 tool. It just feels like it would be more useful and less energy intensive as individual tools than an everything machine
That’s the Unix philosophy you are describing, not Linux. Linux does not follow that, it’s just a common practice among programmers. See linux is a monolithic kernel containing all drivers for all hardware, newer example is systemd
For sys admin tasks LLMs already have a tendency to write a script to carry out what is asked of it. We lean into that for exactly the reasons you give
I mean people are. All the time. They just don’t get the attention things like lmk have.
For example, SAM 3 exactly what I think you are asking, but for images.
But there is another point in here about how"actually, just bigger model better" and that’s the thing with transformers. Them basically becoming chat bots through clever training and massive size and training datasets wasnt expected. You don’t get that behavior from much smaller transformers. And so there was an apparently emergent phenomena in this case. A small network isn’t going to do what you think it is going to do precisely because it’s over constrained.
Thanks, I think this is what I’m getting at. Is there an inherent advantage to all in one over modular? And it sounds like they’re is. I know over constraining is an issue with training and there is no scenario with ML or LLM where you get to 100% accuracy. It’s just not the point of the technology. But I could focus on getting an image editing tool 95-99% of the way there and test that vs. having that functionality bundled up with everything else and potentially have that function suffer as we improve another area. If a bigger transformer is benefiting from the other areas of expertise, that is interesting. I still believe you have to hit a point of diminishing returns where more bigger no longer equals more better
So I have a book on my shelf on complex systems analysis and that might be a place to start, but this concept of emergent properties in complex systems isn’t a new one, and it’s well established in complex systems theory, and especially true in network and graph theory.
Basically, complex systems, and especially networked systems, develop different emergent properties as they scale.
There are examples like Spleeter, it separates music into individual tracks (drums, vocals, piano, guitar etc) Based on its training model.
But for marketing a paid product people expect one thing does it all.
Is that what mcp is? For tool integration, an mcp connection allows the LLM to ask a tool questions. Every tool vendor is integrating an mcp server.
For example if I ask an LLM about a bug report it may have no way of figuring it out. But since I configured an mcp service to our ticketing system, my LLM can ask that ticketing system about the bug and get an exact answer
There are plenty of small models that you can run locally, and they can be fine tuned on different types of content.
