Locally run AI should be the standard. All you have to do is download a .gguf file from hugging face (around 8 gig) and load it into Koboldcpp’s GUI… Or AnythingLLM if Kobolcpp is too technical for you.
Some models run faster, others run slower (depending on hardware, model size, and structure). Models like LFM2.5 for instance runs pretty fast because it’s more like 3 small models cowriting responses.
Locally run AI should be the standard. All you have to do is download a .gguf file from hugging face (around 8 gig) and load it into Koboldcpp’s GUI… Or AnythingLLM if Kobolcpp is too technical for you.
Some models run faster, others run slower (depending on hardware, model size, and structure). Models like LFM2.5 for instance runs pretty fast because it’s more like 3 small models cowriting responses.