Right!? I’ll be the first to admit I don’t understand context, windows, or whatever the limit is, but I can’t understand how these large language models have probably exabytes worth of data hoovered up from all over but they can’t remember what I told it either a minute ago or in another chat. I’ve taken to trying to get it to write markdown files to help its own memory, but I don’t know if those are counting against me for context or whatever either.
Not to pile on, but my real grape is the nanny nature of some of these, especially nannythropic. I can’t tell of our times they told me outright that “I won’t do XYZ. This is a hard limit you can’t get around it.” And I’m sitting here like dude you just created that test record to verify the piping you can’t go back through and delete it because it’s a record delete? Le sigh…
The other chat is really f****** easy. Every chat is a unique instance.
Go get a piece of paper and write a word on it now go get another piece of paper and write a word on that one.
Do both piece of papers have both words on them? No, they don’t. They’re unique instances.
If you don’t load previous information into the new chat, then it’s not going to have that information to reference. You’re resetting to zero every single time.
If you are using an llm that doesn’t have this preset up as a built-in function, then you’re using a s***** LLM. You shouldn’t have to set this up yourself at this point.
The only thing you should have to do is tell it what you want in the memory. It should have a preset up system to be able to handle the rest itself.
They key thing to know is that they don’t actually remember anything. They are completely stateless. Every question, every reply, every interaction needs to have the entirety of the conversation history fed back in to it. That’s actually what the “context” is. It would be like talking to a person, and every time you spoke you had to repeat everything you’ve ever said to them (and what they said to you!) before you could say something new. It’s insanely inefficient and why they require data centers the size of a small city. “Agents” are even worse, because they talk to themselves and each other in loops, but still have to repeat that context for every single interaction.
Right!? I’ll be the first to admit I don’t understand context, windows, or whatever the limit is, but I can’t understand how these large language models have probably exabytes worth of data hoovered up from all over but they can’t remember what I told it either a minute ago or in another chat. I’ve taken to trying to get it to write markdown files to help its own memory, but I don’t know if those are counting against me for context or whatever either.
Not to pile on, but my real grape is the nanny nature of some of these, especially nannythropic. I can’t tell of our times they told me outright that “I won’t do XYZ. This is a hard limit you can’t get around it.” And I’m sitting here like dude you just created that test record to verify the piping you can’t go back through and delete it because it’s a record delete? Le sigh…
The other chat is really f****** easy. Every chat is a unique instance.
Go get a piece of paper and write a word on it now go get another piece of paper and write a word on that one.
Do both piece of papers have both words on them? No, they don’t. They’re unique instances.
If you don’t load previous information into the new chat, then it’s not going to have that information to reference. You’re resetting to zero every single time.
If you are using an llm that doesn’t have this preset up as a built-in function, then you’re using a s***** LLM. You shouldn’t have to set this up yourself at this point.
The only thing you should have to do is tell it what you want in the memory. It should have a preset up system to be able to handle the rest itself.
They key thing to know is that they don’t actually remember anything. They are completely stateless. Every question, every reply, every interaction needs to have the entirety of the conversation history fed back in to it. That’s actually what the “context” is. It would be like talking to a person, and every time you spoke you had to repeat everything you’ve ever said to them (and what they said to you!) before you could say something new. It’s insanely inefficient and why they require data centers the size of a small city. “Agents” are even worse, because they talk to themselves and each other in loops, but still have to repeat that context for every single interaction.
Please do pile on. I agree with everything you said there. The whole point of these posts is so people can pile it on.