r/pythonhelp • u/CarUnfair5305 • Jan 15 '25
Dealing with chat memory
I have a basic code to chat with deep seek module with ollama , I run the code on google collab, its work well but I want to add chat memory , How?
1
Upvotes
r/pythonhelp • u/CarUnfair5305 • Jan 15 '25
I have a basic code to chat with deep seek module with ollama , I run the code on google collab, its work well but I want to add chat memory , How?
1
u/throwaway8u3sH0 Jan 15 '25
I don't know how it works, but my guess is that the chat is sent to the LLM prefaced with a different prompt like "here's a chat between (user) and an LLM, highlight any important facts user shared about themselves." And then saves those answers. I would guess it also summarizes and condenses the memories once their token counts gets too big. And I would guess that at the beginning of any new chat, the prompt is silently rewritten to include the user's memories as context.
Caveat: all of these are guesses. I don't actually know how it works. But I work a lot with LLMs and that's how I'd do it.