In the evolving world of artificial intelligence, ChatGPT has emerged as one of the most revolutionary tools humanity has ever created. But at times it can be a bit frustrating! As you delve deeper into extended conversations, you’re going to encounter some annoying messages from the AI and you’ll probably come across a prompt suggesting you start a new conversation due to length constraints. This will say “The conversation is too long, please start a new one.”
Understanding ChatGPT’s Memory Limitations
At the heart of ChatGPT’s operation is a token-based memory system. Each token represents a chunk of text, either a single character or a word. ChatGPT’s memory is capped at 32,768 tokens. This limitation means that ChatGPT can only retain the most recent 32,768 tokens in a conversation, which translates roughly to the last 10,000 words shared between you and the AI.
This really isn’t a lot when you really want to try to get into a topic! I often reach this limit when I am trying to figure something out. Especially if it’s something that no one else has found the answer to already.
ChatGPT’s Rolling Window Mechanism
ChatGPT employs a rolling window mechanism, akin to a first-in-first-out buffer. As new tokens are generated, the oldest ones are discarded to make room for incoming information. This process is important for the system to maintain the efficiency and performance of the AI. It’s not that ChatGPT is ignoring earlier parts of the conversation; it’s more about prioritising recent interactions to stay relevant and responsive.
If OpenAI didn’t incorporate this it would probably end up costing them an absolute fortune. But it’s bloody annoying when it happens.
Why Start A New ChatGPT Chat?
Starting a fresh chat with ChatGPT is like speaking to someone who has no memory of previous interactions. This approach is beneficial when you wish to discuss an entirely new topic, ensuring that the AI does not inadvertently mix contexts or apply irrelevant information from past conversations to the current one.
Of course, if you’ve put a lot of effort and time into essentially training ChatGPT to understand a topic then when you hit the limit you are going to feel a bit cheated. I pay for ChatGPT 4 and I am constantly hitting the limit, which is starting to grate on me a little bit.
Resuming Old Chats: Keeping Context Intact
Conversely, resuming an old chat allows ChatGPT to maintain the context of the conversation, facilitating a smooth continuation without the need to reiterate previous details. This feature is particularly useful when you are in the midst of a detailed or complex topic and wish to pick up where you left off. But it doesn’t help in this scenario because you’ll have already hit the memory limit of the chat. What you can do is copy the conversation by clicking the Share button in the top right corner and then copying the content, adding it to Word or Google Docs and then downloading it and uploading it back to ChatGPT in a new chat.
The only problem is you really have to make it read the document you’ve shared with it. Otherwise it just kind of skims it and doesn’t really take it all in. It’s a pain in the neck and nowhere near as good or useful as it used to be.
Another workaround for the conversation is too long message is to ask ChatGPT to summarise the conversation every few thousand words. You’ve just got to be on it because sometimes it’s just not useful and you can spend ages going backwards and forwards trying to get it to summarise things.
Before you even start getting ChatGPT to answer prompts, another good piece of advice is to ask it to write concisely and not comprehensively. ChatGPT can be really wordy when it chooses to be, and this can really burn through your memory limit.
Trying MemoryGPT And Other Memory-based Solutions
I’ve begun experimenting with MemoryGPT and exploring other innovative memory-based solutions. These tools are essential in addressing the memory limitations of standard ChatGPT, providing external storage and retrieval mechanisms for extended conversation content.
MemoryGPT is designed to adaptively remember and integrate relevant parts of past conversations. Its approach, which uses embedding and indexing of previous interactions, allows for a more contextually rich and personalised chat experience. This method significantly extends the memory reach of ChatGPT, enabling it to recall and build upon past dialogues effectively.
Alongside MemoryGPT, there are other notable solutions like Redis-based ChatGPT Memory and projects utilising vector databases for memory enhancement. These systems store historical user interactions externally and use advanced techniques to fetch contextually relevant information for ongoing conversations. For instance, the Redis vector database approach caches interactions and employs semantic search to retrieve the most appropriate historical data. This not only aids in overcoming the token limits of ChatGPT but also adds a layer of sophistication to AI conversations by ensuring that relevant past interactions are not lost.
I like what I’ve seen so far with MemoryGPT but it’s still early days and it’s definitely in beta at the moment. So we will have to see where it goes.
Striking The Balance With ChatGPT
The key is finding a balance between starting new chats and revisiting old ones. If you’re switching topics completely, a new chat is advisable. However, for minor shifts within a related theme or to continue an unfinished conversation, resuming the old chat is more practical. Time gaps between chats also play a role; a short break might not necessitate a new chat, but after a lengthy period, starting fresh might be more effective.
ChatGPT’s Learning Curve And User Adaptation
As users interact with ChatGPT, they tend to develop their own rhythm for managing conversations. Understanding ChatGPT’s limitations and experimenting with different approaches can enhance the quality of interactions. Remember, ChatGPT is also continually learning and adapting, which is part of the journey in AI communication.
Despite its memory constraints, ChatGPT is a groundbreaking tool in AI. It’s capable of discussing vast ranges of topics, recalling context within its token limit and expressing empathy. Understanding how to navigate its limitations can lead to more productive, insightful and entertaining conversations.
The message to start a new conversation in ChatGPT arises due to its token-based memory limitations. By managing new and old chats effectively, you can optimise your experience with this revolutionary AI tool. Whether it’s discussing complex topics or exploring new ideas, understanding the mechanics of ChatGPT’s memory system is crucial for a seamless and engaging interaction.
