Types of Conversational Memory in Large-Langauge-Models(LLM)

Understand the pros & cons of memory types and improve the Large-Language-Model user experience.

Lavasa City, Pune

Hi there, today i will introduce the different types of Memory available in the LLM space that we can use to enhance the user experience of LLM.

Let’s get started

1. ConversationBufferMemory:

Purpose: To retain the entire conversation history, ensuring full context in a chatbot’s interactions.

When to Use: Ideal when maintaining the full historical context is essential, and memory limitations are not a significant issue.

Advantages:

• Preserves the complete conversation history.
• Enables precise references to previous interactions.
• Maintains contextual understanding throughout the conversation.
• Improves response quality by leveraging the full context.

Disadvantages:

• Higher memory consumption.
• Potential performance issues with large conversation buffers.
• Challenges in scaling for very long conversations.
• Raises privacy concerns due to the storage of the entire conversation history.

from langchain.memory import ConversationBufferMemory
from langchain.agents import load_tools

#load existing tool
tools = load_tools(["llm-math",], llm=llm)
#Appending New Tool------------------
@tool
def get_word_length(word: str) -> int:
    """Returns the length of a word."""
    return len(word)

tools_new = [get_word_length]
tools_new.append(tools[0])

#intialize the LLM
llm = ChatOpenAI(temperature=0.0)
#intialize the memory--------------*******
memory = ConversationBufferMemory() 
#-----------------------------------*******

#Initializing the Agent---------------------
from langchain.agents import AgentType
agent = initialize_agent(tools_new, 
                         llm, max_iterations = 3,
                         agent=AgentType.CONVERSATIONAL_REACT_DESCRIPTION, 
                         verbose=True,
                         memory=memory)
results = agent.invoke("tel me about conversation memory in llm")

#Example 2--------------------------------
agent = create_react_agent(llm, tools_new, prompt) #add a prompt template
#similarly with AgentExecutor
agent_executor = AgentExecutor(
    agent=agent,
    tools=tools,
    verbose=True,
    max_iterations=5,
    memory = memory2,
    #max_execution_time=max_execution_time,
    #early_stopping_method=early_stopping_method,
    handle_parsing_errors=True)
results1 = agent_executor.invoke({"input":"tel me about conversation memory in llm"})

2. ConversationBufferWindowMemory

Well, we have already used this in our previous articles. Here’s the link

LLM Tools and Agents Simplified
Enhance the capabilities of LLMs by allowing them to interact with various tools and data sources, and functions…bobrupakroy.medium.com

Purpose: To efficiently manage memory by retaining recent interactions and preventing token count growth. It stores a specified number (‘k’) of recent messages in the conversation.

When to Use: Suitable when historical context is less important, and controlling memory usage is a priority.

Advantages:

• Optimizes memory usage.
• Reduces token count, lowering memory consumption.
• Maintains context for recent interactions without modification.
• Ensures conversations are current and relevant.

Disadvantages:

• Limited historical context due to the intentional omission of older interactions.
• Potential loss of older information.
• Decreased depth of understanding without the complete conversation history.
• Risk of losing relevance in context as older interactions are dropped.

from langchain.memory import ConversationBufferWindowMemory
memory = ConversationBufferWindowMemory(memory_key="chat_history",
                                  k=3,return_messages = True)

3.ConversationTokenBufferMemory

Purpose: To efficiently manage memory by monitoring token count and avoiding token limitations as it stores recent interactions based on the specified token count.

When to Use: Appropriate when controlling token count is essential for optimal model performance.

Advantages:

• Efficient memory management by considering token length.
• Adjustable buffer size to accommodate different conversation lengths.
• Precise threshold setting for clearing interactions based on token count.
• Enhances overall system performance.

Disadvantages:

• Possible loss of context when interactions are cleared due to token length.
• Challenges in determining the correct token count threshold.
• Difficulty in maintaining long-term context.
• Potential decline in response quality in scenarios requiring high context retention.

from langchain.memory import ConversationTokenBufferMemory 
memory = ConversationTokenBufferMemory(llm=llm, max_token_limit=60)

4.ConversationSummaryMemory

Purpose: To avoid exceeding token count limits by summarizing parts of the conversation. In simple words, it tries to create a summary of conversation snippets to manage token count effectively.

When to Use: Ideal when managing token count efficiently is important, and detailed context is not always required.

Advantages:

• Effective memory management with a condensed conversation history.
• Enhanced processing efficiency for the language model.
• Prevents surpassing token count limits.
• Preserves key information in a summarized format.

Disadvantages:

• Possible loss of detail due to summarization.
• Depends on the accuracy of the summarization process.
• The limited historical context might reduce the depth of understanding.
• Less granularity compared to the original conversation.

from langchain.memory import ConversationSummaryMemory 
memory = ConversationSummaryMemory(llm=llm) #it will use llm to summarize

5.ConversationEntityMemory

Purpose: To track and utilize specific entities or information within a conversation. It is a memory type designed to store and manage conversation entities.

When to Use: Ideal when the goal is to extract and retain specific entities or data points.

Advantages:

• Efficient storage and retrieval of specific conversation entities.
• Focused memory usage tailored to particular information.
• Enhanced understanding of context related to specific entities.
• Improved relevance in responses concerning the stored entities.

Disadvantages:

• Limited to tracking only predefined entities.
• May not be effective for applications that require a broader context.
• Challenges may arise if entities are not clearly defined or change over time.

from langchain.memory import ConversationEntityMemory 
memory = ConversationEntityMemory()

6. VectorStoreRetrieverMemory

Purpose: To enable efficient and rapid retrieval of information through vector representations, facilitating quicker context access.

When to Use: Ideal when fast and accurate context retrieval is a priority.

Advantages:

• Efficient storage and retrieval using vector-based representations.
• Quicker context access compared to traditional text-based memory.
• Well-suited for scenarios where speed is essential.
• Enables similarity-based retrieval of relevant information.

Disadvantages:

• Relies on effective vectorization techniques for precise retrieval.
• May not preserve the fine-grained details found in raw text.
• Effectiveness is highly dependent on the quality of the vector representations.
• Limited usefulness in cases where exact text matching is necessary.

from langchain.memory import VectorStoreRetrieverMemory 
memory = VectorStoreRetrieverMemory()

7.ConversationKGMemory

This type of memory uses a knowledge graph to recreate memory.

from langchain.memory import ConversationKGMemory
memory = ConversationKGMemory(llm=llm)

Thanks again for your time. i hope you enjoyed this. I tried my best to gather details across and simply as much as possible i could.

Next, we will learn PEFT(Perimeter Efficient Fine-Tunning) to tune the model & PAL (Program-aided Language models)

Until then feel free to reach out. Thanks for your time, if you enjoyed this short article there are tons of topics in advanced analytics, data science, and machine learning available in my medium repo. https://medium.com/@bobrupakroy

Some of my alternative internet presences are Facebook, Instagram, Udemy, Blogger, Issuu, Slideshare, Scribd, and more.

Also available on Quora @ https://www.quora.com/profile/Rupak-Bob-Roy

Let me know if you need anything. Talk Soon.

Chill…enjoy machine learning! #Home