Imagine a customer chatting with your chatbot about their order. They have further questions, but the bot immediately forgets which order they’re talking about, creating a frustrating customer experience. To prevent this, you will want the chatbot to remember the message history, and that’s exactly what this component does.
However, having AI hold on to previous messages costs tokens. Moreover, many uses of Flows do not require memory. That’s why this component is optional and allows you to limit the memory according to your needs while not breaking the bank.
What is the Chat History component?
The Chat history component ensures the chatbot remembers a set number of previous messages. It allows the chatbot to hold a coherent conversation while limiting the number of remembered messages and token usage.
Chat history can be found in the Memory category of components and is optional:
Last Messages Count
It controls how many previous messages the chatbot should remember. The default value is 5, which should cover the conversation around any basic query. If your use case requires the bot to hold a coherent conversation for longer, feel free to increase the limit, but don’t forget to increase the Max Tokens setting, too.
Max Tokens
Each past message is tokenized into smaller units of text, which the model processes to generate relevant and context-aware responses. The Max Tokens setting limits the number of tokens used to remember previous messages. Token usage varies with models, and a single token can be anything from words or subwords to a single character.
Message Type
This drop-down setting allows you to filter which messages to remember: only human messages, only AI messages, or all.
How to connect the Chat History component to your flow
The history must be connected to the LLM and the user input to function properly. However, you’ll notice that the component doesn’t have the incoming handle, only the outgoing one.
The Chat History component’s output type is chat history, and it connects to the components asking for history, which are Prompts and Splitters.
For example, in the Follow-up questions component, the answer is provided by connecting an LLM generator, the context by chat history, and the Input text by human Chat Input.
Frequently Asked Questions
What is the Chat History component?
The Chat history component ensures the chatbot remembers a set number of previous messages. It allows the chatbot to hold a coherent conversation while limiting the number of remembered messages and token usage.
How does the bot behave without chat history?
The bot will not be able to use previous information. Try giving it your name and asking for it in the next message. A bot with no history will apologize for not being able to answer, while a bot with history will simply call you by your name.
What happens when the history limit is reached?
The bot will forget messages past the limit. For example, if your limit is five messages and you state your name in the first one, the bot won’t be able to remember it by the 6th message. However, if the bot calls you by your name later in the conversation, the name will still be within the limits of its memory.
