This tutorial provides a comprehensive guide to implementing conversational AI systems with memory capabilities using LangChain in two main approaches.
1. Creating a chain to record conversations
Creates a simple question-answering chatbot using ChatOpenAI.
Implements a system to store and retrieve conversation history based on session IDs.
Uses RunnableWithMessageHistory to incorporate chat history into the chain.
2. Creating a RAG chain that retrieves information from documents and records conversations
Builds a more complex system that combines document retrieval with conversational AI.
Processes a PDF document , creates embeddings, and sets up a vector store for efficient retrieval.
Implements a RAG chain that can answer questions based on the document content and previous conversation history.
Table of Contents
References
Environment Setup
[Note]
langchain-opentutorial is a package that provides a set of easy-to-use environment setup, useful functions and utilities for tutorials.
You can alternatively set API keys such as OPENAI_API_KEY in a .env file and load them.
[Note] This is not necessary if you've already set the required API keys in previous steps.
# Load API keys from .env file
from dotenv import load_dotenv
load_dotenv(override=True)
True
Creating a Chain that remembers previous conversations
1. Adding Chat History to the Core Chain
Implement MessagesPlaceholder to incorporate conversation history
Define a prompt template that handles user input queries
Initialize a ChatOpenAI instance configured to use the ChatGPT model
Construct a chain by connecting the prompt template, language model, and output parser
Implement StrOutputParser to format the model's response as a string
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser
# Define the chat prompt template with system message and history placeholder
prompt = ChatPromptTemplate.from_messages(
[
(
"system",
"You are a Question-Answering chatbot. Please provide an answer to the given question.",
),
# Note: Keep 'chat_history' as the key name for maintaining conversation context
MessagesPlaceholder(variable_name="chat_history"),
# Format user question as input variable {question}
("human", "#Question:\n{question}"),
]
)
# Initialize the ChatGPT language model
llm = ChatOpenAI()
# Build the processing chain: prompt -> LLM -> string output
chain = prompt | llm | StrOutputParser()
Creating a Chain with Conversation History (chain_with_history)
Initialize a dictionary to store conversation session records
Create the function get_session_history that retrieves chat history by session ID and creates a new ChatMessageHistory instance if none exists
Instantiate a RunnableWithMessageHistory object to handle persistent conversation history
# Initialize an empty dictionary to store conversation sessions
store = {}
# Get or create chat history for a given session ID
def get_session_history(session_ids):
print(f"[Conversation Session ID]: {session_ids}")
if session_ids not in store:
# Initialize new chat history for this session
store[session_ids] = ChatMessageHistory()
return store[session_ids] # Return existing or newly created chat history
# Configure chain with conversation history management
chain_with_history = RunnableWithMessageHistory(
chain,
get_session_history,
input_messages_key="question", # User input variable name
history_messages_key="chat_history", # Conversation history variable name
)
Process the initial input.
chain_with_history.invoke(
# User input message
{"question": "My name is Jack."},
# Configure session ID for conversation tracking
config={"configurable": {"session_id": "abc123"}},
)
[Conversation Session ID]: abc123
'Hello Jack! How can I help you today?'
Handle Subsequent Query.
chain_with_history.invoke(
# User follow-up question
{"question": "What is my name?"},
# Use same session ID to maintain conversation context
config={"configurable": {"session_id": "abc123"}},
)
[Conversation Session ID]: abc123
'Your name is Jack.'
2. Implementing RAG with Conversation History Management
Build a PDF-based Question Answering system that incorporates conversational context.
Create a standard RAG Chain, ensuring to include {chat_history} in the prompt template at step 6.
(step 1) Load PDF documents using PDFPlumberLoader
(step 2) Segment documents into manageable chunks with RecursiveCharacterTextSplitter
(step 3) Create vector embeddings of text chunks using OpenAIEmbeddings
(step 4) Index and store embeddings in a FAISS vector database
(step 5) Implement a retriever to query relevant information from the vector database
(step 6) Design a QA prompt template that incorporates conversation history , user queries, and retrieved context with response instructions
(step 7) Initialize a ChatOpenAI instance configured to use the GPT-4o model
(step 8) Build the complete chain by connecting the retriever, prompt template, and language model
The system retrieves relevant document context for user queries and generates contextually informed responses.
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import PDFPlumberLoader
from langchain_community.vectorstores import FAISS
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import PromptTemplate
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory
from operator import itemgetter
loader = PDFPlumberLoader("data/A European Approach to Artificial Intelligence - A Policy Perspective.pdf")
docs = loader.load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=50)
split_documents = text_splitter.split_documents(docs)
embeddings = OpenAIEmbeddings()
vectorstore = FAISS.from_documents(documents=split_documents, embedding=embeddings)
retriever = vectorstore.as_retriever()
prompt = PromptTemplate.from_template(
"""You are an assistant for question-answering tasks.
Use the following pieces of retrieved context to answer the question.
If you don't know the answer, just say that you don't know.
#Previous Chat History:
{chat_history}
#Question:
{question}
#Context:
{context}
#Answer:"""
)
llm = ChatOpenAI(model_name="gpt-4o", temperature=0)
chain = (
{
"context": itemgetter("question") | retriever,
"question": itemgetter("question"),
"chat_history": itemgetter("chat_history"),
}
| prompt
| llm
| StrOutputParser()
)
Implementing Conversation History Management
Initialize the store dictionary to maintain conversation histories indexed by session IDs, and create the get_session_history function to retrieve or create session records
Create a RunnableWithMessageHistory instance to enhance the RAG chain with conversation tracking capabilities, handling both user queries and historical context
# Dictionary for storing session records
store = {}
# Retrieve session records by session ID
def get_session_history(session_ids):
print(f"[Conversation Session ID]: {session_ids}")
if session_ids not in store:
# Initialize new ChatMessageHistory and store it
store[session_ids] = ChatMessageHistory()
return store[session_ids]
# Create RAG chain with conversation history tracking
rag_with_history = RunnableWithMessageHistory(
chain,
get_session_history, # Session history retrieval function
input_messages_key="question", # Template variable key for user question
history_messages_key="chat_history", # Key for conversation history
)
Process the first user input.
rag_with_history.invoke(
# User query for analysis
{"question": "What are the three key components necessary to achieve 'trustworthy AI' in the European approach to AI policy?"},
# Session configuration for conversation tracking
config={"configurable": {"session_id": "rag123"}},
)
[Conversation Session ID]: rag123
"The three key components necessary to achieve 'trustworthy AI' in the European approach to AI policy are: (1) compliance with the law, (2) fulfillment of ethical principles, and (3) robustness."
Execute the subsequent question.
rag_with_history.invoke(
# Request for translation of previous response
{"question": "Please translate the previous answer into Spanish."},
# Session configuration for maintaining conversation context
config={"configurable": {"session_id": "rag123"}},
)
