How to Build Your Own Chatbot with LangChain

Chatbots have become one of the most valuable components of businesses in recent years. Almost every other website offers a conversational chatbot that you can interact with to resolve your queries. However, a chatbot's features are not restricted to basic conversations; you can extend its functionality by providing it with domain-specific information.

Building your own chatbot has become easier than ever, as multiple publicly available frameworks allow you to do so. One such framework is LangChain, which enables you to use large language models (LLMs) to create chatbots.

This article highlights two different methods that you can follow to create your own LangChain chatbot.

LangChain Chatbot Architecture

Creating a chatbot involves multiple techniques and implementations. This is especially true when you are building a bot that can handle diverse questions and provide accurate answers. For better domain-specific question answers, chatbots are usually built upon Retrieval-Augmented Generation (RAG).

RAG is the process of augmenting LLM with domain-specific knowledge. The public training data limits the capabilities of an LLM by making it too vague about certain topics. This is where RAG comes into play, providing appropriate information and inserting it into the LLM.

You must train your LLM model on data gathered from multiple sources to ensure relevant information is present for each question. With the data coming from various sources, it becomes necessary to store it in a single data repository like a data warehouse. To achieve this, you can use tools like Airbyte to perform data integration.

The key component of a chatbot architecture is its memory, which helps create strong conversation capabilities by better managing chat history. However, optimizing memory and chat history might come with the trade-off of increased latency and complexity.

Build an LLM LangChain Chatbot

Chatbots are becoming essential components of business, enabling you to enhance customer experience and increase efficiency by automating redundant tasks. Let’s explore the steps required to create a basic chatbot with LangChain.

Step 1: Installing Necessary Packages

The first step in building your own chatbot is installing and importing the necessary libraries to execute the code for building a LangChain chatbot. The common Python libraries include:

• ‘os’ to interact with the operating system.
• ‘utils’ that offer helper functions.
• ‘streamlit’ library that creates a web application for the chatbot.
• ‘streaming’ library to stream data between application and source.
• ‘langchain’ to develop an LLM app.

pip install utils, streamlit, streaming, langchain

import os
import utils
import streamlit as st
from streaming import StreamHandler

from langchain.chains import ConversationChain
from langchain_openai import OpenAI

Step 2: Setting up Streamlit

After importing the libraries, you can configure the page title, icon, header, and a display that conveys the chatbot's purpose.

st.set_page_config(page_title="Chatbot", page_icon="💬")
st.header('Basic Chatbot')
st.write('Allows users to interact with the LLM')

Step 3: Defining the Chatbot Class

This step involves creating a chatbot class that contains its key logic. The init function is the constructor that runs every time the chatbot instance is called. It contains the code to synchronize the streamlit session and assign an LLM model to the instance of the class.

class BasicChatbot:

    def __init__(self):
        utils.sync_st_session()
        self.llm = self.configure_llm()

The configure_llm function fetches your OpenAI API key from the environment variable. If the key is not found, it displays a value error message on the screen. Accessing the API key allows you to create an OpenAI LLM instance and assign it to a variable, as done in the code below. You can mention your own LLM model by providing its API key if you prefer not to use OpenAI for bot creation.

    def configure_llm(self):
        api_key = os.getenv("OPENAI_API_KEY")
        
        if not api_key:
            raise ValueError(“OpenAI API key not found in the env variables”)
        
        llm = OpenAI(model_name='gpt-3.5-turbo',
                              temperature=0.7)
        return llm

Using the configured LLM model, the setup_chain method creates a conversation chain—a LangChain function. This chain manages conversation history and context, which results in the production of relevant answers.

    def setup_chain(self):
        chain = ConversationChain(llm=self.llm, verbose=False)
        return chain

The main function orchestrates the chatbot interactions by managing conversation history. It then sets up a conversation chain and gathers your input. Your message gets displayed while the streamlit chat message is created for the bot’s response. StreamHandler maintains real-time updates, and the bot’s response gets added to the session state for history.

    @utils.enable_chat_history
    def main(self):
        chain = self.setup_chain()
        user_query = st.chat_input(placeholder="Ask me anything!")
        if user_query:
            utils.display_msg(user_query, 'user')
            with st.chat_message("assistant"):
                st_cb = StreamHandler(st.empty())
                result = chain.invoke(
                    {"input":user_query},
                    {“callbacks”:[st_cb]}
                )
                response = result["response"]
                st.session_state.messages.append({"role": "assistant", "content": response})
                utils.print_qa(self, user_query, response)

Step 4: Executing the Chatbot Code

Finally, you can execute the chatbot code to create your own assistant that can provide you answers to the prompts you input. Run the code below in a code editor or command line interface (CLI):

if __name__ == "__main__":
    obj = BasicChatbot()
    obj.main()

Live Example

Let’s explore how the LangChain chatbot you just created works. Execute the code in your CLI by replacing ‘your_script’ with the name of the file where you have saved your data.

streamlit run your_script.py

This code will run the streamlit UI on your default web browser, redirecting you to a page as shown below.

In the ‘Ask me anything!’ search box, you can enter your prompt or the question you want the chatbot to answer. Let’s assume you want to know what a chatbot is, you can enter the prompt below to generate a response from the bot.

After entering the prompt, click Enter or click on the execution button on the right of the search box. The chatbot will generate a response and display it on your screen. This is how easy it is to create your own LangChain chatbot.

Additional Considerations

While the LangChain chatbot that you have created works really well, it’s not the only way to build a chatbot. You can utilize other tools, LLM models, and methods to create elegant chatbots that cater to your specific needs.

Another way to develop a chatbot is using a graph database like Neo4j and utilizing retrieval-augmented generation (RAG) with LangChain. The RAG lets you fetch structured and unstructured data from neo4j database, which you can use to train the chatbot. To deploy this LangChain chatbot, you can use FastAPI with Streamlit.

You can consider building a healthcare chatbot with LangChain RAG. This bot will allow you to retrieve patients’ data, insurance information, and nearest hospitals using natural language. Adapting chatbots into the healthcare system is revolutionizing, as it automates day-to-day tasks.

How Airbyte Enhances the Chatbot Creation Process?

In this section, you will explore how leveraging no-code tools like Airbyte can help you easily build chatbots. Airbyte is a data replication and integration tool that simplifies your AI workflow, enabling you to load semi-structured and unstructured data into prominent vector stores. Some of the most common vector stores include Pinecone, Milvus, and Weaviate.

Here are some of the key features Airbyte offers:

• GenAI Workflow Support: Airbyte supports RAG-specific transformation features like chunking and embedding that allow you to transform and store data with a single operation.‍
• Connector Options: It has 350+ pre-built data connectors that you can utilize to extract data from multiple sources into the destination of your choice. Additionally, you can leverage the Connector Development Kit (CDK) to develop custom connectors.‍
• Flexible Deployment Options: Airbyte offers flexibility in terms of deployment options, providing self-hosted, cloud-based, and hybrid methods.

Building a LangChain Chatbot Using Airbyte and Pinecone

This section showcases how you can extract unstructured data from dispersed sources and load it into a vector database, Pinecone. Migrating data into Pinecone will help you prepare it for LLM training. Finally, you will create a chatbot using OpenAI that responds to all your GitHub queries.

Follow the steps below to chat with your data utilizing RAG with LangChain.

• Login to your Airbyte account, and you will be redirected to Airbyte’s dashboard.

• Click on Sources on the left panel, and search for GitHub.

• Select the available GitHub connector. On the next page, authenticate your GitHub account and click on Set up source.

• After configuring the source, select Pinecone connector as the destination.

• Airbyte offers multiple text splitting, embedding, and indexing options that you can access on the Pinecone destination page.

• Configure the connection between the GitHub source and Pinecone destination. Airbyte also allows updates for the vectors with the issues that have changed since the last sync, ensuring the data is up-to-date.

• The final step is to create a chatbot interface using LangChain as an orchestration framework. Follow the code in the next set of steps to create your own chatbot utilizing GitHub issues to answer questions.
• Install and import all the necessary libraries.

pip install pinecone-client langchain openai tiktoken

• Create a Python file chatbot.py and copy/paste the code below:

import os
from langchain.embeddings import OpenAIEmbeddings
from langchain.chains import RetrievalQA
from langchain.vectorstores import Pinecone
import pinecone
from langchain.llms import OpenAI

• Create the simplest version of a chatbot using LangChain's RetrievalQA class, which helps build up the basic interactions.

embeddings = OpenAIEmbeddings()

• Initialize a Pinecone client:

pinecone.init(api_key=os.environ["PINECONE_KEY"], environment=os.environ["PINECONE_ENV"])

• You can define the Pinecode index to organize your vector data:

index = pinecone.Index(os.environ["PINECONE_INDEX"])
vector_store = Pinecone(index, embeddings.embed_query, "text")


qa = RetrievalQA.from_chain_type(llm=OpenAI(temperature=0), chain_type="stuff", retriever=vector_store.as_retriever())


print("Connector development help bot. What do you want to know?")
while True:
    query = input("")
    answer = qa.run(query)
    print(answer)
    print("\nWhat else can I help you with:")

• Provide the necessary OpenAI and Pinecone credentials to run the code.

export OPENAI_API_KEY=...
export PINECONE_KEY=...
export PINECONE_ENV=...
export PINECONE_INDEX=...
python chatbot.py

• Fine-tune your chatbot model by providing a prompt, which helps the model to be more specific according to the use case.

prompt_template = “““You are a question-
answering bot operating on Github issues and 
documentation pages for a product called connector 
builder. The documentation pages document what can be 
done; the issues document future plans and bugs. Use the 
following pieces of context to answer the question at 
the end. If you don't know the answer, just say that you 
don't know; don't try to make up an answer. State where 
you got this information from (and the GitHub issue 
number if applicable), but only do so if you used the 
information in your answer.


{context}


Question: {question}
Helpful Answer:”””
prompt = PromptTemplate(
    template=prompt_template, input_variables=["context", "question"]
)
class ContextualRetriever(VectorStoreRetriever):
    def _get_relevant_documents(self, query: str, *, run_manager):
        docs = super()._get_relevant_documents(query, run_manager=run_manager)
        return [self.format_doc(doc) for doc in docs]


    def format_doc(self, doc: Document) -> Document:
        if doc.metadata["_ab_stream"] == "issues":
            doc.page_content =  f" Excerpt from Github issue: {doc.page_content}, issue number: {int(doc.metadata['number']):d}, issue state: {doc.metadata['state']}"
        return doc
qa = RetrievalQA.from_chain_type(llm=OpenAI(temperature=0), chain_type="stuff", retriever=ContextualRetriever(vectorstore=vector_store), chain_type_kwargs={"prompt": prompt})

The full script can also be found on GitHub.‍

Result:‍

Here’s how your chatbot can respond to GitHub-specific queries; for example, let’s ask your bot a question.

Connector development help bot. What do you want to know?

> Can you give me information about how to authenticate via a login endpoint that returns a session token?‍

Output:‍

You can use the GenericSessionTokenAuthenticator to authenticate through a login endpoint that returns a session token. This is documented in the Connector Builder documentation with an example of how the request flow functions (e.g., metabase). This issue is not closed yet - the feature might not be shipped yet (Github issue #26341).

Following the above steps, you can extract data from GitHub, transform it into a format compatible with LLM training, and use it to train your chatbot. To learn more about how you can chat with your data, refer to Langchain chatbot using Airbyte.

Summary

Through this article, you have understood how to utilize LangChain to create custom chatbots that fulfill your organization’s specific requirements. There are many ways to improve your chatbot’s conversational performance. One such method involves leveraging SaaS-based tools like Airbyte, which can streamline your data integration process and help you extract data from multiple sources.

FAQs

How Can I Stream Only the Final Result from an Agent in Streamlit?

If you are facing a problem while streaming the final result from an agent in Streamlit, you must use the “.write_stream” function. After using this function, you must check if the agent’s final output is of type AgentFinish using the “isInstance” method.

Why Does LangChain's BaseModel Sometimes Output a Copy of the Pydantic Field Description Instead of the Expected Value?

If you don’t want to use Pydantic, you can define schema using TypedDict class. You must ensure to use an updated model instead of an old model. Additionally, you can use Annotated syntax supported by LangChain to specify default option values for the schema.

Why Does EmbeddingStoreContentRetriever Not Output Directly Score in Java LangChain?

In Java LangChain or LangChain4j, EmbeddingStoreContentRetriever returns TextSegment objects based on criteria like minimum score. However, you can add a custom retriever to generate score details.

How to Customize the Chat Format LangChain Uses for My Specific LLM?

Customizing the LangChain chat format involves implementing your own custom chat model or using low-level text-in/text-out classes with string prompts. The Hugging Face Transformers library has built-in methods to perform these functionalities.