How to Train a Chatbot Using RAG and Custom Data

?

RAG, which stands for Retrieval-Augmented Era, describes a course of by which an LLM (Massive Language Mannequin) could be optimized by coaching it to tug from a extra particular, smaller data base slightly than its large unique base. Usually, LLMs like ChatGPT are educated on the complete web (billions of knowledge factors). This implies they’re liable to small errors and hallucinations.

Right here is an instance of a scenario the place RAG might be used and be useful:

I need to construct a US state tour information chat bot, which accommodates common details about US states, reminiscent of their capitals, populations, and most important vacationer sights. To do that, I can obtain Wikipedia pages of those US states and practice my LLM utilizing textual content from these particular pages.

Creating your RAG LLM

One of the vital fashionable instruments for constructing RAG programs is LlamaIndex, which:

• Simplifies the combination between LLMs and exterior knowledge sources
• Permits builders to construction, index, and question their knowledge in a approach that’s optimized for LLM consumption
• Works with many forms of knowledge, reminiscent of PDFs and textual content information
• Helps assemble a RAG pipeline that retrieves and injects related chunks of knowledge right into a immediate earlier than passing it to the LLM for era

Obtain your knowledge

Begin by getting the info you need to practice your mannequin with. To obtain PDFs from Wikipedia (CC by 4.0) in the suitable format, be sure you click on Print after which “Save as PDF.”

Don’t simply export the Wikipedia as a PDF — Llama received’t just like the format it’s in and can reject your information.

For the needs of this text and to maintain issues easy, I’ll solely obtain the pages of the next 5 fashionable states: 

• Florida
• California
• Washington D.C.
• New York
• Texas

Be sure that to avoid wasting these all in a folder the place your mission can simply entry them. I saved them in a single referred to as “knowledge”.

Get obligatory API keys

Earlier than you create your customized states database, there are 2 API keys you’ll must generate.

• One from OpenAI, to entry a base LLM
• One from Llama to entry the index database you add customized knowledge to

After getting these API keys, retailer them in a .env file in your mission. 

#.env file
LLAMA_API_KEY = ""
OPENAI_API_KEY = ""

Create an Index and Add your knowledge 

Create a LlamaCloud account. When you’re in, discover the Index part and click on “Create” to create a brand new index.

An index shops and manages doc indexes remotely to allow them to be queried by way of an API without having to rebuild or retailer them regionally.

Right here’s the way it works:

1. Once you create your index, there might be a spot the place you’ll be able to add information to feed into the mannequin’s database. Add your PDFs right here.
2. LlamaIndex parses and chunks the paperwork.
3. It creates an index (e.g., vector index, key phrase index).
4. This index is saved in LlamaCloud.
5. You possibly can then question it utilizing an LLM via the API.

The subsequent factor you want to do is to configure an embedding mannequin. An embedding mannequin is the LLM that may underlie your mission and be liable for retrieving the related data and outputting textual content.

Once you’re creating a brand new index you need to choose “Create a brand new OpenAI embedding”:

Once you create your new embedding you’ll have to offer your OpenAI API key and identify your mannequin.

After getting created your mannequin, go away the opposite index settings as their defaults and hit “Create Index” on the backside.

It could take a couple of minutes to parse and retailer all of the paperwork, so ensure that all of the paperwork have been processed earlier than you attempt to run a question. The standing ought to present on the suitable facet of the display screen once you create your index in a field that claims “Index Recordsdata Abstract”.

Accessing your mannequin by way of code

When you’ve created your index, you’ll additionally get an Group ID. For cleaner code, add your Group ID and Index Identify to your .env file. Then, retrieve all the mandatory variables to initialize your index in your code:

index = LlamaCloudIndex(
  identify=os.getenv("INDEX_NAME"), 
  project_name="Default",
  organization_id=os.getenv("ORG_ID"),
  api_key=os.getenv("LLAMA_API_KEY")
)

Question your index and ask a query

To do that, you’ll must outline a question (immediate) after which generate a response by calling the index as such:

question = "What state has the very best inhabitants?"
response = index.as_query_engine().question(question)

# Print out simply the textual content a part of the response
print(response.response)

Having an extended dialog along with your bot

By querying a response from the LLM the best way we simply did above, you’ll be able to simply entry data from the paperwork you loaded. Nevertheless, if you happen to ask a comply with up query, like “Which one has the least?” with out context, the mannequin received’t bear in mind what your unique query was. It’s because we haven’t programmed it to maintain monitor of the chat historical past.

With a view to do that, you want to:

• Create reminiscence utilizing ChatMemoryBuffer
• Create a chat engine and add the created reminiscence utilizing ContextChatEngine

To create a chat engine:

from llama_index.core.chat_engine import ContextChatEngine
from llama_index.core.reminiscence import ChatMemoryBuffer

# Create a retriever from the index
retriever = index.as_retriever()

# Arrange reminiscence
reminiscence = ChatMemoryBuffer.from_defaults(token_limit=2000)

# Create chat engine with reminiscence
chat_engine = ContextChatEngine.from_defaults(
    retriever=retriever,
    reminiscence=reminiscence,
    llm=OpenAI(mannequin="gpt-4o"),
)

Subsequent, feed your question into your chat engine:

# To question:
response = chat_engine.chat("What's the inhabitants of New York?")
print(response.response)

This provides the response: “As of 2024, the estimated inhabitants of New York is nineteen,867,248.”

I can then ask a comply with up query:

response = chat_engine.chat("What about California?")
print(response.response)

This provides the next response: “As of 2024, the inhabitants of California is 39,431,263.” As you’ll be able to see, the mannequin remembered that what we have been asking about beforehand was inhabitants and responded accordingly.

Conclusion

Retrieval Augmented Era is an environment friendly solution to practice an LLM on particular knowledge. LlamaCloud affords a easy and simple solution to construct your individual RAG framework and question the mannequin that lies beneath.

The code I used for this tutorial was written in a pocket book, but it surely may also be wrapped in a Streamlit app to create a extra pure backwards and forwards dialog with a chatbot. I’ve included the Streamlit code here on my Github.

Thanks for studying

• Join with me on LinkedIn
• Buy me a coffee to help my work!
• I supply 1:1 knowledge science tutoring, profession teaching/mentoring, writing recommendation, resume critiques & extra on Topmate!