Customize Your RAG Chatbot

1. Chatbot Customization Overview

We offer a range of customization options on the "Settings" tab to help you tailor your chatbot, including:

• LLM Endpoint
• LLM parameters such as max tokens, temperature, and the system message
• Chatbot UI settings, such as
  • The "default dialog", or the welcome message of the chatbot
  • Avatar for the chatbot
  • Preset questions and optionally their answers
  • Color scheme for the chatbot
• Updating the access token of your chatbot

Note that after you change any of these settings, please remember to click the green "Update" button at the bottom of the page. Otherwise, the changes will not be saved.

2. LLM Endpoint

You can change the LLM endpoint for the chatbot any time. We provide standard endpoints such as Llama 3.1 70B Instruct and Llama 3 8B Instruct, both hosted by Theta EdgeCloud. GPT-4o-mini is also supported.

You can also use your own LLM endpoint, for example other OpenAI/Anthropic LLMs not included in the standard list, or an LLM model you trained or finetuned on your own.

To do that, click on "Add your own LLM endpoint", and the following modal should pop up. It has four fields:

• LLM Endpoint: Enter the API endpoint of your LLM here. Note that we require the endpoint to be compatible with OpenAI's chat completions API. For example, if you want to use an OpenAI LLM, you can enter https://api.openai.com/v1/completions.
• LLM Model: Here is the model parameter for your OpenAI compatible LLM endpoint. For instance, if you want to use GPT-4o, just enter gpt-4o.
• API Key: Enter the API key for your LLM endpoint. If the endpoint doesn't require an API key, you can leave this field blank. For OpenAI LLMs, just put in your OpenAI API key created here.
• Endpoint Name: This is an optional field for you to specify a name for the custom LLM endpoint.

3. LLM Parameters

You can change LLM parameters such as max tokens, temperature, and system message. Please refer to this article for the meanings of these parameters. In the example below, we set the max token and temperature of our chatbot to 2000, and 0.2, respectively. And the system message (aka. system prompt) is set to the following paragraph:

• "You are a highly knowledgeable and versatile assistant, specializing in all aspects of Theta Network and Theta EdgeCloud. Your deep understanding of these technologies allows you to provide comprehensive, accurate, and detailed assistance on a wide range of related topics. Whether it's explaining the underlying blockchain infrastructure of Theta Network, detailing the functionality of Theta EdgeCloud, or answering specific technical or operational questions, you are equipped to deliver insightful, helpful, and up-to-date information to users. Your goal is to assist users with clarity and precision, ensuring they gain a thorough understanding of the subject matter."

4. Chatbot UI Settings

4.1 Default Dialog

You can set the "Default Dialog", i.e. the welcome message which will display by default when the chat is opened. Below is what we use for our example:

• "Welcome! I’m your helpful assistant here to provide insights and support on all things related to Theta Network and Theta EdgeCloud. Whether you have questions about blockchain, decentralized video streaming, or the technical workings of Theta's infrastructure, I’m here to help. Feel free to ask anything, and let’s explore together!"

4.2 Chatbot Avatar

You can custom the chatbot's avatar by providing the URL of the avatar icon.

4.3 Preset Questions and Answers

In the "Preset questions" section, you can specify several preset questions, which will be displayed at the bottom of the chatbot. You can optionally provide the answers to some of the preset questions. If you leave it blank, the chatbot will generate the answer dynamically for the corresponding question. Below are the preset questions for our example chatbot.

4.4 Chatbot Color Scheme

You can customize the color scheme of your chatbot. We also provide two preset schemes "Dark" and "Light" you can choose with one click.

5. Change the Access Token of the Chatbot

Each chatbot has its own access token for integration purpose (check out Integrate the RAG Chatbot with Your Apps for more details). You can change the access token by simply clicking the refreshing button as shown below.

6. Conclusions

After making the above setting changes, please click the "Update" button at the bottom of the "Settings" tab to save the changes. Next, go back to the "Playground" tab, you should see the chatbot UI has been updated accordingly. For example, the chatbot now has the Theta logo as its avatar, and displays a welcome message as we set earlier. At the bottom of chatbot box, you can also find the preset questions. Click on any of these preset questions and see whether the chatbot can correctly answer based on the documents uploaded to the knowledge base!

Next, let's Integrate the RAG Chatbot with Your Apps!