Guide: Training your AI chatbot and improving answers

This guide explains step by step how to evaluate answers, maintain sources, refine instructions with the prompt, and, if necessary, create FAQ training. The examples and buttons match what you see in the conversation screen.

Quickstart: The 4‑step protocol

1. Evaluate the answer. Click 👍 or 👎 so accuracy is tracked.

2. Check and fix sources. Open the sources under the answer, update the correct page or training, and recrawl if needed.

3. Refine with prompt. If sources are correct but the answer is still wrong, adjust your prompt.

4. Last resort: Improve answer → FAQ. Rewrite the answer only if step 2 and 3 don’t solve it.

Step 1: registering feedback

• Use 👍 / 👎 on each answer. This builds an accuracy percentage per chatbot and helps set priorities.

• If all answers in the conversation are correct, simply mark it as reviewed and you’re finished. 

Step 2: Check and fix sources

Under each answer you see Sources. These are the documents and pages the chatbot used for this answer. This is the first place to start when something goes wrong, because the assistant can only work with the content it has been given. If the sources are outdated, incomplete, or misleading, no amount of prompt tweaking will fix the answer. By ensuring your knowledge base is correct and current, you solve most issues at the root before moving on to instructions or improving answers manually.

1. Open and inspect the source

• Click a source at the bottom to open it. If it’s a webpage you’ll be taken to that page directly; if it’s a document or text training, it will open in the Training documents view.

• For more detailed insights, open Training documents. Here you can see the same information for all training sources, including score, priority, and the Re‑crawl URL button. You can also navigate between multiple sources using the buttons at the bottom.

1. Scenarios and Solutions

Here are the most common problem cases you may encounter when checking sources, along with the recommended way to fix them:

Scenario A: The information in the source is wrong or misleading

Problem: The answer looks incorrect because the underlying document or page contains outdated, inaccurate, or misleading details. Since the chatbot can only reproduce what’s in the source, these errors flow directly into the response.

Solution:

• Update the source page on your website. After making changes, click Re‑crawl URL to fetch only that page again. If you need to update a broader section of your site, restart the crawler to refresh a larger set of pages (slower).

• If you cannot update the source: Remove it from your knowledge base and add a Text training, Document, or FAQ with the correct info.

Scenario B: The content is missing from your knowledge base

Problem: The chatbot cannot provide the right answer because the required content simply doesn’t exist in your current knowledge base. If the knowledge is not stored anywhere, the assistant has nothing to draw from.

Solution:

• Add a new webpage on your site, or

• Upload a new document, or

• Create a Text training or FAQ training with the required content.

Scenario C: The content exists but doesn’t show among the answer links

Problem: The right information is available in your knowledge base, but it isn’t being picked up in the list of sources under the answer. This means the assistant is overlooking a valid source in favor of others with higher priority or relevance.

Solution:

• Increase the priority of the relevant source in Training documents and test again.

• You can also lower priority of other sources to make space for more relevant ones.

Step 3: Refine Instructions with the Prompt

If your sources are in correct, but the answer is still wrong or incomplete, the issue is usually the prompt.

• Use the Change prompt shortcut at the top of the conversation to edit instructions.

• Add global writing and decision rules, e.g. language, tone, which page to reference in certain cases, answer format, disclaimers.

Important Guideline

• The prompt does not control which sources are used. By the time the prompt is applied, the search engine has already selected sources.

• You can instruct the assistant to ignore certain sources in specific situations, but usually it’s better to remove or apply a lower priority for those sources.

Examples of useful prompt rules

• “Always answer in Englisch, short and active, max three paragraphs.”

• “For billing questions, mention our billing e-mail ( billing@example.com )”

• “When the user identifies as X, use format Y.”

Step 4: Use Improve Answer to Create an FAQ (last resort)

Sometimes you will still encounter wrong or incomplete answers even after fixing sources and refining the prompt. In those rare cases, Improve Answer lets you directly rewrite the response and save it as a FAQ training.

With Improve answer you can manually rewrite an answer. This is powerful, but use it sparingly.

• Each Improve answer creates an FAQ training, with a question pattern and the desired answer.

• FAQ’s must be maintained if information changes. Too many FAQ’s make the knowledge base messy, less flexible and will reduce overall performance.

When to use Improve answer/FAQ:

• For recurring cases where you always want the exact same phrasing.

• For policies, disclaimers, standard texts that rarely change.

• As a temporary workaround while sources or site updates are pending.

When NOT to use it:

• If the info changes often.

• If a good source already exists.

• If it can be better enforced via prompt rules.

Working with Training Documents

Open Training documents from the conversation for extra control:

• Score: how relevant the document was for the query.

• Priority: increase or decrease to steer how likely it is to be used.

• Re‑crawl URL: fetch the latest version of a page after editing.

• Open: click through to inspect the document/page itself.

Quality Control and Maintenance

• Mark as Reviewed. Do this once you’ve fixed an issue in a conversation.

• Use tags to label conversations (e.g., “pricing”, “onboarding”, “bugs”).

• Schedule regular clean‑ups:

  • Remove outdated/duplicate docs.

  • Revisit source priorities.

  • Consolidate multiple FAQ’s into one Text training or webpage.

• Versioning: name documents clearly, e.g. Guide – Billing – v2025‑05.

Troubleshooting: Common Issues

1. Chatbot shows outdated prices

• Update the price page/document.

• Click Re‑crawl URL.

• Check for old PDFs with higher priority.

• Temporary fix: Improve answer → FAQ with current prices, remove later.

1. Internal/restricted pages are referenced

• Remove or lower priority in Training documents.

• Add public summary pages instead.

1. Good source exists but wasn’t selected

• Increase priority of the right source.

• Lower generic/less relevant ones.

• Check for overlapping titles/content.

1. Too many Improve answers, fragmented knowledge

• Merge FAQ’s into a Text training or webpage.

• Update the prompt to enforce answer structure.

• Clean up old/duplicate FAQ’s.

Best Practices per Content Type

Webpages

• Keep titles and headings clear and unique. Put keywords in H1/H2.

• Update existing pages instead of creating duplicates.

• After editing: Re‑crawl URL.

PDF/Documents

• Use a clear title and version number.

• Replace outdated files, delete the old ones.

• Ensure text is searchable, not just images.

Text training

• Use for stable internal knowledge not public on your site.

• Keep it concise, with clear headings and links.

FAQ training

• Use for fixed responses that must always sound the same.

• Add representative question.

• Review FAQ’s regularly and merge when possible.

Checklist: Fixing a Wrong Answer

1. Click 👍/👎.

2. Open sources under the answer.

3. In Training documents: check score/priority, use Re‑crawl URL if updated.

4. Apply Scenario A/B/C: fix/replace sources, add missing content, or adjust priority.

5. If sources are fine: refine prompt.

6. As last resort: Improve answer → FAQ.

7. Mark conversation Reviewed, optionally tag.

FAQ

Can I control source selection in the prompt?
No. Source selection happens before the prompt is applied. Use source maintenance and priorities to steer it.

When do I increase priority vs. create FAQ?
Increase priority if the content is correct and should be used more often. Create FAQ if you need exact phrasing or policy texts.

How long does recrawling take?
Re‑crawl URLis usually quick. Full crawls take longer depending on site size.

What if a source no longer exists?
Delete it. By default, deleted sources are not restored unless you change crawlerconfig.

Following this process keeps your Chathive chatbot reliable, up‑to‑date, and well aligned with your audience. Always start with source/content fixes, then refine with the prompt, and use Improve answer/FAQ only as a final step.