Overview

Sources are the content your widgets search and chat with: your website, documents you upload, answers you write yourself, and more. Open them from Sources in the main menu. You can put several sources on one widget, and choose which ones under Widget sources.

You need the KB Manager or Account Admin role to work with sources.

Source types

Create a source

1. Go to Sources and click Add new source
2. Select the source type and click Next
3. Follow the steps for that type. Each type has its own guide, linked in the table above.

If you have reached your plan’s source limit, you can’t add another source. Delete a source or upgrade your plan to create more.

Scraping process

The crawler goes through the pages of a Web Scrape source, plus your sitemap, to find and scrape all text content. A page can’t be scraped if:

• it isn’t published;
• it has the noindex or nofollow tags (unless Disable Robot Tags is on);
• it isn’t linked from any page and isn’t in the sitemap;
• your site’s robots.txt has a disallow rule for crawlers.

A page that is published but not linked from anywhere can still be scraped: add it under Entrypoints in the source’s Advanced settings.

Sync frequency

After the first crawl, Raffle crawls the website again on the source’s Scrape Plan:

• Weekly: every week;
• Daily: every day, if your plan includes daily sync;
• First crawl only: never again.

Change the plan in the source’s Settings. To get a changed page into Raffle before the next crawl, use Instant Sync or the Watchlist.

Scrape cleaning

When a page is scraped, Raffle removes content that repeats on many pages, such as cookie banners. This is called de-noising.

You decide which part of each page is used. In the source’s Advanced settings, add CSS selectors to the CSS Whitelist to keep only, for example, the article body, or to the CSS Blacklist to leave out breadcrumbs, menus or footers.

Sectioning

Each page is divided into smaller sections. When a visitor searches, Raffle looks for the sections that match best and shows one section per answer as the snippet in the results.

Whitelisting

Some servers only let known visitors in. Then Raffle’s IP addresses must be on your server’s allowlist, which your IT administrators set up. Contact Raffle Support for the list of Raffle IP addresses.

That section is about Raffle reaching your server. For your site allowing the Raffle widget, see Content Security Policy.

Direct access

If crawling your website doesn’t work, you can push the content with an API source instead, or connect a SharePoint site directly.

Incomplete scrape

If a page is missing from a source:

1. Check whether the page has the noindex or nofollow tags. Raffle skips those pages unless Disable Robot Tags is on.
2. Check that the page is in your sitemap, e.g. website.com/sitemap.xml, or linked from another page.
3. Check that the page isn’t excluded by Excluded URL Filters, No Index URL Filters or the pages you excluded during setup.
4. Click Add URL above the list of pages to add that one page, or add it under Entrypoints in Advanced settings so every crawl finds it.

View sources

The Sources page lists your sources with their type, title, scope (Internal or Public), number of pages, when they were added and crawled, and their Status: Ready, or Updating while a crawl runs. Next Update shows when the next crawl is due.

• Filter: type in Filter by title, type, date added or sync.
• Sort: click a column title. Click it again to reverse the order.
• Columns: show or hide columns with the columns button next to the filter.

Right-click a source, or open its options, to View its content, open its Settings or Delete it. For an MCP server, the options are View, Settings and Delete.

Pages in a source

Click View answers under a source’s title, or View in its options, to see the pages in it, with their URL, title, and when they were added and modified.

• Search: type a whole or partial URL or title in the search field.
• Edit: click Edit next to a page to open it. On the right you see the page’s content as Raffle scraped it, divided into sections; hover over the content to see each section. Click View original page to open the page on your website.
• Update: on Web Scrape sources, click Update to re-crawl one page now. See Instant Sync.

Add questions to a page

When you open a page with Edit, the General tab on the left lists the questions you’ve added to it. Type a question in Add a question and press Enter. Click the pencil icon to change a question or the trash icon to remove it, then click Update.

When a visitor’s search matches one of the questions word for word, the page is boosted to the top. Add the questions visitors actually ask, in their words. To choose the top answer for a question while seeing the results, use Re-rank.

Change a source’s settings

Right-click a source, or open its options, and click Settings. The steps you went through when you created the source are there. Click Update to save.

Renaming

Change the Source Title in the Source details step.

Inclusions

For a Web Scrape source, add Entrypoints, Allowed Domains or Included URL Filters under Advanced settings. The main URL of a source can’t be changed.

Exclusions

For a Web Scrape source, tick the pages to leave out in the Exclude pages step, or add regular expressions under Excluded URL Filters in Advanced settings.

Delete a source

1. Right-click the source, or open its options, and click Delete
2. Confirm

You can’t delete a source while widgets still use it. Remove it from those widgets first, under Widget sources.