Connecting External Knowledge Sources

SUPERWISE Chat can pull knowledge directly from websites and document sources into your knowledge collections. Once connected, the AI can search and cite these sources in conversations.

What Are Connectors?

Connectors are integrations that automatically ingest content from an external source into a knowledge collection. Content is chunked, indexed, and made available for semantic search.

Currently supported connectors:

Connector	What it ingests
Web Crawl	Public websites, documentation sites, wikis
File Upload	PDF, DOCX, TXT, Markdown, HTML, CSV

Note: Additional connectors (SharePoint, Google Drive, Confluence) are available on Professional and Enterprise plans. Contact your account manager to enable them.

Setting Up a Web Crawl

Web crawl reads content from a public website into your collection.

Who can do this: users with project_admin role or higher.

1. Navigate to Knowledge in the sidebar.
2. Open or create a collection.
3. Click Add Source → Web Crawl.
4. Enter the starting URL (e.g. https://docs.example.com).
5. Set crawl options:
   • Depth: How many link-levels deep to follow (1–3 recommended)
   • Include paths: Optional URL prefix filter (e.g. /docs/) to limit scope
   • Exclude paths: Paths to skip (e.g. /blog/, /changelog/)
6. Click Start Crawl.

The crawl runs in the background. Progress is shown in the collection’s Sources tab. When complete, the status changes to Ready and the content is immediately searchable.

Scheduling Recrawls

On Professional and Enterprise plans, you can schedule automatic recrawls to keep content fresh.

1. Open the crawl source in your collection.
2. Click Schedule.
3. Choose a frequency: Daily, Weekly.
4. Click Save.

The connector re-crawls changed pages only (incremental sync).

Uploading Files

File upload adds one or more documents directly to a collection.

1. Open a collection.
2. Click Upload.
3. Drag and drop files or click Browse to select.
   • Supported: PDF, DOCX, TXT, MD, HTML, CSV
   • Maximum file size: 50 MB per file
4. Files are processed automatically. Status shows Processing → Ready.

Security Scanning

All uploaded files and crawled pages are automatically scanned before indexing:

• Sensitive data detection: Patterns such as social security numbers, credit card numbers, and API keys are detected and flagged. Files with detected sensitive data are quarantined and not indexed until reviewed.
• Malware scanning: Files are scanned before processing. Infected files are rejected.

Quarantined files appear in the collection’s Sources tab with a Quarantined status. A tenant administrator can review and approve or delete them.

Managing Sources

Viewing Source Status

In a collection, click Sources to see all connected sources and their status:

Status	Meaning
Processing	Being ingested or re-crawled
Ready	Available for search
Quarantined	Flagged by security scan — requires admin review
Error	Crawl or processing failed — see error details

Removing a Source

1. Open the collection → Sources.
2. Click the source you want to remove.
3. Click Remove Source.

Removing a source deletes all indexed content from that source from the collection. Conversations that previously cited the content are not retroactively updated.

Refreshing a Source

To manually trigger a re-crawl or re-index:

1. Open the source.
2. Click Refresh Now.

Attaching Collections to Channels

A collection must be attached to a channel for the AI to use it in conversations in that channel.

1. Go to Settings → Channels → [Channel] → Knowledge.
2. Click Attach Collection.
3. Select the collection(s) to attach.
4. Click Save.

You can attach multiple collections to a channel. During a conversation, the AI searches across all attached collections.

Tips

• Scope your crawls: A focused crawl (e.g. docs.example.com/api/) returns better results than crawling the full site. Start narrow.
• Use collections per topic: One collection per domain or project makes it easier to manage access and keep content relevant.
• Re-crawl regularly: For external sites that change frequently, set a weekly recrawl schedule.
• Check quarantine: If content you expected to find isn’t appearing, check the Sources tab for quarantined files.