Add Web Page Knowledge
- Add any publicly accessible web page by URL — your agent fetches and indexes the content automatically
- Reference external documentation, status pages, runbook sites, and wiki articles during investigations
- One-step setup: provide a URL, name, and optional description
- Content is indexed immediately and available across all future conversations
The problem: external docs stay external
Your team's knowledge isn't all in one place. Runbooks live on wiki sites. Vendor documentation is on external portals. Architecture diagrams and status pages are scattered across different URLs. When your agent investigates an issue, it can't reference these external resources — unless someone manually copies the content and uploads it as a file.
That manual process is tedious and creates stale copies. The original page gets updated, but the uploaded file doesn't. Over time, your agent's knowledge diverges from the actual documentation your team relies on.
How web page knowledge works
When you add a web page as a knowledge source, the agent fetches the page content and stores it for reference:
- You provide a URL, a name, and an optional description
- The agent fetches the page content via an anonymous HTTP request
- The page content is stored and indexed
- Your agent can reference this content during conversations and investigations
The fetch happens at the time you add the URL. Content is stored as a point-in-time snapshot of the page.
The agent fetches pages anonymously — without authentication credentials. Pages that require login, VPN access, or corporate SSO cannot be indexed. If you need to add content from protected pages, copy the content and upload it as a file instead.
What gets indexed
The agent fetches the full page content from the URL you provide. Key details:
| Aspect | Behavior |
|---|---|
| Content fetched | Full page content from the single URL provided |
| Link following | No — only the specified URL is fetched, not linked pages |
| Authentication | Anonymous — no credentials sent with the request |
| Supported protocols | HTTP and HTTPS |
| Fetch timeout | 30 seconds |
| Refresh | Manual — delete and re-add the URL to get updated content |
When to use web page knowledge
Web page knowledge sources work best for:
- Public documentation — vendor docs, API references, cloud service guides
- Status pages — service health dashboards, incident history pages
- Wiki articles — publicly accessible knowledge base articles
- Architecture overviews — publicly hosted architecture diagrams and design docs
- Runbook sites — external runbook repositories accessible without authentication
For internal or protected content that requires authentication, use file uploads instead.
Limitations
| Limitation | Details |
|---|---|
| No authentication | Cannot access pages behind login, VPN, or SSO |
| Single page only | Does not crawl or follow links to other pages |
| Point-in-time snapshot | Content is not automatically refreshed when the source page changes |
| 30-second timeout | Pages that take longer than 30 seconds to load will fail |
| URL format | Must be an absolute HTTP or HTTPS URL |
Related capabilities
| Capability | What it adds |
|---|---|
| Connect Knowledge | → Overview of all knowledge source types — files, web pages, and repositories |
| Upload Knowledge Documents | → Upload files directly or let your agent create documents during conversations |
| File Attachments | → Share files in chat for immediate analysis context |
Get started
→ Add a Web Page Knowledge Source — step-by-step guide to adding your first web page