Websites
Website Knowledge
This knowledge can be used in the Knowledge Step in the AI Agent builder
Adding Website Knowledge to EasyBeam
The Add Website feature in EasyBeam allows you to add knowledge from a website to your AI agent. This feature includes configurable encoding strategies and an optional web crawler to retrieve structured information efficiently.
Opening the Add Website Modal
To add website knowledge:
Open the Knowledge page (the book on the left nav)
Open the Add Website modal.
Enter the URL of the website you want to process.
Configure the encoding strategy and optional crawling settings.
Click Add Website to submit.
Encoding Strategies
EasyBeam offers three encoding strategies to optimize how website content is chunked and stored for retrieval:
Precise
256
64
Technical documentation, code
Balanced
512
128
General content like articles and blogs
Context-Heavy
1024
256
Long-form narrative content, complex explanations
Each strategy balances chunk size and overlap to ensure effective retrieval when querying the AI.
Website Crawler (Beta)
The Website Crawler allows you to fetch multiple pages from a website by following links within a defined scope.
Enabling the Crawler
Toggle the Website Crawler (Beta) switch to enable crawling.
Configure additional crawling options.
Crawl Options
Crawl Depth
Limits how many levels of links are followed (currently 1-2).
Max Pages to Crawl
Defines the number of pages the crawler will retrieve (min: 10, max: 500).
Use JavaScript
Enables JavaScript rendering for dynamically loaded content.
Crawl Scope
Defines which links the crawler will follow.
Crawl Scope Options
Same Domain
Only follows links on the same domain (e.g., example.com).
Same Root Domain
Follows links across subdomains (e.g., *.example.com).
All Links
Follows any external link (use with caution).
Advanced Options
URL Pattern Matching
You can specify a URL pattern (using a regular expression) to limit the pages the crawler follows. Example:
Last updated