URL Datasource

A URL Datasource represents a web-based location or link that provides access to data stored on the internet. It allows retrieving information directly from websites or web services, making it a valuable source for real-time data retrieval and integration with other applications.

Here's how you can create your URL Datasource:

Start by clicking the button from the Datasource section. The next step is to choose the the type of Datasource. Here we will discuss about URL Datasource. Once you select the URL Datasource option, follow the below steps:

1. Add Datasource Information: Input essential details like the Datasource's name and briefly describe what it does in the 'About' section (this is for user's understanding).

2. Insert Public URL: The next step is to add the public URL for indexing and creating the datasource.

3.Configure for Routing: Compose a descriptive text for the Datasource. A Datasource can be used individually or within an App. The Router description is crucial as the App's LLM/Router utilises this description to determine if this Datasource should be invoked within complex workflows.

Within this section, users have the option to persistently save a datasource configuration, especially when a model is not required for their task. Alternatively, users can choose to "Proceed," leading to the introduction of two additional sections(Both are advanced settings). In these subsequent sections, users can specify values for chunking and select an appropriate model tailored to their task requirements.

4. Configure Retreival and Vector Storage: Select the model that the Datasource will leverage to complete it’s querying and reply generation. For example GPT 4, Claude etc. You can also adjust settings like temperature and output tokens for the model.