Alan AI allows you to crawl web resources protected with Basic authentication.

To be able to retrieve data from such resources, add the auth parameter to the corpus() function and define the username and password to access the resource.

Example

corpus({
    title: `Protected resources`,
    urls: [`https://testpages.eviltester.com/styled/auth/basic-auth-results.html`],
    auth: {username: `authorized`, password: `password001`},
    depth: 3,
    maxPages: 1,
    priority: 1,
});