This transport plugin uses standard
fetch() to retrieve remote content from alternative endpoints — that is, HTTPS endpoints that are not in the original domain. This enables retrieving content even if the website on the original domain is down for whatever reason. The list of alternative endpoints is itself retrieved using DNSLink for the original domain.
As per LibResilient architecture, this plugin adds
X-LibResilient-ETag headers to the returned response.
dnslink-fetch plugin supports the following configuration options:
concurrency (default: 3)
Number of alternative endpoints to attempt fetching from simultaneously.
If the number of available alternative endpoints is lower then
concurrency, all are used for each request. If it is higher, only
concurrency of them, chosen at random, are used for any given request.
edns_client_subnetparameter of the DoH JSON API.
When fetching an URL,
dnslink-fetch removes the scheme and domain component. Then, for each alternative endpoint that is used for this particular request (up to
concurrency of endpoints, as described above), it concatenates the endpoint with the remaining URL part. Finally, it performs a
fetch() request for every URL construed in such a way.
Let’s say the plugin is deployed for website
concurrency set to
2 and these are the alternative endpoints specified in relevant TXT records according to the DNSLink specification (so, in multiaddr form):
dnslink-fetch currently only supports a rudimentary, naïve form of multiaddr addresses, which is
/https/domain_name[/optional/path]; full mutiaddr support might be implemented at a later date.
A visitor, who has visited the
https://example.com website at least once before (and so, LibResilient is loaded and working), tries to access it. For whatever reason, the
https://example.com site is down or otherwise inaccessible, and so the
dnslink-fetch plugin kicks in.
The request for
https://example.com/index.html is being handled thus:
concurrencysetting) random alternative endpoints selected:
fetch()request issued simultaneously for URL (so, alternative endpoint concatenated with the path from hte original request):