Pagination using response link headers

[StijnKing]

This question seeks guidance on a specific problem where the response headers are needed for subsequent API requests and how this can be achieved within the constraints of the Tableau Web Data Connector environment.

Problem Summary: While the Fetcher and Parser classes are accessible after the initial import, all other JavaScript functionality is restricted. The primary problem arises when users attempt to refresh data from Tableau Desktop. The lack of access to other DOM functions, including custom fetch methods, poses a significant challenge.
To address this issue, we face a challenge:

Use of Tableau's fetchJson Function: To overcome the limitation of not having access to most JavaScript functions after the initial import, we can leverage Tableau's fetchJson function. This function allows us to make data requests directly from Tableau Desktop.

However, there is a significant limitation to be aware of:

Inability to Access Response Headers: The primary limitation of using fetchJson is that it just returns the response body (response.json) but does not provide access to response headers. This omission is crucial because response headers often contain critical information such as authentication tokens (e.g., x-my-authtoken) required for subsequent API requests. Also response.json() inside fetchJson will throw an error if there is no any response body for example in case of api/auth/login endpoint. Since api/auth/login returns 204 (no content) this response.json() will throw an error because it expects some response.body.

Question: How can we access the response headers when making a data request using the fetch function within the Fetcher class of a Tableau Web Data Connector, particularly in cases where the connector window is closed, and traditional JavaScript methods become inaccessible? Because taco-toolkit's FetcherUtils.fetchJson function only returns response body(response.json()) and we have an authentication endpoint which returns auth token inside headers not in body. I specifically want to use the next link in the link headers returned by the API.