Skip to content

[Bug]: How to make stream cancellation provide usage chunk #149

@SpicyMelonYT

Description

@SpicyMelonYT

Using the fetch API, I am triggering a generation request and I am making sure that it has streaming enabled and usage enabled. And when I provide an abort controller to be able to cancel the stream, I expect the usage to still come at the moment of cancellation. But this is not what I get. Instead the connection just terminates and I get no usage chunks, or any data of any kind.

How am I suppose to be able to support stopping of generation and usage? Is there some other api end point I am supposed to call to get the usage of the generation? I need the usage because this is how I can calculate how much to bill the user.

The main core of the request is here showing that I am requesting for streaming, usage, and providing an abort controller.

const requestBody = {
  model,
  messages,
  stream: true,
  stream_options: {
    include_usage: true,
  }
};

// Create AbortController for OpenRouter request cancellation
const openRouterAbortController = new AbortController();

const fetchOptions = {
  method: "POST",
  headers,
  body: JSON.stringify(requestBody),
  signal: openRouterAbortController.signal,
};

let response;
try {
  response = await fetch(openRouterUrl, fetchOptions);
} catch (fetchError) {
  ...
}

The expected result would be to call the abort controller and the stream of chunks ending by including a final finish reason chunk, usage chunk, and that [DONE] message.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions