[BOT ISSUE] OpenAI: chat completions streaming drops `refusal` delta text from span output

[braintrust-bot]

Summary

When a model refuses a request during a streaming chat completion, the refusal text arrives via delta.refusal on each chunk. The wrapper's _postprocess_streaming_results (lines 288–356 of oai.py) does not accumulate this field, so the assembled span output contains no refusal text. The message dict in the output only includes role, content, and tool_calls.

Non-streaming calls correctly capture refusals because the full choices dict is logged directly (line 204), which includes message.refusal.

What is missing

ChoiceDelta.refusal is a documented Optional[str] field in the OpenAI Python SDK. When a model refuses a request in streaming mode:

1. delta.content is typically None
2. delta.refusal contains the refusal text chunks
3. The wrapper only checks delta.get("content") (line 312) and delta.get("tool_calls") (line 315), never delta.get("refusal")

Result: the span output shows "content": null with no indication the model refused. Users debugging why a streaming call produced no output have no visibility into the refusal reason from the span.

Braintrust docs status

not_found — The OpenAI integration docs do not mention refusal capture.

Upstream sources

• OpenAI Python SDK type: ChoiceDelta.refusal: Optional[str] — "The refusal message generated by the model." (source)
• Non-streaming equivalent: ChatCompletionMessage.refusal: Optional[str]

Local files inspected

• py/src/braintrust/oai.py:
  • _postprocess_streaming_results (lines 288–356): processes delta.get("content") at line 312 and delta.get("tool_calls") at line 315, but never reads delta.get("refusal")
  • Assembled output (lines 343–356): message dict has role, content, tool_calls — no refusal key
  • Non-streaming path (line 204): logs full choices which includes message.refusal — correct
• py/src/braintrust/wrappers/test_openai.py: no test for refusal in streaming mode

[AbhiPrasad]

I don’t think we want to do this unless there is a really good customer use case. Closing.

[AbhiPrasad]

We're implementing this in the JS SDK, so let's do this here too.

[AbhiPrasad]

released with https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.15.0