Summary
The Anthropic instrumentation extracts input_tokens and output_tokens from the Messages API response usage object, but does not capture cache_creation_input_tokens or cache_read_input_tokens. These are standard fields in the Anthropic Messages API response when prompt caching is used, and they are important for understanding cost and cache effectiveness.
What is missing
In InstrumentationSemConv.tagAnthropicResponse() (lines 204–231), the usage extraction block only handles:
if (usage.has("input_tokens")) metrics.put("prompt_tokens", usage.get("input_tokens"));
if (usage.has("output_tokens")) metrics.put("completion_tokens", usage.get("output_tokens"));It should additionally extract:
cache_creation_input_tokens — tokens written to the prompt cache on this requestcache_read_input_tokens — tokens read from an existing prompt cache
These fields are present in every Anthropic Messages API response that uses prompt caching (via cache_control blocks in the request). Without them, users cannot observe cache hit rates or understand the cost breakdown of cached vs. uncached tokens in Braintrust traces.
Note: the full response JSON is stored in braintrust.output_json, so the raw data is technically present — but it is not extracted into braintrust.metrics where Braintrust's UI and cost calculations can use it.
Braintrust docs status
- Braintrust docs generically mention cached token support for cost calculations: unclear whether this is expected to work for Java SDK specifically
- No Java SDK-specific documentation on prompt caching metric extraction
Upstream sources
Local files inspected
braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java — lines 204–231 (tagAnthropicResponse only extracts input_tokens and output_tokens)braintrust-sdk/instrumentation/anthropic_2_2_0/src/main/java/dev/braintrust/instrumentation/anthropic/v2_2_0/TracingHttpClient.java — HTTP-level instrumentation, delegates to InstrumentationSemConvbraintrust-sdk/instrumentation/anthropic_2_2_0/src/test/java/.../BraintrustAnthropicTest.java — no test cases exercise prompt caching responses
Summary
The Anthropic instrumentation extracts
input_tokensandoutput_tokensfrom the Messages API responseusageobject, but does not capturecache_creation_input_tokensorcache_read_input_tokens. These are standard fields in the Anthropic Messages API response when prompt caching is used, and they are important for understanding cost and cache effectiveness.What is missing
In
InstrumentationSemConv.tagAnthropicResponse()(lines 204–231), the usage extraction block only handles:It should additionally extract:
cache_creation_input_tokens— tokens written to the prompt cache on this requestcache_read_input_tokens— tokens read from an existing prompt cacheThese fields are present in every Anthropic Messages API response that uses prompt caching (via
cache_controlblocks in the request). Without them, users cannot observe cache hit rates or understand the cost breakdown of cached vs. uncached tokens in Braintrust traces.Note: the full response JSON is stored in
braintrust.output_json, so the raw data is technically present — but it is not extracted intobraintrust.metricswhere Braintrust's UI and cost calculations can use it.Braintrust docs status
Upstream sources
usageobject: https://docs.anthropic.com/en/api/messages — documentscache_creation_input_tokensandcache_read_input_tokensfieldscom.anthropic:anthropic-java:2.2.0(depended on by this repo)Local files inspected
braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java— lines 204–231 (tagAnthropicResponseonly extractsinput_tokensandoutput_tokens)braintrust-sdk/instrumentation/anthropic_2_2_0/src/main/java/dev/braintrust/instrumentation/anthropic/v2_2_0/TracingHttpClient.java— HTTP-level instrumentation, delegates toInstrumentationSemConvbraintrust-sdk/instrumentation/anthropic_2_2_0/src/test/java/.../BraintrustAnthropicTest.java— no test cases exercise prompt caching responses