Skip to content

Temporarily ignore DeepSeek upstream usage fields and prefer internal token estimation#244

Merged
CJackHwang merged 3 commits intodevfrom
codex/temporarily-switch-to-internal-usage-count
Apr 7, 2026
Merged

Temporarily ignore DeepSeek upstream usage fields and prefer internal token estimation#244
CJackHwang merged 3 commits intodevfrom
codex/temporarily-switch-to-internal-usage-count

Conversation

@CJackHwang
Copy link
Copy Markdown
Owner

@CJackHwang CJackHwang commented Apr 7, 2026

Motivation

  • 上游 DeepSeek 返回的 accumulated_token_usage / token_usage 可能是累计上下文口径,会导致单次输出计数误差非常大,内部估计需要统一口径以避免计数异常。
    -方案是 完全不处理,通过改进算法依靠回归内部估计处理

Description

  • 将 Go SSE 解析器中的 extractAccumulatedTokenUsage 改为始终返回 0,0,并删除/移除对上游解析辅助函数的依赖,以实现忽略上游 usage(internal/sse/parser.go)。
  • 在 JS SSE 解析实现中同步忽略上游 usage,返回 { prompt:0, output:0 } 保持前后端解析口径一致(internal/js/chat-stream/sse_parse_impl.js)。
  • 更新并修正相关单元/兼容测试断言以匹配临时策略(包括 internal/sse/line_test.gointernal/sse/parser_test.gointernal/sse/raw_stream_token_replay_test.gointernal/adapter/openai/stream_status_test.gotests/node/chat-stream.test.jstests/compat/expected/sse_content_filter_status.json 等)。
  • 保持行为为“临时”的注释说明,后续可在研究清楚上游口径后再恢复或改为更精细的合并策略。

Testing

  • 执行 ./scripts/lint.sh,结果通过(golangci-lint bootstrapped,0 issues)。
  • 执行 ./tests/scripts/check-refactor-line-gate.sh,结果通过。
  • 执行 ./tests/scripts/run-unit-all.sh,包含 Go 与 Node 测试,已修正断言后所有测试通过(Go 单元测试和 Node 兼容测试均通过)。
  • 执行 npm run build --prefix webui 成功构建 Web UI 产物。

Codex Task

@vercel
Copy link
Copy Markdown

vercel bot commented Apr 7, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
ds2api Ready Ready Preview, Comment Apr 7, 2026 0:39am

@CJackHwang CJackHwang marked this pull request as ready for review April 7, 2026 12:14
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d9e65c9710

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@CJackHwang CJackHwang merged commit c6a6f1c into dev Apr 7, 2026
3 checks passed
@CJackHwang CJackHwang deleted the codex/temporarily-switch-to-internal-usage-count branch April 7, 2026 12:39
CJackHwang added a commit that referenced this pull request Apr 7, 2026
Merge pull request #244 from CJackHwang/codex/temporarily-switch-to-internal-usage-count

Temporarily ignore DeepSeek upstream usage fields and prefer internal token estimation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant