feat(cli): remove prompt subcommand + add agentv import claude#880
Merged
feat(cli): remove prompt subcommand + add agentv import claude#880
Conversation
The `prompt` subcommand (agentv prompt / agentv eval prompt) is no longer used. Remove the command, its tests, and all references from the CLI registration and preprocessArgv logic. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add a transcript import pipeline that reads Claude Code session JSONL files (~/.claude/projects/) and converts them to AgentV's Message[] format for offline grading without re-running the agent. New modules: - packages/core/src/import/types.ts — TranscriptEntry, TranscriptSource - packages/core/src/import/claude-parser.ts — Claude JSONL parser - packages/core/src/import/session-discovery.ts — session discovery - apps/cli/src/commands/import/claude.ts — CLI handler - apps/cli/src/commands/import/index.ts — subcommand registration Supports --session-id, --discover latest, --project-path, --list, and --output flags. Ref: #872 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix biome lint/format issues: import ordering, template literals, non-null assertion, and describe block indentation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Deploying agentv with
|
| Latest commit: |
756f090
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://0e8b282c.agentv.pages.dev |
| Branch Preview URL: | https://feat-import-claude.agentv.pages.dev |
…de parser Fix three critical bugs found during code review: 1. tool_result blocks appear in user events, not assistant events — restructure parser to pair tool_use IDs from assistant events with tool_result blocks in subsequent user events 2. Token usage is cumulative per requestId — only keep the last value per requestId instead of summing all streaming updates 3. User events containing only tool_result content were silently dropped — now properly processed Also fixes: subagent filtering (isSidechain), timestamp tracking from all events for accurate duration, and requestId added to ClaudeEvent interface (removing unsafe cast). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove all `agentv prompt eval` / `agentv eval prompt` references from running-evals.mdx, agent-skills-evals.mdx, and skill-improvement-workflow.mdx - Add tools/import.mdx documenting the new `agentv import claude` command with all flags and output format - Update code-graders.mdx: replace deprecated field names (output_text → output, input_text → input, expected_output_text → expected_output) in contract, examples, and advanced fields table - Update defineCodeGrader SDK examples to use current API (output as Message[], not outputText string) - Add examples/features/import-claude/ with README, eval YAML, and transcript-quality code grader Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- running-evals.mdx: add "Offline Grading" section pointing to agentv import + transcript workflow - skill-improvement-workflow.mdx: remove stale bun scripts/prompt-eval.ts block (scripts never existed) - agentv-eval-writer/SKILL.md: replace agentv prompt eval commands with agentv eval assert and agentv import claude - migrating-from-skill-creator.md: replace agentv prompt eval and stale bun scripts references Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
promptsubcommand: Deletesagentv prompt/agentv eval promptand all associated code, tests, and referencesagentv import claude: New transcript import pipeline that reads Claude Code session JSONL files (~/.claude/projects/) and converts them to AgentV'sMessage[]format for offline grading without re-running the agentNew modules
packages/core/src/import/types.ts—TranscriptEntry,TranscriptSourceinterfaces +readTranscriptFilehelperpackages/core/src/import/claude-parser.ts— Claude Code JSONL parser (user/assistant events, tool_use/tool_result extraction, usage aggregation, streaming dedup)packages/core/src/import/session-discovery.ts—discoverClaudeSessions()scans~/.claude/projects/with filtering by session ID, project path, and latestapps/cli/src/commands/import/claude.ts— CLI handler with--session-id,--discover latest,--project-path,--output,--listapps/cli/src/commands/import/index.ts— subcommand registrationCLI usage
Ref: #872
Test plan
bun run build— clean compilationbun test— all 1301 core tests pass, 9 preprocess-argv tests passagentv import claude --help— shows expected optionsagentv import claude --discover latest— discovers and imports a real sessionagentv prompt— fails with "Not a valid subcommand name"agentv eval prompt— falls through to eval run (expected)🤖 Generated with Claude Code