-
Notifications
You must be signed in to change notification settings - Fork 345
Closed
Description
Summary
Add a nav agent command that runs an interactive REPL loop on the client, communicates with LLMs through a new Chat gRPC service on the Navigator gateway server, and executes tools by shelling out to nav CLI commands to operate the cluster.
Design
- Agent loop runs client-side. The gRPC
Chatservice handles single LLM calls only (server-streaming RPC). The client orchestrates multi-turn tool-calling loops. - Tools shell out to the CLI. The agent executes tools by running
nav <subcommand> <args>as subprocesses (e.g.,nav sandbox list,nav cluster status). No direct gRPC calls for tools. - Binary resolution: Use
std::env::current_exe()by default, withNAV_AGENT_CLIenv var override for dev. - Cluster context pinning: Every shell-out includes
--cluster <name>so subcommands target the correct cluster. - Provider-agnostic tool calling protocol. Proto defines its own
Tool,ToolCall,ChatMessagemessages. The server translates to/from OpenAI or Anthropic format based on the resolved inference route's protocol. - Route selection via
routing_hint = "agent". The operator creates an inference route withrouting_hint: agent. The Chat service filters routes using the existinglist_sandbox_routesfunction. - Streaming responses.
ChatStreamis a server-streaming RPC that streamsContentDeltaevents as tokens arrive via SSE, then sends a finalChatMessagewith anytool_calls.
Components
Proto (proto/chat.proto)
Chatservice withChatStreamserver-streaming RPCChatStreamRequest,ChatMessage,Tool,ToolCall,ChatStreamEvent,ContentDelta,ChatErrormessages
Server (crates/navigator-server/src/chat/)
ChatService— gRPC implementation with route resolution, protocol detection, spawned streaming taskopenai.rs— OpenAI format translation and SSE streaming with tool call accumulationanthropic.rs— Anthropic format translation (system prompt extraction, tool_use/tool_result blocks) and SSE streaming- Multiplexer extended from 2 to 3 gRPC services
CLI (crates/navigator-cli/src/agent.rs)
- 9 tools:
cluster_status,sandbox_list,sandbox_get,sandbox_create,sandbox_delete,sandbox_logs,provider_list,provider_get,inference_route_list - Binary resolution with
NAV_AGENT_CLIenv var override - Interactive REPL using
rustylinewith tool-calling loop --routing-hintand--system-promptCLI arguments
Acceptance Criteria
-
nav agentstarts an interactive REPL - Agent can call tools and feed results back to the LLM
- Both OpenAI and Anthropic backends are supported
- Streaming token output works correctly
-
cargo check,cargo clippy,cargo test, andcargo fmtall pass
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels