Skip to content

feat(cli): add interactive agent REPL with Chat gRPC service #50

@drew

Description

@drew

Summary

Add a nav agent command that runs an interactive REPL loop on the client, communicates with LLMs through a new Chat gRPC service on the Navigator gateway server, and executes tools by shelling out to nav CLI commands to operate the cluster.

Design

  • Agent loop runs client-side. The gRPC Chat service handles single LLM calls only (server-streaming RPC). The client orchestrates multi-turn tool-calling loops.
  • Tools shell out to the CLI. The agent executes tools by running nav <subcommand> <args> as subprocesses (e.g., nav sandbox list, nav cluster status). No direct gRPC calls for tools.
  • Binary resolution: Use std::env::current_exe() by default, with NAV_AGENT_CLI env var override for dev.
  • Cluster context pinning: Every shell-out includes --cluster <name> so subcommands target the correct cluster.
  • Provider-agnostic tool calling protocol. Proto defines its own Tool, ToolCall, ChatMessage messages. The server translates to/from OpenAI or Anthropic format based on the resolved inference route's protocol.
  • Route selection via routing_hint = "agent". The operator creates an inference route with routing_hint: agent. The Chat service filters routes using the existing list_sandbox_routes function.
  • Streaming responses. ChatStream is a server-streaming RPC that streams ContentDelta events as tokens arrive via SSE, then sends a final ChatMessage with any tool_calls.

Components

Proto (proto/chat.proto)

  • Chat service with ChatStream server-streaming RPC
  • ChatStreamRequest, ChatMessage, Tool, ToolCall, ChatStreamEvent, ContentDelta, ChatError messages

Server (crates/navigator-server/src/chat/)

  • ChatService — gRPC implementation with route resolution, protocol detection, spawned streaming task
  • openai.rs — OpenAI format translation and SSE streaming with tool call accumulation
  • anthropic.rs — Anthropic format translation (system prompt extraction, tool_use/tool_result blocks) and SSE streaming
  • Multiplexer extended from 2 to 3 gRPC services

CLI (crates/navigator-cli/src/agent.rs)

  • 9 tools: cluster_status, sandbox_list, sandbox_get, sandbox_create, sandbox_delete, sandbox_logs, provider_list, provider_get, inference_route_list
  • Binary resolution with NAV_AGENT_CLI env var override
  • Interactive REPL using rustyline with tool-calling loop
  • --routing-hint and --system-prompt CLI arguments

Acceptance Criteria

  • nav agent starts an interactive REPL
  • Agent can call tools and feed results back to the LLM
  • Both OpenAI and Anthropic backends are supported
  • Streaming token output works correctly
  • cargo check, cargo clippy, cargo test, and cargo fmt all pass

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions