# Knowledge System

## Overview

The knowledge system provides agents with runtime access to Microsoft Learn documentation and web search results. It enables agents to look up current Azure documentation, patterns, and troubleshooting information during prototype generation, supplementing their built-in knowledge with up-to-date external sources.

The system has three components: web search functions, an in-memory search cache, and a knowledge contribution command for feeding project-specific learnings back into the knowledge base.

## Web Search

The `knowledge/web_search.py` module provides module-level functions (following the `deploy_helpers.py` pattern) that search the Microsoft Learn API and fetch page content. All functions return empty results on failure -- they never raise exceptions.

### Functions

| Function | Description |
|----------|-------------|
| `search_learn(query, max_results=3)` | Search the Microsoft Learn API. Returns a list of `{title, url, description}` dicts. |
| `fetch_page_content(url)` | Fetch a page and convert HTML to plain text using a minimal stdlib-based HTML parser. |
| `search_and_fetch(query, max_results=3)` | Combines search and fetch: searches Learn, then fetches and concatenates the top results. |
| `format_search_results(results)` | Format search results for injection into agent context. |

The search endpoint is `https://learn.microsoft.com/api/search` with a 10-second HTTP timeout.

## Search Cache

The `SearchCache` class (in `knowledge/search_cache.py`) provides session-scoped caching to avoid redundant HTTP requests when multiple agents search for similar queries.

| Setting | Default | Description |
|---------|---------|-------------|
| TTL | 30 minutes (1800 seconds) | Time-to-live for cached entries |
| Max entries | 50 | Maximum number of cached entries |
| Eviction | LRU | Oldest entry evicted when capacity is reached |

The cache is attached to `AgentContext._search_cache` on first search, so it is shared across all agents within a session. Cache keys are normalized (lowercased, whitespace-collapsed) to improve hit rates for similar queries.

Cache statistics (hits, misses, entry count) are available via `cache.stats()`.

## Knowledge Contributions

The `az prototype knowledge contribute` command allows users and agents to feed learnings back into the knowledge base.

```
az prototype knowledge contribute [options]
```

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `--service` | string | (required) | Azure service name (e.g., `cosmos-db`, `key-vault`). |
| `--description` | string | (required) | Brief description of the knowledge contribution. |
| `--file` | string | (required) | Path to a file containing the contribution content. |
| `--draft` | flag | `false` | Preview the contribution without submitting. |
| `--type` | enum | `Pitfall` | Type of contribution. One of: `Service pattern update`, `New service`, `Tool pattern`, `Language pattern`, `Pitfall`. |
| `--section` | string | None | Target section within the knowledge file. |
| `--json` / `-j` | flag | `false` | Output machine-readable JSON instead of formatted display. |

Contributions are also made automatically during sessions. For example, the build session's QA remediation loop contributes findings from QA review to the knowledge base as fire-and-forget operations.

## Agent Integration

Five agents have web search enabled (`_enable_web_search = True`):

1. **cloud-architect** -- looks up cross-service patterns and Azure architecture guidance
2. **terraform-agent** -- searches for Terraform provider documentation and module references
3. **bicep-agent** -- searches for Bicep template references and ARM API documentation
4. **app-developer** -- looks up SDK documentation and application patterns
5. **qa-engineer** -- searches for error messages, known issues, and troubleshooting guides

## Search Markers

Agents request web searches by including `[SEARCH: query]` markers in their AI responses. The `_resolve_searches()` method in `BaseAgent`:

1. Detects `[SEARCH: query]` markers in the AI response (up to 3 per response)
2. Executes each search via `search_and_fetch()`
3. Re-invokes the AI with the fetched documentation appended to the context
4. Merges token usage from both AI calls

This allows agents to self-direct their information gathering based on what they encounter during task execution.

## POC Mode

The `compose_context(mode="poc")` function strips the `## Production Backlog Items` section from knowledge service files by default. This keeps agent context focused on prototype-relevant information. Production backlog items are extracted separately via `extract_production_items(service)` and used during [backlog generation](Backlog-Generation.md) as deferred epic material.

## Examples

Contribute a pitfall learned during development:

```bash
az prototype knowledge contribute \
  --service cosmos-db \
  --description "Connection string requires AccountEndpoint format for SDK v4" \
  --file ./learnings/cosmos-connection.md \
  --type Pitfall
```

Preview a contribution without submitting:

```bash
az prototype knowledge contribute \
  --service key-vault \
  --description "Soft-delete enabled by default since 2021" \
  --file ./learnings/keyvault-softdelete.md \
  --draft
```

Contribute a new service pattern:

```bash
az prototype knowledge contribute \
  --service container-apps \
  --description "Dapr sidecar configuration pattern" \
  --file ./patterns/dapr-sidecar.md \
  --type "Service pattern update" \
  --section "Configuration"
```

## Related

- [MCP Integration](MCP-Integration.md) -- external tool access for agents
- [Escalation](Escalation.md) -- Level 3 escalation uses web search
- [Error Analysis](Error-Analysis.md) -- QA agent uses web search during diagnosis