Develop

AGENTS.md

@@ -50,6 +50,104 @@ The operating model intentionally follows current agent practice:
   - `docker-compose/debug-docker-compose.yaml` expects `local_edge_node`
 - Kubernetes manifests are not self-consistent enough to treat as production-safe without review; validate names, namespaces, PVCs, and mount paths before use.
 
+### Active Refactors
+- CAR / Deeploy networking refactor planning:
+  - Local working plan lives at `xperimental/container_apps_deeploy_refactor_plan.md`.
+  - Approved sequencing is test-first:
+    1. Add CAR compatibility baseline under `extensions/business/container_apps/tests/`.
+    2. Add Deeploy compatibility baseline under `extensions/business/deeploy/tests/`.
+    3. Introduce normalized CAR `EXPOSED_PORTS` model with backward-compatible mappers.
+    4. Refactor CAR runtime and Deeploy request handling around the normalized model.
+  - Compatibility rules locked during planning:
+    - Keep legacy semaphore exports `HOST`, `PORT`, `URL` unchanged.
+    - Add explicit CAR exports `HOST_IP`, `HOST_PORT`, `CONTAINER_IP`, `CONTAINER_PORT`.
+    - Keep `shmem` backend-only for normal UI; UI should model dynamic env as `static`, `host_ip`, or `container_ip` with provider selection.
+  - Current status:
+    - Phase 0 completed on `2026-03-17`: focused CAR compatibility baseline added under `extensions/business/container_apps/tests/`.
+    - Verified passing command: `python3 -m unittest discover -s extensions/business/container_apps/tests -p "test_*.py"`.
+    - Note: existing top-level module `extensions.business.container_apps.test_worker_app_runner` currently has pre-existing failures on this branch and is not the Phase 0 compatibility gate.
+    - Phase 1 completed on `2026-03-17`: focused Deeploy compatibility baseline added under `extensions/business/deeploy/tests/`.
+    - Verified passing command: `python3 -m unittest discover -s extensions/business/deeploy/tests -p "test_*.py"`.
+    - Note: existing top-level module `extensions.business.deeploy.test_deeploy` currently is not a reliable gate on this branch because importing `DeeployManagerApiPlugin` hits an environment/import-time circular dependency in the installed `naeural_core`.
+    - Phase 2 completed on `2026-03-17`: CAR now parses and validates normalized `EXPOSED_PORTS` config and caches normalized state internally.
+    - Verified passing commands:
+      - `python3 -m unittest discover -s extensions/business/container_apps/tests -p "test_*.py"`
+      - `python3 -m unittest extensions.business.container_apps.tests.test_exposed_ports_model`
+    - Note: runtime port allocation and tunnel orchestration still follow legacy CAR paths; Phase 2 only adds normalized parsing/validation, preserving existing behavior unless `EXPOSED_PORTS` is explicitly provided.
+    - Phase 3 completed on `2026-03-17`: CAR now synthesizes normalized `EXPOSED_PORTS` state from legacy `PORT`, legacy port mappings, and legacy tunnel fields, with explicit precedence and conflict handling.
+    - Verified passing command: `python3 -m unittest discover -s extensions/business/container_apps/tests -p "test_*.py"`.
+    - Note: legacy runtime allocation and tunnel orchestration are still the active execution paths; Phase 3 changes only the internal normalized view used for later refactor steps.
+    - Phase 4 completed on `2026-03-17`: CAR runtime now allocates host ports, derives extra tunnels, and resolves health defaults from normalized `EXPOSED_PORTS` state while preserving `self.port` and `extra_ports_mapping` as compatibility outputs.
+    - Verified passing commands:
+      - `python3 -m unittest discover -s extensions/business/container_apps/tests -p "test_*.py"`
+      - `python3 -m unittest extensions.business.container_apps.tests.test_tunnel_runtime_behavior extensions.business.container_apps.tests.test_health_check_behavior`
+    - Note: main-tunnel and extra-tunnel processes remain separate for compatibility, but both are now driven from the normalized per-port model instead of raw legacy config fields.
+    - Phase 5 completed on `2026-03-17`: CAR now exports explicit semaphore networking keys `HOST_IP`, `HOST_PORT`, `CONTAINER_IP`, and `CONTAINER_PORT` while keeping legacy `HOST`, `PORT`, and `URL` semantics unchanged.
+    - Verified passing commands:
+      - `python3 -m unittest discover -s extensions/business/container_apps/tests -p "test_*.py"`
+      - `python3 -m unittest extensions.business.container_apps.tests.test_semaphore_exports`
+    - Note: explicit semaphore keys resolve from the normalized main port, so they work even when CAR is configured only through `EXPOSED_PORTS`.
+    - Phase 6 completed on `2026-03-17`: Deeploy create/update preparation now explicitly preserves normalized CAR `EXPOSED_PORTS` payloads, and containerized request validation rejects malformed non-dict `EXPOSED_PORTS` before the payload reaches CAR.
+    - Verified passing commands:
+      - `python3 -m unittest discover -s extensions/business/deeploy/tests -p "test_*.py"`
+      - `python3 -m unittest extensions.business.deeploy.tests.test_create_requests extensions.business.deeploy.tests.test_update_requests`
+    - Note: no Deeploy config flattening was needed; the main change was test coverage plus shallow request-shape validation at the translation boundary.
+    - Phase 7 completed on `2026-03-17`: Deeploy now translates a UI-friendly `DYNAMIC_ENV_UI` model into backend `DYNAMIC_ENV`, compiling `container_ip` providers to `shmem(..., CONTAINER_IP)` while keeping explicit raw `DYNAMIC_ENV` as the advanced-precedence path.
+    - Verified passing commands:
+      - `python3 -m unittest discover -s extensions/business/deeploy/tests -p "test_*.py"`
+      - `python3 -m unittest extensions.business.deeploy.tests.test_create_requests extensions.business.deeploy.tests.test_update_requests extensions.business.deeploy.tests.test_dynamic_env_resolution`
+    - Note: the frontend contract for this translation is now defined in backend code, but the actual UI implementation is still outside this repo.
+    - Phase 8 completed on `2026-03-17`: the `deeploy-dapp` frontend now edits generic CAR ports through `exposedPorts` rows, uses `container_ip` provider selection for dynamic env, serializes `EXPOSED_PORTS` / `DYNAMIC_ENV_UI`, and recovers/edits/display these values with legacy fallbacks.
+    - Verified passing commands:
+      - `npm run lint` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npx tsc --noEmit --incremental false` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `google-chrome --headless=new --disable-gpu --window-size=1440,2200 --screenshot=.playwright/container-networking-preview.png http://127.0.0.1:3005/playwright-preview`
+    - Note: frontend preview screenshot confirmed the new exposed-ports and dynamic-env sections; legacy raw `shmem` references outside `CONTAINER_IP` are still not expressible in the simplified UI and remain an advanced compatibility edge case.
+    - Planned follow-up on `2026-03-18`: generalize dynamic-env UI beyond `container_ip` by adding a `plugin_value(provider, key)` source with built-in semaphore-key mappings for known plugin signatures and free-text key entry for custom signatures. Local working plan: `xperimental/dynamic_env_plugin_value_followup_plan.md`.
+    - Follow-up Phase A completed on `2026-03-18`: `deeploy-dapp` now supports `plugin_value` at the type/schema level, with validation requiring both provider and exported key, and a runnable schema test gate (`npm run test:dynamic-env-schema`) added for this follow-up track.
+    - Follow-up Phase B completed on `2026-03-18`: `deeploy-dapp` now has a centralized plugin-signature semaphore-key capability map with preset explicit keys for CAR/WAR, default `PORT` mapping for built-in native signatures, and manual-key fallback for custom or unknown signatures.
+    - Follow-up Phase C completed on `2026-03-18`: the `deeploy-dapp` dynamic-env editor now supports `plugin_value` with provider plus preset-or-manual key selection, and native-app dependency computation now treats `plugin_value` references the same way as `container_ip` references.
+    - Follow-up Phase D completed on `2026-03-18`: `deeploy-dapp` now serializes `plugin_value(provider, key)` through `DYNAMIC_ENV_UI`, and Deeploy compiles those entries to backend `shmem(path=[provider, key])` while preserving existing `container_ip` translation and explicit `DYNAMIC_ENV` precedence.
+    - Follow-up Phase D verification on `2026-03-18`:
+      - `npm run test:dynamic-env-serialization` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npm run test:dynamic-env-ui` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npm run test:plugin-semaphore-keys` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npm run test:dynamic-env-schema` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npm run lint` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npx tsc --noEmit --incremental false` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `python3 -m unittest discover -s extensions/business/deeploy/tests -p "test_*.py"`
+    - Follow-up Phase E completed on `2026-03-18`: `deeploy-dapp` now reverse-maps non-`CONTAINER_IP` semaphore paths to `plugin_value(provider, key)` during recovery, edit hydration, legacy JSON import, and read-only display, eliminating the prior flattening of supported `shmem` entries to empty `static` fragments.
+    - Follow-up Phase E verification on `2026-03-18`:
+      - `npm run test:dynamic-env-roundtrip` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npm run test:dynamic-env-serialization` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npm run test:dynamic-env-ui` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npm run test:plugin-semaphore-keys` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npm run test:dynamic-env-schema` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npm run lint` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npx tsc --noEmit --incremental false` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+    - Follow-up Phase G completed on `2026-03-18`: the `deeploy-dapp` preview harness now visibly exercises all generalized dynamic-env modes, including `container_ip`, built-in native `PORT`, and a custom free-text plugin key, with fresh desktop/mobile screenshots captured from `/playwright-preview`.
+    - Follow-up Phase G verification on `2026-03-18`:
+      - `npm run lint` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npx tsc --noEmit --incremental false` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `google-chrome --headless=new --disable-gpu --window-size=1440,2200 --screenshot=.playwright/dynamic-env-plugin-value-desktop.png http://localhost:3005/playwright-preview`
+      - `google-chrome --headless=new --disable-gpu --window-size=430,2200 --screenshot=.playwright/dynamic-env-plugin-value-mobile.png http://localhost:3005/playwright-preview`
+    - Follow-up Phase F closed on `2026-03-18`: the planned serializer/compiler/round-trip coverage was fully delivered in the Phase D and Phase E commits, so no separate code-bearing phase was needed beyond running the aggregated frontend/backend test gates.
+    - Post-implementation docs cleanup completed on `2026-03-18`: frontend README, container-app README, and Deeploy/CAR inline docs now describe `plugin_value(provider, key)`, normalized `EXPOSED_PORTS`, and the explicit container-app semaphore networking keys that new consumers should prefer.
+    - FastAPI native semaphore follow-up completed on `2026-03-18`: `FastApiWebAppPlugin` now exports default semaphore keys `HOST`, `PORT`, and `URL` using the allocated runtime port (`self.port`), so bare FastAPI native plugins such as `EDGE_NODE_API_TEST` can satisfy `plugin_value(..., "PORT")` dependencies without per-plugin overrides.
+    - FastAPI native semaphore verification on `2026-03-18`:
+      - `python3 -m unittest naeural_core.business.test_framework.test_fastapi_webapp_semaphore_env` (in `/home/vi/work/ratio1/repos/edge_node/naeural_core`)
+      - `python3 -m unittest discover -s extensions/business/deeploy/tests -p "test_*.py"`
+    - FastAPI native host-key follow-up completed on `2026-03-18`: `FastApiWebAppPlugin` now exports explicit `HOST_IP` and `HOST_PORT` alongside legacy `HOST` and `PORT`, and the `deeploy-dapp` native plugin key presets now expose those explicit host keys ahead of legacy `PORT`.
+    - FastAPI native host-key verification on `2026-03-18`:
+      - `python3 -m unittest naeural_core.business.test_framework.test_fastapi_webapp_semaphore_env` (in `/home/vi/work/ratio1/repos/edge_node/naeural_core`)
+      - `python3 -m unittest discover -s extensions/business/deeploy/tests -p "test_*.py"`
+      - `npm run test:plugin-semaphore-keys` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npm run test:dynamic-env-ui` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npm run lint` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+      - `npx tsc --noEmit --incremental false` (in `/home/vi/work/ratio1/repos/deeploy-dapp`)
+    - Native UI key simplification completed on `2026-03-18`: the `deeploy-dapp` standard key list for built-in native providers now shows only `HOST_IP` and `HOST_PORT`; legacy `PORT` remains backend-supported and round-trippable but is no longer advertised as a parallel preset in the normal UI.
+    - FastAPI semaphore guard follow-up completed on `2026-03-18`: the default `FastApiWebAppPlugin` semaphore export now degrades gracefully when localhost IP resolution fails, skipping `HOST`/`HOST_IP`/`URL` export with warnings instead of aborting semaphore readiness for the provider.
+
 ### How To Run
 - Local dev (single node, local image):
   - `./debug.sh`
@@ -571,6 +669,15 @@ Entry format:
 - Verification: Not run (not requested).
 - Links: `extensions/business/deeploy/deeploy_manager_api.py`
 
+- ID: `ML-20260317-001`
+- Timestamp: `2026-03-17T15:30:21Z`
+- Type: `decision`
+- Summary: Approved phased CAR/Deeploy networking refactor plan centered on a normalized `EXPOSED_PORTS` model and test-first compatibility baselines.
+- Criticality: Cross-cutting architecture and migration plan affecting container apps, Deeploy request handling, semaphore exports, and future UI contracts.
+- Details: Planning locked the sequence to separate CAR and Deeploy test baselines before code changes; the target CAR model is a dict-based `EXPOSED_PORTS` keyed by container port with per-port tunnel config and one `is_main_port`; legacy CAR config and legacy semaphore keys remain backward-compatible through translation; new explicit CAR semaphore env keys will be added for unambiguous host/container addressing; UI-facing dynamic env stays higher-level and compiles `container_ip` to backend `shmem(..., CONTAINER_IP)`.
+- Verification: `date -u +"%Y-%m-%dT%H:%M:%SZ"`; `sed -n '1,260p' AGENTS.md`; local plan written to `xperimental/container_apps_deeploy_refactor_plan.md`
+- Links: `AGENTS.md`, `xperimental/container_apps_deeploy_refactor_plan.md`, `extensions/business/container_apps/container_app_runner.py`, `extensions/business/deeploy/deeploy_mixin.py`
+
 - ID: `ML-20260317-001`
 - Timestamp: `2026-03-17T21:14:35Z`
 - Type: `change`