Skip to content

feat: Local DevEx: Component Hot Reloading, component benchmarks for warm/cold builds#1095

Merged
mergify[bot] merged 2 commits intomainfrom
feature/component-benchmark-harness
Mar 30, 2026
Merged

feat: Local DevEx: Component Hot Reloading, component benchmarks for warm/cold builds#1095
mergify[bot] merged 2 commits intomainfrom
feature/component-benchmark-harness

Conversation

@bobbravo2
Copy link
Copy Markdown
Member

@bobbravo2 bobbravo2 commented Mar 30, 2026

Why

Every contributor's first experience with the platform is a cold install — cloning the repo and waiting for dependencies to download and compile. Today there's no systematic way to know whether that experience is getting better or worse. This PR establishes a measurable baseline for contributor setup time and incremental rebuild speed across all components, so regressions are caught before they land and improvements can be tracked over time.

Jira Story: RHOAIENG-55731

Summary

  • Adds a build-time benchmark harness (scripts/benchmarks/) that measures cold-install and warm-rebuild times for every component, compares baseline vs candidate git refs, and emits human/TSV/JSON reports
  • Fixes SIGINT handling: Ctrl+C now recursively kills the full process tree (background jobs, dev servers, dot-progress subshells), cleans up git worktrees and temp dirs, then re-raises SIGINT so make sees exit code 130
  • Guards all cd calls in bench-manifest.sh with || return 1 (benchmark functions) or || return 0 (cleanup functions) to prevent wrong-directory builds (shellcheck SC2164)
  • Adds CI workflow, self-test suite (8 tests), and documentation updates

Changes

File What
scripts/benchmarks/component-bench.sh Core harness: timing, reporting, signal handling
scripts/benchmarks/bench-manifest.sh Per-component build/warm/cleanup functions
scripts/benchmarks/README.md Semantics and caveats
.github/workflows/component-benchmarks.yml CI: self-tests + benchmarks on workflow_dispatch or benchmark label
tests/bench-test.sh 8 self-tests (syntax, coverage, reports, ANSI, fallback)
Makefile make benchmark target
CLAUDE.md Benchmark commands and notes
.claude/skills/dev-cluster/SKILL.md Agent guidance for benchmark usage
.env.local.example, .gitignore Dev env housekeeping

Test plan

  • shellcheck clean on both shell scripts (only pre-existing false-positive SC2034/SC1091/SC2155 remain)
  • bash tests/bench-test.sh passes 8/8 locally
  • GitHub Actions workflow validated locally via act (nektos/act v0.2.86 + Podman) — self-tests pass, benchmarks execute correctly in container
  • Verify Ctrl+C during make benchmark kills all children and exits cleanly
  • Full CI run on GitHub Actions
  • Closes RHOAIENG-55731

Adds a build-time benchmark harness that measures cold-install and
warm-rebuild times for every component, compares baseline vs candidate
refs, and emits human/TSV/JSON reports.

Key implementation details:
- Proper signal handling: Ctrl+C recursively kills process trees
  (bench_kill_tree via pgrep), cleans up git worktrees and temp dirs,
  then re-raises SIGINT so make sees exit code 130.
- All cd calls in bench-manifest.sh guarded with || return 1 (benchmark
  functions) or || return 0 (cleanup functions) per shellcheck SC2164.
- CI workflow (.github/workflows/component-benchmarks.yml) runs
  self-tests and full benchmarks on workflow_dispatch or 'benchmark'
  label.
- Self-test suite (tests/bench-test.sh) validates syntax, function
  coverage, report generation, and ANSI suppression (8 tests).
- CLAUDE.md, dev-cluster SKILL, and .env.local.example updated with
  benchmark usage guidance.

Made-with: Cursor
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 30, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: c35b1070-594f-4ff1-9f7d-6a6321953649

📥 Commits

Reviewing files that changed from the base of the PR and between 8ac0f85 and d3ccb1b.

📒 Files selected for processing (12)
  • .claude/skills/dev-cluster/SKILL.md
  • .env.local.example
  • .github/workflows/component-benchmarks.yml
  • .gitignore
  • CLAUDE.md
  • Makefile
  • e2e/scripts/setup-kind.sh
  • scripts/benchmarks/README.md
  • scripts/benchmarks/bench-manifest.sh
  • scripts/benchmarks/component-bench.sh
  • tests/bench-test.sh
  • tests/local-dev-test.sh

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting


Walkthrough

This PR introduces a comprehensive component benchmarking system, including new Bash scripts for benchmark execution and component setup, a GitHub Actions workflow for automated benchmark runs, significant Makefile enhancements for local development workflows (new make dev orchestration), environment configuration templates, and supporting test infrastructure and documentation.

Changes

Cohort / File(s) Summary
Documentation & Configuration
CLAUDE.md, .claude/skills/dev-cluster/SKILL.md, .env.local.example, .gitignore
Added benchmarking command documentation, expanded environment setup guidance, ignored frontend .env.local, and clarified dev-cluster skill instructions for benchmark invocation patterns and interpretation.
Benchmark Infrastructure
scripts/benchmarks/component-bench.sh, scripts/benchmarks/bench-manifest.sh, scripts/benchmarks/README.md
Implemented core benchmarking harness with per-component setup/cold/warm/cleanup phases, multi-format reporting (human/TSV/JSON), git worktree isolation, timed execution tracking, and documented benchmark semantics, cost strategies, and prerequisite toolchains.
CI/CD & Testing
.github/workflows/component-benchmarks.yml, tests/bench-test.sh, tests/local-dev-test.sh
Added GitHub Actions workflow for manual/pull-request-triggered component benchmarks with result parsing and artifact uploads; added benchmark syntax/function validation tests and script health checks.
Local Development & Cluster Setup
Makefile, e2e/scripts/setup-kind.sh
Refactored Makefile to introduce make dev orchestrator, component-scoped development targets, local cluster preflight checks, environment generation (dev-env), and new benchmarking targets (benchmark, benchmark-ci); updated kind cluster setup messaging to delegate control to Makefile.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/component-benchmark-harness

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@bobbravo2 bobbravo2 changed the title feat: add component benchmark harness with SIGINT handling feat: Better DevEx. Component Hot Reloading, component benchmarks for warm/cold builds Mar 30, 2026
@bobbravo2 bobbravo2 changed the title feat: Better DevEx. Component Hot Reloading, component benchmarks for warm/cold builds feat: Local DevEx: Component Hot Reloading, component benchmarks for warm/cold builds Mar 30, 2026
…ain permissions

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@ktdreyer ktdreyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

concept LGTM - ready to merge when you are.

@bobbravo2 bobbravo2 marked this pull request as ready for review March 30, 2026 17:08
@mergify
Copy link
Copy Markdown

mergify bot commented Mar 30, 2026

Merge Queue Status

This pull request spent 26 seconds in the queue, including 4 seconds running CI.

Required conditions to merge

mergify bot added a commit that referenced this pull request Mar 30, 2026
@mergify mergify bot merged commit 40a1e6c into main Mar 30, 2026
21 of 22 checks passed
@mergify mergify bot deleted the feature/component-benchmark-harness branch March 30, 2026 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants