demo: Add benchmark-react with normalization and ref-stability scenarios#3783
demo: Add benchmark-react with normalization and ref-stability scenarios#3783
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
|
ntucker
force-pushed
the
benchmark-react-normalization
branch
from
af004f6 to
a2cb25d
Compare
|
Size Change: +16 B (+0.02%) Total Size: 80.5 kB
ℹ️ View Unchanged
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3783 +/- ##
==========================================
- Coverage 98.05% 97.99% -0.07%
==========================================
Files 151 151
Lines 2834 2836 +2
Branches 555 556 +1
==========================================
Hits 2779 2779
- Misses 11 13 +2
Partials 44 44 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
ntucker
force-pushed
the
benchmark-react-normalization
branch
from
d8d904f to
98de6fa
Compare
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Benchmark
Details
| Benchmark suite | Current: 6768766 | Previous: a8c10ca | Ratio |
|---|---|---|---|
normalizeLong |
455 ops/sec (±1.07%) |
448 ops/sec (±0.82%) |
0.98 |
normalizeLong Values |
418 ops/sec (±0.38%) |
404 ops/sec (±0.48%) |
0.97 |
denormalizeLong |
287 ops/sec (±3.00%) |
275 ops/sec (±2.97%) |
0.96 |
denormalizeLong Values |
265 ops/sec (±2.30%) |
264 ops/sec (±2.36%) |
1.00 |
denormalizeLong donotcache |
1044 ops/sec (±0.15%) |
1043 ops/sec (±0.15%) |
1.00 |
denormalizeLong Values donotcache |
783 ops/sec (±0.18%) |
759 ops/sec (±0.20%) |
0.97 |
denormalizeShort donotcache 500x |
1570 ops/sec (±0.08%) |
1431 ops/sec (±0.16%) |
0.91 |
denormalizeShort 500x |
856 ops/sec (±2.22%) |
785 ops/sec (±2.10%) |
0.92 |
denormalizeShort 500x withCache |
6372 ops/sec (±0.07%) |
4769 ops/sec (±0.62%) |
0.75 |
queryShort 500x withCache |
2727 ops/sec (±0.14%) |
2611 ops/sec (±0.31%) |
0.96 |
buildQueryKey All |
53854 ops/sec (±0.35%) |
58214 ops/sec (±0.33%) |
1.08 |
query All withCache |
6316 ops/sec (±0.78%) |
5813 ops/sec (±0.25%) |
0.92 |
denormalizeLong with mixin Entity |
277 ops/sec (±2.18%) |
277 ops/sec (±2.02%) |
1 |
denormalizeLong withCache |
6819 ops/sec (±0.12%) |
6431 ops/sec (±0.09%) |
0.94 |
denormalizeLong Values withCache |
5059 ops/sec (±0.15%) |
4658 ops/sec (±0.10%) |
0.92 |
denormalizeLong All withCache |
6210 ops/sec (±0.14%) |
5468 ops/sec (±0.09%) |
0.88 |
denormalizeLong Query-sorted withCache |
6500 ops/sec (±0.31%) |
6294 ops/sec (±0.09%) |
0.97 |
denormalizeLongAndShort withEntityCacheOnly |
1690 ops/sec (±0.21%) |
1551 ops/sec (±0.25%) |
0.92 |
getResponse |
4660 ops/sec (±0.65%) |
3770 ops/sec (±0.23%) |
0.81 |
getResponse (null) |
9440466 ops/sec (±0.93%) |
9815415 ops/sec (±0.77%) |
1.04 |
getResponse (clear cache) |
275 ops/sec (±1.95%) |
272 ops/sec (±2.12%) |
0.99 |
getSmallResponse |
3360 ops/sec (±0.11%) |
3231 ops/sec (±0.39%) |
0.96 |
getSmallInferredResponse |
2457 ops/sec (±0.19%) |
2476 ops/sec (±0.21%) |
1.01 |
getResponse Collection |
4578 ops/sec (±0.30%) |
3763 ops/sec (±0.15%) |
0.82 |
get Collection |
4566 ops/sec (±0.22%) |
3539 ops/sec (±0.31%) |
0.78 |
get Query-sorted |
5109 ops/sec (±0.92%) |
5072 ops/sec (±0.11%) |
0.99 |
setLong |
452 ops/sec (±0.26%) |
452 ops/sec (±0.54%) |
1 |
setLongWithMerge |
257 ops/sec (±0.12%) |
258 ops/sec (±0.29%) |
1.00 |
setLongWithSimpleMerge |
274 ops/sec (±0.12%) |
273 ops/sec (±0.18%) |
1.00 |
setSmallResponse 500x |
957 ops/sec (±0.07%) |
903 ops/sec (±0.17%) |
0.94 |
This comment was automatically generated by workflow using github-action-benchmark.
ntucker
force-pushed
the
benchmark-react-normalization
branch
from
5ad1b81 to
4df9d01
Compare
ntucker
force-pushed
the
benchmark-react-normalization
branch
2 times, most recently
from
c8bc040 to
081e48d
Compare
- Browser benchmark comparing @data-client/react (Playwright, customSmallerIsBetter). - Scenarios: mount, update entity/author, ref-stability (item/author ref counts). - Hot-path (CI) vs with-network (local): simulated delay for overfetch comparison. - CI workflow runs hot-path only; reports to rhysd/github-action-benchmark. Made-with: Cursor
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| ); | ||
| const margin = 1.96 * (stdDev / Math.sqrt(trimmed.length)); | ||
| return (margin / Math.abs(mean)) * 100 <= targetMarginPct; | ||
| } |
There was a problem hiding this comment.
Convergence check uses mean, not median as documented
Low Severity
The isConverged docstring and the RunProfile.targetMarginPct JSDoc both state convergence is checked against the median, but the implementation computes margin / Math.abs(mean). Since computeStats reports the median as the final value, convergence is assessed against a different central tendency than the one reported. For skewed distributions (common with benchmark outliers), mean and median can diverge significantly, causing premature or delayed convergence relative to what the median-based report actually needs.
Additional Locations (1)
…as HTML Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 3 total unresolved issues (including 2 from previous reviews).
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
| const lastPaint = | ||
| paintEvents.length ? | ||
| Math.max(...paintEvents.map(e => e.ts + (e.dur ?? 0))) | ||
| : Math.max(...events.map(e => e.ts + (e.dur ?? 0))); |
There was a problem hiding this comment.
Spread into Math.min/max can stack overflow on large traces
Low Severity
Math.min(...events.map(...)) and Math.max(...) spread the entire events array as function arguments. Chrome traces with the devtools.timeline,blink categories can easily produce tens of thousands of events, exceeding the JS engine's maximum call stack size. The outer try/catch silently returns 0, making trace results unreliable without any warning when this occurs.
1 participant
Motivation
We need a browser-based benchmark for
@data-client/reactthat highlights the performance advantages of normalization and referential equality: when one entity is updated, only components that use that entity get new references; others keep the same object reference so React can skip rerenders. This enables comparison against other React data libraries (e.g. TanStack Query, SWR) and surfaces regressions in CI.Solution
examples/benchmark-react: Playwright-driven benchmark that runs a small React app with shared authors, measures mount/update duration viaperformance.measure(), and reports incustomSmallerIsBetterformat forrhysd/github-action-benchmark.data-clienttrack our regressions; competitor libraries (TanStack Query, SWR, baseline) are benchmarked locally for comparison only.yarn bench:run:compilerbuilds withbabel-plugin-react-compiler(via@anansi/babel-preset'sreactCompileroption) and labels results[compiler]for side-by-side comparison in the report viewer.Scenarios
Hot path (CI, data-client only)
Queryschema memoization vsuseMemosort)Local comparison only
Key design decisions
performance.measure()(JS-driven), React ProfileractualDuration(reconciliation), Chrome trace duration (full rendering pipeline)bench/report-viewer.html): filterable table (base/react-commit/trace), time-series charting via history file loadingScripts
yarn bench:runyarn bench:run:compiler[compiler]yarn build:compileryarn bench:compiler[compiler]label onlyOpen questions
N/A
Made with Cursor
Note
Medium Risk
Adds a large new Playwright-driven benchmark app plus a new CI workflow, and extends the public
Collectionschema API withmoveWith, which could affect endpoint type/behavior expectations if misused.Overview
Adds a new
examples/benchmark-reactbrowser benchmark suite that compares@data-client/reactagainst TanStack Query, SWR, and a baseline, including scenarios for mount/update timing, ref-stability counts, derived/sorted views, list/detail switching, and optional memory/trace measurements.Wires this suite into CI via a new
benchmark-reactGitHub Actions workflow (Playwright +rhysd/github-action-benchmark) and updates existing workflows with concurrency controls and benchmark permissions; also addsbuild:benchmark-reactand includes the workspace in the rootpackage.json.Extends
@data-client/endpointcollections by addingCollection.moveWith()(and corresponding typings) to build custom move schemas, and updates generated playground type definitions accordingly.Written by Cursor Bugbot for commit 6768766. This will update automatically on new commits. Configure here.