fix: replace abort-on-invalid-label with graceful bounds check

[santhsecurity]

The update_label function uses assert() to validate the label index, which calls abort() on failure. In embedding contexts where QuickJS executes untrusted JavaScript (sandboxes, URL detonation engines, server-side JS), crafted scripts can trigger the assertion through pathological code patterns that produce invalid label indices during bytecode optimization.

This turns a logic error into a process-killing SIGABRT, which is a denial-of-service vector for any application embedding QuickJS.

The fix replaces the fatal assert with a bounds check that returns -1. Callers that check the return value (e.g., the if (update_label(...) > 0) pattern in resolve_labels) already handle -1 correctly. Callers that ignore the return value are unaffected. An a -1 return on an invalid label is strictly better than a process abort.

The second assert (ref_count >= 0) is similarly replaced with a clamp to zero, preventing underflow from cascading into further undefined behavior.

Discovered during adversarial testing of the Sear URL detonation engine (https://santh.io/sear), which executes untrusted JavaScript from phishing pages in a QuickJS sandbox.

[saghul]

I'm not sure about this one. The whole bytecode reader was not designed for untrusted input. You can patch some calls but I think there are way more that would surface with specially crafter payloads.

Why not evaluate JS code rather than bytecode?

[bnoordhuis]

crafted scripts can trigger the assertion through pathological code patterns that produce invalid label indices during bytecode optimization

Assuming that's actually possible, then that's a bug and it needs to be fixed, not worked around.

[bnoordhuis]

I don't think we'll be merging this so I'm going to close it. Thanks anyway for the pull request.