Overhaul test infrastructure by lkdvos · Pull Request #389 · QuantumKitHub/TensorKit.jl

lkdvos · 2026-03-24T23:13:15Z

Summary

Moves test dependencies into a dedicated test/Project.toml, separating them from the main package manifest
Replaces the monolithic test runner with ParallelTestRunner.jl, running each test file in its own worker process
Adds --fast mode that skips AD test groups and reduces sector/scalar-type coverage for quick iteration
Adds test/README.md documenting how to run tests, available groups, fast mode, and how to add new test files
Updates CI to auto-discover test groups as matrix jobs; draft PRs run a reduced matrix (ubuntu + Julia 1 only) while ready PRs run the full matrix

.github/workflows/CI.yml

kshyatt

Apart from minor comment LGTM, this will be very helpful for the Enzyme PR

github-actions · 2026-03-25T18:59:06Z

Your PR no longer requires formatting changes. Thank you for your contribution!

kshyatt · 2026-03-26T09:58:51Z

Ope, GPU failures look related...

lkdvos · 2026-03-26T13:20:42Z

Needs #390 first, but should then be ready.

codecov · 2026-03-26T21:44:26Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Jutho · 2026-03-30T19:18:13Z

test/setup.jl

+    return (Vtr, Vℤ₃, VU₁, VfU₁, VCU₁, VSU₂, VfSU₂, VIB_diag, VIB_M)
+end
+
+# Gauge-fixing tangents for AD factorization tests


Clearly we should have these functions in the main MAK module?

I think this is in line with the current MAK approach. I'm happy to migrate this, but probably we should then also do this for MAK itself so we can reuse the functions.

test/setup.jl

test/cuda/factorizations.jl

test/cuda/tensors.jl

test/chainrules/tensoroperations.jl

Jutho · 2026-03-30T19:39:39Z

This generally looks good; I left a few small comments and questions. But clearly, this is too much change for a detailed review. Is there a convenient way to review such code reorganization, i.e. to separate between what has just moved to other files and what are actual changes. I could probably ask some agent, but I don't feel like doing that.

lkdvos · 2026-03-30T20:57:10Z

I don't think there is, but I did try and not actually alter anything except for the organization of the tests.
To summarize:

Split files over separate files/folders to have some parallelization
Reworked the github actions implementation
added a --fast filter that just reduces some of the tests

In principle there is no reason to review the actual contents of the test files, since these are unchanged, which also explains some of the things you commented on.
I'll address those in the meantime.

lkdvos · 2026-03-31T22:08:24Z

Update: really struggling with the CUDA tests here, that are not happy with the updated spacelist. After that, I think the branch rules should be updated to check for the test summary succeeding, and this should be good to go from there. Unfortunately I didn't manage to actually get this done, so if someone feels like continuing, that would be great.

kshyatt · 2026-04-02T10:28:54Z

Picking this back up as I exit the JuliaCon mines

Jutho · 2026-04-02T12:19:20Z

So there definitely seems to be an issue with the FiniteDifferences specification. I don't know how it works with the separate project.toml and test/project.toml files of what needs to go where. However, FiniteDifferences is in this case only appearing in test/project.toml and only has a compat entry in project.toml, so that definitely looks off.

I'll fix this locally and run the tests locally before committing.

kshyatt · 2026-04-02T12:20:14Z

And I'm looking at the CUDA failure!

kshyatt · 2026-04-02T14:15:55Z

CUDA tests are failing because the output of

bs = blocks(t)
iterate(bs)

is nothing for the IsingBimodule space. I'll try to figure out why.

Jutho · 2026-04-02T14:27:10Z

I was wrong; FiniteDifferences is a weakdeps in the main Project.toml, so it makes that its compat entry is in that file, as it is for several other weakdeps, and then this compat entry is not repeated in test/project.toml. Which means I don't understand the FiniteDifferences complaint (I can also not reproduce it locally).

kshyatt · 2026-04-02T14:32:14Z

OK, should be a quick fix..

borisdevos · 2026-04-02T14:36:40Z

CUDA tests are failing because the output of
bs = blocks(t)
iterate(bs)
is nothing for the IsingBimodule space. I'll try to figure out why.

I purposefully avoided testing the VIB_M spaces for certain tests because of the space W = V1 ⊗ V2 ⊗ V3 ⊗ V4 ⊗ V5 being considered. Long story short, it's a hom-space that's forbidden by fusion rules. The new default_spacelist function will take these spaces for every situation.

Edit: turns out I didn't avoid testing them, I just worked around them the same way you did in your recent commit 😄

Jutho · 2026-04-02T14:38:35Z

test/cuda/tensors.jl

    @timedtestset "Tensors with symmetry: $Istr" verbose = true begin
        V1, V2, V3, V4, V5 = V
        @timedtestset "Basic tensor properties" begin
            W = V1 ⊗ V2 ⊗ V3 ⊗ V4 ⊗ V5


The problem is rather that the IsingBimodule spaces have been constructed such that V1 ⊗ V2 ← V3 ⊗ V4 ⊗ V5 is non-empty. I don't think it was ever the goal to construct a TensorMap{5,0} with codomain given by V1 ⊗ V2 ⊗ V3 ⊗ V4 ⊗ V5.

We should probably be a bit more consistent with which tensors are being constructed, to avoid indeed empty tensor or overly large tensors.

kshyatt · 2026-04-02T14:39:23Z

Yeah I figured it out in the end. Might be good to add a comment to the setup file about this.

…

On Thu, Apr 2, 2026 at 4:37 PM Boris De Vos ***@***.***> wrote: *borisdevos* left a comment (QuantumKitHub/TensorKit.jl#389) <#389?email_source=notifications&email_token=AAGKJYZGJRKV3G5WUIQ2AUT4TZ3I3A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJXHAZTMMJQGU3KM4TFMFZW63VHMNXW23LFNZ2KKZLWMVXHJNLQOJPWG33NNVSW45C7N5YGK3S7MNWGSY3L#issuecomment-4178361056> CUDA tests are failing because the output of bs = blocks(t)iterate(bs) is nothing for the IsingBimodule space. I'll try to figure out why. I purposefully avoided testing the VIB_M spaces for certain tests because of the space W = V1 ⊗ V2 ⊗ V3 ⊗ V4 ⊗ V5 being considered. Long story short, it's a hom-space that's forbidden by fusion rules. The new default_spacelist function will take these spaces for every situation. — Reply to this email directly, view it on GitHub <#389?email_source=notifications&email_token=AAGKJYZGJRKV3G5WUIQ2AUT4TZ3I3A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJXHAZTMMJQGU3KM4TFMFZW63VHMNXW23LFNZ2KKZLWMVXHJNLQOJPWG33NNVSW45C7N5YGK3S7MNWGSY3L#issuecomment-4178361056>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGKJY64YTUV24QMFUWPFN34TZ3I3AVCNFSM6AAAAACW6DFFAGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DCNZYGM3DCMBVGY> . You are receiving this because you commented.Message ID: ***@***.***>

Jutho · 2026-04-02T14:42:26Z

Also, locally I know get Aqua ambiguity warnings (on Julia 1.10)

1 ambiguities found. To get a list, set `broken = false`.
Ambiguity #1
findtruncated_svd(values::AbstractVector, strategy::MatrixAlgebraKit.TruncationUnion) @ MatrixAlgebraKit ~/.julia/packages/MatrixAlgebraKit/x7OBV/src/implementations/truncation.jl:147
findtruncated_svd(values::TensorKit.SectorVector, strategy::MatrixAlgebraKit.TruncationStrategy) @ TensorKit.Factorizations ~/.julia/packages/TensorKit/Lo4KY/src/factorizations/truncation.jl:156

Possible fix, define
  findtruncated_svd(::TensorKit.SectorVector, ::MatrixAlgebraKit.TruncationUnion)

I don't understand how this wasn't caught on previous merges, since this cannot be caused
by the current PR

kshyatt · 2026-04-02T16:30:03Z

I just worked around them the same way you did in your recent commit 😄

Great minds etc etc...

kshyatt · 2026-04-02T16:41:12Z

Also, locally I know get Aqua ambiguity warnings (on Julia 1.10)

1 ambiguities found. To get a list, set `broken = false`.
Ambiguity #1
findtruncated_svd(values::AbstractVector, strategy::MatrixAlgebraKit.TruncationUnion) @ MatrixAlgebraKit ~/.julia/packages/MatrixAlgebraKit/x7OBV/src/implementations/truncation.jl:147
findtruncated_svd(values::TensorKit.SectorVector, strategy::MatrixAlgebraKit.TruncationStrategy) @ TensorKit.Factorizations ~/.julia/packages/TensorKit/Lo4KY/src/factorizations/truncation.jl:156

Possible fix, define
  findtruncated_svd(::TensorKit.SectorVector, ::MatrixAlgebraKit.TruncationUnion)

I don't understand how this wasn't caught on previous merges, since this cannot be caused by the current PR

This is after fixing the FD issue? Could the FD be pushed so I can take a look alongside (if you like, of course!)

Jutho · 2026-04-02T18:57:28Z

I am fixing / changing a few other things along the way. I will try to push something later tonight.

kshyatt · 2026-04-02T19:02:12Z

Cool, sorry to rush you!

…

On Thu, Apr 2, 2026 at 8:57 PM Jutho ***@***.***> wrote: *Jutho* left a comment (QuantumKitHub/TensorKit.jl#389) <#389?email_source=notifications&email_token=AAGKJY254LLL7S7FKBY7FU34T2Z23A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJXHE4DMOBYGQ32M4TFMFZW63VHMNXW23LFNZ2KKZLWMVXHJNLQOJPWG33NNVSW45C7N5YGK3S7MNWGSY3L#issuecomment-4179868847> I am fixing / changing a few other things along the way. I will try to push something later tonight. — Reply to this email directly, view it on GitHub <#389?email_source=notifications&email_token=AAGKJY254LLL7S7FKBY7FU34T2Z23A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJXHE4DMOBYGQ32M4TFMFZW63VHMNXW23LFNZ2KKZLWMVXHJNLQOJPWG33NNVSW45C7N5YGK3S7MNWGSY3L#issuecomment-4179868847>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGKJYZRIFBNATBMJEQZ3LD4T2Z23AVCNFSM6AAAAACW6DFFAGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DCNZZHA3DQOBUG4> . You are receiving this because you commented.Message ID: ***@***.***>

Jutho · 2026-04-02T23:17:59Z

Didn't manage to finish this; some tests are still failing, so it feels stupid to strain the CI with that. Will continue tomorrow.

kshyatt reviewed Mar 25, 2026

View reviewed changes

.github/workflows/CI.yml Outdated Show resolved Hide resolved

kshyatt previously approved these changes Mar 25, 2026

View reviewed changes

lkdvos dismissed kshyatt’s stale review via e70866f March 25, 2026 12:47

lkdvos force-pushed the ld-testing branch from e70866f to 9c586f3 Compare March 25, 2026 14:00

kshyatt self-requested a review March 26, 2026 09:58

lkdvos force-pushed the ld-testing branch from 9470fa8 to 41da06f Compare March 26, 2026 13:20

lkdvos force-pushed the ld-testing branch from 7560fd9 to e75ce11 Compare March 26, 2026 21:17

lkdvos force-pushed the ld-testing branch from e75ce11 to 6edd7d0 Compare March 27, 2026 12:50

lkdvos marked this pull request as ready for review March 27, 2026 13:23

lkdvos force-pushed the ld-testing branch 2 times, most recently from ab125b5 to e3ce090 Compare March 30, 2026 12:56