Eliminate the global state from TrainJob reconciler

[mimowo]

What would you like to be cleaned:

I would like to eliminate the global state from the TrainJob reconciler https://github.com/kubernetes-sigs/kueue/blob/main/pkg/controller/jobs/trainjob/trainjob_controller.go#L87

Why is this needed:

This is causing non-obvious problems in the MultiKueue integration tests, as we encountered in the PR: https://github.com/kubernetes-sigs/kueue/pull/8341/changes#r2766310790

[mimowo]

cc @kaisoz @sohankunkerkar @hiboyang