Use contextual logging for events

[Garfield96]

Errors of the event recorder are shown in the wrong format. Example:

E0811 00:00:00.000000       1 event.go:359] "Server rejected event (will not retry!)" err="Unauthorized" event="&Event{ObjectMeta:{...

The event recorder was extended in client-go to support contextual logging (change: kubernetes/client-go@3595e52#diff-401f00d69d10cd2fce62a75c1ce524c04c978d646e13f532f0a64b8149efcfe1)

To implement structured logging in controller-runtime, a context with the logger must be passed to the broadcaster via StartRecordingToSinkWithContext. At the moment, context.Background() is used:

controller-runtime/pkg/internal/recorder/recorder.go

Lines 120 to 125 in adb6465

	ctx, cancel := context.WithCancel(context.Background())
	p.cancelSinkRecordingFunc = cancel
	if err := p.broadcaster.StartRecordingToSinkWithContext(ctx); err != nil {
	p.logger.Error(err, "error starting recording for broadcaster")
	return
	}

To inject the logger, log.IntoContext() can be used:

controller-runtime/pkg/log/log.go

Line 103 in adb6465

func IntoContext(ctx context.Context, log logr.Logger) context.Context {

In addition, it could be an option to use StartLogging (doc: https://pkg.go.dev/k8s.io/client-go@v0.35.0/tools/events#EventBroadcaster, implementation: https://github.com/kubernetes/client-go/blob/9bcb69436287b966d0c5c195efef00aed921fb1b/tools/events/event_broadcaster.go#L330-L340) instead of StartEventWatcher to align the log messages for events with the implementation in client-go:

controller-runtime/pkg/internal/recorder/recorder.go

Lines 102 to 105 in 7a1b16d

	broadcaster.StartEventWatcher(
	func(e *corev1.Event) {
	p.logger.V(1).Info(e.Message, "type", e.Type, "object", e.InvolvedObject, "reason", e.Reason)
	})

This issue is related to #2656, but #2656 mainly discusses the adoption of structured logging for the leader election package.

If you agree to the proposed changes, I can create a PR.

[troy0820]

I think this is reasonable. You can inject the context from what is defined there and add the logger to pass in like you explained. Any thoughts @sbueringer @alvaroaleman ?