tests: add CA rotation integration test and manual testing tools#21158
tests: add CA rotation integration test and manual testing tools#21158aaomidi wants to merge 3 commits intoetcd-io:mainfrom
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: aaomidi The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @aaomidi. Thanks for your PR. I'm waiting for a etcd-io member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
aaomidi
force-pushed
the
ca-reload-tests
branch
from
33a7a5b to
6af0e9d
Compare
Add CAReloader that polls CA files and updates the certificate pool when changes are detected (using SHA256 hashing). This enables zero-downtime CA rotation for mTLS clusters. Key changes: - Add CAReloader struct with Start/Stop lifecycle management - Use sync.RWMutex for thread-safe pool access - Graceful fallback: invalid CA files keep using cached pool - Add Prometheus metrics for reload operations Part 1 of CA hot-reload support (issue etcd-io#11555). Signed-off-by: Amir Omidi <amir@aaomidi.com>
Integrate CAReloader into the transport layer for both server and client TLS configurations. Add server flags to enable the feature. Transport changes: - Add ReloadTrustedCA and CAReloadInterval fields to TLSInfo - Server: use VerifyConnection callback with dynamic CA pool - Client: use DialTLSContext to inject fresh RootCAs per connection - Add caReloaderTracker for proper cleanup on shutdown Server changes: - Add --peer-tls-reload-ca flag - Add --client-tls-reload-ca flag - Add --tls-ca-reload-interval flag (default 10s) - Wire up cleanup in Etcd.Close() The feature is disabled by default and must be explicitly enabled. Part 2 of CA hot-reload support (issue etcd-io#11555). Signed-off-by: Amir Omidi <amir@aaomidi.com>
Add integration test that validates live CA rotation without cluster restart. Also add manual testing infrastructure in hack/mtls/. Integration test covers: - Starting cluster with CA1 - Adding CA2 to trust bundle - Removing CA1 and verifying rejection Manual testing (hack/mtls/): - 4-phase rotation test with 4-node cluster - Makefile for cert generation with cfssl - Procfile for goreman-based cluster Part 3 of CA hot-reload support (issue etcd-io#11555). Signed-off-by: Amir Omidi <amir@aaomidi.com>
aaomidi
force-pushed
the
ca-reload-tests
branch
from
6af0e9d to
6845887
Compare
2 participants
Summary
Add integration test that validates live CA rotation without cluster restart. Also add manual testing infrastructure in
hack/mtls/.Changes
Integration test (
tests/integration/v3_ca_rotation_test.go):Manual testing (
hack/mtls/):test-rotation.sh: 5-phase rotation test with 4-node clusterMakefile: Certificate generation using cfsslProcfile: goreman-based cluster configurationREADME.md: Usage documentationBackground
This PR is a split of #21074 into smaller, reviewable pieces as requested by maintainers.
Stacking
This PR is part of a stacked PR series for CA hot-reload support (issue #11555):
tlsutil: add CAReloader← must be merged firsttransport,embed: integrate CA hot-reload support← must be merged secondtests: add CA rotation integration testPlease review #21156 and #21157 first.
Part 3 of 3 for CA hot-reload support.
Replaces #21074 (split into smaller PRs)
Fixes #11555
Signed-off-by: Amir Omidi amir@aaomidi.com