Skip to content

e2e testing: load images into kind worker nodes in parallel #9052

@mimowo

Description

@mimowo

What would you like to be cleaned:

Looking at the function to load images:

kueue/hack/e2e-common.sh

Lines 363 to 386 in 1d93e45

# $1 cluster
# $2 image
function cluster_kind_load_image {
# check if the command to get worker nodes could succeeded
if ! $KIND get nodes --name "$1" > /dev/null 2>&1; then
echo "Failed to retrieve nodes for cluster '$1'."
return 1
fi
# filter out 'control-plane' node, use only worker nodes to load image
worker_nodes=$($KIND get nodes --name "$1" | grep -v 'control-plane')
if [[ -n "$worker_nodes" ]]; then
# Use docker save + ctr import directly to avoid the --all-platforms
# issue with multi-arch images in DinD environments.
# See: https://github.com/kubernetes-sigs/kind/issues/3795
echo "Loading image '$2' to cluster '$1'"
while IFS= read -r node; do
echo " Loading image to node: $node"
if ! docker save "$2" | docker exec -i "$node" ctr --namespace=k8s.io images import --digests --snapshotter=overlayfs -; then
echo "Failed to load image '$2' to node '$node'"
return 1
fi
done <<< "$worker_nodes"
fi
}

I think we could load the images in parallel.
This will require calling docker save "$2" prior to the parallel loading.

Why is this needed:

To improve time to setup the cluster for testing. This especially will matter for TAS e2e tests which create 8 worker nodes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/cleanupCategorizes issue or PR as related to cleaning up code, process, or technical debt.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions