Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 44 additions & 6 deletions .github/workflows/code-quality.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,16 @@
name: Github Action - Code Quality
name: Code Quality

on:
pull_request:
branches: [main]
types: [opened, reopened, synchronize]
workflow_dispatch:

jobs:
quality:
name: Quality
# The quality pipeline should not take more that 2 minutes.
timeout-minutes: 2
# The quality pipeline should not take more than 10 minutes.
timeout-minutes: 10
runs-on: ubuntu-latest
steps:
- name: Checkout code
Expand All @@ -18,15 +20,51 @@ jobs:

- name: Get changed markdown files
id: changed-markdown
uses: tj-actions/changed-files@v41
uses: tj-actions/changed-files@v45
with:
files: |
**/*.md
.github/workflows/github-action-code-quality.yml
.github/workflows/code-quality.yml

- name: Get changed action and test paths
id: changed-actions
uses: tj-actions/changed-files@v45
with:
files: |
.github/actions/**
tests/**

- name: Lint all Documentation
- name: Lint all documentation
if: steps.changed-markdown.outputs.any_modified == 'true'
uses: DavidAnson/markdownlint-cli2-action@v14
with:
globs: |
**/*.md

- name: Install bats
if: steps.changed-actions.outputs.any_changed == 'true'
run: sudo npm install -g bats

- name: Test - promote-ecr-image
if: >
contains(steps.changed-actions.outputs.all_changed_files, '.github/actions/promote-ecr-image') ||
contains(steps.changed-actions.outputs.all_changed_files, 'tests/unit/promote-ecr-image')
run: bats --verbose-run tests/unit/promote-ecr-image/

- name: Test - test-python
if: >
contains(steps.changed-actions.outputs.all_changed_files, '.github/actions/test-python') ||
contains(steps.changed-actions.outputs.all_changed_files, 'tests/unit/test-python')
run: bats --verbose-run tests/unit/test-python/

- name: Test - update-aws-ecs
if: >
contains(steps.changed-actions.outputs.all_changed_files, '.github/actions/update-aws-ecs') ||
contains(steps.changed-actions.outputs.all_changed_files, 'tests/unit/update-aws-ecs')
run: bats --verbose-run tests/unit/update-aws-ecs/

- name: Test - update-aws-lambda
if: >
contains(steps.changed-actions.outputs.all_changed_files, '.github/actions/update-aws-lambda') ||
contains(steps.changed-actions.outputs.all_changed_files, 'tests/unit/update-aws-lambda')
run: bats --verbose-run tests/unit/update-aws-lambda/
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,8 @@ This repo hold various accelerators for Github workflows (Actions) as well as re

### Linting and Formatting

See [LINTING.md](./LINTING.md) for more information.
See [documentation/LINTING.md](./documentation/LINTING.md) for more information.

### Testing

See [documentation/TESTING.md](./documentation/TESTING.md) for more information.
File renamed without changes.
105 changes: 105 additions & 0 deletions documentation/TESTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# Testing

Unit tests for the shell scripts that power the GitHub Actions in this repository.

## Framework

Tests are written using [bats-core](https://github.com/bats-core/bats-core) — the Bash Automated Testing System.

## Structure

Tests are organised per-action, mirroring the mono-repo structure of the actions themselves.
Each action that contains testable shell scripts has a corresponding directory under `tests/unit/`.

```text
tests/
├── unit/
│ ├── promote-ecr-image/
│ │ ├── test_aws_unset.bats # Tests for the shared aws_unset.sh utility
│ │ ├── test_options_helpers.bats # Tests for the shared options_helpers.sh utility
│ │ └── test_promote_image.bats # Tests for promote_image.sh
│ ├── test-python/
│ │ └── test_configure_pip.bats # Tests for configure_pip.sh
│ ├── update-aws-ecs/
│ │ └── test_update_ecs.bats # Tests for update_ecs.sh
│ └── update-aws-lambda/
│ └── test_update_lambda.bats # Tests for update_lambda.sh
└── helpers/
└── mock_helpers.bash # Shared mock creation and assertion utilities
```

## What Is Tested

| Action | Script | Tests | What's covered |
|--------|--------|-------|----------------|
| `promote-ecr-image` | `options_helpers.sh` | 15 | `has_argument()` and `extract_argument()` parsing logic |
| `promote-ecr-image` | `aws_unset.sh` | 7 | All 4 AWS credential env vars are cleared; no-op when already unset |
| `promote-ecr-image` | `promote_image.sh` | 13 | Every required env var validation (exits 1 for each missing var); `--help` |
| `test-python` | `configure_pip.sh` | 10 | Correct `pip config set` calls per env var; no-op when unset; `--help` |
| `update-aws-ecs` | `update_ecs.sh` | 8 | `--help`, `aws ecs update-service` invocation, `--force-new-deployment`, failure path |
| `update-aws-lambda` | `update_lambda.sh` | 7 | `--help`, `aws lambda update-function-code` invocation, failure path |

### What Is NOT Tested Here

- **Composite action YAML** — action `.yml` files use GitHub Actions expression syntax
(`${{ inputs.xxx }}`) that cannot run outside of a GitHub Actions runner.
- **Live AWS calls** — tests that require actual AWS credentials are integration tests
and must run in a real CI environment with OIDC or stored secrets.

## Mocking Strategy

External commands (`aws`, `pip`, `tput`) are replaced with lightweight mock binaries
that record every invocation to a log file (`$MOCK_DIR/<command>_calls.log`).
Tests assert the correct arguments were passed without hitting real cloud APIs.

`tests/helpers/mock_helpers.bash` provides shared utilities for creating mocks and
making assertions against them.

## Running Locally

### Install bats

```sh
# via npm (recommended — matches the CI install)
npm install -g bats

# via Homebrew
brew install bats-core
```

### Run all tests for a specific action

```sh
bats tests/unit/update-aws-ecs/
```

### Run tests for all actions

```sh
for dir in tests/unit/*/; do bats --verbose-run "$dir"; done
```

### Run with verbose output

```sh
bats --verbose-run tests/unit/promote-ecr-image/
```

## CI

The `code-quality.yml` workflow runs automatically on every PR to `main`.
It uses `tj-actions/changed-files` to detect which action directories have changed
and runs tests only for those actions — each in its own isolated job.

## Writing New Tests

1. Create `tests/unit/<action-name>/test_<script_name>.bats`.
2. Set `REPO_ROOT` relative to `BATS_TEST_DIRNAME` — tests are three levels deep,
so use: `REPO_ROOT="$(cd "$BATS_TEST_DIRNAME/../../.." && pwd)"`
3. In `setup()`, create a `MOCK_DIR`, add mocks for any external commands, and prepend
`$MOCK_DIR` to `PATH` — subshells spawned by `run bash -c "..."` inherit the PATH
automatically, so do not re-export `PATH` inside the subshell.
4. Use `run bash -c "source '...script.sh'"` for tests that need to capture a non-zero
exit code from the script under test.

See existing test files for patterns.
88 changes: 88 additions & 0 deletions tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Tests

Unit tests for the shell scripts that power the GitHub Actions in this repository.

## Framework

Tests are written using [bats-core](https://github.com/bats-core/bats-core) — the Bash Automated Testing System.

## Structure

Tests are organised per-action, mirroring the mono-repo structure of the actions themselves.
Each action that contains testable shell scripts has a corresponding directory under `tests/unit/`.

```text
tests/
├── unit/
│ ├── promote-ecr-image/
│ │ ├── test_aws_unset.bats # Tests for the shared aws_unset.sh utility
│ │ ├── test_options_helpers.bats # Tests for the shared options_helpers.sh utility
│ │ └── test_promote_image.bats # Tests for promote_image.sh
│ ├── test-python/
│ │ └── test_configure_pip.bats # Tests for configure_pip.sh
│ ├── update-aws-ecs/
│ │ └── test_update_ecs.bats # Tests for update_ecs.sh
│ └── update-aws-lambda/
│ └── test_update_lambda.bats # Tests for update_lambda.sh
└── helpers/
└── mock_helpers.bash # Shared mock creation and assertion utilities
```

## What Is Tested

| Action | Script | Tests |
|--------|--------|-------|
| `promote-ecr-image` | `options_helpers.sh` | `has_argument()` and `extract_argument()` parsing logic |
| `promote-ecr-image` | `aws_unset.sh` | All four AWS credential env vars are cleared |
| `promote-ecr-image` | `promote_image.sh` | Env var validation (exits 1 for each missing required var) |
| `test-python` | `configure_pip.sh` | Correct `pip config set` calls for each env var; no-op when unset |
| `update-aws-ecs` | `update_ecs.sh` | AWS CLI invocation, `--force-new-deployment`, empty-response failure |
| `update-aws-lambda` | `update_lambda.sh` | AWS CLI invocation, function name + image URL propagation, failure |

### What Is NOT Tested Here

- **Composite action YAML** — action `.yml` files use GitHub Actions expression syntax
(`${{ inputs.xxx }}`) that cannot run outside of a GitHub Actions runner.
- **Live AWS calls** — tests that require actual AWS credentials are integration tests
and must run in a real CI environment with OIDC or stored secrets.

## Running Locally

### Install bats

```bash
# via npm (recommended)
npm install -g bats

# via Homebrew
brew install bats-core
```

### Run all tests for a specific action

```bash
# From the repo root
bats tests/unit/update-aws-ecs/
```

### Run tests for all actions

```bash
for dir in tests/unit/*/; do bats --verbose-run "$dir"; done
```

### Run with verbose output

```bash
bats --verbose-run tests/unit/promote-ecr-image/
```

## Writing New Tests

1. Create `tests/unit/<action-name>/test_<script_name>.bats`
2. Set `REPO_ROOT` relative to `BATS_TEST_DIRNAME` — tests are three levels deep,
so use: `REPO_ROOT="$(cd "$BATS_TEST_DIRNAME/../../.." && pwd)"`
3. Mock external commands (`aws`, `docker`, `pip`) using `MOCK_DIR` in PATH
4. Use `run bash -c "..."` for tests that expect `exit 1` from the script under test

See existing test files and `tests/helpers/mock_helpers.bash` for patterns.
98 changes: 98 additions & 0 deletions tests/helpers/mock_helpers.bash
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
#!/usr/bin/env bash
# =============================================================================
# mock_helpers.bash
# Shared helper utilities for bats unit tests.
# Provides mock command creation and assertion utilities.
# =============================================================================

# Set up a temporary directory and create mock executables for the given
# commands. Each mock records every call (arguments) to
# "$MOCK_DIR/<command>_calls.log" and exits 0 by default.
#
# Usage:
# source tests/helpers/mock_helpers.bash
# setup_mocks pip tput aws docker
#
# After setup_mocks, $MOCK_DIR is in PATH, so the fakes shadow real binaries.
setup_mocks() {
MOCK_DIR="$(mktemp -d)"
export MOCK_DIR

for cmd in "$@"; do
# Write the mock script
cat > "$MOCK_DIR/$cmd" << 'MOCK_SCRIPT'
#!/usr/bin/env bash
# Record the call
echo "$@" >> "${MOCK_DIR}/${0##*/}_calls.log"
exit 0
MOCK_SCRIPT

# Substitute the actual command name (heredoc can't expand $cmd)
sed -i.bak "s|\${0##\*/}|${cmd}|g" "$MOCK_DIR/$cmd"
rm -f "$MOCK_DIR/$cmd.bak"
chmod +x "$MOCK_DIR/$cmd"
done

# Prepend mock dir to PATH so mocks shadow real binaries
export PATH="$MOCK_DIR:$PATH"
}

# Remove the mock directory created by setup_mocks.
teardown_mocks() {
if [[ -n "${MOCK_DIR:-}" && -d "$MOCK_DIR" ]]; then
rm -rf "$MOCK_DIR"
fi
}

# Assert that a mock was called with the given argument string.
#
# Usage:
# assert_mock_called_with "pip" "config set global.index-url https://example.com"
assert_mock_called_with() {
local cmd="$1"
local expected_args="$2"
local log_file="$MOCK_DIR/${cmd}_calls.log"

if [[ ! -f "$log_file" ]]; then
echo "FAIL: mock '$cmd' was never called (no call log found)" >&2
return 1
fi

if ! grep -qF "$expected_args" "$log_file"; then
echo "FAIL: mock '$cmd' was not called with args: $expected_args" >&2
echo "Actual calls recorded in $log_file:" >&2
cat "$log_file" >&2
return 1
fi

return 0
}

# Assert that a mock was NOT called at all.
#
# Usage:
# assert_mock_not_called "pip"
assert_mock_not_called() {
local cmd="$1"
local log_file="$MOCK_DIR/${cmd}_calls.log"

if [[ -f "$log_file" ]]; then
echo "FAIL: mock '$cmd' was called but should not have been" >&2
echo "Calls recorded:" >&2
cat "$log_file" >&2
return 1
fi

return 0
}

# Return the call count for a given mock command.
mock_call_count() {
local cmd="$1"
local log_file="$MOCK_DIR/${cmd}_calls.log"
if [[ -f "$log_file" ]]; then
wc -l < "$log_file" | tr -d ' '
else
echo "0"
fi
}
Loading
Loading