feat: publish emulator image directly from testing library#196
feat: publish emulator image directly from testing library#196
Conversation
|
🤖 Emulator PR Created A draft PR has been created with locked dependencies: ➡️ https://github.com/aws/aws-durable-execution-emulator/pull/401 The emulator will build binaries using the exact testing SDK commit locked in uv.lock. |
|
🔄 Emulator PR Updated The emulator PR has been updated with locked dependencies: ➡️ https://github.com/aws/aws-durable-execution-emulator/pull/401 |
|
🔄 Emulator PR Updated The emulator PR has been updated with locked dependencies: ➡️ https://github.com/aws/aws-durable-execution-emulator/pull/401 |
|
🔄 Emulator PR Updated The emulator PR has been updated with locked dependencies: ➡️ https://github.com/aws/aws-durable-execution-emulator/pull/401 |
|
🔄 Emulator PR Updated The emulator PR has been updated with locked dependencies: ➡️ https://github.com/aws/aws-durable-execution-emulator/pull/401 |
|
🔄 Emulator PR Updated The emulator PR has been updated with locked dependencies: ➡️ https://github.com/aws/aws-durable-execution-emulator/pull/401 |
|
🔄 Emulator PR Updated The emulator PR has been updated with locked dependencies: ➡️ https://github.com/aws/aws-durable-execution-emulator/pull/401 |
.github/workflows/ci.yml
Outdated
| python -m pip install hatch==1.15.0 | ||
| - name: Install specific version of Virtual Env due to bug with hatch | ||
| run: | | ||
| python -m pip install virtualenv==20.39.0 |
|
|
||
| # Copy and install the wheel | ||
| COPY dist/*.whl /tmp/ | ||
| RUN pip install --no-cache-dir /tmp/*.whl && rm -rf /tmp/*.whl |
There was a problem hiding this comment.
nit: rm wouldn't reduce image size
There was a problem hiding this comment.
you could use a bind mount
RUN --mount=type=bind,source=dist,target=/tmp/dist pip install --no-cache-dir /tmp/dist/*.whl
(then you don't need to COPY before)
| - name: Install Hatch | ||
| run: pip install hatch | ||
|
|
||
| - name: Install specific version of Virtual Env due to bug with hatch |
There was a problem hiding this comment.
for posterity, which bug in hatch? so we know when/what to fix in future
There was a problem hiding this comment.
I couldn't easily find any mention of this but I added this/experienced this on Feb 25th where you can see, there were multiple updates to virtualenv version to try to fix this: https://pypi.org/project/virtualenv/#history. I will see if the tests work without the newest version released on Feb 26.
.github/workflows/ecr-release.yml
Outdated
| run: hatch build | ||
| - name: pip install | ||
| run: | | ||
| pip install -e . |
There was a problem hiding this comment.
why? we're only interested in the whl hatch builds in the previous step?
if it's to extract the version number, can do that without installing like this:
VERSION=$(python -c "import sys; sys.path.insert(0, 'src'); from aws_durable_execution_sdk_python_testing.__about__ import __version__; print(__version__)")
or even just grep it:
VERSION=$(grep "^__version__" src/aws_durable_execution_sdk_python_testing/__about__.py | cut -d'"' -f2)
There was a problem hiding this comment.
yeah you're right, it was only to fetch the the version.
| AWS_SECRET_ACCESS_KEY=bar \ | ||
| AWS_DEFAULT_REGION=us-east-1 | ||
|
|
||
| CMD ["dex-local-runner", "start-server", \ |
There was a problem hiding this comment.
without
EXPOSE 9014
consumers will have to specify -p at run docker run -p 9014:9014 arb/img-name
There was a problem hiding this comment.
I can include EXPOSE 9014.
| # AWS credentials (required for boto3) | ||
| ENV AWS_ACCESS_KEY_ID=foo \ | ||
| AWS_SECRET_ACCESS_KEY=bar \ | ||
| AWS_DEFAULT_REGION=us-east-1 |
There was a problem hiding this comment.
could be a nice addition, but maybe not essential:
HEALTHCHECK --interval=10s --timeout=3s --start-period=5s \
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:9014/health').read()" || exit 1
There was a problem hiding this comment.
I did not include this since the SAM CLI already does this healthcheck. In addition, this was causing the image to fail when running through the SAM CLI. As it's not essential, I opted to not use my time there.
.github/workflows/ecr-release.yml
Outdated
| ECR_REGISTRY: ${{ steps.login-ecr-public.outputs.registry }} | ||
| ECR_REPOSITORY: ${{ env.ecr_repository_name }} | ||
| IMAGE_TAG: "${{ env.image_tag }}v${{ steps.version.outputs.VERSION }}" | ||
| PER_ARCH_IMAGE_TAG: "${{ matrix.arch }}v${{ steps.version.outputs.VERSION }}" |
There was a problem hiding this comment.
maybe standardize to a v in front?
PER_ARCH_IMAGE_TAG: "v${{ steps.version.outputs.VERSION }}-${{ matrix.arch }}"
so
Standard format is v1.2.3-arm64 not arm64v1.2.3.
.github/workflows/ecr-release.yml
Outdated
| id-token: write # This is required for requesting the JWT | ||
|
|
||
| env: | ||
| path_to_dockerfile: "DockerFile" |
There was a problem hiding this comment.
Dockerfile rather than DockerFile.
happens to work because docker -f is not case sensitive.
There was a problem hiding this comment.
I think it is case sensitive. DockerFile is the name of the docker file.
There was a problem hiding this comment.
https://docs.docker.com/build/concepts/dockerfile/
standard is Dockerfile
"The default filename to use for a Dockerfile is Dockerfile, without a file extension. Using the default name allows you to run the docker build command without having to specify additional command flags."
.github/workflows/ecr-release.yml
Outdated
| id-token: write # This is required for requesting the JWT | ||
|
|
||
| env: | ||
| path_to_dockerfile: "DockerFile" |
There was a problem hiding this comment.
https://docs.docker.com/build/concepts/dockerfile/
standard is Dockerfile
"The default filename to use for a Dockerfile is Dockerfile, without a file extension. Using the default name allows you to run the docker build command without having to specify additional command flags."
5 participants
Issue #, if available:
Description of changes: Create emulator image from the python testing library. For each release, we should update the version in the pyproject.poml file in the emulator folder. The image tag will be the aws-durable-sdk-python-testing version with v prefixing it, e.g. v1.1.1.
Dependencies
If this PR requires testing against a specific branch of the Python Language SDK (e.g., for unreleased changes), uncomment and specify the branch below. Otherwise, leave commented to use the main branch.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.