proud-art-41399
04/28/2021, 10:14 AMpulumi_docker.Image
(link) while pushing it to Amazon ECR and then use the Docker image for AWS Batch job definition in a later step. I'm curious how I can leverage caching Docker layers. There's cache_from
argument of pulumi_docker.DockerBuild
, but as per the documentation, it's a list of build stages to use for caching that will be pushed to the target repository. However, I'd like to use local cache on the GitHub Actions runner (as per https://github.com/docker/build-push-action/blob/master/docs/advanced/cache.md#github-cache). Would it work to use BuildKit by specifying DOCKER_BUILDKIT=1
to env
argument of pulumi_docker.DockerBuild
and also supplying --cache-from
and --cache-to
(link) to extra_options
?billowy-army-68599
04/28/2021, 2:41 PMproud-art-41399
04/28/2021, 2:51 PM...
jobs:
provision-infra:
steps:
- uses: actions/checkout@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
with:
install: true
- name: Cache Docker layers
uses: actions/cache@v2
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: eu-central-1
- name: Install Pulumi CLI
uses: pulumi/action-install-pulumi-cli@v1
- name: Create or update stack resources
uses: pulumi/actions@v3
with:
command: up
stack-name: xxx-prod
work-dir: infra
cloud-url: <s3://xxx.xxx.xxx>
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_REGION: eu-central-1
PULUMI_CONFIG_PASSPHRASE: ${{ secrets.PULUMI_CONFIG_PASSPHRASE }}
...
and this is what I have in my Pulumi program (Python):
...
image = docker.Image(
'my-image',
build=docker.DockerBuild(
context='../my-context',
extra_options=[
'--output=type=docker',
'--cache-from=type=local,src=/tmp/.buildx-cache',
'--cache-to=type=local,dest=/tmp/.buildx-cache'
]
),
image_name=repo.repository_url,
registry=registry,
skip_push=False
)
...
Without the docker/setup-buildx-action@v1
with install: true
, the --cache-to
command line option was unknown resulting in an error:
docker:image:Image job-image {"Client":{"Platform":{"Name":""},"Version":"20.10.6+azure","ApiVersion":"1.41","DefaultAPIVersion":"1.41","GitCommit":"370c28948e3c12dce3d1df60b6f184990618553f","GoVersion":"go1.13.15","Os":"linux","Arch":"amd64","BuildTime":"Fri Apr 9 17:01:36 2021","Context":"default","Experimental":true},"Server":{"Platform":{"Name":""},"Components":[{"Name":"Engine","Version":"20.10.6+azure","Details":{"ApiVersion":"1.41","Arch":"amd64","BuildTime":"Fri Apr 9 22:06:18 2021","Experimental":"false","GitCommit":"8728dd246c3ab53105434eef8ffe997b6fd14dc6","GoVersion":"go1.13.15","KernelVersion":"5.4.0-1046-azure","MinAPIVersion":"1.12","Os":"linux"}},{"Name":"containerd","Version":"1.4.4+azure","Details":{"GitCommit":"05f951a3781f4f2c1911b05e61c160e9c30eaa8e"}},{"Name":"runc","Version":"1.0.0-rc93","Details":{"GitCommit":"12644e614e25b05da6fd08a38ffa0cfe1903fdec"}},{"Name":"docker-init","Version":"0.19.0","Details":{"GitCommit":""}}],"Version":"20.10.6+azure","ApiVersion":"1.41","MinAPIVersion":"1.12","GitCommit":"8728dd246c3ab53105434eef8ffe997b6fd14dc6","GoVersion":"go1.13.15","Os":"linux","Arch":"amd64","KernelVersion":"5.4.0-1046-azure","BuildTime":"2021-04-09T22:06:18.000000000+00:00"}}
docker:image:Image job-image Login Succeeded
docker:image:Image job-image warning: WARNING! Your password will be stored unencrypted in /home/runner/.docker/config.json.
docker:image:Image job-image Building image '../my-context'...
docker:image:Image job-image error: unknown flag: --cache-to
billowy-army-68599
04/28/2021, 2:52 PMdocker.Image.get()
proud-art-41399
04/28/2021, 2:58 PMdocker.Image.get()
but I wasn't able to find this in the documentation nor was I able to find any example. Also, I'm a bit confused whether I should use docker.Image
, docker.RemoteImage
or docker.RegistryImage
for this.tall-scientist-89115
05/06/2021, 4:44 AMproud-pizza-80589
05/06/2021, 5:58 AM- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v3.x
- name: Build and push
uses: docker/build-push-action@v2
with:
...
tags: |
<http://ghcr.io/settlemint/launchpad-nest-api:${{|ghcr.io/settlemint/launchpad-nest-api:${{> env.GITHUB_REF_SLUG }}
<http://ghcr.io/xxx/xxx:$|ghcr.io/xxx/xxx:$>{{ env.GITHUB_REF_SLUG }}-build${{ github.run_number }}
and in my pulumi script (we use a helm chart so in the chart values)
image: {
tag: process.env.DEPLOY_TAG || 'latest',
},
and then in different follow up jobs
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v3.x
- name: Pulumi
uses: pulumi/actions@v3
with:
command: up
...
env:
PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
DEPLOY_TAG: ${{ env.GITHUB_REF_SLUG }}-build${{ github.run_number }}
tall-scientist-89115
05/06/2021, 4:08 PMproud-pizza-80589
05/06/2021, 4:17 PMtall-scientist-89115
05/06/2021, 4:29 PM