https://pulumi.com logo
#general
Title
# general
p

proud-art-41399

04/28/2021, 10:14 AM
Hi, I'm using Pulumi with GitHub Actions. In the Pulumi program, I first build a Docker image using
pulumi_docker.Image
(link) while pushing it to Amazon ECR and then use the Docker image for AWS Batch job definition in a later step. I'm curious how I can leverage caching Docker layers. There's
cache_from
argument of
pulumi_docker.DockerBuild
, but as per the documentation, it's a list of build stages to use for caching that will be pushed to the target repository. However, I'd like to use local cache on the GitHub Actions runner (as per https://github.com/docker/build-push-action/blob/master/docs/advanced/cache.md#github-cache). Would it work to use BuildKit by specifying
DOCKER_BUILDKIT=1
to
env
argument of
pulumi_docker.DockerBuild
and also supplying
--cache-from
and
--cache-to
(link) to
extra_options
?
b

billowy-army-68599

04/28/2021, 2:41 PM
Hey @proud-art-41399 - this should work - the docker provider uses the docker API under the hood I believe. Give it a try and report back!
p

proud-art-41399

04/28/2021, 2:51 PM
I've been trying it for a couple of hours but it does not work per se. What I ended up with is this GitHub Actions workflow (the relevant part):
Copy code
...
jobs:
  provision-infra:
    steps:
      - uses: actions/checkout@v2
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1
        with:
          install: true
      - name: Cache Docker layers
        uses: actions/cache@v2
        with:
          path: /tmp/.buildx-cache
          key: ${{ runner.os }}-buildx-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-buildx-
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: eu-central-1
      - name: Install Pulumi CLI
        uses: pulumi/action-install-pulumi-cli@v1
      - name: Create or update stack resources
        uses: pulumi/actions@v3
        with:
          command: up
          stack-name: xxx-prod
          work-dir: infra
          cloud-url: <s3://xxx.xxx.xxx>
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          AWS_REGION: eu-central-1
          PULUMI_CONFIG_PASSPHRASE: ${{ secrets.PULUMI_CONFIG_PASSPHRASE }}
...
and this is what I have in my Pulumi program (Python):
Copy code
...
image = docker.Image(
    'my-image',
    build=docker.DockerBuild(
        context='../my-context',
        extra_options=[
            '--output=type=docker',
            '--cache-from=type=local,src=/tmp/.buildx-cache',
            '--cache-to=type=local,dest=/tmp/.buildx-cache'
        ]
    ),
    image_name=repo.repository_url,
    registry=registry,
    skip_push=False
)
...
Without the
docker/setup-buildx-action@v1
with
install: true
, the
--cache-to
command line option was unknown resulting in an error:
Copy code
docker:image:Image job-image  {"Client":{"Platform":{"Name":""},"Version":"20.10.6+azure","ApiVersion":"1.41","DefaultAPIVersion":"1.41","GitCommit":"370c28948e3c12dce3d1df60b6f184990618553f","GoVersion":"go1.13.15","Os":"linux","Arch":"amd64","BuildTime":"Fri Apr  9 17:01:36 2021","Context":"default","Experimental":true},"Server":{"Platform":{"Name":""},"Components":[{"Name":"Engine","Version":"20.10.6+azure","Details":{"ApiVersion":"1.41","Arch":"amd64","BuildTime":"Fri Apr  9 22:06:18 2021","Experimental":"false","GitCommit":"8728dd246c3ab53105434eef8ffe997b6fd14dc6","GoVersion":"go1.13.15","KernelVersion":"5.4.0-1046-azure","MinAPIVersion":"1.12","Os":"linux"}},{"Name":"containerd","Version":"1.4.4+azure","Details":{"GitCommit":"05f951a3781f4f2c1911b05e61c160e9c30eaa8e"}},{"Name":"runc","Version":"1.0.0-rc93","Details":{"GitCommit":"12644e614e25b05da6fd08a38ffa0cfe1903fdec"}},{"Name":"docker-init","Version":"0.19.0","Details":{"GitCommit":""}}],"Version":"20.10.6+azure","ApiVersion":"1.41","MinAPIVersion":"1.12","GitCommit":"8728dd246c3ab53105434eef8ffe997b6fd14dc6","GoVersion":"go1.13.15","Os":"linux","Arch":"amd64","KernelVersion":"5.4.0-1046-azure","BuildTime":"2021-04-09T22:06:18.000000000+00:00"}}
      docker:image:Image job-image  Login Succeeded
      docker:image:Image job-image  warning: WARNING! Your password will be stored unencrypted in /home/runner/.docker/config.json.
      docker:image:Image job-image  Building image '../my-context'...
      docker:image:Image job-image  error: unknown flag: --cache-to
It would be really cool it there was a native support for Buildx
b

billowy-army-68599

04/28/2021, 2:52 PM
we are sort of restricted by the upstream provider there I think 😞
you might have better luck building your image and then using
docker.Image.get()
p

proud-art-41399

04/28/2021, 2:58 PM
Yeah, that's what I initially wanted to try as an alternative way. I guess it would also be faster (see https://github.com/pulumi/pulumi-docker/issues/14 and https://github.com/pulumi/pulumi-docker/issues/32).
@billowy-army-68599 would you please give me a hint how to achieve what you were suggesting? Let's say I build the Docker image and push it to Docker registry (Amazon ECR) in a separate job of GitHub Actions workflow. Now I'd like to reference the Docker image in Pulumi program (instead of building it during the Pulumi program run). You suggested using
docker.Image.get()
but I wasn't able to find this in the documentation nor was I able to find any example. Also, I'm a bit confused whether I should use
docker.Image
,
docker.RemoteImage
or
docker.RegistryImage
for this.
🎯 1
t

tall-scientist-89115

05/06/2021, 4:44 AM
@billowy-army-68599 I share Tomáš's question. What I'd really like is a registry lookup for the hash of the image so my deployments know to cycle the containers without downloading the image content.
👍 1
the fanout is lovely, vs the proposed solution above forcing it all to be in the same job..
p

proud-pizza-80589

05/06/2021, 5:58 AM
What I do is use env vars
Copy code
- name: Inject slug/short variables
        uses: rlespinasse/github-slug-action@v3.x
      - name: Build and push 
        uses: docker/build-push-action@v2
        with:
          ...
          tags: |
            <http://ghcr.io/settlemint/launchpad-nest-api:${{|ghcr.io/settlemint/launchpad-nest-api:${{> env.GITHUB_REF_SLUG }}
            <http://ghcr.io/xxx/xxx:$|ghcr.io/xxx/xxx:$>{{ env.GITHUB_REF_SLUG }}-build${{ github.run_number }}
and in my pulumi script (we use a helm chart so in the chart values)
Copy code
image: {
          tag: process.env.DEPLOY_TAG || 'latest',
        },
and then in different follow up jobs
Copy code
- name: Inject slug/short variables
        uses: rlespinasse/github-slug-action@v3.x
      - name: Pulumi
        uses: pulumi/actions@v3
        with:
          command: up
          ...
        env:
          PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
          DEPLOY_TAG: ${{ env.GITHUB_REF_SLUG }}-build${{ github.run_number }}
t

tall-scientist-89115

05/06/2021, 4:08 PM
@proud-pizza-80589 Thank you! Why do you use the build run number vs just the slug?
p

proud-pizza-80589

05/06/2021, 4:17 PM
Because the slug is e.g “staging” and a new build will not trigger an update if in k8s your deployment does not use “Always” as a pull policy
t

tall-scientist-89115

05/06/2021, 4:29 PM
oh that makes sense, I was thinking that was the short commit hash for some reason, thank you!
109 Views