broad-eve-12764
05/18/2021, 7:27 PMpulumi up
locally, but re-builds each layer with no caching when run in GitHub Actions.
Excerpt from Pulumi `index.ts`:
const image = repo.buildAndPushImage({
target: 'app',
cacheFrom: { stages: ['deps', 'files', 'app'] },
context: '../',
})
Dockerfile is essentially:
FROM external.image as deps
# install dependencies
FROM deps as files
# copy in some files, precompile assets
FROM files as app
# copy in other files, build app
When run locally, it builds each layer and uploads to the repo. When I run it in GitHub Actions, it does this:
• builds deps
• builds deps
again as part of building files
• builds files
• builds deps
again as part of building app
• builds files
again as part of building app
• builds app
Not sure why caching is failing to work only in CI. Any thoughts?bored-oyster-3147
05/18/2021, 7:46 PMbroad-eve-12764
05/18/2021, 7:52 PMbored-oyster-3147
05/18/2021, 7:55 PMbroad-eve-12764
05/18/2021, 7:56 PMrepo.buildAndPushImage
)bored-oyster-3147
05/18/2021, 7:57 PMdocker
CLI so if docker supports an environment variable to specify the directory you could probably just set that and pulumi doesn't need to be aware of the differencerepo.buildAndPushImage
accepts DockerBuild
arguments? Like this: https://www.pulumi.com/docs/reference/pkg/docker/image/#dockerbuildbroad-eve-12764
05/18/2021, 8:10 PMconst image = repo.buildAndPushImage({
target: 'app',
cacheFrom: { stages: ['deps', 'files', 'app'] },
context: '../',
})
where the stages are the names of the targets in my multi-stage build (see dockerfile example above)bored-oyster-3147
05/18/2021, 8:16 PMbroad-eve-12764
05/18/2021, 8:17 PM- name: Add Pulumi CLI
uses: pulumi/action-install-pulumi-cli@v1
- name: Install Pulumi CLI
run: cd deploy && npm install
- uses: pulumi/actions@v3
with:
command: up
stack-name: ${{ env.STACK_NAME }}
work-dir: deploy
env:
PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
bored-oyster-3147
05/18/2021, 8:18 PMbroad-eve-12764
05/18/2021, 8:58 PMdeps
correctly, but then will end up building files
and app
twice stillquiet-motorcycle-76742
05/18/2021, 9:18 PM- uses: docker/setup-buildx-action@v1
with:
install: true
broad-eve-12764
05/18/2021, 9:19 PMbuildx
and also added some options to the buildAndPushImage
call:
In GitHub Actions workflow:
- uses: docker/setup-buildx-action@v1
with:
install: true
driver: docker
buildkitd-flags: --debug
In Pulumi file:
checkForBuildx() // (see write-up linked below)
const image = repo.buildAndPushImage({
target: 'app',
cacheFrom: { stages: ['deps', 'files'] },
extraOptions: ['--load'],
args: { BUILDKIT_INLINE_CACHE: '1' },
})
Success! It was a bit of a struggle to get to this solution so I wrote it up in a bit more detail here for everyone's reference.