I’m having tons of problems with

pulumi-docker

4.x For one, it seems that Pulumi team created a hashing function that hashes all the Docker context (minus the .dockerignore) before actually running the build with docker’s engine : is there a way to disable this and instead let Docker build the image and compute the actual digest ? (source: https://github.com/pulumi/pulumi-docker/blob/1966f086164d1e1b87d96c9a9da81efef1167816/provider/provider.go#L162) In our case, we have a structure like this:

projects/
libs/

So we usually set the Docker context at

.

and use multi-stage builds to build our different projects : this way we can share the libs that are built at earlier stages in some of the projects Although the new docker provider does support the

target

argument, the

hashContext

function does not care for multi-stage builds : it calculates a single hash for the whole context... no matter which stage we actually intend to build The other -much more problematic- issue we’re having is that altering a file (such as adding a comment

# foo

somewhere) does not trigger an update when running

pulumi up

I can’t know for sure what’s happening, but I know for a fact that running

docker build

manually does pick up the change and update the digest of the image, whereas pulumi-docker’s pre-hash seems to block the docker engine from actually running This is preventing us from moving forward on our prod releases 😞 @limited-rainbow-51650 is there anybody at Pulumi that could chime in on this ?

Hi Louis! I’ve pinged some of the internal team for the Docker provider with a link to this post. If they don’t respond here directly, I’ll relay the feedback. Thanks for sharing your concerns! It might also be good to go ahead and open an issue (if you haven’t already) with all these details as well (if you don’t mind).

Hey @bright-orange-69401 - can you confirm that you're on the latest version of the provider? We've made some very specific fixes to context hashing, other steps. Regarding build stages, I'm not sure what you mean? All build stages utilize the same context, I've confirmed this by comparing our provider to the Docker CLI's implementation. We now support caching multi-stage builds via

cacheFrom

, and setting the build arg

BUILDKIT_INLINE_CACHE=1

so I don't think you need to build specific stages any longer to obtain caching benefits. There is a caveat: there is an outstanding bug in Docker that causes inline caching to fail every other build, but we may have a workaround in a coming release.

Hey @orange-policeman-59119

can you confirm that you’re on the latest version of the provider?

Yes, as of this morning at least

I'd be happy to hop on a call to talk through the changes in v4 and create issue(s) tracking anything we think is a defect. Computing the build context hash ourselves is an integral part of the provider, though - it is necessary in order for us to detect the correct operation that will be performed in

preview

.

Regarding build stages, I’m not sure what you mean? All build stages utilize the same context, I’ve confirmed this by comparing our provider to the Docker CLI’s implementation.

That’s my point : when you build a

foo

stage, your image digest is not gonna be the same as a

bar

stage on the same Dockerfile, even though they have the same context So the

hashContext

function that calculates the context’s hash and then decides not to actually run the docker engine is actually causing problems for us

Hello again @orange-policeman-59119 @shy-arm-32391 I’ve managed to do a minimal repro here using pure YAML: https://github.com/LouisAmon/pulumi-docker-4-bug There’s 100% something fishy going on : the bug seems to happen depending on the level of complexity of the directory structure as illustrated in the repro

Thank you for this repro - it was so helpful and user friendly! :meow-party: I am filing an issue for this and I have a potential fix in the works.