Hi guys, I've been trying to build multi-arch docker images, i.e. images built for both linux/arm64 and linux/amd64. I have been trying to build docker image using the pulumi docker SDK for go. The approach which I am trying to follow is: Approach; 1. Create a docker.Image to build a single-arch image for e.g. arm64. 2. Create another docker.Image to build a single-arch image for e.g. amd64. 3. Create a pulumi.Command.local (Pulumi Command provider) to run docker manifest to create the multi-arch image. this was suggested by @stocky-restaurant-98004 in GH issues thread i.e. issue here is a function in Go responsible for building and pushing the docker image to dockerhub. However, currently, it's not working. I am also not getting any meaningful logs from pulumi's docker.NewImage to know where its failing. also is it possible that while the image built, rather than just being pushed to dockerhub (or any registry), its pushed to local docker aswell. This is required as this will allow me to write guardrails like vuln. scans before i push this created image to dockerhub. current function responsible for building and pushing docker image to dockerhub [not working❌]

func buildDockerImage(ctx *pulumi.Context, imageName string, platform string, dockerfilePath string) error {
	if ctx == nil {
		log.Error("Pulumi context is nil")
		return fmt.Errorf("Pulumi context is nil")
	}
	if _, err := os.Stat(dockerfilePath); os.IsNotExist(err) {
		log.WithField("dockerfilePath", dockerfilePath).Error("Dockerfile does not exist")
		return fmt.Errorf("Dockerfile does not exist at the specified path")
	}

	username := os.Getenv("DOCKERHUB_USERNAME")
	password := os.Getenv("DOCKERHUB_PASSWORD")
	server := "<https://index.docker.io/v1>"
	if username == "" || password == "" || server == "" {
		return fmt.Errorf("DOCKERHUB_USERNAME, DOCKERHUB_PASSWORD and DOCKERHUB_SERVER environment variables must be set")
	}

	builtImage, err := docker.NewImage(ctx, imageName, &docker.ImageArgs{
		Build: &docker.DockerBuildArgs{
			Context:    pulumi.String("."),
			Dockerfile: pulumi.String(dockerfilePath),
			Args: pulumi.StringMap{
				"BUILDKIT_INLINE_CACHE": pulumi.String("1"),
				"platform":              pulumi.String(platform),
			},
			Platform: pulumi.String(platform),
		},
		ImageName: pulumi.String(imageName),
		Registry: &docker.RegistryArgs{
			Server:   pulumi.String(server),
			Username: pulumi.String(username),
			Password: pulumi.String(password),
		},
	})
	if err != nil {
		log.WithFields(log.Fields{
			"imageName": imageName,
			"platform":  platform,
		}).Errorf("Error building Docker image: %s", err)
		return fmt.Errorf("Error building Docker image: %w", err)
	} else {
		builtImage.ImageName.ApplyT(func(name string) error {
			log.WithFields(log.Fields{
				"imageName": imageName,
				"platform":  platform,
				"imageID":   name,
			}).Infof("Successfully built and pushed Docker image with ID %s", name)
			return nil
		})
		return nil
	}
}

ref. doc: https://www.pulumi.com/registry/packages/docker/api-docs/image/

A few questions: 1. Can you post some details about what's failing? (Which step, and what's the Pulumi output?) 2. Does it have to be done in Pulumi, at least right now? Could you write a bash script that does what you need and optionally call it with the Command provider if it needs to be in Pulumi itself?

1. Josh, the image is not getting created/pushed to dockerhub. from the logs I get from this function, it shows the config being used by pulumi to create the docker image i.e.

+ docker:index/image:Image: (create)
        [urn=urn:pulumi:dev::<stack name>::docker:index/image:Image::<username>/node:18-buster-slim-arm-debian-latest]
        build    : {
            args         : {
                BUILDKIT_INLINE_CACHE: "1"
                platform             : "linux/amd64"
            }
            context      : "."
            contextDigest: <digest>
            dockerfile   : "Dockerfile.dev"
            platform     : "linux/amd64"
        }
        imageName: <image-name>
        registry : {
            password: <password>
            server  : "<https://index.docker.io/v1>"
            username: <username>
        }
        skipPush : false
msg="Successfully built and pushed Docker image with ID <username>/node:18-buster-slim-amd-debian-latest" imageID="<username>/node:18-buster-slim-amd-debian-latest" imageName="<username>/node:18-buster-slim-amd-debian-latest" platform=linux/amd64

2. This project involves a set of steps/configs which are added to the docker image. We are trying to write it in go as much as possible. We were prev. using the docker client SDK for go to build the image, but the docker SDK for go does not support creating manifests (for building multi-arch images). From the github issue, I thought that pulumi's SDK allows us to create manifest and so we thought of using pulumi.

Hi @sparse-umbrella-88444 - I am wondering if you are running into a version of https://github.com/pulumi/pulumi-docker/issues/812 or possibly awaiting changes underway in https://github.com/pulumi/pulumi-docker/pull/828. If you have verified an image is not being pushed to the registry despite logs telling you otherwise, it is worth filing a bug.

@sparse-umbrella-88444 Did I send you the link to how we are doing those multi-platform images for our own containers? Of course, this is in GHA, but you might find it useful as a reference for what you're trying to do: https://github.com/pulumi/pulumi-docker-containers/blob/main/.github/workflows/release.yml#L157-L230

@shy-arm-32391 could be this issue itself https://github.com/pulumi/pulumi-docker/issues/812 i.e. there's no error logs and I can't find the image in the registry.

Create the manifests using the

docker

CLI, since that seems to be the only way.

It sounds like there's two issues here: • unable to annotate local images with manifests (this is not something Pulumi can do at this moment and is marked as experimental in the official Docker docs) • image not getting pushed to registry I'm not confident this is an instance of https://github.com/pulumi/pulumi-docker/issues/812. If you have a minimum viable repro for us, we would appreciate it very much, because so far we have not been able to reproduce #812 reliably. I will look into the status of https://github.com/pulumi/pulumi-docker/pull/828 and see if we can release a patch once merged. Once you can push images, a manifest will be created automatically and you can query it via

docker manifest inspect

and you should be able to leverage pulumi-command to create the multi-arch image in the way you'd expect.