I am also having and issue where I am able to build docker i Pulumi Community #general

I am also having and issue where I am able to buil...

shy-author-33795

05/18/2021, 8:02 PM

I am also having and issue where I am able to build docker images locally with

pulumi up

but fails to build on Buildkite. To tell you more about what I have going on... I have the following `ComponentResource`:

Copy code

class ECRRepo(ComponentResource):        
    ComponentResource.__init__(
    self, "awsx:ecr:ECRRepo", name, {
    "name": name,
    "docker_build_dir": docker_build_dir,
    "docker_image": docker_image,
    "extra_protected_tags": extra_protected_tags,
    "image_tag_mutability": image_tag_mutability,
    "max_images": max_images,
    "scan_on_push": scan_on_push,
    "tags": tags
    }, opts)

Here's how I grab registry information:

Copy code

class ECRRepo(ComponentResource):
    @staticmethod
    def get_registry_info(rid):
        creds = ecr.get_credentials(registry_id=rid)
        decoded = base64.b64decode(creds.authorization_token).decode()
        parts = decoded.split(':')
        if len(parts) != 2:
            raise Exception("Invalid credentials")
        return ImageRegistry(creds.proxy_endpoint, parts[0], parts[1])

Here's how I'm creating a repo:

Copy code

def _create(self):
        repo = ecr.Repository(
            resource_name = self.name,
            image_scanning_configuration = ecr.RepositoryImageScanningConfigurationArgs(
                scan_on_push = self.scan_on_push,
            ),
            name = self.name,
            image_tag_mutability = self.image_tag_mutability,
            tags = self.tags,
            opts = ResourceOptions(parent=self)
        )

Here's how I'm building the image:

Copy code

custom_image = "{image}".format(image=self.docker_image)
directory = "{dir}".format(dir=self.docker_build_dir)

Image(
    self.name,
    image_name=pulumi.Output.concat(repo.repository_url, ":v1.0.0"),
    build=f'{directory}/{custom_image}',            
    registry=repo.registry_id.apply(ECRRepo.get_registry_info),
        opts = ResourceOptions(parent=repo)
)

shy-author-33795

05/18/2021, 8:04 PM

again, this works locally, but fails with the following on Buildkite:

Copy code

+  docker:image:Image jsonlint create Building image './docker/jsonlint'...
    pulumi:pulumi:Stack ecr-us-west-2 running error: Program failed with an unhandled exception:
    pulumi:pulumi:Stack ecr-us-west-2 running error: Traceback (most recent call last):
    pulumi:pulumi:Stack ecr-us-west-2 running sys:1: RuntimeWarning: coroutine 'Output.future.<locals>.get_value' was never awaited
    pulumi:pulumi:Stack ecr-us-west-2 running RuntimeWarning: Enable tracemalloc to get the object allocation traceback
    pulumi:pulumi:Stack ecr-us-west-2 running error: an unhandled error occurred: Program exited with non-zero exit code: 1
    pulumi:pulumi:Stack ecr-us-west-2  3 errors; 2 messages
 
Diagnostics:
    error: Program failed with an unhandled exception:
    error: Traceback (most recent call last):
      File "/pulumi/bin/pulumi-language-python-exec", line 85, in <module>
        loop.run_until_complete(coro)
      File "/usr/local/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
        return future.result()
      File "/workdir/projects/ecr_repos/stacks/us-west-2/venv/lib/python3.9/site-packages/pulumi/runtime/stack.py", line 110, in run_in_stack
        await run_pulumi_func(lambda: Stack(func))
      File "/workdir/projects/ecr_repos/stacks/us-west-2/venv/lib/python3.9/site-packages/pulumi/runtime/stack.py", line 45, in run_pulumi_func
        await wait_for_rpcs()
      File "/workdir/projects/ecr_repos/stacks/us-west-2/venv/lib/python3.9/site-packages/pulumi/runtime/stack.py", line 94, in wait_for_rpcs
        raise exception
      File "/workdir/projects/ecr_repos/stacks/us-west-2/venv/lib/python3.9/site-packages/pulumi/output.py", line 178, in run
        transformed: Input[U] = func(value)
      File "/workdir/projects/ecr_repos/stacks/us-west-2/venv/lib/python3.9/site-packages/pulumi_docker/image.py", line 254, in <lambda>
        lambda args: get_image_data(_ImageArgs(*args))
      File "/workdir/projects/ecr_repos/stacks/us-west-2/venv/lib/python3.9/site-packages/pulumi_docker/image.py", line 231, in get_image_data
        unique_target_name = build_and_push_image(
      File "/workdir/projects/ecr_repos/stacks/us-west-2/venv/lib/python3.9/site-packages/pulumi_docker/docker.py", line 240, in build_and_push_image
        build_result = build_image(base_image_name, path_or_build, log_resource, cache_from)
      File "/workdir/projects/ecr_repos/stacks/us-west-2/venv/lib/python3.9/site-packages/pulumi_docker/docker.py", line 434, in build_image
        docker_build(image_name, build, log_resource, cache_from)
      File "/workdir/projects/ecr_repos/stacks/us-west-2/venv/lib/python3.9/site-packages/pulumi_docker/docker.py", line 489, in docker_build
        return run_command_that_must_succeed("docker", build_args, log_resource, env=build.env)
      File "/workdir/projects/ecr_repos/stacks/us-west-2/venv/lib/python3.9/site-packages/pulumi_docker/docker.py", line 596, in run_command_that_must_succeed
        command_result = run_command_that_can_fail(
      File "/workdir/projects/ecr_repos/stacks/us-west-2/venv/lib/python3.9/site-packages/pulumi_docker/docker.py", line 658, in run_command_that_can_fail
        process = subprocess.Popen(cmd, env=env, stdout=subprocess.PIPE,
      File "/usr/local/lib/python3.9/subprocess.py", line 951, in __init__
        self._execute_child(args, executable, preexec_fn, close_fds,
      File "/usr/local/lib/python3.9/subprocess.py", line 1821, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    FileNotFoundError: [Errno 2] No such file or directory: 'docker'
    error: an unhandled error occurred: Program exited with non-zero exit code: 1
 
    sys:1: RuntimeWarning: coroutine 'Output.future.<locals>.get_value' was never awaited
    RuntimeWarning: Enable tracemalloc to get the object allocation traceback

shy-author-33795

05/18/2021, 8:06 PM

Here's the invocation:

Copy code

from ecr_repo import ECRRepo as ecr_repository
repo = ecr_repository(
        name=jsonlint,
        tags=tags,
        extra_protected_tags=['latest'],
        docker_build_dir='./docker',
        docker_image=jsonlint
    )

shy-author-33795

05/18/2021, 8:10 PM

I'd like to think that

No such file or directory: 'docker'

means that the buildkite process can't find my directory, but I have confirmed that the directory is there. There may be some subprocess shenanigans going on, but I'm a little lost on getting to the bottom of that

bored-oyster-3147

05/18/2021, 8:22 PM

My interpretation of

No such file or directory: 'docker'

would be that the docker CLI is not available to your build agent

🙌 1

👍 1

shy-author-33795

05/18/2021, 8:24 PM

ooooohhhhh, let me try volume mounting the docker socket in the container running

pulumi up

(and install docker in the image)

shy-author-33795

05/18/2021, 9:05 PM

that was indeed the fix. thank you for your help @bored-oyster-3147

🎉 1

2 Views

Open in Slack

Previous Next