Hello < CRH5ENVDX|aws> I am getting crazy in trying to use p Pulumi Community #aws

Hello <#CRH5ENVDX|aws>, I am getting crazy in tryi...

sparse-caravan-37954

09/13/2023, 2:03 PM

Hello #aws, I am getting crazy in trying to use parameters in AWS job definition. A minimal example would look like this

Copy code

container_properties = {
        "jobRoleArn": batch_role.arn,
        "command": [
            'sh', 
            '-c',
            'echo Ref::param1'
        ],
        "image": image_uri,
        "memory": 1024*2,
        "vcpus": 1,
    }


    job_definition = aws.batch.JobDefinition("jobDefinition",
        type="container",
        container_properties=pulumi.Output.json_dumps(container_properties)
        ),
        parameters={
            "param1": "pippo",
        },
    )

When the job is run, however, the output is

Ref::param1

instead of

pippo

. I think the problem is in the fact that the container properties definition contains pulumi Output objects (

batch_role

and

image_uri

are resources defined before in the full code) and therefore I need the

pulumi.Output.json_dumps

story when using it in the job definition, and this is somehow avoiding the parameters to be picked up. Any idea? (edited)

cuddly-nightfall-71097

09/13/2023, 2:06 PM

a tip: to dump json use

pulumi.Output.json_dumps

(it's python, right?). It returns

Output

and produces string when all keys and values for the dict are ready.

cuddly-nightfall-71097

09/13/2023, 2:10 PM

E.g. my definition looks like

Copy code

pformat = pulumi.Output.format
to_json = pulumi.Output.json_dumps

            container_definitions=to_json(
                [
                    {
                        "name": service.container_name,
                        "image": pformat("{}:{}", image, service.image_tag),
                        "portMappings": [
                            {"containerPort": service.grpc_port},
                            {"containerPort": service.http_port},
                        ],
                        "environment": environment,
                        "logConfiguration": {
                            "logDriver": "awslogs",
                            "options": {
                                "awslogs-group": log_group.name,
                                "awslogs-region": get_region().name,
                                "awslogs-stream-prefix": service.image_tag,
                            },
                        },
                    }
                ]
            ),

sparse-caravan-37954

09/13/2023, 2:12 PM

Thanks. This is indeed very convenient. I did changed all that messy things with just

pulumi.Output.json_dumps(container_properties)

, but the problem persists unfortunately

sparse-caravan-37954

09/13/2023, 2:12 PM

I have edited the initial message with your suggestion 👍

cuddly-nightfall-71097

09/13/2023, 2:14 PM

This is the second part which unfortunately I can't help with 😞 Looks like Batch doesn't replace your

Ref::param1

so it is used literally. I didn't work with Batch 😞 I would try fixing it in web console, to make it working, then see what is the difference between web version and what Pulumi generates.

sparse-caravan-37954

09/13/2023, 2:15 PM

I'll try, thanks

cuddly-nightfall-71097

09/13/2023, 2:17 PM

The only difference from documentation I can see is that they use placeholders as stand-alone parameters, not as a part of a string.

sparse-caravan-37954

09/13/2023, 2:18 PM

Yeah I noticed this as well, but didn't think it was important

sparse-caravan-37954

09/13/2023, 2:27 PM

I have also tried to use the parameter as environment variable

Copy code

container_properties = {
        "jobRoleArn": batch_role.arn,
        "command": [
            'sh', 
            '-c',
            'echo $VAR'
        ],
        "environment": [
            {"name": "VAR", "value": "Ref::param1"},
        ],
        "image": image_uri,
        "memory": 1024*2,
        "vcpus": 1,
    }


    job_definition = aws.batch.JobDefinition("jobDefinition",
        type="container",
        container_properties=pulumi.Output.json_dumps(container_properties)
        ),
        parameters={
            "param1": "pippo",
        },
    )

but nothing, the value of

VAR

is `Ref::param1"

cuddly-nightfall-71097

09/13/2023, 2:29 PM

You can try calling

echo

directly instead of

sh -c

(since it's usually a program rather than shell alias). like

["echo", "Ref::param1"]

sparse-caravan-37954

09/13/2023, 2:33 PM

It worked! But the

echo

command was just a simplification. In reality I have someting like

sh -c python my_script.py --param Ref::param1

sparse-caravan-37954

09/13/2023, 2:45 PM

I found I work around defining the parameters as environment variables and then using containerOverride when firing the job. But still I do not get why it is not working with parameters

cuddly-nightfall-71097

09/13/2023, 5:16 PM

You can call python the same way. You don't need to prefix with

sh

Open in Slack

Previous Next