https://pulumi.com logo
Powered by
#aws
Title
s

sparse-caravan-37954

09/13/2023, 2:03 PM
Hello #aws, I am getting crazy in trying to use parameters in AWS job definition. A minimal example would look like this
Copy code
container_properties = {
        "jobRoleArn": batch_role.arn,
        "command": [
            'sh', 
            '-c',
            'echo Ref::param1'
        ],
        "image": image_uri,
        "memory": 1024*2,
        "vcpus": 1,
    }


    job_definition = aws.batch.JobDefinition("jobDefinition",
        type="container",
        container_properties=pulumi.Output.json_dumps(container_properties)
        ),
        parameters={
            "param1": "pippo",
        },
    )
When the job is run, however, the output is 
Ref::param1
instead of 
pippo
. I think the problem is in the fact that the container properties definition contains pulumi Output objects (
batch_role
and 
image_uri
are resources defined before in the full code) and therefore I need the 
pulumi.Output.json_dumps
story when using it in the job definition, and this is somehow avoiding the parameters to be picked up. Any idea? (edited)
c

cuddly-nightfall-71097

09/13/2023, 2:06 PM
a tip: to dump json use 
pulumi.Output.json_dumps
(it's python, right?). It returns 
Output
and produces string when all keys and values for the dict are ready.
E.g. my definition looks like
Copy code
pformat = pulumi.Output.format
to_json = pulumi.Output.json_dumps

            container_definitions=to_json(
                [
                    {
                        "name": service.container_name,
                        "image": pformat("{}:{}", image, service.image_tag),
                        "portMappings": [
                            {"containerPort": service.grpc_port},
                            {"containerPort": service.http_port},
                        ],
                        "environment": environment,
                        "logConfiguration": {
                            "logDriver": "awslogs",
                            "options": {
                                "awslogs-group": log_group.name,
                                "awslogs-region": get_region().name,
                                "awslogs-stream-prefix": service.image_tag,
                            },
                        },
                    }
                ]
            ),
s

sparse-caravan-37954

09/13/2023, 2:12 PM
Thanks. This is indeed very convenient. I did changed all that messy things with just 
pulumi.Output.json_dumps(container_properties)
, but the problem persists unfortunately
I have edited the initial message with your suggestion 👍
c

cuddly-nightfall-71097

09/13/2023, 2:14 PM
This is the second part which unfortunately I can't help with 😞 Looks like Batch doesn't replace your 
Ref::param1
so it is used literally. I didn't work with Batch 😞 I would try fixing it in web console, to make it working, then see what is the difference between web version and what Pulumi generates.
s

sparse-caravan-37954

09/13/2023, 2:15 PM
I'll try, thanks
c

cuddly-nightfall-71097

09/13/2023, 2:17 PM
The only difference from documentation I can see is that they use placeholders as stand-alone parameters, not as a part of a string.
s

sparse-caravan-37954

09/13/2023, 2:18 PM
Yeah I noticed this as well, but didn't think it was important
I have also tried to use the parameter as environment variable
Copy code
container_properties = {
        "jobRoleArn": batch_role.arn,
        "command": [
            'sh', 
            '-c',
            'echo $VAR'
        ],
        "environment": [
            {"name": "VAR", "value": "Ref::param1"},
        ],
        "image": image_uri,
        "memory": 1024*2,
        "vcpus": 1,
    }


    job_definition = aws.batch.JobDefinition("jobDefinition",
        type="container",
        container_properties=pulumi.Output.json_dumps(container_properties)
        ),
        parameters={
            "param1": "pippo",
        },
    )
but nothing, the value of 
VAR
is `Ref::param1"
c

cuddly-nightfall-71097

09/13/2023, 2:29 PM
You can try calling 
echo
directly instead of 
sh -c
(since it's usually a program rather than shell alias). like 
["echo", "Ref::param1"]
s

sparse-caravan-37954

09/13/2023, 2:33 PM
It worked! But the 
echo
command was just a simplification. In reality I have someting like 
sh -c python my_script.py --param Ref::param1
I found I work around defining the parameters as environment variables and then using containerOverride when firing the job. But still I do not get why it is not working with parameters
c

cuddly-nightfall-71097

09/13/2023, 5:16 PM
You can call python the same way. You don't need to prefix with 
sh
.
Powered by