https://pulumi.com logo
Join Slack
Powered by
Hi, I've got a question reagarding the apply() fun...
# python
s
Hi, I've got a question reagarding the apply() function in Pulumi-Python, as I assume I have a problem with concurrent execution of my stack. My code does similar things as the example of @hallowed-animal-47023 from Sunday. I create a VPC and the the whole network setup. Including the attachment of the the newly created vpc and it's subnet to our TransitGateway .. For the aws.ec2transitgateway.VpcAttachment() I need the private subnets in my vpc .. as there is the ec2.get_subnet_ids() function I call it to filter my subnets:
Copy code
private_subnet_ids = aws.ec2.get_subnet_ids(
    vpc_id=vpc.id, # created as the first resource in the stack
    tags={"visibility": "private"} # filter by tags for the correct subnets
)
But, this consistently fails with the error message of:
Copy code
Exception: invoke of aws:ec2/getSubnetIds:getSubnetIds failed: Missing required argument: The argument "vpc_id" is required, but no definition was found. ()
This worked before, while developing the code, as the VPC was created some days ago, but then we deleted the stack, and now everything should be created in a single run. I guess the call to aws.ec2.Vpc() hasn't returned yet with an vpc and its id when the call to get_subnet_ids() is being issued.
I tried to enforce the code to wait for the vpc by encapsulating the whole vpc attachment calls in a function and invoke the function like this:
Copy code
vpc_id = vpc.id.apply(lambda x: x)
create_transitgateway_attachment(tgw_config_paramenters, vpc_id, tags)
Same error
Even doing some printf debugging with
Copy code
vpc_id = vpc.id.apply(lambda x: x)
print(vpc id after apply(): ", vpc_id)
create_transit_gateway_attachment(tgw_config_parameters, vpc_id, tags)
Leads to the astonishing output of:
Copy code
vpc id after apply():  <pulumi.output.Output object at 0x7fa5c8475400>

error: Program failed with an unhandled exception:
error: Traceback (most recent call last):
<snip>
Exception: invoke of aws:ec2/getSubnetIds:getSubnetIds failed: Missing required argument: The argument "vpc_id" is required, but no definition was found. ()
The error still exists ..
h
@sticky-bear-14421 even if your depends_on?
Oh wait wait sorry misreading your code
s
The get_subnet_ids call has no opts field
sadly
h
Yeah
s
my current approach is to encapsulate the tgw.create_my_stuff wrapper inside the vpc.id.apply(lambda id: tgw.create_my_stuff_wrapper(id))
And this did work
p
are you creating a new vpc and subnets in it as part of this pulumi stack or importing existing ones ?
s
Hi @purple-plumber-90981 The intention for this stack is to create a vpc with nat/igw and TransitGateway setups. Its the base deployment of a full network setup.
The codebase is split into different files and the function call to collect the subnets is in an function where I do the whole TGW setup process.
I thought about something like in the example of @hallowed-animal-47023 with 
private_networks=[]
and then put these as parameter in my transitgateway() function, but the code base has a second use/intent as an example for new pulumi users in our organisation. So I put the get_subnet_ids() call in this as an example for querying resources from the api with the use of proper tagging.
This would become then something like this:
Copy code
vpc_id = vpc.id.apply(lambda x: x)
create_transitgateway_attachment(tgw_config_paramenters, vpc_id, private_networks, tags)
But other resources inside this function need the vpc.id aswell, so the concurrency problem would remain.
p
right, i have been having similar experience when trying to reference future output objects (pulumi.output.Output object) outside of a resource definition
so i was wondering if you were referencing pulumi resources not created in the same stack
s
Yepp, I have experienced this with Pulumi and Go aswell, it's just that with Go and maybe even more with TypeScript this concurrency behavior is more promiment/obvious.
p
im only python . . . so i cant speak across languages
the bit which caught me out is that i was trying to do the transforms outside a resource object but then use the result as input to another resource
i have since learned this is not the pulumi way
and that the transforms need to be encapsualted in a pulumi resource else transformation operations on them fail
wondered if this was similar to what you were experiencing
s
yeah, these boxed types are a complex thing
Open in Slack
PreviousNext
Powered by