https://pulumi.com logo
Title
h

helpful-receptionist-73337

03/21/2023, 9:47 PM
Hey everyone, I have a resource in mongodb atlas that is made outside pulumi. I use one of the get functions to get the information on the resource, the specifics of this is getting a connection string. Additionally I am creating a privatelink from aws to mongodb atlas. When the privatelink is created the connection string is populated with the values I need. And then i take those values and save it to the secrets manager. i have my secret dependent on the privatelink being created and for the
secret_string
param i am using an
apply
function to transform the Output. The issue i am having is that event though the secret is dependent on the private link, the
apply
runs before the privatelink finishes causing my build to error out. Does anyone know how to make the apply wait until a resource is finished being created? Here is a mock code snippet to help with context:
aws_endpoint_service = aws.ec2.VpcEndpoint(
    f"{environment}-aws-vpc-endpoint",
    vpc_id=vpc_configs["vpc_id"],
    service_name=mongo_private_link_endpoint.endpoint_service_name,
    vpc_endpoint_type="Interface",
    subnet_ids=vpc_configs["subnet_ids"],
    security_group_ids=vpc_configs["security_group_ids"],
)
mongo_private_link_endpoint_service = mongodbatlas.PrivateLinkEndpointService(
    f"{environment}-PrivateLinkEndpointService",
    project_id=mongo_private_link_endpoint.project_id,
    private_link_id=mongo_private_link_endpoint.private_link_id,
    endpoint_service_id=aws_endpoint_service.id,
    provider_name="AWS",
)
cluster = mongodbatlas.get_cluster(
    name=mongo_configs["cluster_name"], project_id=mongo_configs["project_id"]
)

secret = aws.secretsmanager.Secret(
        resource_name=f"{environment}_mongo_connections",
        name=f"{environment}_mongo_connections",
        opts=ResourceOptions(delete_before_replace=True),
        recovery_window_in_days=0,
    )
secret_version = aws.secretsmanager.SecretVersion(
        resource_name=f"{environment}_mongo_connections",
        secret_id=secret.arn,
        # the id does not exist until the endpoint service is created
        # but the apply runs before it finishes causing a failure
        secret_string=cluster.connection_strings.apply(
            lambda x: json.dumps({"private_connection": f"{x[0].get(mongo_private_link_endpoint_service.id).split('://')[1]}"}
                                 )
            ),
        opts=ResourceOptions(depends_on=[mongo_private_link_endpoint_service]),
    )
l

little-cartoon-10569

03/21/2023, 9:56 PM
Are you sure the problem is that the PrivateLink isn't finished being built? There's a bug in there, which is that you're running split on an output. Maybe fix that and see if the problem goes away?
I don't know how to do it in Python, but there's probably an equivalent to
pulumi.all()
, which resolves multiple outputs at once?
h

helpful-receptionist-73337

03/21/2023, 9:58 PM
Yea basically if i rerun it after the link is built the code finishes successfully
l

little-cartoon-10569

03/21/2023, 9:58 PM
Really? It shouldn't.
This code is dependent on two outputs, but you're only applying one of them:
secret_string=cluster.connection_strings.apply(
            lambda x: json.dumps({"private_connection": f"{x[0].get(mongo_private_link_endpoint_service.id).split('://')[1]}"}
                                 )
            ),
You need to apply
mongo_private_link_endpoint_service.id
too.
h

helpful-receptionist-73337

03/21/2023, 10:03 PM
yea sorry i was trying to make a simplified code snippet but kind of botched that, the real code is more like
secret_string=Output.all(
     cluster.connection_strings,
     mongo_private_link_endpoint_service.interface_endpoint_id,
     ).apply(
             lambda x: 
             json.dumps({
                 "private_connection": f"{x[0].get(x[1]).split('://')[1]}"
                 })
         )
the issue is that the id for the private link exists before it is finished being created and the cluster connection strings also exist but a new value is populated only after the private link finishes
l

little-cartoon-10569

03/21/2023, 10:06 PM
So applying the id is causing the error output? That sounds like a bug in the SDK. I can understand that making a connection to something might fail until it has finished setting up, but getting its id should simply wait until the id is available...
Yes, I've looked at the API, and it's normal. The output should simply wait until the value behind it is available. That's the point of outputs...
h

helpful-receptionist-73337

03/21/2023, 10:09 PM
yea definitely. the id here does actually exist, it just doesnt exist inside the dictionary object that is the connection strings until the private link is created so since all those outputs technically exist it runs instead of waiting
wasnt sure if theres a way for me to get the apply to wait until the resource is finished i was looking through the source code and docs and havent been able to find anything
l

little-cartoon-10569

03/21/2023, 10:11 PM
It looks like you're doing everything right. The depends_on shouldn't be necessary: the interface_endpoint_id.apply() achieves that. I would raise an issue here: https://github.com/pulumi/pulumi-mongodbatlas/issues
h

helpful-receptionist-73337

03/21/2023, 10:11 PM
thank you! I appreciate the help