https://pulumi.com logo
Powered by
m

mysterious-egg-7415

01/26/2020, 7:33 PM
How are folks working with StackReference and complex types (eg, an array/list of Route53 zones, VPC subnets, etc.)?
Here's an example of how it fails to work with complex types
In the first stack (route53-private-zones), provision 2 Route53 private hosted zones. A dummy VPC is associated with the zones to make them private. Export the list of private zones.
Copy code
import pulumi
import pulumi_aws as aws

dummy_vpc = aws.ec2.Vpc(
    resource_name="dummy-vpc",
    cidr_block="10.0.0.0/16",
    tags={
        "Name": "dummy-vpc",
        "Purpose": "to support provisioning Route53 private hosted zones",
    }
)

domains = ["<http://example.io|example.io>", "<http://example.com|example.com>"]

zones = []

for domain in domains:
    zone = aws.route53.Zone(
    resource_name=domain,
    name=domain,
    comment=f"Private Zone for {domain}",
    force_destroy=True,
    vpcs=[{
        "vpc_id": dummy_vpc.id
    }],
    tags={
        "Name": domain
    },
    opts=pulumi.ResourceOptions(
        depends_on=[dummy_vpc],
        ignore_changes=["vpcs"]
    )
)

    zones.append(zone)

pulumi.export("private-zones", zones)
This is what the outputs looks like:
Copy code
Outputs:
    private-zones: [
        [0]: {
            comment          : "Private Zone for <http://example.io|example.io>"
            force_destroy    : true
            id               : "Z0821544235H0134DJISV"
            name             : "<http://example.io|example.io>."
            name_servers     : [
                [0]: "<http://ns-0.awsdns-00.com|ns-0.awsdns-00.com>."
                [1]: "<http://ns-1024.awsdns-00.org|ns-1024.awsdns-00.org>."
                [2]: "<http://ns-1536.awsdns-00.co.uk|ns-1536.awsdns-00.co.uk>."
                [3]: "<http://ns-512.awsdns-00.net|ns-512.awsdns-00.net>."
            ]
            tags             : {
                Name: "<http://example.io|example.io>"
            }
            urn              : "urn:pulumi:route53-private-zones::zones::aws:route53/zone:Zone::<http://example.io|example.io>"
            vpcs             : [
                [0]: {
                    vpc_id    : "vpc-0540adf940a96ff5f"
                    vpc_region: "us-west-2"
                }
            ]
            zone_id          : "Z0821544235H0134DJISV"
        }
        [1]: {
            comment          : "Private Zone for <http://example.com|example.com>"
            force_destroy    : true
            id               : "Z08216062GEL3399WZSXA"
            name             : "<http://example.com|example.com>."
            name_servers     : [
                [0]: "<http://ns-0.awsdns-00.com|ns-0.awsdns-00.com>."
                [1]: "<http://ns-1024.awsdns-00.org|ns-1024.awsdns-00.org>."
                [2]: "<http://ns-1536.awsdns-00.co.uk|ns-1536.awsdns-00.co.uk>."
                [3]: "<http://ns-512.awsdns-00.net|ns-512.awsdns-00.net>."
            ]
            tags             : {
                Name: "<http://example.com|example.com>"
            }
            urn              : "urn:pulumi:route53-private-zones::zones::aws:route53/zone:Zone::<http://example.com|example.com>"
            vpcs             : [
                [0]: {
                    vpc_id    : "vpc-0540adf940a96ff5f"
                    vpc_region: "us-west-2"
                }
            ]
            zone_id          : "Z08216062GEL3399WZSXA"
        }
    ]
In the second stack, provision a new VPC and associate it with all of the Route53 private zones provisioned earlier.
Copy code
import pulumi
import pulumi_aws as aws

stack_reference = pulumi.StackReference("route53-private-zones")

private_zones = stack_reference.require_output("private-zones")

main_vpc = aws.ec2.Vpc(
    resource_name="main-vpc",
    cidr_block="10.100.0.0/16",
    tags={
        "Name": "main-vpc"
    }
)

for zone in private_zones:
    zone_name = zone["name"]
    zone_id = zone["zone_id"]

    aws.route53.ZoneAssociation(
        resource_name=zone_name,
        vpc_id=main_vpc.id,
        zone_id=zone_id,
        opts=pulumi.ResourceOptions(delete_before_replace=True)
    )
The above will hang indefinitely
Here's an extremely inadequate workaround:
Copy code
import pulumi
import pulumi_aws as aws

stack_reference = pulumi.StackReference("route53-private-zones")

private_zones = stack_reference.require_output("private-zones")

main_vpc = aws.ec2.Vpc(
    resource_name="main-vpc",
    cidr_block="10.100.0.0/16",
    tags={
        "Name": "main-vpc"
    }
)

pulumi.export("vpc", main_vpc)

zone_ids = []

zone_ids.append(private_zones[0]["zone_id"])
zone_ids.append(private_zones[1]["zone_id"])

for i, zone_id in enumerate(zone_ids):
    aws.route53.ZoneAssociation(
        resource_name=f"zone-association-{i}",
        vpc_id=main_vpc.id,
        zone_id=zone_id,
        opts=pulumi.ResourceOptions(delete_before_replace=True)
    )
I've tried provisioning the resources within an 
apply()
but that doesnt seem to work:
Copy code
import pulumi
import pulumi_aws as aws


def vpc_route53_private_zone_association(args):
    vpc_id   = args[0]
    zone_ids = args[1]

    for i, zone in enumerate(zone):
        zone_id = zone["zone_id"]

        aws.route53.ZoneAssociation(
            resource_name=f"zone-association-{i}",
            vpc_id=vpc_id,
            zone_id=zone_id,
            opts=pulumi.ResourceOptions(delete_before_replace=True)
        )


stack_reference = pulumi.StackReference("route53-private-zones")

private_zones = stack_reference.require_output("private-zones")

main_vpc = aws.ec2.Vpc(
    resource_name="main-vpc",
    cidr_block="10.100.0.0/16",
    tags={
        "Name": "main-vpc"
    }
)

pulumi.export("vpc", main_vpc)

pulumi.Output.all(
    main_vpc.id,
    private_zones
).apply(vpc_route53_private_zone_association)
f

flat-insurance-25294

01/27/2020, 11:39 AM
It looks like you’re trying to replicate the internal datastructures to store in your stack, wy not just store ids and reference them using the static 
get
functions to get them from AWS?
w

white-balloon-205

01/27/2020, 5:07 PM
Yes - there is a serialization boundary here - so you do have to turn data into a format which can be safely serialized across the boundary (effectively as JSON). You cannot pass “live JavaScript objects” across this boundary. So exporting ids and rehydrating with 
get
as @flat-insurance-25294 suggests is definitely the recommended approach.
m

mysterious-egg-7415

01/27/2020, 10:17 PM
Can you share an example?
In Terraform, as of v0.12, the entire attributes of a resource can be exported ain an output variable. For example:
Copy code
resource "aws_route53_zone" "example_io" {
  name          = "<http://example.io|example.io>
  comment       = "Root Zone"
  force_destroy = true
}

output "route53_example_io_zone" {
  value = aws_route53_zone.example_io
}
In Pulumi, I have been able to do the same and it's worked fine for me except with array/list of resources. It's just not able to loop over it.
f

flat-insurance-25294

01/28/2020, 10:49 AM
aws.route53.getZone({ name: "<http://example.io|example.io>" })
m

mysterious-egg-7415

01/28/2020, 12:22 PM
That example won't work because I do not know the name of the domains to be able to do the lookup. I have to fetch the list of domains from the state, but even exporting the list of domains or zone IDs does not work because Pulumi will not correctly loop over a list retrieved by StackReference.
f

flat-insurance-25294

01/28/2020, 12:22 PM
How are you creating the domain?
So export the domain name in the stack that creates it. And use its value for getting it.
Regardless, you’re trying to serialize and deserialize values between stacks, that’s a pretty brittle way and will break hard. Try to export ids for referencing resources between stacks which is your initial problem.
m

mysterious-egg-7415

01/28/2020, 12:25 PM
That might work but it will lead to code duplication. In my actual use case, I have a ComponentResource that accepts a domain and creates multiple Route53 zones and exports the private zones, so that they can be associated with VPCs in another ComponentResource
f

flat-insurance-25294

01/28/2020, 12:25 PM
huh? Dude, you’re literally exporting entire objects and trying to remake them. Export an id, and refer to them in your second stack.
m

mysterious-egg-7415

01/28/2020, 12:29 PM
I dont think the issue is with exporting entire objects since they are output correctly to the state as outputs. I can interact with dicts/maps from the state just fine. Even if I output just a list of IDs, Pulumi will not allow me to fetch them from the state and loop over the list. I'm not sure how exporting individual IDs would solve my problem.
I will try a couple different approaches and report back
f

flat-insurance-25294

01/28/2020, 12:30 PM
If you say so.
2 Views
Powered by