https://pulumi.com logo
Powered by
#python
Title
# python
r

rhythmic-branch-12845

11/24/2022, 7:57 AM
question for the Pulumi folks (and whoever’s in the know): are there plans to have a list be exportable in Pulumi? I’m hoping to be able to export a list from a stack so that I can simply iterate through the list from another stack - instead of having to essentially maintain the same list of names (“list-item-1”, “list-item-2", etc.) across 2 projects. My use case is to be able to export a list of subnet ids from a 
vpc
project, to another project that needs the list of subnet ids. Based on https://www.pulumi.com/docs/reference/pkg/python/pulumi/#stack-exports-1 and seeing 
pulumi.export(_name: str_, _value: Any_)
, I was thinking that I could export a list… but nope:
Copy code
pulumi:pulumi:Stack (acm-us):
    error: Program failed with an unhandled exception:
    Traceback (most recent call last):
      File "/Users/jf/.../pulumi/acm/./__main__.py", line 40, in <module>
        pulumi.export(cert_arns, cert_arns)
      File "/Users/jf/.../pulumi/venv/lib/python3.10/site-packages/pulumi/resource.py", line 1282, in export
        res.output(name, value)
      File "/Users/jf/.../pulumi/venv/lib/python3.10/site-packages/pulumi/runtime/stack.py", line 158, in output
        self.outputs[name] = value
    TypeError: unhashable type: 'list'
UPDATE: this has been resolved. A big thank you to @billowy-army-68599 for the help! Looking back at the code above again, I also see where I went wrong: 
pulumi.export(cert_arns, cert_arns)
should have been 
pulumi.export("cert_arns", cert_arns)
! argh….
I can think of 2 workarounds. Both would work, but not really ideal, especially the 2nd: 1. read the output stack’s config (the list of names is maintained in that stack’s YAML file) from the input stack 2. roll my own state backend on the side for exporting / importing this list (no!)
argh, I thought I had it with this last method (exporting a list of values), but am getting stuck. Reading https://www.pulumi.com/docs/intro/concepts/inputs-outputs/, I thought I could push the list of subnet ids into a list (which really is a list of outputs), and then 
.apply
on each one. But nope!:
Copy code
pulumi:pulumi:Stack (vpc-us):
    error: Program failed with an unhandled exception:
    Traceback (most recent call last):
      File "/Users/jf/…/pulumi/vpc/./__main__.py", line 106, in <module>
        pulumi.export('public_subnet_ids', yaml.dump( [ sid.apply(lambda v: f'{ v }') for sid in public_subnet_ids ] ))
      File "/Users/jf/…/pulumi/venv/lib/python3.10/site-packages/yaml/__init__.py", line 253, in dump
        return dump_all([data], stream, Dumper=Dumper, **kwds)
      File "/Users/jf/…/pulumi/venv/lib/python3.10/site-packages/yaml/__init__.py", line 241, in dump_all
        dumper.represent(data)
      File "/Users/jf/…/pulumi/venv/lib/python3.10/site-packages/yaml/representer.py", line 27, in represent
        node = self.represent_data(data)
      File "/Users/jf/…/pulumi/venv/lib/python3.10/site-packages/yaml/representer.py", line 48, in represent_data
        node = self.yaml_representers[data_types[0]](self, data)
      File "/Users/jf/…/pulumi/venv/lib/python3.10/site-packages/yaml/representer.py", line 199, in represent_list
        return self.represent_sequence('tag:<http://yaml.org|yaml.org>,2002:seq', data)
      File "/Users/jf/…/pulumi/venv/lib/python3.10/site-packages/yaml/representer.py", line 92, in represent_sequence
        node_item = self.represent_data(item)
      File "/Users/jf/…/pulumi/venv/lib/python3.10/site-packages/yaml/representer.py", line 52, in represent_data
        node = self.yaml_multi_representers[data_type](self, data)
      File "/Users/jf/…/pulumi/venv/lib/python3.10/site-packages/yaml/representer.py", line 317, in represent_object
        reduce = data.__reduce_ex__(2)
    TypeError: 'Output' object is not callable
What I am doing wrong? Instead of doing the export, I’ve also tried to 
print
with 
print( public_subnet_ids[0].apply(lambda v: f"{ v }") )
. While this does not give me any explicit errors, it seems that Pulumi still has issues because I get this output under `Diagnostics:`:
Copy code
Diagnostics:
  pulumi:pulumi:Stack (vpc-us):
    Calling __str__ on an Output[T] is not supported.
    To get the value of an Output[T] as an Output[str] consider:
    1. o.apply(lambda v: f"prefix{v}suffix")
    See <https://pulumi.io/help/outputs> for more details.
    This function may throw in a future version of Pulumi.
It seems that it does NOT recognize that I have done precisely what it’s telling me to do? This is the output for 
print(public_subnet_ids)
, btw: 
[<pulumi.output.Output object at 0x1048b5f30>, <pulumi.output.Output object at 0x1048da950>, <pulumi.output.Output object at 0x104903430>]
ok rereading https://www.pulumi.com/docs/intro/concepts/inputs-outputs/ again, it seems like all an output is ever good for is to ultimately be the input for another resource. I cannot otherwise access the value of an output directly (like I can in Terraform!), even if the value is already computed and present. So there is no elegant way to get rid of this coupling of the knowledge of the contents of a list between 2 projects. Arrrrgh….
b

billowy-army-68599

11/24/2022, 3:18 PM
1. you can definitely output a list 2. if you output anything, you need to access it inside an apply
this code is wrong
Copy code
print( public_subnet_ids[0].apply(lambda v: f"{ v }") )
it needs to be
Copy code
public_subnet_ids.apply(lambda v: print v)
note how I’m doing the print inside the apply
r

rhythmic-branch-12845

11/24/2022, 3:33 PM
hmmm: • 
public_subnet_ids
is a list • if I were to do a “public_subnet_ids.apply” call, I get 
AttributeError: ‘list’ object has no attribute ‘apply’
• unless you’re using python 2 (I’m not sure Pulumi supports that now), you will get a complaint: 
SyntaxError: Missing parentheses in call to ‘print’. Did you mean print(...)?
b

billowy-army-68599

11/24/2022, 3:34 PM
ah it’s a list of 
output<T>
so you’d need to do
Copy code
public_subnet_ids[0].apply(lambda v: print v)
sorry yeah I’m not testing it, you’ll need to do 
print(v)
r

rhythmic-branch-12845

11/24/2022, 3:39 PM
to reiterate, I’m hoping to be able to export a list from a stack so that I can simply iterate through the list from another stack - instead of having to essentially maintain the same list of names (“list-item-1”, “list-item-2", etc.) across 2 projects. My use case is to be able to export a list of subnet ids from a 
vpc
project, to another project that needs the list of subnet ids. In this case then, I want to export a list. 
print(v)
just simply prints the text to 
Diagnostics:
(which I dont need) but does not export the list for use from another project
b

billowy-army-68599

11/24/2022, 3:41 PM
you can access the output of a value directly via an 
apply
- you have to resolve it.
let me throw an example together
r

rhythmic-branch-12845

11/24/2022, 3:43 PM
thank you. My issue is that I havent been able to do the 
apply
and then export the list. I could do a 
pulumi.export("public_subnet_ids", public_subnet_ids)
… but Pulumi just gives me this under the output after running:
Copy code
Outputs:
  + public_subnet_ids      : [
  +     [0]: <null>
  +     [1]: <null>
  +     [2]: <null>
    ]
I doubt this means I can access it from another project
(and I just confirmed it at app.pulumi.com. Whereas the other outputs give proper values, 
public_subnet_ids
is just a list of `null`s):
b

billowy-army-68599

11/24/2022, 3:45 PM
okay, that’s an issue with your code, not the output system
can you share your full code in a gist?
r

rhythmic-branch-12845

11/24/2022, 3:46 PM
whatever the case, can I say that this is a problem with the docs too?
b

billowy-army-68599

11/24/2022, 3:47 PM
what would you say is the issue with the docs?
r

rhythmic-branch-12845

11/24/2022, 3:51 PM
I keep bumping into issues where the docs are wrong, or in this case, not clear enough. Some examples: • https://www.pulumi.com/registry/packages/aws/api-docs/acm/certificatevalidation/#dns-validation-with-route-53 ’s Python example has 
aws.route53.CertificateValidation
. This is wrong. There is no 
aws.route53.CertificateValidation
. It should be 
aws.acm.CertificateValidation
• https://www.pulumi.com/registry/packages/aws/api-docs/lb/listener/#inputs: the description for 
protocol
says: 
Protocol. Valid values are HTTP, HTTPS, or #{protocol}. Defaults to #{protocol}.
It actually defaults to 
HTTP
.
b

billowy-army-68599

11/24/2022, 3:52 PM
those both look like valid bugs yes, we auto generate our docs so there may be some issues
r

rhythmic-branch-12845

11/24/2022, 3:53 PM
as I expected, because it really looks like it. The docs are / have been autogenerated… and they dont seem to have gone through a human verifier
but back to what I started this thread for, I’m hoping this little snippet of code should be sufficient to show something of what I’m trying to do
Copy code
public_subnet_ids = []
for i in range(0, vars.az_count):
  public_subnet = aws.ec2.Subnet(...)
  public_subnet_ids.append( public_subnet.id )

# hoping to export a proper list
pulumi.export('public_subnet_ids', public_subnet_ids)
if it’s easier you could create a list of buckets, rather than a list of subnets that requires a whole bunch of other resources to be created as well, so that it’s easier to test things out. Again, the point is that I would like to export a list of values that I can read from another project
b

billowy-army-68599

11/24/2022, 3:58 PM
this appears to work correctly to me
Copy code
import pulumi_aws
import pulumi

# Create a VPC
vpc = pulumi_aws.ec2.Vpc('my-vpc', cidr_block="172.16.0.0/16")

# Get all the availability zones
zones = pulumi_aws.get_availability_zones()

subnet_ids = []

# Loop through all the zones and create a subnet in each
for zone in zones.names:
    vpc_subnet = pulumi_aws.ec2.Subnet(
        f'vpc-subnet-{zone}',
        vpc_id=vpc.id,
        cidr_block=f'172.16.{len(subnet_ids)}.0/24',
        availability_zone=zone
    )
    subnet_ids.append(vpc_subnet.id)

pulumi.export('subnet_ids', subnet_ids)
Copy code
Updating (dev)

View Live: <https://app.pulumi.com/jaxxstorm/vpc_with_subnets_python/dev/updates/2>

     Type                 Name                         Status               Info
     pulumi:pulumi:Stack  vpc_with_subnets_python-dev
 +-  ├─ aws:ec2:Vpc       my-vpc                       replaced (0.72s)     [diff: ~cidrBlock]
 +   ├─ aws:ec2:Subnet    vpc-subnet-us-west-2a        created (0.83s)
 +   ├─ aws:ec2:Subnet    vpc-subnet-us-west-2b        created (0.84s)
 +   ├─ aws:ec2:Subnet    vpc-subnet-us-west-2c        created (0.91s)
 +   └─ aws:ec2:Subnet    vpc-subnet-us-west-2d        created (0.91s)


Outputs:
  + subnet_ids: [
  +     [0]: "subnet-0e0b8c4ef854516e2"
  +     [1]: "subnet-02351fc61ddbce3af"
  +     [2]: "subnet-00efaf2162127a2f2"
  +     [3]: "subnet-09d3ec89cf51b6388"
    ]
r

rhythmic-branch-12845

11/24/2022, 4:07 PM
thank you. It looks like this does work for me too. I’m trying to debug this right now, but I’m suspecting a “depends_on” that may be affecting the output. Otherwise I dont see anything special or different about the subnets that I create
b

billowy-army-68599

11/24/2022, 4:07 PM
if you’re able to share the full code we might be able to figure out where the issue is
r

rhythmic-branch-12845

11/24/2022, 4:10 PM
pls hold on. I’m trying to find a good-enough small snippet to demonstrate the issue
b

billowy-army-68599

11/24/2022, 4:11 PM
I threw a quick example together of exporting vpc ids and then consuming them from a stack reference, you can also see the way to print a list as well https://github.com/jaxxstorm/pulumi-examples/tree/main/python/aws/eks_platform
r

rhythmic-branch-12845

11/24/2022, 4:41 PM
ok I think i’ve found the issue. The crux of the issue is that you cannot export a value twice. If it is already exported… the second export will produce the null. This is basically the same code like you provide above, but with the crucial addition of the additional export. Forgive the other edits; I did all those in an effort to try to find the issue:
Copy code
import pulumi_aws
import pulumi

# Create a VPC
vpc = pulumi_aws.ec2.Vpc('my-vpc', cidr_block="172.16.0.0/16", tags = { 'Name': 'test' })

# Get all the availability zones
zones = pulumi_aws.get_availability_zones(all_availability_zones = True)

subnet_ids = []


# Loop through all the zones and create a subnet in each
for i in range(0, 3):
	subnet_name = f'vpc-subnet-{i}'

	vpc_subnet = pulumi_aws.ec2.Subnet(subnet_name,
		vpc_id = vpc.id,

		availability_zone = zones.names[i],
		cidr_block=f'172.17.{len(subnet_ids)}.0/24',

		tags = { 'Name': f'test-subnet-{i}' }
	)

	pulumi.export(subnet_name, vpc_subnet.id) # export vpc_subnet.id once
	subnet_ids.append(vpc_subnet.id)

pulumi.export('subnet_ids', subnet_ids)
b

billowy-army-68599

11/24/2022, 4:42 PM
r

rhythmic-branch-12845

11/24/2022, 4:45 PM
great. One last question: I see at https://github.com/jaxxstorm/pulumi-examples/blob/main/python/aws/eks_platform/cluster/__main__.py that you use 
.require_output
. I’ve not seen this in the docs that I’ve come across, only 
get_output
. Is this a synonym? I’m not able to find any info on 
.require_output
just reporting that I seem to be able to get by with 
.get_output
b

billowy-army-68599

11/24/2022, 5:12 PM
require_output
throws an error if the upstream stack reference doesn’t exist 
get_output
will return null if the upstream stack reference doesn’t exist
r

rhythmic-branch-12845

11/24/2022, 5:14 PM
great. Thank you!! Is this documented anywhere?
b

billowy-army-68599

11/24/2022, 5:19 PM
r

rhythmic-branch-12845

11/24/2022, 5:28 PM
4 Views
Powered by