Is there a way from within python / while running pulumi to access all the objects of a stack? Concrete example of what I am trying to do - I create some subnets, a few weeks later I want to add a subnet, I want to read the objects that were created before and make sure my new subnet is at the end of the previous. Instead of specifying every cidr I want to just say, make 5 /21 and have it math it out itself but I need to be able to see what was created before which in theory should all be in the state? (Using the pulumi saas for state if that makes any difference)

If it's in the same project, then you should try very hard not to have an workflow like this. Essentially you're trying to make something declarative appear to be imperative. It's just not designed this way.

I want declarative, but compositionally declarative. So, in a VPC object I want to be able to say I want 3 public subnets, and 5 private subnets - and then I want that to be calculated. The issue is say I loop to create the subnet objects, and I do public then private - but later I add more public subnets it will break without some kind of awareness of what existed previously. If there isn't a pulumi native way to get state info I might try pickling the objects.

Later, you cannot add more public subnets. If you did that, then you'd need to manage the VPC in two projects, which is not to be recommended.

After fighting with it, I am accepting that pulumi is pythonic but not python and submitting myself to projects/stacks and stack references - so you were right.

private_subnets = []
for subnet in range(1, number_of_azs + 1):
    ec2_subnet = aws.ec2.Subnet(
        f"{env}-ec2Subnet-{subnet}-{suffix}",
        availability_zone=available_azs.names[subnet - 1],
        cidr_block=f"10.27.{subnet}.0/24",
        vpc_id=vpc.id,
        map_public_ip_on_launch=False,
        tags={
            "Type": "Private",
            "Name": f"{env}-ec2Subnet-{subnet}-private-{suffix}",
        },
        opts=pulumi.ResourceOptions(provider=provider),
    )
    private_subnets.append(ec2_subnet)

will this work? I need to get the subnet ids at the end..However I am still getting each element as type Output... So this wont work?

This works. You cannot get the subnet id at this top code level though. You can only get it inside an apply().