https://pulumi.com logo
#python
Title
# python
b

better-actor-92669

09/17/2019, 9:36 AM
Hi everyone! I want to separate my GCP cloud resources by categories. I use pulumi's
gcp-python
. For instance, I want to store my buckets definitions in a separate directory or in python terms, simply in a separate python package, instances in another one, vpn and network definition in yet another one, and so on. With the directory structure below, I managed to import packages' code into pulumi's
__main__.py
and make it work. Nevertheless, I don't like that I have to do import statements
Copy code
import pulumi
from pulumi_gcp import storage
both in a package module and in
__main__.py
itself. I know that am probably doing this wrong, so my question is: What is the best practice to separate pulumi resources as python packages? I would really appreciate some examples of how some of you do that. tree structure:
Copy code
(pulumi_venv) some_user@MAC-C02YG37SJGH some-pulumi (master) $ tree
.
├── Pulumi.dev.yaml
├── Pulumi.yaml
├── __main__.py
├── __pycache__
│   └── __main__.cpython-37.pyc
├── buckets
│   ├── __init.py__
│   ├── __pycache__
│   │   ├── bucket_func.cpython-37.pyc
│   │   └── bucket_plain.cpython-37.pyc
│   ├── bucket_func.py
│   └── bucket_plain.py
└── requirements.txt

3 directories, 10 files
buckets/bucket_func.py which has a bucket creation within a function:
Copy code
import pulumi
from pulumi_gcp import storage


def create_bucket5():
    # Create a GCP resource (Storage Bucket)
    bucket = storage.Bucket('pulumi-test-bucket-5', name="pulumi-test-bucket-5")

    # Export the DNS name of the bucket
    pulumi.export('bucket_name',  bucket.url)
buckets/bucket_plain.py which is a plain code file not wrapped into a function or something:
Copy code
import pulumi
from pulumi_gcp import storage

# Create a GCP resource (Storage Bucket)
bucket = storage.Bucket('pulumi-test-bucket-5', name="pulumi-test-bucket-5")

# Export the DNS name of the bucket
pulumi.export('bucket_name',  bucket.url)
and the main.py for the buckets/bucket_func
Copy code
import pulumi
from pulumi_gcp import storage

-->>import buckets.bucket_func

# State bucket
# Create a GCP resource (Storage Bucket)
bucket1 = storage.Bucket('pulumi-test-bucket-2')

# Export the DNS name of the bucket
pulumi.export('bucket_name',  bucket1.url)

-->>buckets.bucket_func.create_bucket5()
and also the main.py for the second case buckets/bucket_plain:
Copy code
import pulumi
from pulumi_gcp import storage

-->>import buckets.bucket_plain

# State bucket
# Create a GCP resource (Storage Bucket)
bucket1 = storage.Bucket('pulumi-test-bucket-2')

# Export the DNS name of the bucket
pulumi.export('bucket_name',  bucket1.url)
In both cases it works and creates a
pulumi-test-bucket-5
bucket. Please note the
pulumi-test-bucket-2
in
__main__.py
is just a state bucket for pulumi.
Copy code
(pulumi_venv) some_user@MAC-C02YG37SJGH some-pulumi (master) $ pulumi up
Enter your passphrase to unlock config/secrets
    (set PULUMI_CONFIG_PASSPHRASE to remember):
Previewing update (dev):

     Type                   Name                  Plan
     pulumi:pulumi:Stack    lodgify-pulumi-dev
 +   └─ gcp:storage:Bucket  pulumi-test-bucket-5  create

Resources:
    + 1 to create
    2 unchanged

Do you want to perform this update?
  yes
> no
  details
n

numerous-easter-93888

09/17/2019, 7:22 PM
Hi dz, I believe your best bet is to create a master project, then multiple stacks within that project
To create stacks that share a single project, use the same Pulumi.yaml file in each project's folder. When you run
pulumi up
it will look for the "nearest Pulumi.yaml file" and use that as its project
b

better-actor-92669

09/18/2019, 8:48 AM
thank you for the reply
but i don't think having a lot of stacks is the best approach, since in my opinion stack is for different purposes
n

numerous-easter-93888

09/18/2019, 1:16 PM
Ah, I see now. Apologies, I misunderstood the question
As far as I'm aware that's how you'll have to do it. There's a way to configure setup.py so it will import and install modules for you upon installing your package, but beyond that, what you are doing is more or less "best practice." I'd love to learn along with you if someone knows a better way
b

better-actor-92669

09/18/2019, 1:46 PM
ok, thank you @numerous-easter-93888. Let's hope that someone else will reply as well )