Hello and Happy new Year :tada: I'm trying to load...
# python
s
Hello and Happy new Year šŸŽ‰ I'm trying to load a JSON file using pulumi-python and I'm struggling to find a way to use a relative path. context: I'd like to load this JSON file and then use the data to create pulumi resources. I placed the JSON file next to the
Pulumi.yaml
and the stack files and I'd like to be able to store a relative path inside the stack
config
. Let's say it looks like this:
Copy code
.
ā”œā”€ā”€ Pulumi.main.yaml
ā”œā”€ā”€ Pulumi.yaml
ā””ā”€ā”€ auth0-users.json
and inside `Pulumi.main.yaml`:
Copy code
config:
  vault_config: ./auth0-users.json
According to the Paths doc and the Assets & archives doc, pulumi is capable of resolving relative paths
they are always relative to the working directory
, but it seems that pulumi does not expose any method or function to get the path to the working directory Any idea how to achieve something like this? Or maybe there is a method that can expose the path to the working directory and I missed it?
m
Would something like this work?
Copy code
import json
from pulumi_aws import s3

with open("buckets.json") as bucket_data:
    buckets = json.load(bucket_data)

for bucket in buckets:
    s3.Bucket(bucket)
... where
buckets.json
just contains a JSON array of strings:
Copy code
[
    "one",
    "two",
    "three"
]
When I run this, I get this, for example:
If you want the actual working directory, you can probably get that from Python directly -- like with
os.getcwd()
. But based on what you've described, I don't think you'll need it.
If you wanted to read the file path from a stack-config file, then, it'd just be something like this:
Copy code
import json
import pulumi
from pulumi_aws import s3

config = pulumi.Config()
bucket_list = config.get("bucket_path")

with open(bucket_list) as bucket_data:
    buckets = json.load(bucket_data)

for bucket in buckets:
    s3.Bucket(bucket)
with a config file like:
Copy code
config:
  aws:region: us-west-2
  python-json:bucket_path: ./buckets.json
Hope that helps and that I've understood you right!
s
ā€¢ I looked into using
os.getcwd()
but unfortunately, pulumi changes the current directory to the program directory so it does not give an accurate value šŸ˜ž Using the project structure below, if I am inside the
configs/dev
directory when I run
pulumi up
, if I add
print(os.getcwd())
inside the
__main__.py
, it returns the path to the
src
folder and not the configs folder ā€¢ Your example works because the
Pulumi.yaml
and the python program are in the same folder. I guess that's on me for not being more detailed šŸ˜„ This is my project structure:
Copy code
ā”œā”€ā”€ configs
ā”‚   ā”œā”€ā”€ dev
ā”‚   ā”‚   ā”œā”€ā”€ Pulumi.main.yaml
ā”‚   ā”‚   ā”œā”€ā”€ Pulumi.yaml
ā”‚   ā”‚   ā””ā”€ā”€ auth0-users.yaml
ā”œā”€ā”€ requirements.txt
ā””ā”€ā”€ src
    ā”œā”€ā”€ __init__.py
    ā”œā”€ā”€ __main__.py
    ā”œā”€ā”€ __pycache__
    ā”œā”€ā”€ auth0_database.py
and the
Pulumi.yaml
file has
main: ../../src
attribute
m
There may be options for adjusting some of this at the project file level -- e.g., here: https://www.pulumi.com/docs/concepts/projects/project-file/#attributes
If the users yaml/json file is truly infra config, you might also consider just putting that data right into one of the Pulumi YAML files
That'd get you around having to work around the default behavior and standard conventions
Also curious what error(s) you're getting
I think I have something that works with your directory structure though
s
> If the users yaml/json file is truly infra config, you might also consider just putting that data right into one of the Pulumi YAML files I thought about this but I did not want to bundle all the configuration inside the one yaml file because it'd make it quite big and difficult to maintain šŸ˜• --- so far the only option I found is the use
os.getcwd()
inside
__main__.py
and manually compute the path back to the configs folder like:
../configs/dev
where
dev
would be stored in a variable so I can dynamically adjust the path
your example seems similar to what I had in mind looks like: manually compute the path between the pulumi program and the initial config folder
m
Yeah -- I mean the Python program isn't running in the
configs
folder, it's running in the
src
folder, so this makes sense, relative to where the program code is
s
I'm sad that pulumi clearly knows this path (it uses it to compute the absolute path when using
FileAsset
resource) but does not expose it anywhere šŸ˜ž Maybe I could open a feature request for this
I do very much appreciate the time you spent trying to figure out a solution with me šŸ™
m
You bet! Always fun trying to make these things work šŸ™‚