https://pulumi.com logo
Join Slack
Powered by
I was having trouble finding any information on th...
# general
l
I was having trouble finding any information on this online: Is it at all reasonable to use a database as a source of data when building infra from pulumi? E.G. I want to build a Cloud Storage Bucket for each user organization that signs up with us, information which we store in a database table "organization". Is there a recommended way to approach this kind of situation?
r
Hey! You could use automation API for something like this. Your automation API server could read from the database, or be triggered on some sort of event based mechanism when the table is updated. You could then create an inline program with data passed in from the db query. There are examples here: https://github.com/pulumi/automation-api-examples Of interest might be the pulumi over http examples.
l
Thank you! I'll take a look at this.
Is there any built-in way to still load values from a Pulumi.yaml config?
Or do I need to load the YAML file and manually set everything?
r
You can still access stack configuration using stack.GetConfig or stack.SetConfig. The thing to note though, is that if you're using an inline program, that program is by default executed in a temporary directory, so you either need to specify the directory you're working in (if you have a stack configuration yaml) or set all of your configuration values using 
setConfig
What's your language of choice? I can point to the right examples if I know what lang you're using
The pulumi program itself will still consume configuration as it normally does.
l
Ah, thank you. I was trying to do an inline program when what I want is a local program, just overriding some config values with things selected from our DB.
I will certainly miss the nicely formatted output on the Pulumi CLI, but it's great how easy this is. Thanks for the help.
One last question @red-match-15116, sorry! So via the automation API, is it not possible to use 
set_config
to set a value to an array or object? 
pulumi config set
has the option for a --path that enables this. Not seeing anything analogous for 
set_config
in the Python API ..
r
You can do 
path
with 
set_config
too! I think that functionality was added not too long ago. Lemme find the link.
l
AH.. I do see it in the source code for the lib actually! Thanks! Is there a reference/documentation for the libraries anywhere? I was trying to find it on the pulumi site but I couldn't..
r
Yeah, it exists but it's a little funky: https://www.pulumi.com/docs/reference/pkg/python/pulumi/#module-pulumi.automation:~:text=ConfigMap-,set_config,-(stack_name%3A There is an open issue to move this to readthedocs style python sdk documentatoin
l
oh this is wonderful regardless, thank you!!
one day soon I will have time to contribute back to the Pulumi community. Great stuff, thank you very much for the help here
Okay -- one final question, actually! Code complete, but having some trouble getting this to run properly in a GHA, and I'm worried this is a tricky one:
Copy code
pulumi:pulumi:Stack my-org-prod  Installing dependencies in virtual environment...
    pulumi:pulumi:Stack my-org-prod  Looking in indexes: <https://pypi.org/simple>, <https://us-central1-python.pkg.dev/my-org-test-01/my-org-python-repo/simple>, <https://us-central1-python.pkg.dev/my-org-prod-01/my-org-python-repo/simple>
@ Previewing update....
    pulumi:pulumi:Stack my-org-prod  User for us-central1-python.pkg.dev: 
    pulumi:pulumi:Stack my-org-prod  error: ERROR: Exception:
    pulumi:pulumi:Stack my-org-prod 
 stderr: error: failed to discover plugin requirements: installing dependencies via '/home/runner/work/universe/universe/venv/bin/python -m pip install -r requirements.txt': exit
We have dependencies on internal python libs that we host in GCP. As a result, we need to install 
keyring
and 
keyrings.google-artifactregistry-auth
before running 
pip install -r requirements.txt
so that 
pip
knows how to find our internal Python libraries... Is there any way to get Pulumi with a local program to first install a separate 
requirements-prereqs.txt
file before it tires to install the main 
requirements.txt
in its virtual environment?
r
Heh yeah this is a bit of a tricky one. There's no automation api mechanism for this but you can always use python builtins to run shell commands at the start of your auto api script: https://www.geeksforgeeks.org/executing-shell-commands-with-python/#
l
Unless I'm missing something, I don't see how that will help -- I already have the dependencies installed when I run the automation. But it seems with a Local Program in Pulumi it creates a new virtual subenvironment as part of its process. What I need to do is have THAT virtual environment install these additional dependencies. Is there a way I can have a shell subprocess run WITHIN the virtual env, before it tries to install 
requirements.txt
?
2 Views
Open in Slack
PreviousNext
Powered by