https://pulumi.com logo
#general
Title
# general
p

purple-lion-47621

07/31/2023, 9:49 AM
Hi Experts !! 👋 We are trying to evaluate whether we can use Pulumi Automation API to orchestrate infrastructure creation through a REST API. • We will host Pulumi state on our own cloud storage bucket. • We write a python based middle-ware service which integrates with Pulumi automation API. • This middle-ware service will eventually work with a fronted. • Our python middle-ware service will expose endpoints to create resources (e.g.: an endpoint to create a given Azure key vault). ◦ e.g.:
User1
sends in a request to create
vault1
.
User2
sends in a request to create
vault2
. Is this architecture supported by Pulumi automation API ? Or does Pulumi strictly requires the entire state to be defined in the source of truth (100% IaaC) ?
e

echoing-dinner-19531

07/31/2023, 3:34 PM
I know lots of users do expose automation api via rest apis. So I'd think this should be fine. Your last comment is confusing me a little, because yes Pulumi does strictly require the entire state to be defined, how else would it work, but you can do that with automation api just fine.
h

hallowed-horse-57635

07/31/2023, 9:07 PM
We have used a similar architecture four our commercial software with a UI and No Code technologies ( drag and drop, templates, smart wizrad). This uses the automation api but we do use pulumi cloud for state management and other advanced rbac capabilities. <advt> qmcloud.io
p

purple-lion-47621

08/01/2023, 4:11 AM
Thanks for the responses @echoing-dinner-19531 and @hallowed-horse-57635 ! How do you handle the below scenario for example ? Let's say we have the below generic endpoint to create a secret placeholder in GCP. •
User1
sends a request to create the secret
placeholder-1
(fine it will be created). •
User2
sends a request to create secret
placeholder-2
. New
placeholder-2
will be created but it will destroy
placeholder-1
, correct ? How do to handle this scenario ? Do you have your own layer implemented to reflect the complete state ? Or instead of generic endpoints do you define everything explicitly ?
Copy code
@app.get('/gcp/{project}/{stack}/secret/add/{secret_id}')
async def add_secret(project: str, stack: str, secret_id: str):
    try:
        selected_stack = auto.select_stack(stack_name=stack, project_name=project, program=GcpSecret(secret_id).provision)
        selected_stack.export_stack()
        selected_stack.up(debug=True, on_output=print)
        out = "success"
    except Exception as e:
        out = str(e)
        <http://logging.info|logging.info>(out)

    return UJSONResponse(content={
        "status": out
    })
e

echoing-dinner-19531

08/01/2023, 7:55 AM
I think there's two common choices there. Option 1 is you create new stacks for each request, so stack-1 has placeholder-1 in it, and stack-2 has placeholder-2 in it. Option 2 is you append each request into some state description and get your program to define every resource requested so far based on that state.
p

purple-lion-47621

08/01/2023, 11:14 AM
For the Option 2, are there any already built tools/libraries around ? 🤔
e

echoing-dinner-19531

08/01/2023, 11:15 AM
No, you'd have to build that yourself because it's will be very use case specific
p

purple-lion-47621

08/01/2023, 11:30 AM
The other option we have been considering is the Pulumi code generation through an Automation UI. Users don't directly interact with the GitHub repository but rather they will select what they require from an Automation UI. Automation UI++ will generate the Pulumi code in the repository. GitHub Actions (GHA) takes care of the orchestration. What do you think of that design ? Is that a common design implemented around Pulumi and is there any library/tool support around Pulumi code generation ?
e

echoing-dinner-19531

08/01/2023, 11:39 AM
You could do that. If you were doing code generation I'd probably recommend just using pulumi-yaml and write out yaml files to describe the resources rather than trying to generator python/typescript/etc
h

hallowed-horse-57635

08/01/2023, 1:46 PM
We do generate code from the ui and save and we can initiate an action based on some trigger. If the user 2 was oknto see the user 1s view, you could use the same automation api++ (or qmcloud in our case) and the code will just append the user 2s stuff....
p

purple-lion-47621

08/01/2023, 1:49 PM
@hallowed-horse-57635 do you generate code and push to GitHub ?
Also curious to know what you are generating ? yaml/some-language-code or some-other-format (json) ?
h

hallowed-horse-57635

08/01/2023, 2:04 PM
Yes to github. Its js/ts program with config file in yaml for now.
p

purple-lion-47621

08/01/2023, 2:24 PM
By config file, do you mean Pulumi.yaml ?
h

hallowed-horse-57635

08/01/2023, 2:28 PM
yes
2 Views