Hi Experts !! 👋 We are trying to evaluate whether we can use Pulumi Automation API to orchestrate infrastructure creation through a REST API. • We will host Pulumi state on our own cloud storage bucket. • We write a python based middle-ware service which integrates with Pulumi automation API. • This middle-ware service will eventually work with a fronted. • Our python middle-ware service will expose endpoints to create resources (e.g.: an endpoint to create a given Azure key vault). ◦ e.g.:

User1

sends in a request to create

vault1

.

User2

sends in a request to create

vault2

. Is this architecture supported by Pulumi automation API ? Or does Pulumi strictly requires the entire state to be defined in the source of truth (100% IaaC) ?

I know lots of users do expose automation api via rest apis. So I'd think this should be fine. Your last comment is confusing me a little, because yes Pulumi does strictly require the entire state to be defined, how else would it work, but you can do that with automation api just fine.

We have used a similar architecture four our commercial software with a UI and No Code technologies ( drag and drop, templates, smart wizrad). This uses the automation api but we do use pulumi cloud for state management and other advanced rbac capabilities. <advt> qmcloud.io

Thanks for the responses @echoing-dinner-19531 and @hallowed-horse-57635 ! How do you handle the below scenario for example ? Let's say we have the below generic endpoint to create a secret placeholder in GCP. •

User1

sends a request to create the secret

placeholder-1

(fine it will be created). •

User2

sends a request to create secret

placeholder-2

. New

placeholder-2

will be created but it will destroy

placeholder-1

, correct ? How do to handle this scenario ? Do you have your own layer implemented to reflect the complete state ? Or instead of generic endpoints do you define everything explicitly ?

@app.get('/gcp/{project}/{stack}/secret/add/{secret_id}')
async def add_secret(project: str, stack: str, secret_id: str):
    try:
        selected_stack = auto.select_stack(stack_name=stack, project_name=project, program=GcpSecret(secret_id).provision)
        selected_stack.export_stack()
        selected_stack.up(debug=True, on_output=print)
        out = "success"
    except Exception as e:
        out = str(e)
        <http://logging.info|logging.info>(out)

    return UJSONResponse(content={
        "status": out
    })

I think there's two common choices there. Option 1 is you create new stacks for each request, so stack-1 has placeholder-1 in it, and stack-2 has placeholder-2 in it. Option 2 is you append each request into some state description and get your program to define every resource requested so far based on that state.

For the Option 2, are there any already built tools/libraries around ? 🤔

No, you'd have to build that yourself because it's will be very use case specific

The other option we have been considering is the Pulumi code generation through an Automation UI. Users don't directly interact with the GitHub repository but rather they will select what they require from an Automation UI. Automation UI++ will generate the Pulumi code in the repository. GitHub Actions (GHA) takes care of the orchestration. What do you think of that design ? Is that a common design implemented around Pulumi and is there any library/tool support around Pulumi code generation ?

You could do that. If you were doing code generation I'd probably recommend just using pulumi-yaml and write out yaml files to describe the resources rather than trying to generator python/typescript/etc

We do generate code from the ui and save and we can initiate an action based on some trigger. If the user 2 was oknto see the user 1s view, you could use the same automation api++ (or qmcloud in our case) and the code will just append the user 2s stuff....

@hallowed-horse-57635 do you generate code and push to GitHub ?

Yes to github. Its js/ts program with config file in yaml for now.

By config file, do you mean Pulumi.yaml ?

yes