Hello! I need some help here with my use case. We...
# pulumi-deployments
f
Hello! I need some help here with my use case. We have written a library that we use internally where we have the Pulumi Automation code, we send some parameters so we can dynamically create resources when calling that library from multiple repositories. This is the code I have in that library in order to create resources:
Copy code
def deploy_code_to_aws(metadata):
    """
    Deploy all the resources to aws using Pulumi.
    """

    def pulumi_program():
        pass
    project_name = metadata.bucket
    stack_name = "Dev"
    stack = auto.create_or_select_stack(
        stack_name=stack_name, project_name=project_name, program=pulumi_program
    )
    print("successfully initialized stack")
    print("installing plugins...")
    stack.workspace.install_plugin("aws", "v4.0.0")
    print("plugins installed")
    print("setting up config")
    stack.set_config("aws:region", auto.ConfigValue(value=metadata.region))
    stack.set_config(
        "aws:profile", auto.ConfigValue(value=metadata.profile_name)
    ) 
    print("config set")
    print("refreshing stack...")
    stack.refresh(on_output=print)
    print("refresh complete")

    print("updating stack...")
    up_res = stack.up(on_output=print)
    print(f"update summary: \n{json.dumps(up_res.summary.resource_changes, indent=4)}")
My question is: How can I update this code so we can start using Remote Deployments? (We already requested access) From this example I know we have the option to call
create_or_select_remote_stack_git_source()
but we don’t have a “git source” we just wanted to grab the pulumi program/stack that we wrote and send that to a Pulumi Deployment remote server so it can process the stack instead of processing everything on our local machines. Is that even possible? Is there another way we could accomplish that with Remote Deployments?
l
Today we only support
git
as a source mechanism. We'd like to support more and expect that in the fullness of time we will also support: • Specifying a YAML program directly in the inline payload • Specifying a pre-signed blob URL, for instance S3 • Maybe others we haven't thought of yet. We don't currently have any plans to try and support inline automation api programs via deployments (it isn't clear that this is possible). Do any of the options above seem interesting? If so would you mind opening an issue at github.com/pulumi/service-requests We're definitely interested in adding this capability if we have users who are ready to consume it! It is true that today we only support the
Another option, which is a bit clunky, is to clone effectively an empty repo and then use pre-run commands to
cat
the program and the stack yaml into the directory. Here is an example of doing that with a yaml program: https://github.com/pulumi/deploy-demos/blob/main/deployment-drivers/nodejs/typescript-driver/index.ts#L140-L176
f
@lemon-agent-27707 thanks! I was thinking this would work like an “API” behind the scenes, and that we could send the parameters in a JSON object so pulumi could process it. Do you recommend a way to accomplish something similar with existing Pulumi’s features? We were also thinking about putting our pulumi auto api code in a lambda function so we can send that JSON to it and deploy our resources. (Do you see any potential issues with this approach?)
r
We were also thinking about putting our pulumi auto api code in a lambda function so we can send that JSON to it and deploy our resources. (Do you see any potential issues with this approach?)
The thing to keep in mind here is that the Pulumi CLI needs to be present for automation api, so you'll need to use a custom image as the base of your lambda. Otherwise I don't foresee any issues.
l
@fresh-minister-66960 You could use the deployments REST API directly. Here is an example of running a refresh in a lambda: https://github.com/pulumi/deploy-demos/blob/main/pulumi-programs/drift-detection/index.ts#L24-L75 Here's a simpler example that runs an update: https://github.com/pulumi/deploy-demos/blob/main/deployment-drivers/nodejs/typescript-driver/index.ts#L116-L135
Do you see any potential issues with this approach?
The primary issue with wrapping normal Automation API in a lambda vs using Pulumi Deployments (whether the REST API or Remote workspaces) is that many pulumi programs take multiple minutes to run. Longer than is reasonable for a synchronous HTTP request. That is part of the benefit of deployments. You could create a lambda that does the following: 1. Receives an HTTP request to deploy some resources 2. Translate that into a call to the Deployments API 3. The deployments API will immediately return a URL to track the deployment to completion (app.pulumi.com/{org}/{project}/{stack}/deployments/{deploymentNumber} 4. You can return that URL immediately to users to they can monitor progress of the deployment to completion.