pulumi-community #automation-api

automation-api

better-shampoo-48884

03/21/2021, 2:13 PM

(well not really stateless of course, but putting the state in a central repository outside of the app itself.. you know what I mean.. I hope).

better-shampoo-48884

03/21/2021, 2:14 PM

for instance using a mongodb instance to store pulumi.yaml and stacks - would my program need to pull these and push them to disk first before the automation would do its thing?

better-shampoo-48884

03/21/2021, 9:50 PM

I have a feeling I may have missed something or misunderstood some part of the conf here.. Given this:

let currentStack = "dev-devsecops"
    const stack = await pulumi.LocalWorkspace.createOrSelectStack({
        projectName: "k8s-baseline",
        program: pulumiProgram,
        stackName: currentStack
    },{ 
        projectSettings: {
            name: "k8s-baseline",
            author: "It's a me",
            runtime: "nodejs",
            backend: {
                url: "<azblob://pulumi-state>"
            },            
            description: "Baseline Kubernetes configuration for xxxxx"
        },
        secretsProvider: "<azurekeyvault://xxxxxvault.azure.net/keys/pulumi>"
    })

I'm getting:

commandResult: CommandResult {
    stdout: '',
    stderr: 'error: getting secrets manager: passphrase must be set with PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE environment variables\n',
    code: 4294967295,
    err: undefined

There is no Pulumi.yaml or stack yml in the directory where I'm running this btw. Could there be something of the stack previously cached or something? I would have thought providing the secretsProvider explicitly removed the need for the passphrase?

better-shampoo-48884

03/25/2021, 12:52 PM

Does anyone have an example where they have automation reference a stack in a different project? Ideally such that that project's pulumi.yaml and stack config are not accessible to the automation? To put this in context: I have a project called "baseline-infra" which is your traditional pull-request based IaC with automation pipeline. Next, I'm going to have a k8s-components which, ideally, should have stacks which reference their reciprocal stacks.. i.e. Baseline-Infra::dev.infra.project1 is the source stack for k8s-components::dev.k8s.project1 .. if it helps, both projects will keep their state in the same azure blob.

better-shampoo-48884

03/26/2021, 3:36 PM

I'm wondering.. I've defined my "traditional" pulumi project as such:

import * as pulumi from "@pulumi/pulumi";
import * as azure from "@pulumi/azure-native";

export const createFunction = async () => { // edit: exported createFunction in addition to stackInfo based on discussion in thread :)
    ....pulumistuff...
/** edit: Adding more fluff to show how I actually return something from the created variable **/
        const rg = new azure.resources.ResourceGroup("myResourceGroup", {
            location: "location",
            resourceGroupName: "myResourceGroup",
        },{
            protect: false // prevents this resource from being deleted. unprotect with `pulumi state unprotect <resource URN>`
        })
        let created : any = {
            parameters: {
                name:  rg.name.apply(name => name),
                location: rg.location.apply(location => location), 
                id: rg.id.apply(id => id),
                resource: rg
            }
        }
    ....pulumistuff...
    return created   
}
export const stackInfo = createFunction();

would I be able to import createFunction as my pulumiProgram in automation?

better-shampoo-48884

03/26/2021, 3:47 PM

with a slight modification - need to export the createFunction as well 😉

better-shampoo-48884

03/27/2021, 6:39 AM

Yeah.. can't get over this hurdle.. genuinely don't know what I could do any longer to get past it.

λ ts-node automation.ts
CommandError: code: 4294967295
 stdout:
 stderr: error: decrypting secrets: passphrase must be set with PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE environment variables

 err?:

    at Object.createCommandError (c:\<path>\node_modules\@pulumi\pulumi\x\automation\errors.js:71:17)
    at ChildProcess.<anonymous> (c:\<path>\node_modules\@pulumi\pulumi\x\automation\cmd.js:63:40)

See code in post above for source, and in the thread for the automation.ts code as well.. Basically I'm running automation on top of a preexisting standard pulumi-cli set of stacks in a project. Previously, I got the message when I failed to set AZURE_KEYVAULT_AUTH_VIA_CLI to true - but now I've both got it in the environment I'm executing in AND the envVars of the LocalWorkspace settings. Also - the CommandError: code: is a bit weird because there are quite a lot of hits for that exact code.. which is really weird.. and that code returns results relating from electron stuff to .msp upgrades of adobe software to IIS stuff.. The most useful result was the IIS one which has this as the title:

ERROR_FILE_IN_USE msdeploy.exe failed with return code: 4294967295

- could this be a filesystem issue somehow?! Using pulumi

2.23.2

if it helps.

better-shampoo-48884

03/27/2021, 9:48 AM

A side note - which may be related - any way of adding a "suppress-outputs" parameter (equivalent to

--suppress-outputs

for the cli) to automation stack operations? Think that might actually be the cause of some of my issues..

better-shampoo-48884

03/27/2021, 9:52 AM

As it stands I think the error 4294967295 is related to the absolutely massive size of my output 🙂 I'll try to minimize the output a bit in a new run just to see what happens, but yeah..

better-shampoo-48884

03/27/2021, 10:51 AM

Nope - made the output tiny - didn't help.

better-shampoo-48884

03/27/2021, 10:51 AM

stack.info() works fine. stack.preview() does not.

better-shampoo-48884

03/27/2021, 10:54 AM

cli-based preview and up work fine as well.

better-shampoo-48884

03/27/2021, 12:15 PM

finally nearing the end of this saga/debugging nightmare.. thankfully automation IS debuggable 😄 It seems it reacted to a simple condition that

pulumi-cli

did not! WHY there's a difference between the two in this case, I have NO idea, the config is the same regardless. Deep in the code where I create storageAccounts, I had this:

} else if (!saConfiguration.common.hasOwnProperty("subnetReference")) {

Naturally, this is no good when

.common

does not exist on

saConfiguration

, as was the case with this stack. Pulumi-cli doesn't care apparently. However, changing it to this made everything work just fine:

} else if (!saConfiguration.common?.hasOwnProperty("subnetReference")) {

(adding a

to the .common). How the hell this relates to error code 4294967295 is still a mystery though. Anyway.. now I get to go thorugh a few thousand lines of code and try to see if I have any more of these potentially invalid references.. sigh..

better-shampoo-48884

03/27/2021, 4:27 PM

Any idea how I can make sure automation actually loads Pulumi.<stackname>.yaml from disk? Or is the intention of automation to get rid of it? I really thought it had succeded as it didn't prompt me for passphrase for secrets + my secretsprovider is configured on each stack.. but it seems like it's completely ignoring the config otherwise in the running of the program. Edit: stack.getConfig() got me the confirmation I needed. Issue was that the modules for each type of infra I was provisioning had the pulumi.config() and config.requireObject() stuff outside of the exported function. Sigh.

better-shampoo-48884

03/27/2021, 6:00 PM

Yeah. progress is killing me today. Every Single Step Hurts.

Error: The Pulumi runtime detected that 904 promises were still active
at the time that the process exited. There are a few ways that this can occur:
  * Not using `await` or `.then` on a Promise returned from a Pulumi API
  * Introducing a cyclic dependency between two Pulumi Resources
  * A bug in the Pulumi Runtime

This works fine a) with pulumi cli, b) automation locally with

tsc

, c) automation locally with

ts-node

. Yet not in azure pipelines with either

tsc

ts-node

. And yes - if there was an alternative I could use I would.

plain-quill-22197

03/30/2021, 1:11 PM

hello there! just picking up on the Automation API and I have tried a few samples, also have a multi-stack deployment with Node API, all fine

plain-quill-22197

03/30/2021, 1:11 PM

today I am doing a GitLab CI deployment automation, and got this error, which I spent like three hours fighting with no success

plain-quill-22197

03/30/2021, 1:12 PM

error: could not create stack: provided project name "roomote-cal-sync" doesn't match Pulumi.yaml

plain-quill-22197

03/30/2021, 1:12 PM

as I understand, the local workspace has its own project settings and stack settings. how is it possible it could complain about the project name mismatch if I do it all inline? it's literally a copy-paste from a working example

plain-quill-22197

03/30/2021, 1:13 PM

var program   = PulumiFn.Create(() => {
                    var autoDevOps = new DefaultStack();
                    return new Dictionary<string, object> {{"Urn", autoDevOps.Urn}};
                }
            );
            var stackArgs = new InlineProgramArgs(Defaults.ProjectName, stack, program);

            using var appStack = await LocalWorkspace.CreateOrSelectStackAsync(stackArgs);

plain-quill-22197

03/30/2021, 1:13 PM

and it crashes on that last line

plain-quill-22197

03/30/2021, 1:21 PM

allright, what I found is exactly what I thought could be an issue but didn't believe in it

plain-quill-22197

03/30/2021, 1:22 PM

as soon as I changed the project name from

roomote-cal-sync

dummy

- it started to progress further

brave-planet-10645

03/30/2021, 1:23 PM

What happens if you actually hard code the project name in

InlineProgramArgs

to the same as what's in the Pulumi.yaml file?

plain-quill-22197

03/30/2021, 1:24 PM

I hard coded

dummy

as a name. I didn't have the Pulumi.yaml file as none of the automation API samples have it. I also was running samples and it worked. I also tried to add the yaml file with the project name matching, but I don't think the local workspace uses it at all

plain-quill-22197

03/30/2021, 1:25 PM

it looks something like "you cannot use project name with dashes, but we won't tell you this and throw a cryptic error instead"

plain-quill-22197

03/30/2021, 1:26 PM

I know for a fact that names like "aaa-bbb" work but I never tries using two dashes 🙂

brave-planet-10645

03/30/2021, 1:27 PM

of course... sorry you don't need that. I'm just trying something on my machine...

plain-quill-22197

03/30/2021, 1:27 PM

now I am getting

Duplicate resource URN 'urn:pulumi:production::dummy::pulumi:pulumi:Stack::dummy-production'; try giving it a unique name

but at least I moved beyond the project name mismatch issue 🙂

plain-quill-22197

03/30/2021, 1:30 PM

the worst part, of course, that the same stack deploys just fine when I use it without the automation API