https://pulumi.com logo
Join the conversationJoin Slack
Channels
announcements
automation-api
aws
azure
blog-posts
built-with-pulumi
cloudengineering
cloudengineering-support
content-share
contribex
contribute
docs
dotnet
finops
general
getting-started
gitlab
golang
google-cloud
hackathon-03-19-2020
hacktoberfest
install
java
jobs
kubernetes
learn-pulumi-events
linen
localstack
multi-language-hackathon
office-hours
oracle-cloud-infrastructure
plugin-framework
pulumi-cdk
pulumi-crosscode
pulumi-deployments
pulumi-kubernetes-operator
pulumi-service
pulumiverse
python
registry
status
testingtesting123
testingtesting321
typescript
welcome
workshops
yaml
Powered by Linen
automation-api
  • b

    better-shampoo-48884

    03/21/2021, 2:13 PM
    (well not really stateless of course, but putting the state in a central repository outside of the app itself.. you know what I mean.. I hope).
  • b

    better-shampoo-48884

    03/21/2021, 2:14 PM
    for instance using a mongodb instance to store pulumi.yaml and stacks - would my program need to pull these and push them to disk first before the automation would do its thing?
    b
    • 2
    • 1
  • b

    better-shampoo-48884

    03/21/2021, 9:50 PM
    I have a feeling I may have missed something or misunderstood some part of the conf here.. Given this:
    let currentStack = "dev-devsecops"
        const stack = await pulumi.LocalWorkspace.createOrSelectStack({
            projectName: "k8s-baseline",
            program: pulumiProgram,
            stackName: currentStack
        },{ 
            projectSettings: {
                name: "k8s-baseline",
                author: "It's a me",
                runtime: "nodejs",
                backend: {
                    url: "<azblob://pulumi-state>"
                },            
                description: "Baseline Kubernetes configuration for xxxxx"
            },
            secretsProvider: "<azurekeyvault://xxxxxvault.azure.net/keys/pulumi>"
        })
    I'm getting:
    commandResult: CommandResult {
        stdout: '',
        stderr: 'error: getting secrets manager: passphrase must be set with PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE environment variables\n',
        code: 4294967295,
        err: undefined
    There is no Pulumi.yaml or stack yml in the directory where I'm running this btw. Could there be something of the stack previously cached or something? I would have thought providing the secretsProvider explicitly removed the need for the passphrase?
    b
    l
    • 3
    • 7
  • b

    better-shampoo-48884

    03/25/2021, 12:52 PM
    Does anyone have an example where they have automation reference a stack in a different project? Ideally such that that project's pulumi.yaml and stack config are not accessible to the automation? To put this in context: I have a project called "baseline-infra" which is your traditional pull-request based IaC with automation pipeline. Next, I'm going to have a k8s-components which, ideally, should have stacks which reference their reciprocal stacks.. i.e. Baseline-Infra::dev.infra.project1 is the source stack for k8s-components::dev.k8s.project1 .. if it helps, both projects will keep their state in the same azure blob.
    l
    g
    b
    • 4
    • 16
  • b

    better-shampoo-48884

    03/26/2021, 3:36 PM
    I'm wondering.. I've defined my "traditional" pulumi project as such:
    import * as pulumi from "@pulumi/pulumi";
    import * as azure from "@pulumi/azure-native";
    
    export const createFunction = async () => { // edit: exported createFunction in addition to stackInfo based on discussion in thread :)
        ....pulumistuff...
    /** edit: Adding more fluff to show how I actually return something from the created variable **/
            const rg = new azure.resources.ResourceGroup("myResourceGroup", {
                location: "location",
                resourceGroupName: "myResourceGroup",
            },{
                protect: false // prevents this resource from being deleted. unprotect with `pulumi state unprotect <resource URN>`
            })
            let created : any = {
                parameters: {
                    name:  rg.name.apply(name => name),
                    location: rg.location.apply(location => location), 
                    id: rg.id.apply(id => id),
                    resource: rg
                }
            }
        ....pulumistuff...
        return created   
    }
    export const stackInfo = createFunction();
    would I be able to import createFunction as my pulumiProgram in automation?
    b
    l
    • 3
    • 17
  • b

    better-shampoo-48884

    03/26/2021, 3:47 PM
    with a slight modification - need to export the createFunction as well šŸ˜‰
  • b

    better-shampoo-48884

    03/27/2021, 6:39 AM
    Yeah.. can't get over this hurdle.. genuinely don't know what I could do any longer to get past it.
    Ī» ts-node automation.ts
    CommandError: code: 4294967295
     stdout:
     stderr: error: decrypting secrets: passphrase must be set with PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE environment variables
    
     err?:
    
        at Object.createCommandError (c:\<path>\node_modules\@pulumi\pulumi\x\automation\errors.js:71:17)
        at ChildProcess.<anonymous> (c:\<path>\node_modules\@pulumi\pulumi\x\automation\cmd.js:63:40)
    See code in post above for source, and in the thread for the automation.ts code as well.. Basically I'm running automation on top of a preexisting standard pulumi-cli set of stacks in a project. Previously, I got the message when I failed to set AZURE_KEYVAULT_AUTH_VIA_CLI to true - but now I've both got it in the environment I'm executing in AND the envVars of the LocalWorkspace settings. Also - the CommandError: code: is a bit weird because there are quite a lot of hits for that exact code.. which is really weird.. and that code returns results relating from electron stuff to .msp upgrades of adobe software to IIS stuff.. The most useful result was the IIS one which has this as the title:
    ERROR_FILE_IN_USE msdeploy.exe failed with return code: 4294967295
    - could this be a filesystem issue somehow?! Using pulumi
    2.23.2
    if it helps.
    l
    • 2
    • 8
  • b

    better-shampoo-48884

    03/27/2021, 9:48 AM
    A side note - which may be related - any way of adding a "suppress-outputs" parameter (equivalent to
    --suppress-outputs
    for the cli) to automation stack operations? Think that might actually be the cause of some of my issues..
  • b

    better-shampoo-48884

    03/27/2021, 9:52 AM
    As it stands I think the error 4294967295 is related to the absolutely massive size of my output šŸ™‚ I'll try to minimize the output a bit in a new run just to see what happens, but yeah..
    h
    • 2
    • 2
  • b

    better-shampoo-48884

    03/27/2021, 10:51 AM
    Nope - made the output tiny - didn't help.
  • b

    better-shampoo-48884

    03/27/2021, 10:51 AM
    stack.info() works fine. stack.preview() does not.
  • b

    better-shampoo-48884

    03/27/2021, 10:54 AM
    cli-based preview and up work fine as well.
  • b

    better-shampoo-48884

    03/27/2021, 12:15 PM
    finally nearing the end of this saga/debugging nightmare.. thankfully automation IS debuggable šŸ˜„ It seems it reacted to a simple condition that
    pulumi-cli
    did not! WHY there's a difference between the two in this case, I have NO idea, the config is the same regardless. Deep in the code where I create storageAccounts, I had this:
    } else if (!saConfiguration.common.hasOwnProperty("subnetReference")) {
    Naturally, this is no good when
    .common
    does not exist on
    saConfiguration
    , as was the case with this stack. Pulumi-cli doesn't care apparently. However, changing it to this made everything work just fine:
    } else if (!saConfiguration.common?.hasOwnProperty("subnetReference")) {
    (adding a
    ?
    to the .common). How the hell this relates to error code 4294967295 is still a mystery though. Anyway.. now I get to go thorugh a few thousand lines of code and try to see if I have any more of these potentially invalid references.. sigh..
    l
    • 2
    • 10
  • b

    better-shampoo-48884

    03/27/2021, 4:27 PM
    Any idea how I can make sure automation actually loads Pulumi.<stackname>.yaml from disk? Or is the intention of automation to get rid of it? I really thought it had succeded as it didn't prompt me for passphrase for secrets + my secretsprovider is configured on each stack.. but it seems like it's completely ignoring the config otherwise in the running of the program. Edit: stack.getConfig() got me the confirmation I needed. Issue was that the modules for each type of infra I was provisioning had the pulumi.config() and config.requireObject() stuff outside of the exported function. Sigh.
    l
    • 2
    • 1
  • b

    better-shampoo-48884

    03/27/2021, 6:00 PM
    Yeah. progress is killing me today. Every Single Step Hurts.
    Error: The Pulumi runtime detected that 904 promises were still active
    at the time that the process exited. There are a few ways that this can occur:
      * Not using `await` or `.then` on a Promise returned from a Pulumi API
      * Introducing a cyclic dependency between two Pulumi Resources
      * A bug in the Pulumi Runtime
    This works fine a) with pulumi cli, b) automation locally with
    tsc
    , c) automation locally with
    ts-node
    . Yet not in azure pipelines with either
    tsc
    or
    ts-node
    . And yes - if there was an alternative I could use I would.
    b
    • 2
    • 5
  • p

    plain-quill-22197

    03/30/2021, 1:11 PM
    hello there! just picking up on the Automation API and I have tried a few samples, also have a multi-stack deployment with Node API, all fine
  • p

    plain-quill-22197

    03/30/2021, 1:11 PM
    today I am doing a GitLab CI deployment automation, and got this error, which I spent like three hours fighting with no success
  • p

    plain-quill-22197

    03/30/2021, 1:12 PM
    error: could not create stack: provided project name "roomote-cal-sync" doesn't match Pulumi.yaml
  • p

    plain-quill-22197

    03/30/2021, 1:12 PM
    as I understand, the local workspace has its own project settings and stack settings. how is it possible it could complain about the project name mismatch if I do it all inline? it's literally a copy-paste from a working example
  • p

    plain-quill-22197

    03/30/2021, 1:13 PM
    var program   = PulumiFn.Create(() => {
                        var autoDevOps = new DefaultStack();
                        return new Dictionary<string, object> {{"Urn", autoDevOps.Urn}};
                    }
                );
                var stackArgs = new InlineProgramArgs(Defaults.ProjectName, stack, program);
    
                using var appStack = await LocalWorkspace.CreateOrSelectStackAsync(stackArgs);
  • p

    plain-quill-22197

    03/30/2021, 1:13 PM
    and it crashes on that last line
  • p

    plain-quill-22197

    03/30/2021, 1:21 PM
    allright, what I found is exactly what I thought could be an issue but didn't believe in it
  • p

    plain-quill-22197

    03/30/2021, 1:22 PM
    as soon as I changed the project name from
    roomote-cal-sync
    to
    dummy
    - it started to progress further
  • b

    brave-planet-10645

    03/30/2021, 1:23 PM
    What happens if you actually hard code the project name in
    InlineProgramArgs
    to the same as what's in the Pulumi.yaml file?
  • p

    plain-quill-22197

    03/30/2021, 1:24 PM
    I hard coded
    dummy
    as a name. I didn't have the Pulumi.yaml file as none of the automation API samples have it. I also was running samples and it worked. I also tried to add the yaml file with the project name matching, but I don't think the local workspace uses it at all
    b
    • 2
    • 1
  • p

    plain-quill-22197

    03/30/2021, 1:25 PM
    it looks something like "you cannot use project name with dashes, but we won't tell you this and throw a cryptic error instead"
  • p

    plain-quill-22197

    03/30/2021, 1:26 PM
    I know for a fact that names like "aaa-bbb" work but I never tries using two dashes šŸ™‚
  • b

    brave-planet-10645

    03/30/2021, 1:27 PM
    of course... sorry you don't need that. I'm just trying something on my machine...
  • p

    plain-quill-22197

    03/30/2021, 1:27 PM
    now I am getting
    Duplicate resource URN 'urn:pulumi:production::dummy::pulumi:pulumi:Stack::dummy-production'; try giving it a unique name
    but at least I moved beyond the project name mismatch issue šŸ™‚
  • p

    plain-quill-22197

    03/30/2021, 1:30 PM
    the worst part, of course, that the same stack deploys just fine when I use it without the automation API
Powered by Linen
Title
p

plain-quill-22197

03/30/2021, 1:30 PM
the worst part, of course, that the same stack deploys just fine when I use it without the automation API
View count: 1