https://pulumi.com logo
Join the conversationJoin Slack
Channels
announcements
automation-api
aws
azure
blog-posts
built-with-pulumi
cloudengineering
cloudengineering-support
content-share
contribex
contribute
docs
dotnet
finops
general
getting-started
gitlab
golang
google-cloud
hackathon-03-19-2020
hacktoberfest
install
java
jobs
kubernetes
learn-pulumi-events
linen
localstack
multi-language-hackathon
office-hours
oracle-cloud-infrastructure
plugin-framework
pulumi-cdk
pulumi-crosscode
pulumi-deployments
pulumi-kubernetes-operator
pulumi-service
pulumiverse
python
registry
status
testingtesting123
testingtesting321
typescript
welcome
workshops
yaml
Powered by Linen
automation-api
  • p

    plain-quill-22197

    03/30/2021, 4:34 PM
    the deployment automation application is started from the repo root
    • 1
    • 1
  • l

    lemon-agent-27707

    03/30/2021, 4:38 PM
    Just as a heads up. If you are trying to use inline programs with a local
    pulumi.yaml
    that exists on disk there is currently a bug. Basically, if you don't specify a
    projectSettings
    then we will overwrite anything in the workdir with a default project (just for inline programs) https://github.com/pulumi/pulumi/issues/6632
    b
    r
    • 3
    • 4
  • p

    prehistoric-nail-50687

    03/31/2021, 8:42 AM
    The automation API also seems to support
    importStack
    , same as the CLI - does the automation API also generate the code for the imported resource, as the CLI does?
    b
    l
    d
    • 4
    • 11
  • p

    plain-quill-22197

    03/31/2021, 9:20 AM
    It doesn't seem that the automation API supports setting stack tags?
    r
    • 2
    • 2
  • l

    limited-rainbow-51650

    04/01/2021, 7:30 PM
    Hello, I’m using the Automation API to implement an integration for Pulumi with Concourse CI: https://github.com/ringods/pulumi-resource/ Any resource (Concourse term) you use in a build pipeline is configured in YAML. You have the identification of the resource and some parameters. For example, to apply a Pulumi stack, this is the snippet I intend to use:
    resource_types:
    - name: pulumi
      type: docker-image
      source:
        repository: <http://ghcr.io/ringods/pulumi-resource|ghcr.io/ringods/pulumi-resource>
        tag: v0.0.6
    
    resources:
      - name: myinfracode
        type: pulumi
        source:
          organization: companyname
          project: network
          stack: staging
          token: pul-XXXXXXXXXXXXXXXXX
    
    jobs:
      - name: nextstack
        plan:
          - task: build-artifact
            ...
          - put: myinfracode
            params:
              aws:region: eu-west-1
              network:cidr: 'x.x.x.x'
    In my Concourse resource, I want to allow anything in the
    params
    section above which is valid Pulumi stack config (nested et all). In my resource implementation, I get this passed as JSON though:
    {
      "source": {
        "organization": "company",
        "project": "network",
        "stack": "staging",
        "token": "pul-XXXXXXXXXXXXXXXXX"
      },
      "params": {
        "aws:region": "eu-west-1",
        "network:cidr": "x.x.x.x"
      }
    }
    The
    source
    section is specific to Pulumi, but the
    params
    is the JSON equivalent of what I configured as YAML in my pipeline. Is there any function in the Pulumi code base that can Unmarshal this to a Pulumi type representing this Config?
    p
    • 2
    • 1
  • f

    fresh-summer-65887

    04/05/2021, 8:54 PM
    So I've been exploring using the automation API with some automated integration tests with dynamodb-local (localstack in future). - For isolation reasons each test uses it's own stack and thus infra resources. This facilitates running tests in parallel. - Using a single shared LocalWorkspace across tests. - Using local backend (setting backend url to
    file://~
    ) - A stack with a single resource, a simple dynamodb table. - So everything is local and should not be making any external network calls. Non-scientific performance numbers: -
    WorkspaceStack.CreateOrSelectAsync
    ~ 1.4s -
    WorkspaceStack.SetConfigAsync
    ~ 0.7s -
    WorkspaceStack.UpAsync
    ~ 6.5s Overall each test has ~9s overhead. Not bad to be honest but it'll add up as the test count increases. If I use the AWSSDK directly to deploy a table it is significantly faster (<1s). However with this approach I'd have to maintain some sort of table definition abstraction that would be used both in the test fixtures and pulumi component to achieve alignment / re-use. This would add a complexity cost. Thought I'd mention the above in the hope that there could be some performance improvements made to the Automation API. In any case, it's really nice stuff :)
    🍺 1
    🙌🏽 1
    🙌 4
    ☝️ 1
    l
    r
    +2
    • 5
    • 5
  • b

    better-shampoo-48884

    04/07/2021, 6:47 AM
    So.. using automation with kuberenetes, I'm getting tons of "unhandled rejections" every time it fails.. should I be wrapping my pulumi program resource operations in try/catch?
    🙌 1
    w
    l
    • 3
    • 4
  • b

    better-shampoo-48884

    04/09/2021, 10:47 AM
    I'm starting to become slightly disillusioned with my automation adventures.. 😕 I'm finding it really hard to navigate these promise explosion issues, as well as almost any other error in my pulumiProgram through the automation. Noticing some extreme resource utilization as well - my automation is basically just throwing some helm charts into a kubernetes cluster, and node is consuming about 2gb of ram. No concrete request for help here - just a slight venting of frustration 😞 I really -really- love automation, and I feel that's one of the major differentiators with Pulumi versus anything else out there, just so frustrating to try to get it to work in a stable way.
    🙌 1
    b
    • 2
    • 2
  • l

    limited-rainbow-51650

    04/09/2021, 11:17 AM
    Is this the right approach to extend the
    PATH
    used by pulumi to find the language runtime binaries (e.g.
    node
    ), given I mount a NodeJS docker image as volume
    /volume
    ?
    stack.Workspace().SetEnvVar("PATH", os.Getenv("PATH")+":/volume/bin:/volume/usr/bin:/volume/usr/local/bin")
  • p

    proud-pizza-80589

    04/11/2021, 9:07 PM
    Anyone seen this before? Using the automation API in node.
    +  kubernetes:core/v1:Namespace aaa-7f creating error: configured Kubernetes cluster is unreachable: unable to load schema information from the API server: Get "...": getting credentials: exec: fork/exec /usr/local/bin/aws: no such file or directory
    Pulumi running in a docker container. First i thought it was because I was using alpine, but in a debian image it does the same thing. Manually running the aws cli works fine
    root@alpha-launchpad-5d89b45bdc-nz5pp:/usr/src/app# aws --version
    aws-cli/2.1.36 Python/3.8.8 Linux/5.4.89+ exe/x86_64.debian.9 prompt/off
    root@alpha-launchpad-5d89b45bdc-nz5pp:/usr/src/app# /usr/local/bin/aws --version
    aws-cli/2.1.36 Python/3.8.8 Linux/5.4.89+ exe/x86_64.debian.9 prompt/off
    I’ve seen similar things happen before when shelling out from node for other apps if I did not force the shelled out command to use bash as a shell.
    b
    • 2
    • 16
  • c

    clever-cartoon-41433

    04/19/2021, 6:46 PM
    We have a game-breaking bug with pulumi automation where every once in a while the apply will hang with a few "unhandled rejection" errors. Normally we would just retry in these cases but the hanging makes it completely block further use of our application. Not quite sure if this is my issue or yours.
    r
    l
    • 3
    • 62
  • l

    limited-rainbow-51650

    04/21/2021, 5:05 PM
    How should I configure the Pulumi platform access token using the Automation API? What I have now:
    stack.Workspace().SetEnvVar("PULUMI_ACCESS_TOKEN", req.Source.Token)
    But I still get this error:
    error: PULUMI_ACCESS_TOKEN must be set for login during non-interactive CLI sessions
    b
    b
    r
    • 4
    • 15
  • b

    busy-soccer-65968

    04/21/2021, 10:45 PM
    this might be a better place for this https://pulumi-community.slack.com/archives/CCWP5TJ5U/p1619044534108300
    b
    l
    r
    • 4
    • 56
  • a

    average-businessperson-33238

    04/23/2021, 12:52 PM
    So, got the automation API running in production (on K8s)
    😛artypus-8bit: 5
    🙌 1
    🙌🏽 1
    • 1
    • 2
  • d

    dazzling-scientist-80826

    04/26/2021, 12:05 AM
    I'm trying out the Automation API for the first time and I'm a little confused about workspace implementations. I see how
    LocalWorkspace
    works, but it's not quite clear how one would go about implementing a custom workspace. In particular, it looks like there is some machinery around a "language runtime service" and other things that is necessary to make refresh/up implementations. Is there a guide or example for how to do this?
    b
    r
    +4
    • 7
    • 56
  • l

    little-orange-65618

    04/26/2021, 9:09 PM
    Two questions: 1. How do folks test creating AKV's for encrypting resources (which requires soft_deletion, and purge detection), I'm trying to test resource creation based on API input, so I've been deleting the resources post test run (but then the protection kicks in since I am not randomly generating names) 2. In a similar vein I'm attempting to (the problem is E, where it tries to create a new SA rather than update the existing): a. Create Storage Account b. Get SA identity c. Create AKV, with policy to allow SA identity to "get, wrapkey, unwrapkey" d. Add key e. Update SA with encryption specifying new AKV and key
    **creating failed** error: Duplicate resource URN
    Thoughts? (using azure-native 1.0.1 for reference) The equivalent AZ CLI command for E would be:
    az storage account update
        --name <storage-account> \
        --resource-group <resource_group> \
        --encryption-key-name <key> \
        --encryption-key-source Microsoft.Keyvault \
        --encryption-key-vault $key_vault_uri
  • h

    handsome-state-59775

    05/02/2021, 8:27 AM
    I'm trying to create an CI/CD version of a currently CLI-driven Python program, but would like to retain CLI debuggability. 1. From the Go example for an inline/local hybrid, what is the Python equivalent of 
    pulumi.Run()
     ? 2. Can I overlay configuration and secrets on top of a default stack's imported configuration? This would let me check in common secrets/config for a project, and have the CI/CD set env vars to provide optional overrides.
    l
    • 2
    • 3
  • w

    wooden-animal-34285

    05/03/2021, 5:46 PM
    Hi, I'm trying out the Automation API and I am stuck as
    createOrSelectStack()
    blocks indefinitly. It does not timeout or writes any errors to the console.
    export async function createInstance(): Promise<UpResult> {
        const program = programs.getProgram()
    
        // Create our stack
        const args: InlineProgramArgs = {
            stackName: "dev",
            projectName: "inlineNode",
            program,
        };
    
        // create (or select if one already exists) a stack that uses our local program
        <http://console.info|console.info>("initializing stack...");
        const stack = await pulumiAutoAPI.LocalWorkspace.createOrSelectStack(args, {
    
        });
        <http://console.info|console.info>("successfully initialized stack");
    
        <http://console.info|console.info>("setting up config");
        await stack.setConfig("aws:region", { value: "us-east-2" });
        <http://console.info|console.info>("config set");
    
        // is this step necessary because of the config step?
        // possible to setup config as part of create stack?
        <http://console.info|console.info>("refreshing stack...");
        await stack.refresh({ onOutput: <http://console.info|console.info> });
        <http://console.info|console.info>("refresh complete");
    
        <http://console.info|console.info>("updating stack...");
        const upRes = await stack.up({ onOutput: <http://console.info|console.info> });
        <http://console.info|console.info>(`update summary object: ${JSON.stringify(upRes.summary)}`);
        console.log(`update summary: \n${JSON.stringify(upRes.summary.resourceChanges, null, 4)}`);
        console.log(`outputs: ${JSON.stringify(upRes.outputs)}`);
    
        return upRes
    }
    I couldn't find any reference for such behaviour. Has anybody an idea what the cause could be? Thanks! System:
    NodeJS: "14.16.1"
    Pulumi: "3.1.0"
    r
    l
    • 3
    • 9
  • t

    tall-scientist-89115

    05/03/2021, 11:50 PM
    Hey folks, hitting a weird one with a stack that was working fine and seemingly randomly broke (I was messing around with
    pulumi logs -f -s
    simultaneously maybe?)
    FEATURE=staging PROJECT_NAME=qa npm run create-db-seed destroy
    
    > qa@ create-db-seed /Users/daniel/code/RedJade/infra/scripts
    > tsc && node ./bin/scripts/create-db-seed.js "destroy"
    
    project name qa
    feature name staging
    (node:74675) ExperimentalWarning: Conditional exports is an experimental feature. This feature could change at any time
    CommandError: code: 255
     stdout: 
     stderr: error: invalid character 'y' after top-level value
    
     err?: 
    
        at Object.createCommandError (/Users/daniel/code/RedJade/infra/scripts/node_modules/@pulumi/pulumi/automation/errors.js:71:17)
        at ChildProcess.<anonymous> (/Users/daniel/code/RedJade/infra/scripts/node_modules/@pulumi/pulumi/automation/cmd.js:63:40)
        at ChildProcess.emit (events.js:321:20)
        at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12) {
      commandResult: CommandResult {
        stdout: '',
        stderr: "error: invalid character 'y' after top-level value\n",
        code: 255,
        err: undefined
      },
      name: 'CommandError'
    }
    Same thing happened to me yesterday but I was able to get around it by moving my automation code out of a folder shared with another stack, so was able to hand wave it away -- but then it popped up again today. It was working, working, then not working. Has anyone seen this before?
    l
    b
    r
    • 4
    • 47
  • l

    little-orange-65618

    05/05/2021, 12:42 AM
    If you are using the default not configured local workspace and inline programs, where is the "state" (aka list of stacks) stored. (Curious how to make something truly stateless to run on kubernetes for example)
    auto.LocalWorkspace()
    r
    l
    +2
    • 5
    • 19
  • l

    little-orange-65618

    05/05/2021, 8:53 AM
    Interesting, I'd been passing along subscription variables for creating resources in multiple subscriptions but it appears like azure-native.resources.ResourceGroup doesn't support a subscription input so everything ended up in what I assume is set via my az cli (ARM_SUBSCRIPTION_ID). Is there a way to handle that elegantly (One API call -> One Pulumi Inline Program Run -> Resources in 2+ Subscriptions)? Can I swap the pulumi config back and forth every-time I go to create a resource?
    l
    t
    • 3
    • 9
  • h

    handsome-state-59775

    05/06/2021, 6:25 AM
    is it safe to operate on a different stack than the one currently selected?
    stack_1 = auto.select_stack(...)
    stack_2 = auto.create_or_select_stack(...)  # currently selected stack
    stack_1.refresh_config()  # ok to do this now?
    stack_1_config = stack1.get_all_config()  # ok to do this now?
    ✅ 2
    l
    b
    r
    • 4
    • 7
  • c

    colossal-plastic-46140

    05/07/2021, 12:23 AM
    Hi Folks, I am trying create a simple program in python, that previews all the stacks in a given project, but I am running into trouble with the secretsprovider bits, any advice would be appreciated.
    r
    • 2
    • 6
  • l

    limited-rain-96205

    05/07/2021, 1:16 AM
    Is it possible to use gcpkms as the secrets provider with the Automation API? I've got my project configured to use it (via the secretsProvider field in LocalWorkspaceOptions), but attempting to update a secret results in the following error:
    stderr: error: passphrase must be set with PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE environment variables
    This error doesn't make sense because those environment variables are not required for Stacks that use the gcpkms secrets provider.
    r
    • 2
    • 5
  • c

    colossal-plastic-46140

    05/07/2021, 1:08 PM
    In the Go API, is there a decent way to get a jsonfied response of a preview, ie the equivalent of pulumi preview --json?
    l
    • 2
    • 1
  • e

    echoing-angle-67526

    05/08/2021, 1:55 PM
    According to the GitHub page the kubernetes operator is marked as experimental. Do we have an idea when it will become GA?
  • l

    lemon-agent-27707

    05/09/2021, 4:47 PM
    🤖 Pulumi as a nodejs REPL via automation api: https://twitter.com/_Evan_Boyle/status/1391431732084166660
    🙌 1
    😛artypus-8bit: 1
    r
    • 2
    • 1
  • m

    miniature-leather-70472

    05/11/2021, 2:02 PM
    Possibly dumb question, but does the Automation API .net version use Kestrel when running? Seeing a weird error when getting to the refresh stage where it crashes stating that kestrel cannot bind to the porta as it is already in use, which it is, by the web app I'm trying to run Pulumi from
    p
    b
    • 3
    • 7
  • l

    limited-rain-96205

    05/12/2021, 3:21 AM
    Is it normal to get a bunch of "unhandled rejection" errors when the inline program doesn't return all of the resources created? In other words, it is a requirement to output all resources?
    l
    b
    • 3
    • 12
  • b

    bored-oyster-3147

    05/12/2021, 7:11 PM
    Hey everyone - finally got to the point internally that we are now testing the Automation API portion of our in-house CD API. Having an issue that I thought had already been addressed but may be misremembering. Consistently getting the
    error: PULUMI_ACCESS_TOKEN must be set for login during non-interactive CLI sessions
    message. Trying to use a self-managed backend so I don't think I should be hitting this, or may be missing something. I currently have the backend set both in the project settings and in
    PULUM_BACKEND_URL
    f
    l
    b
    • 4
    • 7
Powered by Linen
Title
b

bored-oyster-3147

05/12/2021, 7:11 PM
Hey everyone - finally got to the point internally that we are now testing the Automation API portion of our in-house CD API. Having an issue that I thought had already been addressed but may be misremembering. Consistently getting the
error: PULUMI_ACCESS_TOKEN must be set for login during non-interactive CLI sessions
message. Trying to use a self-managed backend so I don't think I should be hitting this, or may be missing something. I currently have the backend set both in the project settings and in
PULUM_BACKEND_URL
I see a thread here https://pulumi-community.slack.com/archives/C019YSXN04B/p1619024754146000 about it but it looks like Ringo was using the pulumi service backend so he was able to just provide an access token
Any insight on this that I'm missing?
f

faint-table-42725

05/13/2021, 1:28 PM
I don’t think so? I haven’t tried this in .NET, but in NodeJS, for example, I had no issues setting the backend via project settings (and don’t do anything with the env var)
l

lemon-agent-27707

05/13/2021, 3:10 PM
Could it be that your backend URL is malformed? That could force the CLI to try and log you in to the service: https://github.com/pulumi/pulumi/blob/73a66f48ead301bacf1d7d2833c016cf803b57c1/pkg/cmd/pulumi/login.go#L128
the check https://github.com/pulumi/pulumi/blob/73a66f48ead301bacf1d7d2833c016cf803b57c1/pkg/backend/filestate/backend.go#L94
b

bored-oyster-3147

05/13/2021, 7:25 PM
I believe that was the issue. I was using local filestate for testing purposes and my path was
file://{path}
but when you use
Path.Combine(...)
in dotnet it uses back slashes instead of forward slashes, so pulumi wasn't liking that. Thanks guys!
😛artypus-8bit: 1
b

bored-activity-40468

05/15/2021, 12:03 AM
@faint-table-42725 I've verified that it works in .NET.
🙏 1
👍 1
View count: 4