Hello, I'm just wondering if anyone has faced similar issue. I store 500 secrets in ESC. When I use that environment, in IAC stack, it takes ages until preview or up command shows any kind of output

Hi @bitter-scooter-78136 👋 let me try to rule this out, do you also experiment a similar delay when running

pulumi env open ...

or any other context?

env open is fast, it's only when used in iac stack. https://github.com/pulumi/pulumi/issues/20338 It sends separate api call for each secret, which takes 300-500ms 😞

Is your ESC environment using any dynamic secret providers? I'm not able to reproduce this when importing an ESC environment in an iac stack

No • I have Pulumi stack with 500 secrets

config:
  env:secrets:
    LICENSE_KEY:
      secure: <ENCRYPTED_KEY>
    ....

# index.ts
const secrets = config.getObject<{ [key: string]: string }>("secrets") || {};
const secretEntries = Object.entries(secretsObj).map(([key, value]) => {code which creates yaml file and saves it esc environment}

This takes really long to get running. I was hoping that when everything is in esc other stack will load fast enough

#Pulumi.production.yaml
config:
...
environment:
  - org/production

But other stack then is taking same time to load and decrypts secrets one by one

I'm not able to reproduce with an essentially blank iac stack importing an ESC environment with >500 secrets. All it is doing is an Open Environment call and it shouldn't be decrypting one by one. Is your stack doing something else? If you have a minimal reproduction example, that would be super helpful. Thanks!

1. pulumi new 2. for i in {1..500}; do pulumi config set --path "application.$i" "$i" --secret; done index.ts

import * as pulumi from "@pulumi/pulumi";

export const config = new pulumi.Config();

const secrets = config.requireSecretObject<Record<string, any>>("application");

pulumi.all([secrets]).apply(([secretsObj]) => {
  const secretEntries = Object.entries(secretsObj)
    .map(([key, value]) => {
      return `    ${key}:\n      fn::secret: "${value}"`;
    })
    .join("\n");

  let yamlContent = `values:
  pulumiConfig:`;
  if (secretEntries) {
    yamlContent += `\n${secretEntries}`;
  }
  return new pulumi.asset.StringAsset(yamlContent);
});

pulumi up

Ok thanks, I see that if you're using stack secrets, it encrypts/decrypts each one. But if your secrets are in an ESC environment, you should be able to import that into your iac stack pretty quickly

Thank You, but it happens the same if I import it with

environment:

- org/production

Right, I agree that it should be working with setting

environment

normally. I suspect it's still loading the stack secrets

No, I've removed them. Trust me, I've tried everything 😄

Ah, thanks for your patience, I'm finally able to reproduce 🙂 will look into it

Thanks Eimantas! we'll take a look into this issue to improve the integration. We can follow up on the github issue. Thank you so much for all the detailed explanation. I'm glad that you have found a workaround until this gets properly fixed 🧑‍💻