Every once in a while when I run pulumi up, the preview shows that it will replace a lot of resource...
y

Yair Halevi

9 months ago
Every once in a while when I run pulumi up, the preview shows that it will replace a lot of resources due to a provider change. Sometimes this happens when I upgrade the AWS provider, and in the past this tended to happen more when the project has evolved a lot between updates. In some of these cases I can avoid replacement by breaking down the changes to the project into smaller units and applying them in order instead of in one large update. But today it seems to be happening even though the provider version is the same. There are some resources with additional changes, but many resources being replaced only have a provider diff, but it's the same version. Attaching an excerpt from the preview details here. Why is this? And how can I avoid it?
++aws:s3/bucket:Bucket: (create-replacement)
        [id=xxxx-dev]
        [urn=urn:pulumi:dev::datalake-iac::aws:s3/bucket:Bucket::xxxx]
        [provider: urn:pulumi:dev::datalake-iac::pulumi:providers:aws::default_6_48_0::1b1bc0e4-8f0c-4ca7-98eb-e9f2d2c280ad => urn:pulumi:dev::datalake-iac::pulumi:providers:aws::default_6_48_0::output<string>]
        acl         : "private"
        bucket      : "xxxx-dev"
        forceDestroy: false
        tags        : {
            abadai:component     : "Artifacts"
            abadai:env           : "dev"
            abadai:ops:owner     : "Engineering"
            abadai:pulumi:project: "datalake-iac"
            abadai:pulumi:stack  : "dev"
            abadai:service       : "Datalake"
        }
        tagsAll     : {
            abadai:component     : "Artifacts"
            abadai:env           : "dev"
            abadai:ops:owner     : "Engineering"
            abadai:pulumi:project: "datalake-iac"
            abadai:pulumi:stack  : "dev"
            abadai:service       : "Datalake"
        }

    +-aws:s3/bucket:Bucket: (replace)
        [id=xxxx-dev]
        [urn=urn:pulumi:dev::datalake-iac::aws:s3/bucket:Bucket::xxxx]
        [provider: urn:pulumi:dev::datalake-iac::pulumi:providers:aws::default_6_48_0::1b1bc0e4-8f0c-4ca7-98eb-e9f2d2c280ad => urn:pulumi:dev::datalake-iac::pulumi:providers:aws::default_6_48_0::output<string>]
      - accelerationStatus               : ""
      - arn                              : "arn:aws:s3:::xxxx-dev"
      - bucketDomainName                 : "xxxx-dev.s3.amazonaws.com"
      - bucketRegionalDomainName         : "xxxx-dev.s3.amazonaws.com"
      - corsRules                        : []
      - grants                           : []
      - hostedZoneId                     : "REDACTED"
      - id                               : "xxxx-dev"
      - lifecycleRules                   : []
      - loggings                         : []
      - region                           : "us-east-1"
      - requestPayer                     : "BucketOwner"
      - serverSideEncryptionConfiguration: {
          - rule: {
              - applyServerSideEncryptionByDefault: {
                  - kmsMasterKeyId: ""
                  - sseAlgorithm  : "AES256"
                }
              - bucketKeyEnabled                  : false
            }
        }
      - tagsAll                          : [secret]
      + tagsAll                          : {
          + abadai:component     : "Artifacts"
          + abadai:env           : "dev"
          + abadai:ops:owner     : "Engineering"
          + abadai:pulumi:project: "datalake-iac"
          + abadai:pulumi:stack  : "dev"
          + abadai:service       : "Datalake"
        }
      - versioning                       : {
          - enabled  : true
          - mfaDelete: false
        }
Hi all, first time user evaluating pulumi to use for our infrastructure. I'm facing issues testing ...
m

Marwan Mustafa Fikrat

10 months ago
Hi all, first time user evaluating pulumi to use for our infrastructure. I'm facing issues testing out my current setup with
pulumi preview
and the typescript compilation. I receive this error:
TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts"
I'm pretty sure this is due to pulumi trying to execute the index file directly without transpiling it with ts. I'm using a monorepo and different configs for different accounts, like so:
/
packages/
  aws/
    tsconfig.json
    package.json
    accounts/
       account1/
         Pulumi.yaml
         index.ts
tsconfig.json
package.json
I
cd
into
packages/aws/accounts/account1
and run
pulumi preview
as a first test.. but I can't get it working. I can't find any resources on how pulumi handles typescript under the hood to understand how this works. EDIT: Seems like pulumi relies on the
package.json
entry point to read the IaC? Is the best way here to defer the transpilation to something else of my choosing, creating a template package.json and using that? EDIT2: Whatever I tried didn't work with the way pulumi resolves the typescript project under the hood. I had a go at transpiling the typescript outside of pulumi and then running the CLI against the transpiled code. This isn't really pretty.. but it seems to be working so far. I'll leave the example bundler config for anyone who might look into doing this. Post compilation looks something like
cd dist/accounts/account1 && pulumi preview
I really hope there's a better way of doing this - I don't want to think about transpiling the code for use with pulumi - i'd expect the pulumi cli to do this automatically. Can I suggest maybe looking into
tsx
or using a similar transpilation step under the hood?
// rollup.config.js

import { globSync } from 'glob';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import fs from 'node:fs';
import typescript from '@rollup/plugin-typescript';
import { nodeResolve } from '@rollup/plugin-node-resolve';
import commonjs from '@rollup/plugin-commonjs';
import json from '@rollup/plugin-json';
import copy from 'rollup-plugin-copy';

const outputDir = 'dist';

/** @type {import('rollup').RollupOptions} */
export default {
  // We want to ensure the preservation of the directory structure.
  // This example is taken from <https://rollupjs.org/configuration-options/#input>
  input: Object.fromEntries(
    globSync('**/*.ts').map((file) => [
      // This remove `src/` as well as the file extension from each
      // file, so e.g. src/nested/foo.js becomes nested/foo
      file.slice(0, file.length - path.extname(file).length),
      // This expands the relative paths to absolute paths, so e.g.
      // src/nested/foo becomes /project/src/nested/foo.js
      fileURLToPath(new URL(file, import.meta.url)),
    ])
  ),
  output: {
    format: 'cjs',
    dir: outputDir,
  },
  // This will probably require modification once we start importing
  // libraries from within the monorepo since pnpm symlinks packages
  // and packages will be resolved as raw .ts and expected to be
  // transpiled as part of this process.
  external: /node_modules/,
  plugins: [
    typescript(),
    nodeResolve({ preferBuiltins: true }),
    json(),
    commonjs(),
    stubPackageJson(),
    // Copy over the Pulumi configuration files as the cli 
    // will be executed against the transpiled code.
    // NOTE: This targets only accounts/ dir, otherwise node_modules 
    // would be resolved.
    // Other projects might want to include files under src/
    copy({
      targets: [
        {
          src: ['accounts/**/Pulumi(.*)?.yaml'],
          dest: outputDir,
          rename: (_name, _extension, fullPath) => {
            return path.join('..', fullPath);
          },
        },
      ],
      flatten: false,
    }),
  ],
};

/**
 * The repository is a ESM project, but pulumi does not support ESM out of the box.
 * This forces the nodejs process to recognize the output as commonjs.
 */
function stubPackageJson() {
  return {
    name: 'stub-package-json-commonjs',
    writeBundle(options) {
      const filePath = path.join(
        options.dir || path.dirname(options.file),
        'package.json'
      );
      fs.writeFileSync(filePath, `{ "type": "commonjs" }`);
      console.log(`File written: ${filePath}`);
    },
  };
}