Are there any existing modules or tools to glob a ...
# typescript
l
Are there any existing modules or tools to glob a few files and create a hash or mtime checks to trigger a replacement? I would like to refine my
command.local.Command
triggers to only rerun if I need to rebuild some source files.
l
Probably, but if you wanted to do it in pure Pulumi, you could create a FileArchive or FileAsset, and have your Command dependOn that.
l
The issue is I am using
cargo lambda
which already generates a .zip bundle as the output, so I did not wanna re-archive. But because of the upstream, the build changes the hash of that whenever its run. So the lambda .zip changes if I rerun the command which triggers a change each time. If I index on the source files I can scope that down a little bit more to only rerun on material changes to the code / deps.
m
cargo lambda build --output-format binary then zip it up yourself? i do a similar thing with esbuild, and i just have to set the mtime on the zip to a constant and then pass that to FileArchive and i take a hash of the zip and pass it to
sourceCodeHash
Copy code
export class ESBuildNodeFunction extends aws.lambda.Function {
  constructor(name: string, args: NodeFunctionArgs) {
    const options = deepmerge<esbuild.BuildOptions>(
      esbuildDefaultOpts,
      args.esbuild || {}
    );

    const outdir = fs.mkdtempSync(path.join(os.tmpdir(), '/'));

    const { outputFiles } = esbuild.buildSync({
      entryPoints: [args.entry],
      ...options,
      outdir, // important or all filenames become <stdout>
      write: false,
      // plugins: [commonjs()], // don't work in sync builds
    });

    const filenamesAndContents = Object.values(outputFiles).reduce(
      function collectFilenameAndContents(acc, curr) {
        return { ...acc, [path.basename(curr.path)]: curr.contents };
      },
      {}
    );

    // the mtime causes the zip file hash to be deterministic
    // 0 should work but Pulumi has some weird validation where "date not
    // in range 1980-2099", so I picked the best date during that range
    const zipContent = fflate.zipSync(filenamesAndContents, {
      os: 0,
      mtime: '1987-12-26',
    });

    const zipFile = path.join(outdir, 'lambda.zip');

    // we have to write this to disk because FileArchive requires a zip
    // and using StringAsset doesn't support reading in the buffer even
    // when it's a string for whatever reason
    fs.writeFileSync(zipFile, zipContent);

    // handler format is file-without-extension.export-name so the .ts
    // messes this up and we need to remove it from the filename
    const entry = path.basename(args.entry, path.extname(args.entry));
    const method = args.handler || 'default';
    const handler = `${entry}.${method}`;

    // Check that the expected method is exported by the module otherwise it
    // bundles, then lambda fails to call it and its hard to spot until runtime
    import(args.entry).then((mod) => {
      if (mod[method as string] === undefined) {
        throw new Error(`${method} is not exported by ${args.entry}`);
      }
    });

    // this will override NODE_OPTIONS if set by the caller so really
    // this needs more complicated logic to add this option to
    // NODE_OPTIONS if present
    const environment = deepmerge<typeof args.environment>(
      args.environment || {},
      {
        variables: {
          NODE_OPTIONS: options.sourcemap ? '--enable-source-maps' : '',
        },
      }
    );

    super(name, {
      architectures: ['arm64'],
      runtime: 'nodejs20.x',
      ...args,
      code: new pulumi.asset.FileArchive(zipFile),
      handler,
      packageType: 'Zip',
      environment,
      sourceCodeHash: crypto
        .createHash('sha256')
        .update(zipContent)
        .digest('base64'),
    });
  }
}
l
Thank you! That helps a absolute ton.
m
no problem
g
@miniature-arm-21874 This is an interesting solution, would you mind sharing
NodeFunctionArgs
? We build our TS functions beforehand and zip them but your solutions seem to be building the code directly. We have a shared library that we use to populate a few things for pulumi runtime as well as it is part of the lambda code itself. Do you think that your solution can be applied to this use-case as well?
m
Yeah I can share it this eve when I’m at my laptop, no problem
Hard to say whether or not your shared library could work without knowing what it does and your use case but you can use things from the runtime in my example for sure by passing in environment variables as I’ve done that. In terms of using them directly in the code I guess it depends how typescript determines their necessity within the ts program
@great-sunset-355 apologies i forgot to send this the other day Here's the missing bits of the code
Copy code
import * as pulumi from '@pulumi/pulumi';
import * as aws from '@pulumi/aws';
import * as esbuild from 'esbuild';
import * as fs from 'node:fs';
import * as os from 'node:os';
import * as path from 'node:path';
import * as crypto from 'node:crypto';

import * as fflate from 'fflate';
import deepmerge from 'deepmerge';

interface NodeFunctionArgs extends aws.lambda.FunctionArgs {
  /**
   * The file path for the lambda
   */
  entry: string;

  /**
   * A custom esbuild configuration
   */
  esbuild?: esbuild.BuildOptions;

  /**
   * Zip the bundled function into a zip archive called lambda.zip
   * @default true
   */
  zip?: boolean;
}

const esbuildDefaultOpts: esbuild.BuildOptions = {
  bundle: true,
  minify: false,
  sourcemap: true,
  platform: 'node',
  format: 'cjs',
  target: 'esnext',
  // outExtension: { '.js': '.mjs' },
};
and an example of its use with how you might pass in some pulumi values into env vars etc
Copy code
export const exampleLambda = new ESBuildNodeFunction('example', {
  entry: path.resolve(__dirname, 'handler.ts'),
  role: role.arn,
  timeout: 8,
  memorySize: 128,
  environment: {
    variables: {
      DB_HOST: db.address,
      DB_USER: db.username,
      DB_PASS: dbPassword,
      DB_PORT: db.port.apply((port) => port.toString()),
      DB_NAME: db.dbName,
    },
  },
  vpcConfig: {
    securityGroupIds: [lambdaSecurityGroup.id],
    subnetIds: publicSubnetIds,
  },
  esbuild: {
    external: [...knexExternals],
  },
});
g
@miniature-arm-21874 awesome, thanks!