https://pulumi.com logo
Title
c

cold-toothbrush-60276

12/07/2022, 4:17 PM
which results from:
// ../node_modules/@pulumi/pulumi/proto/status_pb.js
var require_status_pb = __commonJS({
  "../node_modules/@pulumi/pulumi/proto/status_pb.js"(exports2) {
    var jspb2 = require_google_protobuf();
    var goog2 = jspb2;
    var global2 = function() {
      return this || window || global2 || self || Function("return this")(); <-----------------
    }.call(null);
    var google_protobuf_any_pb = require_any_pb();
    goog2.object.extend(proto, google_protobuf_any_pb);
    goog2.exportSymbol("proto.google.rpc.Status", null, global2);
    proto.google.rpc.Status = function(opt_data) {
      jspb2.Message.initialize(this, opt_data, 0, -1, proto.google.rpc.Status.repeatedFields_, null);
    };
w

worried-rain-74420

12/08/2022, 1:17 AM
Ah, I remember having that same issue with ESBuild. ESBuild produces an incompatible translation with the pulumi protobuffers in the SDK. My solution was to add an ESBuild transform to replace
window
with another identifier, I can't remember which.
Maybe
this
for instance
c

cold-toothbrush-60276

12/08/2022, 9:35 AM
hmm I’ll have a look into that… but
this
is already present…
question is why have
window
as an option in the first place… looks like a copy paste job to me…
window
should never even be an option in something never meant to run in a browser…
w

worried-rain-74420

12/08/2022, 4:19 PM
The identifier is autogenerated by protobuf, probably for compatibility, but as you've noted it breaks when bundled.
I'm not suggesting you us
this
, just that I believe you can safely replace
window
with
this
to sidestep the bug, since
this
should always be defined anyway. Essentially, we're just looking for a way to remove the
|| window
expression from that statement, since we know
window
will never be present and will only cause issues. By replacing
|| window
with
|| this
, you end up with the harmless code
return this || this || global2 ...
which is unhygenic, but lets you work around this bug. I believe the bug originates upstream.
Hope that clarifies things some 😄
c

cold-toothbrush-60276

12/10/2022, 7:12 PM
for the minified version 😉
sed -i -e "s/window||d2e||self/d2e/g" path/to/index.js
hmmm no dice... as soon as i only have the bundled and rolled up script pulumi bails asking if the sdk was installed...
this is extremely frustrating. looks like we are forced to keep the kitchen sink in the deployment because the pulumi binary doesn't get that the sdk has been bundled into one file...
w

worried-rain-74420

12/11/2022, 8:31 PM
Hello again Christian! I saw your comment on a related issue, I’ll follow up there too. I just wanted to give you a quick explanation to unblock you, but I’m trying to disconnect from this Slack over the weekend to keep a nice work-life balance for myself. To work around the error you’re getting, you still need to have the Pulumi SDK installed. When Pulumi boots, it looks for
@pulumi/pulumi/cmd/run
and executes that file, which is the entrypoint for NodeJS execution. You can bundle your program with ESBuild, but you still have to have
@pulumi/pulumi
installed unbundled so we can boot the runtime itself.
c

cold-toothbrush-60276

12/11/2022, 8:32 PM
I tried that... it complains about other missing packages then 😞
I'll try again in case I made a mistake, thanks for the heads up!
w

worried-rain-74420

12/11/2022, 8:34 PM
The issue that you commented on would let us tree-shake out most of the code in
@pulumi/pulumi
— e.g. automation api, tests, etc. But we will always need some entrypoint into the NodeJS runtime. One other solution I’ve thought of is to install the runtime with the other language plugins — located at
$PULUMI_HOME/bin
. Then, user programs would be able to fully bundle.
That might be a breaking change for some users --- namely, it would change the runtime version from whatever the user installed in node_modules to whatever version of the CLI they’re using.
c

cold-toothbrush-60276

12/11/2022, 8:34 PM
that sounds like a good idea! we're using nix to build most of our stuff...
w

worried-rain-74420

12/11/2022, 8:45 PM
Nice! I’m a nix fan from afar — I don’t know virtually anything but I use
nix profile install
to grab most of my CLI tools lmao
I tried that... it complains about other missing packages then
I believe I got this to work in the past. I might look again at #10210 over the Christmas break. I want to try bundling with SWC if I can, hoping for a performance improvement. If there’s no perf improvement, I doubt we’d ship the change. I’ll also make a note to open an issue about decoupling the NodeJS runtime from the installed
@pulumi/pulumi
package (moving the runtime into
$PULUMI_HOME/bin
). It’s possible that has to wait until v4.0 since it could be breaking. I’m not sure what other languages do. I think Python is the same, which makes me think maybe all of our languages are the same. As an aside, I’ve also looked int minifying (but not bundling!) providers code. If you’re using e.g. AWS Native, the Node code for AWS Native is not minified. (We don’t want to bundle it because then using an S3 bucket would import every single AWS resource, which would kill performance). I figured if we minify the providers, then we’d improve performance (I was also able to preserve source maps after minification, so debugging doesn’t suffer). However, I found that the minified code produced by SWC is slower than unminified code produces by TSC. I’ve got a few bugs in my analysis, but that’s where I got hung up. (More info here.) Curious to hear your thoughts on this! Signing off for the weekend! Good chatting with you! I’ll follow up on the issue tomorrow.
c

cold-toothbrush-60276

12/11/2022, 8:50 PM
thanks!
happy to help on the nix front, learning curve but worth it !
@worried-rain-74420 yup, just confirming that leaving the @pulumi/pulumi sdk in node_modules whilst the rest is bundled doesn’t work. The SDK relies on a whole host of other dependencies like @grpc/grpc-js which it tries to import and they are obviously not there… so currently no dice running minified code…
w

worried-rain-74420

12/13/2022, 3:51 PM
Gotcha. I'll take a look and see if I can get bundling of the runtime working. Suppose you bundle your program, then delete
node_modules
, then
npm install @pulumi/pulumi
. That should pull in the remaining dependencies, right? e.g.
grpc/grpc-js
c

cold-toothbrush-60276

12/13/2022, 3:52 PM
that might work yes, will give it a go. for now we are just bundling our own code and keeping node_modules as external
I’ll try your suggestion however
just btw. we’re building our docker images using nix. pulumi included
w

worried-rain-74420

12/13/2022, 3:57 PM
Nice! One of my collegues wrote a flaketo install
pulumictl
and I think Pulumi itself?
c

cold-toothbrush-60276

12/13/2022, 3:58 PM
we need pulumi to step up a little in the nix community and ensure the packages are kept up to date… just doing brew isn’t going to cut it I’m afraid
w

worried-rain-74420

12/13/2022, 3:58 PM
Here's the flake to install
pulumictl
. https://github.com/t0yv0/pulumi-nix-devenv-flake
Yeah, I'd personally love to have a Pulumi nix package. Right now I don't think we have enough employees using nix for that. @enough-garden-22763 is one of the few people dangerous with nix. I'd like to actually learn the fundamentals over the Christmas break.
c

cold-toothbrush-60276

12/13/2022, 4:01 PM
nix is a quirky language, simple, but worth learning. nix is powerful and very useful.
w

worried-rain-74420

12/13/2022, 4:14 PM
Have you played with any of the tools/languages built on top of nix? Like Riff or Flox? Another forthcoming weekend project for me 😅
I don't know how to feel about tools built on top of nix. They intend to paper over the challenges the language poses, but I feel like in practice you might have a leaky abstraction. I haven't tried them myself so I can't speak authoritatively of course!
e

enough-garden-22763

12/13/2022, 4:33 PM
I'm sticking to straight Nix for now. Unfortunately in case of Pulumi there's two areas were I run into Nix trouble - it's Node and Python. I think it's a bit more than just a steep learning curve, the interaction with Node and Python ecosystems is not so streamlined with the Nix model I think. In case of Node,
npm ci
directives like to go and issue HTTP requests breaking Nix sandboxing model - there was at least one program I've ran into that I could not package at all. And in the case of Python,
venv
is probably a bit better behaved but still has some scary interactions with the Nix model especially when native dependencies are involved, which may have been installed systemwide.
I do think Nix has some staying power though so perhaps this will be streamlined over time. I intend to keep being a casual user for now and keep trying to figure it out.
c

cold-toothbrush-60276

12/13/2022, 4:57 PM
we’ve also taken the pulumi-bin package, made it our own and stripped out a few things. but we use it in anger because it ensures deterministic builds, I also use it to package node applications using a simple derivation, as long as there is a package.json and yarn.lock involved it won’t have side-effects and nix doesn’t moan.
in any case, if I can help with nix, let me know.
these are the new packages https://github.com/NixOS/nixpkgs/tree/nixos-unstable/pkgs/tools/admin/pulumi-packages https://github.com/NixOS/nixpkgs/tree/nixos-unstable/pkgs/tools/admin/pulumi pulumi-packages needs some tlc… seems like they only added the ones they needed themselves (package maintainers)
w

worried-rain-74420

12/14/2022, 3:46 AM
@cold-toothbrush-60276 here's a WIP PR to bundle the NodeJS runtime. It doesn't get you all the way to bundling user programs, but it should let you remove non-
@pulumi/pulumi
packages from
node_modules
(I tested this locally with success).
With this PR, bundling should work at this step, meaning you won't need transitive deps in
node_modules
anymore.