This is really bugging me but I m using < C019YSXN04B|automa Pulumi Community #golang

This is really bugging me, but I’m using <#C019YSX...

clever-byte-21551

11/04/2020, 6:55 PM

This is really bugging me, but I’m using #automation-api and have a program that uses pulumi-kubernetes and pulumi-aws and my binaries are huge (600>mb) and compilation takes 30-60 seconds because of the CGO compiler. Are you guys going to address that somehow? It’s really annoying to run my program during testing.

lemon-agent-27707

11/04/2020, 11:37 PM

Do you see any improvement if you

go install

your pulumi/pulumi, pulumi/auto and pulumi provider dependencies? Not ideas, but does this help with compilation times?

lemon-agent-27707

11/04/2020, 11:38 PM

Curious to see what output you're observing that points to cgo?

clever-byte-21551

11/05/2020, 6:26 AM

I suspect CGO because the linker is working for a long time during the build. go installing doesn’t effect compilation because it needs to bundle everything together (and I already had them install)

clever-byte-21551

11/05/2020, 6:48 AM

I tried inspecting the binary using

goweight

and got this:

clever-byte-21551

11/05/2020, 6:48 AM

Copy code

533 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/core/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/core/v1>
  406 MB <http://github.com/pulumi/pulumi-aws/sdk/v3/go/aws/ec2|github.com/pulumi/pulumi-aws/sdk/v3/go/aws/ec2>
  213 MB <http://github.com/pulumi/pulumi/sdk/v2/go/pulumi|github.com/pulumi/pulumi/sdk/v2/go/pulumi>
  161 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/extensions/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/extensions/v1beta1>
  124 MB <http://github.com/pulumi/pulumi-aws/sdk/v3/go/aws/s3|github.com/pulumi/pulumi-aws/sdk/v3/go/aws/s3>
   90 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apps/v1beta2|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apps/v1beta2>
   81 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apps/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apps/v1>
   69 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/autoscaling/v2beta2|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/autoscaling/v2beta2>
   67 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/flowcontrol/v1alpha1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/flowcontrol/v1alpha1>
   66 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/networking/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/networking/v1>
   61 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apps/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apps/v1beta1>
   59 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/policy/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/policy/v1beta1>
   59 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiextensions/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiextensions/v1>
   56 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiextensions/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiextensions/v1beta1>
   55 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/meta/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/meta/v1>
   51 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/autoscaling/v2beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/autoscaling/v2beta1>
   47 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/storage/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/storage/v1beta1>
   47 MB <http://github.com/pulumi/pulumi-aws/sdk/v3/go/aws/eks|github.com/pulumi/pulumi-aws/sdk/v3/go/aws/eks>
   47 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/storage/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/storage/v1>
   40 MB <http://github.com/pulumi/pulumi-aws/sdk/v3/go/aws|github.com/pulumi/pulumi-aws/sdk/v3/go/aws>
   38 MB <http://github.com/aws/aws-sdk-go/service/ec2|github.com/aws/aws-sdk-go/service/ec2>
   37 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authorization/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authorization/v1beta1>
   37 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/networking/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/networking/v1beta1>
   37 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authorization/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authorization/v1>
   33 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/rbac/v1alpha1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/rbac/v1alpha1>
   33 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/rbac/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/rbac/v1beta1>
   32 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/rbac/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/rbac/v1>
   27 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/admissionregistration/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/admissionregistration/v1beta1>
   27 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/admissionregistration/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/admissionregistration/v1>
   26 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/auditregistration/v1alpha1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/auditregistration/v1alpha1>
   24 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/autoscaling/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/autoscaling/v1>
   24 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authentication/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authentication/v1>
   21 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/storage/v1alpha1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/storage/v1alpha1>
   20 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiregistration/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiregistration/v1beta1>
   20 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiregistration/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiregistration/v1>
   18 MB <http://k8s.io/api/core/v1|k8s.io/api/core/v1>
   18 MB <http://github.com/pulumi/pulumi-aws/sdk/v3/go/aws/iam|github.com/pulumi/pulumi-aws/sdk/v3/go/aws/iam>
   18 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/discovery/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/discovery/v1beta1>
   18 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/batch/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/batch/v1>
   18 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/batch/v2alpha1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/batch/v2alpha1>
   18 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/batch/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/batch/v1beta1>
   18 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/certificates/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/certificates/v1beta1>
   17 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/certificates/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/certificates/v1>
   17 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/node/v1alpha1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/node/v1alpha1>
   14 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/node/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/node/v1beta1>
   13 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authentication/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authentication/v1beta1>
   12 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/settings/v1alpha1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/settings/v1alpha1>
   12 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/events/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/events/v1beta1>
   11 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/events/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/events/v1>
   11 MB <http://github.com/pulumi/pulumi-aws/sdk/v3/go/aws/kms|github.com/pulumi/pulumi-aws/sdk/v3/go/aws/kms>
   10 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/coordination/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/coordination/v1beta1>
  9.9 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/coordination/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/coordination/v1>

clever-byte-21551

11/05/2020, 6:48 AM

You can clearly see the culprits here.

lemon-agent-27707

11/06/2020, 6:51 PM

I believe the linking time is due to the huge number of types that get linked in (work around for no generics in Go), not anything CGO specific.

lemon-agent-27707

11/06/2020, 6:55 PM

What does your developer workflow look like? When you

pulumi up

are you using the default invocation

go run

style? There is an option to specify a prebuilt binary for pulumi to invoke via the

options.binary

field in pulumi.yaml/project settings. Do you see any performance improvement in incremental builds if you go install your program?

lemon-agent-27707

11/06/2020, 6:55 PM

Ah, if I remember correctly you are using Automation API? The above advice is for CLI driven on-disk programs.

clever-byte-21551

11/09/2020, 6:59 AM

Yeah I’m using the automation API - Compiling and testing my microservice takes forever because of the compilation time - also my docker image is huge because of that.

3 Views

Open in Slack

Previous Next