https://pulumi.com logo
#golang
Title
# golang
c

clever-byte-21551

11/04/2020, 6:55 PM
This is really bugging me, but I’m using #automation-api and have a program that uses pulumi-kubernetes and pulumi-aws and my binaries are huge (600>mb) and compilation takes 30-60 seconds because of the CGO compiler. Are you guys going to address that somehow? It’s really annoying to run my program during testing.
l

lemon-agent-27707

11/04/2020, 11:37 PM
Do you see any improvement if you
go install
your pulumi/pulumi, pulumi/auto and pulumi provider dependencies? Not ideas, but does this help with compilation times?
Curious to see what output you're observing that points to cgo?
c

clever-byte-21551

11/05/2020, 6:26 AM
I suspect CGO because the linker is working for a long time during the build. go installing doesn’t effect compilation because it needs to bundle everything together (and I already had them install)
I tried inspecting the binary using
goweight
and got this:
Copy code
533 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/core/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/core/v1>
  406 MB <http://github.com/pulumi/pulumi-aws/sdk/v3/go/aws/ec2|github.com/pulumi/pulumi-aws/sdk/v3/go/aws/ec2>
  213 MB <http://github.com/pulumi/pulumi/sdk/v2/go/pulumi|github.com/pulumi/pulumi/sdk/v2/go/pulumi>
  161 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/extensions/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/extensions/v1beta1>
  124 MB <http://github.com/pulumi/pulumi-aws/sdk/v3/go/aws/s3|github.com/pulumi/pulumi-aws/sdk/v3/go/aws/s3>
   90 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apps/v1beta2|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apps/v1beta2>
   81 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apps/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apps/v1>
   69 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/autoscaling/v2beta2|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/autoscaling/v2beta2>
   67 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/flowcontrol/v1alpha1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/flowcontrol/v1alpha1>
   66 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/networking/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/networking/v1>
   61 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apps/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apps/v1beta1>
   59 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/policy/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/policy/v1beta1>
   59 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiextensions/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiextensions/v1>
   56 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiextensions/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiextensions/v1beta1>
   55 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/meta/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/meta/v1>
   51 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/autoscaling/v2beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/autoscaling/v2beta1>
   47 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/storage/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/storage/v1beta1>
   47 MB <http://github.com/pulumi/pulumi-aws/sdk/v3/go/aws/eks|github.com/pulumi/pulumi-aws/sdk/v3/go/aws/eks>
   47 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/storage/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/storage/v1>
   40 MB <http://github.com/pulumi/pulumi-aws/sdk/v3/go/aws|github.com/pulumi/pulumi-aws/sdk/v3/go/aws>
   38 MB <http://github.com/aws/aws-sdk-go/service/ec2|github.com/aws/aws-sdk-go/service/ec2>
   37 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authorization/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authorization/v1beta1>
   37 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/networking/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/networking/v1beta1>
   37 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authorization/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authorization/v1>
   33 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/rbac/v1alpha1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/rbac/v1alpha1>
   33 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/rbac/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/rbac/v1beta1>
   32 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/rbac/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/rbac/v1>
   27 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/admissionregistration/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/admissionregistration/v1beta1>
   27 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/admissionregistration/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/admissionregistration/v1>
   26 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/auditregistration/v1alpha1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/auditregistration/v1alpha1>
   24 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/autoscaling/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/autoscaling/v1>
   24 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authentication/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authentication/v1>
   21 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/storage/v1alpha1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/storage/v1alpha1>
   20 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiregistration/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiregistration/v1beta1>
   20 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiregistration/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apiregistration/v1>
   18 MB <http://k8s.io/api/core/v1|k8s.io/api/core/v1>
   18 MB <http://github.com/pulumi/pulumi-aws/sdk/v3/go/aws/iam|github.com/pulumi/pulumi-aws/sdk/v3/go/aws/iam>
   18 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/discovery/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/discovery/v1beta1>
   18 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/batch/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/batch/v1>
   18 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/batch/v2alpha1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/batch/v2alpha1>
   18 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/batch/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/batch/v1beta1>
   18 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/certificates/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/certificates/v1beta1>
   17 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/certificates/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/certificates/v1>
   17 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/node/v1alpha1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/node/v1alpha1>
   14 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/node/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/node/v1beta1>
   13 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authentication/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/authentication/v1beta1>
   12 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/settings/v1alpha1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/settings/v1alpha1>
   12 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/events/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/events/v1beta1>
   11 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/events/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/events/v1>
   11 MB <http://github.com/pulumi/pulumi-aws/sdk/v3/go/aws/kms|github.com/pulumi/pulumi-aws/sdk/v3/go/aws/kms>
   10 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/coordination/v1beta1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/coordination/v1beta1>
  9.9 MB <http://github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/coordination/v1|github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/coordination/v1>
You can clearly see the culprits here.
l

lemon-agent-27707

11/06/2020, 6:51 PM
I believe the linking time is due to the huge number of types that get linked in (work around for no generics in Go), not anything CGO specific.
What does your developer workflow look like? When you
pulumi up
are you using the default invocation
go run
style? There is an option to specify a prebuilt binary for pulumi to invoke via the
options.binary
field in pulumi.yaml/project settings. Do you see any performance improvement in incremental builds if you go install your program?
Ah, if I remember correctly you are using Automation API? The above advice is for CLI driven on-disk programs.
c

clever-byte-21551

11/09/2020, 6:59 AM
Yeah I’m using the automation API - Compiling and testing my microservice takes forever because of the compilation time - also my docker image is huge because of that.
3 Views