elegant-crayon-4967
06/29/2021, 4:54 PMsalmon-engine-64802
06/29/2021, 5:29 PMicy-nest-77417
06/29/2021, 6:34 PMcreate
, delete
, update
operations for that service (it is a simple CRUD API for managing a single internal resource)). The important part is that we want developers there to be able to import the plugin from a package manager, and then interact with our custom resource service without any hurdle.
I've found 2 or 3 solutions online for this use case so far:
1. Pulumi Packages
2. Dynamic Providers
In the last days, I tried to clone the TypeScript component package boilerplate and added an update
member in the provider class. However, after compiling the code, only the construct
member seems to be taken into consideration when using pulumi up
or pulumi preview
, which is not enabling me to support the CRUD nature of our internal service. I've found on Github multiple classes implementing the pulumi.ProviderResource
interface, but in all of them, only the construct
member is defined (i.e. not the update
member), which makes me think this is not the right tool for my use case.
We're trying to move away from Terraform, so the bridge solution described in the Packages documentation page doesn't seem appropriate either.
I've also considered using a Dynamic Provider instead of the former TypeScript boilerplate. My idea was to simply put the classes implementing the pulumi.dynamic.ResourceProvider
and pulumi.dynamic.Resource
interfaces in a package manager. I wonder whether there are any major issue with this approach, other than the source code not scaling to more than one language (we would prefer to support more languages than just TypeScript, but it is not a hard requirement).
What is the best way to support our custom CRUD service? I have listed some possible directions above, but I'm lacking pointers. I'm not sure why, but there doesn't seem to exist a lot of documentation about this topic online. Any help would be very appreciated! Also, if I am unaware of a more suitable solution for this case, please don't hesitate to let me know.orange-byte-86458
06/29/2021, 7:15 PMeks.Cluster
- I created it about a week ago or so. Things were fine for a while, but then something (default AMI id I guess) changed and produced a diff for Pulumi. Now if I try to pulumi up
it fails saying it’s unable to delete an old Launch Configuration because it’s still attached to an ASG (which is true). I’m not sure what exactly went wrong, there were no interrupted updates or anything. I believe it’s just doing it in the wrong order, as I see new Launch Configurations are already created, they’re just not associated with an ASG yet. I think it should first attach the new configs and then delete the old ones, not vice versa.
Anyway, my real question is what’s the best way out of this situation?
Should I manually edit the state? I’m not quite sure how would I edit it though as these things are somewhat entangled.
I’ve attached a screenshot of the error I’m getting as well as the code I’m using to create the cluster. Thanks for any help!straight-intern-54129
06/29/2021, 8:03 PMpulumi import
that was protected. Now when I do pulumi up
it gives an error saying that the stack itself is a duplicate, and I can't get it back to a working state.
pulumi:pulumi:Stack (PulumiArm-hco):
error: Duplicate resource URN 'urn:pulumi:hco::PulumiArm::pulumi:pulumi:Stack::PulumiArm-hco'; try giving it a unique name
shy-exabyte-78907
06/29/2021, 8:17 PMpulumi up
?gorgeous-eye-54797
06/29/2021, 8:34 PMbland-florist-21412
06/29/2021, 11:20 PMDownloadFileInputs
. However, I have to pass an auth header which changes over time and the destination file for the download is a tempfile
since I just need it temporarily. As a result Pulumi thinks that resource has changed and needs to be updated on each run. Is there a way to tell it to ignore certain aspects of the DownloadFileInputs
like the header and the dest file?rich-engineer-35554
06/29/2021, 11:49 PMincalculable-action-69391
06/30/2021, 3:45 AMincalculable-action-69391
06/30/2021, 3:45 AMbillowy-vr-96461
06/30/2021, 3:59 AMbetter-shampoo-48884
06/30/2021, 6:56 AMpulumi adopt
- functions like preview/refresh+import, but will preview all resources, and check if it might exist in the target, and if it does (and not in stack), import it.
Alternatively, give an option on a failed pulumi up
to automatically `import`resources into the stack that fail with the error ...already exists
- should be fairly cleear/standard on quite a few providers.early-truck-24471
06/30/2021, 7:23 AMsalmon-engine-64802
06/30/2021, 8:10 AMred-football-97286
06/30/2021, 9:26 AMbrash-kite-78002
06/30/2021, 11:25 AMtall-beard-99429
06/30/2021, 1:07 PMtall-beard-99429
06/30/2021, 1:07 PMupdate
gorgeous-country-43026
06/30/2021, 1:44 PMpulumi up
fails for me because Pulumi insists it has to replace all Kubernetes namespaces but when it tries to do that it fails since it apparently tries to create a new namespace with equal name first (since I have forced explicit names for a good reason) and that naturally won't work. Digged into the diff and it shows this
[provider: urn:pulumi:dev::my-project::pulumi:providers:kubernetes::k8s-provider::efaab349-6f4f-42ff-a607-ee6ab55cbb61 => urn:pulumi:dev::my-project::pulumi:providers:kubernetes::default_3_1_2::7b6466de-7864-49f9-8c33-0ece6217519a]
Which makes absolutely no sense to me. That is the only thing it is trying to change. This is 100% Pulumi internal stuff, apparently for some reason the provider has changed?breezy-greece-88371
06/30/2021, 1:52 PMicy-jordan-58549
06/30/2021, 2:45 PMError: failed to read resource #dns/stage 'dns/stage' [pulumi:pulumi:StackReference]: Resource monitor is terminating
- any ideas what could cause this issue?enough-truck-34175
06/30/2021, 3:54 PM<http://github.com/pulumi/pulumi/pkg/v2/testing/integration|github.com/pulumi/pulumi/pkg/v2/testing/integration>
. When I add that package, go mod also adds these two indirect dependencies
<http://opkg.in/airbrake/gobrake.v2|opkg.in/airbrake/gobrake.v2> v2.0.9 // indirect
<http://gopkg.in/gemnasium/logrus-airbrake-hook.v2|gopkg.in/gemnasium/logrus-airbrake-hook.v2> v2.1.2 // indirect
Then, when I run go test .
, I get the following error message
go test . [.]
panic: proto: duplicate enum registered: pulumirpc.EnforcementLevel
goroutine 1 [running]:
<http://github.com/golang/protobuf/proto.RegisterEnum(0x187e661|github.com/golang/protobuf/proto.RegisterEnum(0x187e661>, 0x1a, 0xc0005fbe60, 0xc0005fbe90)
/home/vscode/go/pkg/mod/github.com/golang/protobuf@v1.4.3/proto/registry.go:104 +0x11f
<http://github.com/pulumi/pulumi/sdk/v3/proto/go.init.0()|github.com/pulumi/pulumi/sdk/v3/proto/go.init.0()>
/home/vscode/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.5.1/proto/go/analyzer.pb.go:963 +0x52
FAIL <http://github.com/relativityone/r1-compute-infra/pkg/test|github.com/relativityone/r1-compute-infra/pkg/test> 0.027s
FAIL
prehistoric-kite-30979
06/30/2021, 4:36 PMambitious-queen-83446
06/30/2021, 4:50 PM-y
does not auto-accept the prompt when a stack doesn’t exist?icy-nest-77417
06/30/2021, 5:12 PMUpdate
function to be called when updating a resource instance. Any idea of how to turn off the delete -> create behaviour?proud-pizza-80589
06/30/2021, 6:43 PMpulumi destroy --if-failed-just-delete-from-stack
or a script that can “deep” delete everything from the stack export that depends on the stack reference. Anyone have such a such a script by any chance?stale-autumn-24797
06/30/2021, 7:04 PMazure-native:containerservice:AgentPool (dg-gpuspot-agentpool-cpu-staging-dev):
error: Code="PropertyChangeNotAllowed" Message="Changing property 'properties.nodeTaints' is not allowed." Target="properties.nodeTaints"
We weren’t trying to update the node taints. It seems like Pulumi knows it was a change to maxCount:
~ azure-native:containerservice:AgentPool dg-gpuspot-agentpool-cpu-staging-dev updating [diff: ~maxCount]; error: Code="PropertyChangeNotAllowed" Message="Changing property 'properties.nodeTaints' is not allowed." Target="properties.nodeTaints"
~ azure-native:containerservice:AgentPool dg-gpuspot-agentpool-cpu-staging-dev **updating failed** [diff: ~maxCount]; error: Code="PropertyChangeNotAllowed" Message="Changing property 'properties.nodeTaints' is not allowed." Target="properties.nodeTaints"
Taking a look at the stack output for this resource, I think the issue is that Pulumi is trying to remove the taints that Azure automatically adds to spot node pools. In this case we haven’t been able to get this to work even after editing the stack.
{
"urn": "urn:pulumi:spark-staging::spark-deployment::azure-native:containerservice:AgentPool::dg-gpuspot-agentpool-cpu-staging-dev",
"custom": true,
"id": "/subscriptions/a8c3fdb1-94c2-4db4-bc18-470696fa4bd4/resourcegroups/cp-staging/providers/Microsoft.ContainerService/managedClusters/dg-spark-kubernetescluster-cpu-staging-dev/agentPools/gpuspot",
"type": "azure-native:containerservice:AgentPool",
"inputs": {
"agentPoolName": "gpuspot",
"count": 1,
"enableAutoScaling": true,
"maxCount": 6,
"minCount": 0,
"mode": "User",
"nodeTaints": [
"sku=gpu:NoSchedule"
],
"resourceGroupName": "cp-staging",
"resourceName": "dg-spark-kubernetescluster-cpu-staging-dev",
"scaleSetPriority": "Spot",
"spotMaxPrice": -1,
"type": "VirtualMachineScaleSets",
"vmSize": "Standard_NC8as_T4_v3",
"vnetSubnetID": "/subscriptions/a8c3fdb1-94c2-4db4-bc18-470696fa4bd4/resourceGroups/cp-staging/providers/Microsoft.Network/virtualNetworks/dg-spark-vnet-cpu-staging-dev/subnets/default"
},
"outputs": {
"__inputs": {
"4dabf18193072939515e22adb298388d": "1b47061264138c4ac30d75fd1eb44270",
"ciphertext": "redacted"
},
"count": 1,
"enableAutoScaling": true,
"enableFIPS": false,
"id": "/subscriptions/a8c3fdb1-94c2-4db4-bc18-470696fa4bd4/resourcegroups/cp-staging/providers/Microsoft.ContainerService/managedClusters/dg-spark-kubernetescluster-cpu-staging-dev/agentPools/gpuspot",
"kubeletDiskType": "OS",
"maxCount": 6,
"maxPods": 110,
"minCount": 0,
"mode": "User",
"name": "gpuspot",
"nodeImageVersion": "AKSUbuntu-1804gen2containerd-2021.06.02",
"nodeLabels": {
"<http://kubernetes.azure.com/scalesetpriority|kubernetes.azure.com/scalesetpriority>": "spot"
},
"nodeTaints": [
"sku=gpu:NoSchedule",
"<http://kubernetes.azure.com/scalesetpriority=spot:NoSchedule|kubernetes.azure.com/scalesetpriority=spot:NoSchedule>"
],
"orchestratorVersion": "1.19.11",
"osDiskSizeGB": 128,
"osDiskType": "Ephemeral",
"osSKU": "Ubuntu",
"osType": "Linux",
"powerState": {
"code": "Running"
},
"provisioningState": "Succeeded",
"scaleSetEvictionPolicy": "Delete",
"scaleSetPriority": "Spot",
"spotMaxPrice": -1,
"type": "VirtualMachineScaleSets",
"vmSize": "Standard_NC8as_T4_v3",
"vnetSubnetID": "/subscriptions/a8c3fdb1-94c2-4db4-bc18-470696fa4bd4/resourceGroups/cp-staging/providers/Microsoft.Network/virtualNetworks/dg-spark-vnet-cpu-staging-dev/subnets/default"
straight-magician-37467
06/30/2021, 10:11 PM/infra
(see conversation here). Now, I’m following a tutorial that covers using separate projects for building a Docker image and pushing it to my cloud provider of choice:
https://www.pulumi.com/docs/tutorials/gcp/gcp-ts-docker-gcr-cloudrun/
In this case, would it make sense to have two separate Pulumi folders? (/build
and /deploy
or something like that) Or should I always be building and deploying in a single step? I’m coming at this with some experience using Docker and docker-compose, so I’m used to building, pushing to a registry, and then pulling the image as three separate steps.bright-sandwich-93783
06/30/2021, 11:37 PM