https://pulumi.com logo
Powered by
Title
b

breezy-butcher-78604

09/16/2020, 9:39 AM
i keep getting the following panic when trying to to update the list of security group IDs on an `aws.ec2.instance`: 
panic: fatal: An assertion has failed: Expected diff to not require deletion or replacement during Update of <instance-urn>
    goroutine 659 [running]:
    <http://github.com/pulumi/pulumi/sdk/v2/go/common/util/contract.failfast(...)|github.com/pulumi/pulumi/sdk/v2/go/common/util/contract.failfast(...)>
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v2@v2.9.1-0.20200825190708-910aa96016cd/go/common/util/contract/failfast.go:23
    <http://github.com/pulumi/pulumi/sdk/v2/go/common/util/contract.Assertf(0xc000f52c00|github.com/pulumi/pulumi/sdk/v2/go/common/util/contract.Assertf(0xc000f52c00>, 0x6005215, 0x48, 0xc000bcb7a8, 0x1, 0x1)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v2@v2.9.1-0.20200825190708-910aa96016cd/go/common/util/contract/assert.go:33 +0x197
    <http://github.com/pulumi/pulumi-terraform-bridge/v2/pkg/tfbridge.(*Provider).Update(0xc0007fe1c0|github.com/pulumi/pulumi-terraform-bridge/v2/pkg/tfbridge.(*Provider).Update(0xc0007fe1c0>, 0x6a97a00, 0xc00187a690, 0xc001136700, 0xc0007fe1c0, 0x5295201, 0xc001d66140)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi-terraform-bridge/v2@v2.7.3/pkg/tfbridge/provider.go:971 +0x81d
    <http://github.com/pulumi/pulumi/sdk/v2/proto/go._ResourceProvider_Update_Handler.func1(0x6a97a00|github.com/pulumi/pulumi/sdk/v2/proto/go._ResourceProvider_Update_Handler.func1(0x6a97a00>, 0xc00187a690, 0x5d22400, 0xc001136700, 0x5d19580, 0xaa6b360, 0x6a97a00, 0xc00187a690)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v2@v2.9.1-0.20200825190708-910aa96016cd/proto/go/provider.pb.go:1920 +0x89
    <http://github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1(0x6a97a00|github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1(0x6a97a00>, 0xc00187a330, 0x5d22400, 0xc001136700, 0xc0007ee3e0, 0xc0007ee400, 0x0, 0x0, 0x69d9820, 0xc0004ae010)
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/grpc-opentracing@v0.0.0-20180507213350-8e809c8a8645/go/otgrpc/server.go:57 +0x2eb
    <http://github.com/pulumi/pulumi/sdk/v2/proto/go._ResourceProvider_Update_Handler(0x5ddd380|github.com/pulumi/pulumi/sdk/v2/proto/go._ResourceProvider_Update_Handler(0x5ddd380>, 0xc0007fe1c0, 0x6a97a00, 0xc00187a330, 0xc00125f560, 0xc0007ee080, 0x6a97a00, 0xc00187a330, 0xc000195500, 0x290a)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v2@v2.9.1-0.20200825190708-910aa96016cd/proto/go/provider.pb.go:1922 +0x14b
    <http://google.golang.org/grpc.(*Server).processUnaryRPC(0xc0007fe000|google.golang.org/grpc.(*Server).processUnaryRPC(0xc0007fe000>, 0x6abc2e0, 0xc001220480, 0xc000883000, 0xc0007ec390, 0xaa2d518, 0x0, 0x0, 0x0)
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.30.0/server.go:1171 +0x50a
    <http://google.golang.org/grpc.(*Server).handleStream(0xc0007fe000|google.golang.org/grpc.(*Server).handleStream(0xc0007fe000>, 0x6abc2e0, 0xc001220480, 0xc000883000, 0x0)
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.30.0/server.go:1494 +0xccd
    <http://google.golang.org/grpc.(*Server).serveStreams.func1.2(0xc0008244e0|google.golang.org/grpc.(*Server).serveStreams.func1.2(0xc0008244e0>, 0xc0007fe000, 0x6abc2e0, 0xc001220480, 0xc000883000)
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.30.0/server.go:834 +0xa1
    created by <http://google.golang.org/grpc.(*Server).serveStreams.func1|google.golang.org/grpc.(*Server).serveStreams.func1>
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.30.0/server.go:832 +0x204
i have no idea whats causing it. any ideas?
it then leaves my stack super broken and I have to tear it all down and start again
b

broad-dog-22463

09/16/2020, 9:41 AM
Hi @breezy-butcher-78604 Sorry about this error - what version of Pulumi and what version of pulumi-aws are you using?
b

breezy-butcher-78604

09/16/2020, 9:42 AM
v2.10.0 of the CLI and v3.2.1 of 
@pulumi/aws
whenever i update the 
vpcSecurityGroupIds
property of an 
aws.ec2.Instance
i seem to get that panic
b

broad-dog-22463

09/16/2020, 9:45 AM
so the issue seems to be suggesting that something else is also changing that is causing a potential replacement of the instance
but I am going to confirm if that's just a rogue error now
b

breezy-butcher-78604

09/16/2020, 9:46 AM
the only property that showed up in the diff for that resource was 
vpcSecurityGroupIds
when this happened
b

broad-dog-22463

09/16/2020, 9:47 AM
ok, testhing this for you now
b

breezy-butcher-78604

09/16/2020, 9:47 AM
thats what showed up in the console
if that helps
the rules were all marked to be replaced because they have 
deleteBeforeReplace
enabled, and as you can see I changed the 
sourceSecurityGroupId
let me know if there's any other information i can provide
b

broad-dog-22463

09/16/2020, 9:51 AM
so just to check, did you change the descript of the security group and that's what caused this?
b

breezy-butcher-78604

09/16/2020, 9:52 AM
no, just the 
sourceSecurityGroupId
of a bunch of 
aws.ec2.SecurityGroupRule
resources
the group itself was unchanged
oh wait, i can see one description has changed in that diff...
b

broad-dog-22463

09/16/2020, 9:53 AM
yeah I think that could be the issue - if the group is going to be replaced and the rules are deleteBeforeCreate then that may be a problem
I have been able to change securityGroupIds no issues here
e.g.
Previewing update (dev)

View Live: <https://app.pulumi.com/stack72/vgw-bug/dev/previews/c4721176-75ed-4450-acef-669f9b6b9b1d>

     Type                      Name              Plan
 +   pulumi:pulumi:Stack       vgw-bug-dev       create
 +   ├─ aws:ec2:SecurityGroup  mysecuritygroup2  create
 +   ├─ aws:ec2:SecurityGroup  mysecuritygroup1  create
 +   └─ aws:ec2:Instance       web-server-www    create

Resources:
    + 4 to create

Do you want to perform this update? yes
Updating (dev)

View Live: <https://app.pulumi.com/stack72/vgw-bug/dev/updates/1>

     Type                      Name              Status
 +   pulumi:pulumi:Stack       vgw-bug-dev       created
 +   ├─ aws:ec2:SecurityGroup  mysecuritygroup1  created
 +   ├─ aws:ec2:SecurityGroup  mysecuritygroup2  created
 +   └─ aws:ec2:Instance       web-server-www    created

Resources:
    + 4 created

Duration: 50s


~/code/vgw-demo
▶ pulumi up
Previewing update (dev)

View Live: <https://app.pulumi.com/stack72/vgw-bug/dev/previews/2c789837-25c2-4d0b-b4bd-0fa375109511>

     Type                 Name            Plan       Info
     pulumi:pulumi:Stack  vgw-bug-dev
 ~   └─ aws:ec2:Instance  web-server-www  update     [diff: ~vpcSecurityGroupIds]

Resources:
    ~ 1 to update
    3 unchanged

Do you want to perform this update? yes
Updating (dev)

View Live: <https://app.pulumi.com/stack72/vgw-bug/dev/updates/2>

     Type                 Name            Status      Info
     pulumi:pulumi:Stack  vgw-bug-dev
 ~   └─ aws:ec2:Instance  web-server-www  updated     [diff: ~vpcSecurityGroupIds]

Resources:
    ~ 1 updated
    3 unchanged

Duration: 12s
b

breezy-butcher-78604

09/16/2020, 9:56 AM
so you think this might have been caused by the security group needing to be replaced (due to the changed description) and some kind of w eird interaction between it and it's rules (which have 
deleteBeforeReplace
enabled?
b

broad-dog-22463

09/16/2020, 9:57 AM
I think that could well be the issue here
changing securityGroupIds of an instance works nps at atll
trying to create it now
b

breezy-butcher-78604

09/16/2020, 11:08 AM
ok just had the problem happen again but under different circumstances. this time no groups were being modified, i just swapped out one security group ID for another.
something else I just thought of that might be related, i have specified an 
ebsBlockDevice
on the EC2 instances like so: 
ebsBlockDevices: [{
            deviceName: "/dev/sda1",
            volumeSize: 50,
            deleteOnTermination: true
        }],
I've been working on this template for a couple days now and a few days I noticed that every 
pulumi up
the EC2 instances were being marked for replacement, even if the template hadn't changed. Examining the diff I noticed that the 
volumeSize
property was always the culprit. it always showed that it was adding that property (as it didn't exist on Pulumi's internal state). I didn't have time to troubleshoot it at the time so I just added 
ignoreChanges: [ "ebsBlockDevices" ]
to the resources and moved on. is it possible that the initial diff only shows an update due to the security group IDs, but another part of the code isn't respecting the 
ignoreChanges
option and flagging the instance for replacement which causes the panic?
hey @broad-dog-22463 did you get any further on this one? is there more information I can provide to assist? should I create an issue on github to track this?
it's not blocking me at the moment so don't need any urgent response etc just wanted to make sure i could help confirm or resolve the bug if possible
#typescriptJoin Slack
Powered by