https://pulumi.com logo
Powered by
m

millions-judge-24978

02/08/2019, 8:26 PM
Does Pulumi perform a deep diff/compare of things like the 
spec
property of a k8s Ingress resource? https://pulumi.io/reference/pkg/nodejs/@pulumi/kubernetes/extensions/v1beta1/#Ingress-spec I am finding that when updating the JSON of this field, which looks like 
{ rules: ... }
, where the 
...
changes, Pulumi is showing no changes.
Accidentally posted in wrong channel initially, sorry!
c

creamy-potato-29402

02/08/2019, 8:28 PM
@millions-judge-24978 yes, it definitely should
on every resource type, not just 
Ingress
can you share more details about what you’re seeing?
m

millions-judge-24978

02/08/2019, 8:28 PM
It's odd, I think I may just be hitting a weird bug. As is, it shows no diffs. however the value that is being input is definitely different than what appears in the Pulumi web app.
I artificially changed the input and I see a diff now
When I run a 
preview
locally, there is no diff calculated, however if I 
console.log()
the value, it differs from the value in the "Resources" tab on app.pulumi.com The state of my infrastructure agrees with app.pulumi.com, and is missing what my input should add.
c

creamy-potato-29402

02/08/2019, 8:31 PM
I was just going to ask that
hmmm
m

millions-judge-24978

02/08/2019, 8:31 PM
It's as if the local CLI has a weird cache that is incorrect.
c

creamy-potato-29402

02/08/2019, 8:31 PM
cc @gorgeous-egg-16927
the client uses an in-memory cache, but that shouldn’t be the issue if you invoke the CLI many times.
super weird, can you share the code and the desired state?
m

millions-judge-24978

02/08/2019, 8:34 PM
I wouldn't be able to share the full code, for the very least for practical reasons. Let me think how I can slim down to pertinent info
Copy code
Previewing update (XXX/YYY-staging):

     Type                              Name                             Plan       Info
     pulumi:pulumi:Stack               YYY-YYY-staging             30 messages
 ~   └─ kubernetes:extensions:Ingress  external-rpc                     update     [diff: ~spec]

Diagnostics:
  pulumi:pulumi:Stack (YYY-YYY-staging):
    [ { host: '<http://0171.rpc.YYY.staging.ZZZ.org|0171.rpc.YYY.staging.ZZZ.org>',
        http: { paths: [Array] } },
      { host: '<http://01701.rpc.YYY.staging.ZZZ.org|01701.rpc.YYY.staging.ZZZ.org>',
        http: { paths: [Array] } },
      { host: '<http://0163.rpc.YYY.staging.ZZZ.org|0163.rpc.YYY.staging.ZZZ.org>',
        http: { paths: [Array] } },
      { host: '<http://1100.rpc.YYY.staging.ZZZ.org|1100.rpc.YYY.staging.ZZZ.org>',
        http: { paths: [Array] } },
      { host: '<http://61651.rpc.YYY.staging.ZZZ.org|61651.rpc.YYY.staging.ZZZ.org>',
        http: { paths: [Array] } },
      { host: '<http://01300.rpc.YYY.staging.ZZZ.org|01300.rpc.YYY.staging.ZZZ.org>',
        http: { paths: [Array] } },
      { host: '<http://01234.rpc.YYY.staging.ZZZ.org|01234.rpc.YYY.staging.ZZZ.org>',
        http: { paths: [Array] } },
      { host: '<http://0163.rpc.YYY.staging.ZZZ.org|0163.rpc.YYY.staging.ZZZ.org>',
        http: { paths: [Array] } },
      { host: '<http://0872.rpc.YYY.staging.ZZZ.org|0872.rpc.YYY.staging.ZZZ.org>',
        http: { paths: [Array] } },
      { host: '<http://227.rpc.YYY.staging.ZZZ.org|227.rpc.YYY.staging.ZZZ.org>',
        http: { paths: [Array] } },
      { host: '<http://229.rpc.YYY.staging.ZZZ.org|229.rpc.YYY.staging.ZZZ.org>',
        http: { paths: [Array] } },
      { host: '<http://229.rpc.YYY.staging.ZZZ.org|229.rpc.YYY.staging.ZZZ.org>',
        http: { paths: [Array] } },
      { host: '<http://0161.rpc.YYY.staging.ZZZ.org|0161.rpc.YYY.staging.ZZZ.org>',
        http: { paths: [Array] } },
      { host: '<http://100.rpc.YYY.staging.ZZZ.org|100.rpc.YYY.staging.ZZZ.org>',
        http: { paths: [Array] } },
      { host: '<http://100.rpc.YYY.staging.ZZZ.org|100.rpc.YYY.staging.ZZZ.org>',
        http: { paths: [Array] } } ]

Resources:
    ~ 1 to update
    115 unchanged
This is the output when I have artificially changed the rules being input to duplicate the item that is missing
Note the 
229
exists twice. Normally it only is there once. Its addition is the thing being added, but not being detected as new or a diff
The code builds up that rules array which is printed out, 
externalIngressRules
, and then does
Copy code
new k8s.extensions.v1beta1.Ingress("external-rpc", {
      metadata: {
        annotations: { ... }
        }
      },
      spec: {
        rules: externalIngressRules
      }
    }))
Looking at the value of 
spec
in app.pulumi.com, it contains the array without any instance of 
229
c

creamy-potato-29402

02/08/2019, 8:42 PM
and if you delete 229, the diff disappears?
one of the 229s
m

millions-judge-24978

02/08/2019, 8:42 PM
The code is artificially inserting it a second time
Yes correct
c

creamy-potato-29402

02/08/2019, 8:42 PM
baffling
m

millions-judge-24978

02/08/2019, 8:42 PM
If I get rid of the extra there is no diff
😄
c

creamy-potato-29402

02/08/2019, 8:42 PM
I’m trying to think about how I’d reproduce this
what did you do leading up to this
m

millions-judge-24978

02/08/2019, 8:43 PM
I had several failed applies, which could well be the cause
Mostly due to k8s stateful sets not becoming ready in time
c

creamy-potato-29402

02/08/2019, 8:43 PM
can you try a 
pulumi refresh
?
m

millions-judge-24978

02/08/2019, 8:43 PM
Yeah I"ll try that
c

creamy-potato-29402

02/08/2019, 8:44 PM
state could have gotten into a strange state
m

millions-judge-24978

02/08/2019, 8:44 PM
That seems most likely. Is there some difference between what's used by the CLI and what would show at app.pulumi.com?
c

creamy-potato-29402

02/08/2019, 8:44 PM
no
m

millions-judge-24978

02/08/2019, 8:48 PM
our 
refresh
fails, due to
Copy code
Diagnostics:
  kubernetes:apps:StatefulSet (0171):
    error: Preview failed: 2 errors occurred:

    * Resource '0171' was created but failed to initialize
    * Failed to observe the expected number of ready replicas
This is somewhat expected for us, as we take a replica out of the set for "maintenance". Still figuring out best way to make this work well with pulumi. Not sure if it could be part of the problems we're having
I am trying the 
preview
again now
c

creamy-potato-29402

02/08/2019, 8:49 PM
cc @gorgeous-egg-16927 @microscopic-florist-22719 this is useful feedback
m

millions-judge-24978

02/08/2019, 8:49 PM
Same result from preview.
c

creamy-potato-29402

02/08/2019, 8:49 PM
can you ask pulumi to forget about the statefulset?
e.g. with 
pulumi state delete
Joe Beda is doing a TGIK8s at 1 pacific — I’ll have to come back to this if it’s not resolved by then
m

millions-judge-24978

02/08/2019, 8:54 PM
No worries, I am trying to think what effect deleting from state would have
c

creamy-potato-29402

02/08/2019, 8:54 PM
should not affect what’s running,
it would cause Pulumi to “forget” about the resource, but that might not be what you want.
you’d have to get it into pulumi again somehow
actually that’s probably more trouble than it’s worth
ok, I’ll be back in a bit…
g

gorgeous-egg-16927

02/08/2019, 8:56 PM
BTW, if you run into behavior in pulumi that surprises you, or doesn't seem to be working, don't hesitate to file an issue https://github.com/pulumi/pulumi-kubernetes/issues
m

millions-judge-24978

02/08/2019, 8:57 PM
👍 if I can convince myself I haven't just done something dumb I'd be more than happy to file one.
I was able to resolve this with some manual state surgery. 1. 
pulumi stack export
2. Manually delete 
229
out of the rules array anywhere it appeared (inputs and ouputs) 3. 
pulumi stack import
4. 
pulumi update
c

creamy-potato-29402

02/08/2019, 10:35 PM
@millions-judge-24978 oh I see, so the problem was state drift?
m

millions-judge-24978

02/08/2019, 10:41 PM
I guess so? Odd that what output from 
stack export
seemed different than what I saw on app.pulumi.com though
Powered by