Hey friends, I am having so much trouble, I really...
# aws
c
Hey friends, I am having so much trouble, I really need help and I'm willing to pay $100 for someone to help me. I've been stuck on this for a week. I am using Pulumi via SST v3, basically I try to diff or deploy it thinks there are changes in my cognito user pool attributes even though there are no changes and havent been for months. Because it thinks there are changes it crashes due to this issue: https://github.com/hashicorp/terraform-provider-aws/issues/39992 So I need to figure out how to convince it there are actually no changes. Refresh did not help. Thank you anyone for the help!
Copy code
schemas: [
              {
                name: "focus_id",
                attributeDataType: "String",
                mutable: true,
                required: false,
                stringAttributeConstraints: { minLength: "1", maxLength: "50" },
                developerOnlyAttribute: true,
              },
              {
                name: "tags",
                attributeDataType: "String",
                mutable: true,
                required: false,
                stringAttributeConstraints: {
                  minLength: "1",
                  maxLength: "2048",
                },
                developerOnlyAttribute: true,
              },
              {
                name: "divergence_id",
                attributeDataType: "String",
                developerOnlyAttribute: false,
                mutable: true,
                required: false,
              },
              {
                name: "resilience",
                attributeDataType: "Boolean",
                developerOnlyAttribute: false,
                mutable: true,
                required: false,
              },
              {
                name: "resilience_assess",
                attributeDataType: "String",
                mutable: true,
                required: false,
                developerOnlyAttribute: false,
              },
              {
                name: "neuro",
                attributeDataType: "Boolean",
                developerOnlyAttribute: false,
                mutable: true,
                required: false,
              },
              {
                name: "neuro_assessment",
                attributeDataType: "String",
                mutable: true,
                required: false,
                developerOnlyAttribute: false,
              },
              {
                name: "vision_id",
                attributeDataType: "String",
                developerOnlyAttribute: false,
                mutable: true,
                required: false,
              },
            ],
Copy code
*  UserPool sst:aws:CognitoUserPool → UserPoolUserPool aws:cognito:UserPool
   * schemas[2].attributeDataType = String
   + schemas[2].developerOnlyAttribute = false
   * schemas[2].mutable = true
   * schemas[2].name = divergence_id
   + schemas[2].required = false
   * schemas[2].stringAttributeConstraints
   - schemas[3].attributeDataType = String
   - schemas[3].mutable = true
   - schemas[3].name = neuro_assessment
   * schemas[3].stringAttributeConstraints
   * schemas[4].attributeDataType = String
   + schemas[4].developerOnlyAttribute = false
   * schemas[4].mutable = true
   * schemas[4].name = resilience_assess
   + schemas[4].required = false
   * schemas[4].stringAttributeConstraints
   - schemas[5].attributeDataType = String
   - schemas[5].mutable = true
   - schemas[5].name = vision_id
   * schemas[5].stringAttributeConstraints
   + schemas[6].attributeDataType = String
   + schemas[6].developerOnlyAttribute = true
   + schemas[6].mutable = true
   + schemas[6].name = focus_id
   + schemas[6].required = false
   + schemas[7].attributeDataType = String
   + schemas[7].developerOnlyAttribute = true
   + schemas[7].mutable = true
   + schemas[7].name = tags
   + schemas[7].required = false
s
A couple of options: • ignoreChanges: https://github.com/sst/sst/issues/4117#issuecomment-2427548166 • Downgrade to a previous version of the AWS provider. • Remove the user pool from IaC altogether until the issue is fixed
For the last option, I mean like "remove it from your program and state file so that it doesn't get deleted but also doesn't get diffed/updated"
c
IgnoreChanges is not working unfortunately, I think because of instead of a regular error, terraform is crashing so its not properly getting to the ignoreChanges part. In my SST config I have set AWS provider to which is the last version it worked, would you recommend going back further or a sepcific version? Ill delete the cognito pool as. a last resort 😞
Copy code
version: "6.58.0"
s
If the cognito pool isn't changing, you don't lose anything by not having it under IaC.
You can always import it later even if it does change.
c
@stocky-restaurant-98004 you are an amazing person, thank you so much for taking a look with me! I am brand new to the Pulumi/Terraform/SST world. I see theres a state file in an s3 bucket, but I am not fully confident on what I need to delete. Could I pay you to screenshare with me while we edit it?
Understandable if not 🙂 just been super frustrated with this
s
So it looks like TF provider 5.70.0 is ok. Pulumi (bridged, but not native) providers take a TF provider as an underlying dependency. Looks like pulumi-aws v 6.59.0 uses TF provider v 5.70.0: https://github.com/pulumi/pulumi-aws/releases/tag/v6.59.0
So you actually may want to upgrade, not downgrade.
c
Ok let me try this
s
No need to pay! (I'm not sure I could ethically accept anyway since I'm a Pulumi employee, but I'm not sure I have the time to screenshare either.)
I have not used SST, so I'm not entirely familiar with how it works relative to vanilla Pulumi, but removing a resource from your state file is not terribly difficult. It's just
pulumi state delete <urn>
, and then you delete from your code, and then it'll be under manual management.
c
Still crashing with both 6.59.0 and 6.59.1, so it is still incorrectly detecting a diff 😞 Would this be a Pulumi issue? Or a Terraform one? I know SST uses Pulumi and I know the Pulumi team works with SST team but I am not sure where the boundries lay.
From your expereince with these tools what would cause it to incorrectly diff like this? And as you saw from the github issue you posted its happened to other people
I think I can get ignoreChanges to work if the terraform stops crashing, is there a way to have pulumi use an old terraform version or do I just need to switch to an older pulumi?
l
Pretty sure it's not incorrectly diffing. My guess is that the data is stored in an ordered array, and your code is storing the same data in a differently-ordered array. If you changed the ordering of the items in your code to match the values returned from AWS, it may resolve itself.
It may well be that some logic within the provider or plugin is falling over because of the ordering difference.
You have focus_id at 0, tags at 1. The diff is saying that it's expecting focus_id at 6 and tags at 7. Etc.
s
Storing data for things like users in an array like this is not usually the way you want to do it b/c it gets enumerated like Paul said, which can cause huge diffs. You want this data to be unordered.
c
https://www.pulumi.com/registry/packages/aws/api-docs/cognito/userpool/#schemas_nodejs So you're saying this is probably a bug in the pulumi provider?
l
No. This is carried over from the Terraform schema used to build the Pulumi provider. In order to remain compatible, it definitely has to be an array. It's just that an array is not a good type to use. If the providers were to be redesigned to be more YAML-y, then a map would work better. Same thing applied to security group rules and NACL rules. I've had sooo many bugs in my own code because those are both stored in arrays.
Unfortunately the solution has always been that I've had to figure out the logic that is used to order the objects in the array, and re-implement that myself when building the array. So that my code and the Pulumi code match.
s
You might want to consider adding the data (the users) out of band, separate from Pulumi, much the same way you'd pre-load a DB outside of Pulumi. If you need that coordination, you can use the Command provider to do whatever you need at the CLI.
c
Sorry to be clear these aren't users, they are just Cognito user attributes
s
Oh, they're schema elements - I get it.
c
@stocky-restaurant-98004 If you're ever bored/have free time you should check out SST, I know they're partnered with Pulumi so I'm sure more people will be asking questions about it in here over time :)
s
Y'know... from an app design standpoint, it looks like you're putting clinical attributes on your end users, correct? If that's so, those attribs aren't really connected to user identity per se. They're more app data, and probably belong with the other app data in a data store other than Cognito. There is obviously a huge amount of context I'm missing since IDK what the other elements of your app are, but it might be worth exploring whether those attribs should be in Cognito at all.
(Apologies if that suggestion comes off as wildly uninformed.)
c
You're right, I've already moved them to Postgres :) but I am inheriting this code/setup from someone else so our mobile app is grabbing them from Cognito until I can get them to grab it from our API instead. I am trying to reduce cognito related things as much as possible, I appreciate your suggestion and I agree :D
I was able to get this all working by doing ignoreChanges = ["schemas"].... Very very annoyingly ignoreChanges = ["*"] was for some reason not ignoring changes in Schema
s
That's weird.
But I believe you, and I'm glad you got an answer to your issue!
c
I was very fortunate I got one of the founders of SST to help me fix it, he also didn't understand why * didn't work
a
Thank you @creamy-mechanic-23690!
c
@adamant-continent-54884 what happened, you had the same issue?
a
Yes, stuck on it for a while but the
ignoreChanges: ['schemas']
was what I needed
c
You using SST or pulumi?
a
both
but in this case my component was pulumi
c
Happy to help 👍 the issue shut me down for days