This is a total hack and I haven't tested if it actually works as a workaround, and even if it did work it's not something I'd recommend, but:
What if you make the value you want to ignore a Stack Output and instead of using IgnoreChanges, you have a ternary that computes "if this should be ignored, instead use value from the Stack Output from this Stack itself (self-referential StackReferences are possible), but otherwise use the new computed value.