AWS had a CloudFront API issue today, which happen...
# awsh
handsome-truck-95168
12/27/2019, 7:50 PMAWS had a CloudFront API issue today, which happened to strike while I was deploying. My deploy created a CloudFront distribution, but their API reported an error ("error: error creating CloudFront Distribution: DistributionAlreadyExists: The caller reference that you are using to create a distribution is associated with another distribution.").
handsome-truck-95168
12/27/2019, 7:51 PMRunning the deployment again would fail, because Pulumi didn't know that the CloudFront distribution existed, and would fail when it tried to create it again.
handsome-truck-95168
12/27/2019, 7:52 PMThe fix was pretty painful. I had to export the state of my test stack (which had worked the night before), extract the CloudFront distribution state there, update all that info to point to my production stack, and then copy that into the state of the production stack.
handsome-truck-95168
12/27/2019, 7:52 PMOnce I did that, though, pulumi didn't try to re-create the distribution again and my deployment succeeded.
handsome-truck-95168
12/27/2019, 7:52 PMBut during the outage, my deployment had removed a bunch of services and DNS records so I had an hour of downtime.
handsome-truck-95168
12/27/2019, 7:53 PMWas quite painful!!! But fortunately solvable.
handsome-truck-95168
12/27/2019, 7:54 PMWOuld have been easier to solve if I didn't have to hand-edit the cloudfront distribution object. If I didn't have an example to go from, it would have been impossible.
handsome-truck-95168
12/27/2019, 7:54 PMBut I'm stll glad I'm not deploying by hand, I love pulumi don't worry ❤️
4 Views