https://pulumi.com logo
Join Slack
Powered by
:wave: Hello, team! We've been using Pulumi Go sdk...
# aws
w
👋 Hello, team! We've been using Pulumi Go sdk to provision infrastructure, and one of the last steps is creating ec2 routes inside route tables. After creating successfully, the pulumi operator wants to delete 3 and create 9, after approving, it wants to create 9 and delete 3, and this goes forever. I was wondering why does it want to create them when they are clearly created successfully already? We do check if the route already exist and skip creation. How the routes are defined: • We loop over every 
(routeTableID, CIDR)
pair and create an 
aws.ec2.Route
with a stable name: • 
fmt.Sprintf("%s-vpc-peering-route-%s", tenant, stableHash(rt+cidr))
Has anyone hit this “flip-flop” behaviour before? Guessing Pulumi can’t see the resources during preview because they’re created inside the 
ApplyT
, so every reconcile looks new, but I’m not sure of the right fix (import IDs? move the route creation out of 
ApplyT
?). Cheers!
Events from Kubernetes
Copy code
Events:
  Type    Reason  Age    From                           Message
  ----    ------  ----   ----                           -------
  Normal  Plan    3m36s  config-controller  Infrastructure plan created successfully: 3 to create:
    - tenant-acme-vpc-peering-route-e608a8b5 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-4abb09ac (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-fb233da6 (type: aws:ec2:Route)
9 to delete:
    - tenant-acme-vpc-peering-route-e8f79f00 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-7db2a39a (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-0b80b7b0 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-b6c55fdb (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-87e2c6f1 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-2b989b32 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-ec64ea05 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-dcafce79 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-68d484d6 (type: aws:ec2:Route)
  Normal  Plan  2m47s  config-controller  Infrastructure plan created successfully: 3 to create:
    - tenant-acme-vpc-peering-route-e608a8b5 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-4abb09ac (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-fb233da6 (type: aws:ec2:Route)
9 to delete:
    - tenant-acme-vpc-peering-route-68d484d6 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-87e2c6f1 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-e8f79f00 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-dcafce79 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-b6c55fdb (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-0b80b7b0 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-2b989b32 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-ec64ea05 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-7db2a39a (type: aws:ec2:Route)
  Normal  Reconciled          2m32s                  config-controller  config reconciled successfully
  Normal  Reconciled          2m25s (x4 over 3m39s)  config-controller  Infrastructure created successfully
  Normal  WaitingForApproval  2m22s (x2 over 3m36s)  config-controller  Preview completed. Add the annotation 'apply-approved: true' to proceed.
  Normal  Plan                2m22s                  config-controller  Infrastructure plan created successfully: 9 to create:
    - tenant-acme-vpc-peering-route-b6c55fdb (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-68d484d6 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-e8f79f00 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-87e2c6f1 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-0b80b7b0 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-2b989b32 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-7db2a39a (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-ec64ea05 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-dcafce79 (type: aws:ec2:Route)
3 to delete:
    - tenant-acme-vpc-peering-route-e608a8b5 (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-4abb09ac (type: aws:ec2:Route)
    - tenant-acme-vpc-peering-route-fb233da6 (type: aws:ec2:Route)
Code snippet
Copy code
vpc.ID().ApplyT(func(vpcId string) (ec2.GetRouteTablesResult, error) {
	routeTables, err := getRouteTables(ctx, vpcId, provider)

	sortedRouteTables := make([]string, len(routeTables.Ids))
	copy(sortedRouteTables, routeTables.Ids)
	sort.Strings(sortedRouteTables)

	// Add routes to our route tables pointing to customer VPC CIDR via peering connection
	for _, rt := range sortedRouteTables {
		for _, route := range localRoutes {
			// Use a stable name format that won't change between reconciliations
			routeName := fmt.Sprintf("%s-vpc-peering-route-%s",
				config.Spec.Name,
				generateStableHash(rt+route), // Generate a stable hash based on route table ID and CIDR
			)
			exists, _ := ec2.LookupRoute(ctx, &ec2.LookupRouteArgs{
				DestinationCidrBlock: &route,
				RouteTableId:         rt,
			}, pulumi.Provider(provider))
			if exists != nil {
				<http://log.Info|log.Info>().Msgf("found existing route for %s for dest %s", exists.RouteTableId, exists.DestinationCidrBlock)
				continue
			}
			_, err = ec2.NewRoute(ctx, routeName, &ec2.RouteArgs{
				RouteTableId:           pulumi.String(rt),
				DestinationCidrBlock:   pulumi.String(route),
				VpcPeeringConnectionId: pcx.ID(),
			}, pulumi.Provider(provider),
				pulumi.DependsOn([]pulumi.Resource{
					customerVpc,
					pcx,
				}))

			if err != nil {
				return ec2.GetRouteTablesResult{}, fmt.Errorf("failed to add route to route table %s for VPC peering: %v", rt, err)
			}
			<http://log.Info|log.Info>().Msgf("Created/Updated route in table %s to VPC CIDR %s via VPC peering connection",
				rt, route)
		}
	}

	return *routeTables, err
})
l
Yes, move the route creation outside of the apply. It should be possible to use the apply inside the constructor, rather than the constructor inside the apply.
Why are you getting existing route tables? Just manage the route tables in Pulumi and none of this would be necessary. Pulumi is declarative, just define what you want and let Pulumi sort it out
2 Views
Open in Slack
PreviousNext
Powered by