a

    able-train-72108

    5 months ago
    Hi, as suggested by @billowy-army-68599, I'm posting my issue with a stack that I have. I want to update an AKS stack, when I look at the details of the update, I have this (see first image). I don't understand why pulumi wants to delete the subnet from the vnet (did another Dev did a refresh or something else, I don't know). If I try to do the update, Azure is not happy, because the subnet contains agent pool from AKS (see second image). I'm kind of stucked. I tried stack export, change json, import the stack, did worked. I'm not sure how to "fix" this.
    b

    billowy-army-68599

    5 months ago
    @able-train-72108 can you also share your code? did you remove something from your pulumi program?
    a

    able-train-72108

    5 months ago
    this is the vnet+subnet part: var vnet = new AzureNative.Network.VirtualNetwork("cluster-network", new AzureNative.Network.VirtualNetworkArgs { Tags = tags, VirtualNetworkName = $"akp-{stack}-vnet", ResourceGroupName = resourceGroup.Name, AddressSpace = new AzureNative.Network.Inputs.AddressSpaceArgs { AddressPrefixes = new[] { "10.0.0.0/8" } } }); this.VNetName = vnet.Name; this.VNetId = vnet.Id; var subnet = new AzureNative.Network.Subnet("cluster-subnet", new AzureNative.Network.SubnetArgs { Name = $"akp-{stack}-cluster-subnet", ResourceGroupName = resourceGroup.Name, VirtualNetworkName = vnet.Name, AddressPrefix = "10.0.0.0/16" // nodes and pods are using this, by default Azure creates a /16 }); looking at the console, I see that someone did a pulumi refresh and then a couple of imports, the person may have broke the stack state 😕
    we did a test: spawn a new environment (stack) for the same project (aks-cluster). Do an export (dev-freshnew.json). Then update all the other stacks (like 5 or 6) that depend on aks-cluster. Do a refresh on aks-cluster, export, remove the subnet from the json, import, up -> we have the problem I describe. Then we exported the stack again in dev.json But, if we do an import of dev-freshnew.json and then pulumi up, we don't have the issue anymore. We diff the export files dev-freshnew.json and dev.json, the only difference is the ciphertext and one etag.
    b

    billowy-army-68599

    5 months ago
    how are you defining the cluster? it seems like the cluster itself is creating a subnet in the virtual network
    a

    able-train-72108

    5 months ago
    diff of both export files
    b

    billowy-army-68599

    5 months ago
    your AKS cluster is creating its own subnet inside your virutal network, I think
    a

    able-train-72108

    5 months ago
    we are creating the vnet, then the subnet and then we give the subnet.Id to the aks cluster
    vnet + subnet:
    var vnet = new AzureNative.Network.VirtualNetwork("cluster-network", new AzureNative.Network.VirtualNetworkArgs
            {
                Tags = tags,
                VirtualNetworkName = $"akp-{stack}-vnet",
                ResourceGroupName = resourceGroup.Name,
                AddressSpace = new AzureNative.Network.Inputs.AddressSpaceArgs
                {
                    AddressPrefixes = new[] { "10.0.0.0/8" }
                }
            });
    
            this.VNetName = vnet.Name;
            this.VNetId = vnet.Id;
    
            var subnet = new AzureNative.Network.Subnet("cluster-subnet", new AzureNative.Network.SubnetArgs
            {
                Name = $"akp-{stack}-cluster-subnet",
                ResourceGroupName = resourceGroup.Name,
                VirtualNetworkName = vnet.Name,
                AddressPrefix = "10.0.0.0/16" // nodes and pods are using this, by default Azure creates a /16
            });
    agent profile:
    var agentProfile = new InputList<ManagedClusterAgentPoolProfileArgs>
            {
                new ManagedClusterAgentPoolProfileArgs
                {
                    MaxPods = 50,
                    EnableAutoScaling = true,
                    Count = 3,
                    MinCount = 3,
                    MaxCount = 10,
                    Mode = AgentPoolMode.System,
                    Name = "agentpool",
                    OsDiskSizeGB = 30,
                    OsType = OSType.Linux,
                    Type = AgentPoolType.VirtualMachineScaleSets,
                    VmSize = "Standard_D4s_v4",
                    AvailabilityZones = new[] { "1", "2", "3" },
                    VnetSubnetID = subnet.Id,
                    Tags = tags,
                },
            };
    Hello @billowy-army-68599, we tried something simpler to narrow down the problem.
    internal class MyStack : Stack
    {
        public MyStack()
        {
            var stack = Pulumi.Deployment.Instance.StackName;
            var resourceGroup = new ResourceGroup($"rgSubnetDeletionBug", new ResourceGroupArgs());
            var vnet = new AzureNative.Network.VirtualNetwork("a-vnet", new AzureNative.Network.VirtualNetworkArgs
            {
                VirtualNetworkName = $"a-vnet",
                ResourceGroupName = resourceGroup.Name,
                AddressSpace = new AzureNative.Network.Inputs.AddressSpaceArgs
                {
                    AddressPrefixes = new[] { "10.0.0.0/8" }
                },
                //run pulumi up once. then uncomment this and run pulumi up again
                /*Subnets =
                {
                    new AzureNative.Network.Inputs.SubnetArgs
                    {
                        AddressPrefix = "10.1.0.0/24",
                        Name = "subnet-created-with-vnet",
                    },
                },*/
            });
    
            var subnet = new AzureNative.Network.Subnet("this-subnet-gets-deleted", new AzureNative.Network.SubnetArgs
            {
                Name = $"this-subnet-gets-deleted",
                ResourceGroupName = resourceGroup.Name,
                VirtualNetworkName = vnet.Name,
                AddressPrefix = "10.0.0.0/16"
            });
    
            //create a NIC attached to this-subnet-gets-deleted to get azure to throw error when it tries to delete this-subnet-gets-deleted
            var networkInterface = new AzureNative.Network.NetworkInterface("networkInterface", new AzureNative.Network.NetworkInterfaceArgs
            {
                EnableAcceleratedNetworking = true,
                IpConfigurations =
                {
                    new AzureNative.Network.Inputs.NetworkInterfaceIPConfigurationArgs
                    {
                        Name = "ipconfig1",
                        Subnet = new AzureNative.Network.Inputs.SubnetArgs
                        {
                            Id = subnet.Id
                        },
                    },
                },
                NetworkInterfaceName = "test-nic",
                ResourceGroupName = resourceGroup.Name,
            });
    
        }
    }
    with this piece of code, you can recreate the problem. run pulumi up once, then uncomment the Subnets part in the VNET and then run pulumi up again and you will be stucked not being able to update (as in my original stack problem)
    I think the issue comes from the fact that you can have subnet inside vnet args but also as a separate entity by referencing only the vnet name
    should I flag a github issue with the code snipet?