https://pulumi.com logo
Join Slack
Powered by
Hi, as suggested by <@UCWG26BH7>, I'm posting my i...
# azure
a
Hi, as suggested by @billowy-army-68599, I'm posting my issue with a stack that I have. I want to update an AKS stack, when I look at the details of the update, I have this (see first image). I don't understand why pulumi wants to delete the subnet from the vnet (did another Dev did a refresh or something else, I don't know). If I try to do the update, Azure is not happy, because the subnet contains agent pool from AKS (see second image). I'm kind of stucked. I tried stack export, change json, import the stack, did worked. I'm not sure how to "fix" this.
b
@able-train-72108 can you also share your code? did you remove something from your pulumi program?
a
this is the vnet+subnet part: var vnet = new AzureNative.Network.VirtualNetwork("cluster-network", new AzureNative.Network.VirtualNetworkArgs { Tags = tags, VirtualNetworkName = $"akp-{stack}-vnet", ResourceGroupName = resourceGroup.Name, AddressSpace = new AzureNative.Network.Inputs.AddressSpaceArgs { AddressPrefixes = new[] { "10.0.0.0/8" } } }); this.VNetName = vnet.Name; this.VNetId = vnet.Id; var subnet = new AzureNative.Network.Subnet("cluster-subnet", new AzureNative.Network.SubnetArgs { Name = $"akp-{stack}-cluster-subnet", ResourceGroupName = resourceGroup.Name, VirtualNetworkName = vnet.Name, AddressPrefix = "10.0.0.0/16" // nodes and pods are using this, by default Azure creates a /16 }); looking at the console, I see that someone did a pulumi refresh and then a couple of imports, the person may have broke the stack state 😕
we did a test: spawn a new environment (stack) for the same project (aks-cluster). Do an export (dev-freshnew.json). Then update all the other stacks (like 5 or 6) that depend on aks-cluster. Do a refresh on aks-cluster, export, remove the subnet from the json, import, up -> we have the problem I describe. Then we exported the stack again in dev.json But, if we do an import of dev-freshnew.json and then pulumi up, we don't have the issue anymore. We diff the export files dev-freshnew.json and dev.json, the only difference is the ciphertext and one etag.
b
how are you defining the cluster? it seems like the cluster itself is creating a subnet in the virtual network
a
diff of both export files
b
your AKS cluster is creating its own subnet inside your virutal network, I think
a
we are creating the vnet, then the subnet and then we give the subnet.Id to the aks cluster
vnet + subnet:
Copy code
var vnet = new AzureNative.Network.VirtualNetwork("cluster-network", new AzureNative.Network.VirtualNetworkArgs
        {
            Tags = tags,
            VirtualNetworkName = $"akp-{stack}-vnet",
            ResourceGroupName = resourceGroup.Name,
            AddressSpace = new AzureNative.Network.Inputs.AddressSpaceArgs
            {
                AddressPrefixes = new[] { "10.0.0.0/8" }
            }
        });

        this.VNetName = vnet.Name;
        this.VNetId = vnet.Id;

        var subnet = new AzureNative.Network.Subnet("cluster-subnet", new AzureNative.Network.SubnetArgs
        {
            Name = $"akp-{stack}-cluster-subnet",
            ResourceGroupName = resourceGroup.Name,
            VirtualNetworkName = vnet.Name,
            AddressPrefix = "10.0.0.0/16" // nodes and pods are using this, by default Azure creates a /16
        });
agent profile:
Copy code
var agentProfile = new InputList<ManagedClusterAgentPoolProfileArgs>
        {
            new ManagedClusterAgentPoolProfileArgs
            {
                MaxPods = 50,
                EnableAutoScaling = true,
                Count = 3,
                MinCount = 3,
                MaxCount = 10,
                Mode = AgentPoolMode.System,
                Name = "agentpool",
                OsDiskSizeGB = 30,
                OsType = OSType.Linux,
                Type = AgentPoolType.VirtualMachineScaleSets,
                VmSize = "Standard_D4s_v4",
                AvailabilityZones = new[] { "1", "2", "3" },
                VnetSubnetID = subnet.Id,
                Tags = tags,
            },
        };
Hello @billowy-army-68599, we tried something simpler to narrow down the problem.
Copy code
internal class MyStack : Stack
{
    public MyStack()
    {
        var stack = Pulumi.Deployment.Instance.StackName;
        var resourceGroup = new ResourceGroup($"rgSubnetDeletionBug", new ResourceGroupArgs());
        var vnet = new AzureNative.Network.VirtualNetwork("a-vnet", new AzureNative.Network.VirtualNetworkArgs
        {
            VirtualNetworkName = $"a-vnet",
            ResourceGroupName = resourceGroup.Name,
            AddressSpace = new AzureNative.Network.Inputs.AddressSpaceArgs
            {
                AddressPrefixes = new[] { "10.0.0.0/8" }
            },
            //run pulumi up once. then uncomment this and run pulumi up again
            /*Subnets =
            {
                new AzureNative.Network.Inputs.SubnetArgs
                {
                    AddressPrefix = "10.1.0.0/24",
                    Name = "subnet-created-with-vnet",
                },
            },*/
        });

        var subnet = new AzureNative.Network.Subnet("this-subnet-gets-deleted", new AzureNative.Network.SubnetArgs
        {
            Name = $"this-subnet-gets-deleted",
            ResourceGroupName = resourceGroup.Name,
            VirtualNetworkName = vnet.Name,
            AddressPrefix = "10.0.0.0/16"
        });

        //create a NIC attached to this-subnet-gets-deleted to get azure to throw error when it tries to delete this-subnet-gets-deleted
        var networkInterface = new AzureNative.Network.NetworkInterface("networkInterface", new AzureNative.Network.NetworkInterfaceArgs
        {
            EnableAcceleratedNetworking = true,
            IpConfigurations =
            {
                new AzureNative.Network.Inputs.NetworkInterfaceIPConfigurationArgs
                {
                    Name = "ipconfig1",
                    Subnet = new AzureNative.Network.Inputs.SubnetArgs
                    {
                        Id = subnet.Id
                    },
                },
            },
            NetworkInterfaceName = "test-nic",
            ResourceGroupName = resourceGroup.Name,
        });

    }
}
with this piece of code, you can recreate the problem. run pulumi up once, then uncomment the Subnets part in the VNET and then run pulumi up again and you will be stucked not being able to update (as in my original stack problem)
I think the issue comes from the fact that you can have subnet inside vnet args but also as a separate entity by referencing only the vnet name
should I flag a github issue with the code snipet?
7 Views
Open in Slack
PreviousNext
Powered by