Azure Container Job with Service Bus Queue Trigger Question: I have a chicken and egg problem with creating an Azure Container Job with a Service Bus Trigger. In order for the provisioning of the Container Job to finish, the SystemAssigned user id of the Container job must be given reader privileges on the Service Bus (confirmed this with Azure Support yesterday). Chicken. But I can't get the System Assigned identity of the Container Job in Pulumi until the Job finishes provisioning. Egg. I tried creating the job with a Manual trigger, then getting the System Assigned id from there, assigning it to the Service Bus, then calling another method to alter the definition of the job to set the trigger to Event trigger, by setting the CustomResourceOptions passing in the Urn of the original ContainerJob, but that doesn't do anything. Id I leave the URN off, I get a duplicate resource issue.

new CustomResourceOptions
                {
                    Provider = Context.Provider,
                    ReplaceOnChanges = { "TriggerType", },
                    Urn = new Urn(urn),
                });

The other option I tried was to create a UserAssignedIdentity, but that failed due to a "A Subscription ID must be configured when authenticating as a Service Principal using a Client Secret." which I think is related to how the AzureAD provider works... and something that is not easy for us to fix because we have a multitenant solution that deploys to dozens of subscriptions... anyway... Is there a way to tell pulumi to take the existing Container Job definition and alter it after it has been created, and await the provisioning of the Service Bus queue and role assignments?

You generally won't have a good time trying to back over resources this way. The UserAssignedIdentity is likely the path of least resistance here. Then you could handle the queue and any perms ahead of creating the job. Do you have a more complete code example I could review?

Here are the two methods I built as an attempt... First one creates the container as a manually triggered job. Then, after creating the service bus and assigning the identity of the container to the service bus, I tried to do an update... but not sure of the syntax for doing that exactly.

/// <summary>
        /// Method to create Azure Container App Job in azure.
        /// </summary>
        /// <typeparam name="TContainerOptions">The type of container options.</typeparam>
        /// <param name="containerOptions">Container configuration options.</param>
        /// <param name="resourceGroupName">Resource group name.</param>
        /// <param name="envArgList">Environment variables.</param>
        /// <param name="appSecrets">Optional application secrets to include in the container app.</param>
        /// <returns>Reference to the created container app.</returns>
        public async Task<Job> DeployContainerJob<TContainerOptions>(TContainerOptions containerOptions,
                                                                     Input<string> resourceGroupName,
                                                                     InputList<EnvironmentVarArgs> envArgList,
                                                                     IEnumerable<SecretArgs>? appSecrets = null)
            where TContainerOptions : ContainerOptions
        {
            if (containerOptions.TriggerOptions == null)
            {
                throw new Exception("ContainerOptions.TriggerOptions for a job cannot be null.");
            }

            var containerRegistryId = await Context.RequireStampOutput<string>(ResourceOutputs.ContainerRegistryId);
            var containerAppEnvironmentId = await Context.RequireCustomerInstanceOutput<string>(ResourceOutputs.ContainerAppsEnvironmentId);
            var containerRegistryFqdn =
                await Context.RequireStampOutput<string>(ResourceOutputs.ContainerRegistryFullyQualifiedDomainName);
            var (username, password) = await DeploymentContext.GetRegistryCredentials(containerRegistryId);

            var jobConfigurationArgs = new JobConfigurationArgs
            {
                ReplicaTimeout = 300,
                ReplicaRetryLimit = 1,
                TriggerType = "Manual",
                Registries = new RegistryCredentialsArgs
                {
                    Server = containerRegistryFqdn,
                    Username = username,
                    PasswordSecretRef = "registry-secret",
                },
                Secrets = BuildSecretsList(password, appSecrets),
            };

            var jobApp = new Job($"{containerOptions.Name}-job",
                new JobArgs
                {
                    ResourceGroupName = resourceGroupName,
                    Configuration = jobConfigurationArgs,
                    JobName = containerOptions.Name,
                    Identity = new ManagedServiceIdentityArgs
                    {
                        Type = ManagedServiceIdentityType.SystemAssigned,
                    },
                    Template = new JobTemplateArgs
                    {
                        Containers = new[]
                        {
                        new ContainerArgs
                        {
                            Name = containerOptions.Name,
                            Image = $"{containerRegistryFqdn}/{containerOptions.Image}",
                            Resources = new ContainerResourcesArgs
                            {
                                Cpu = containerOptions.Cpu ?? 0.17,
                                Memory = containerOptions.Memory ?? "0.5Gi",
                            },
                            Probes = new[]
                            {
                                containerOptions.StartupProbe.GetProbe(),
                                containerOptions.ReadinessProbe.GetProbe(),
                                containerOptions.LivenessProbe.GetProbe(),
                            },
                            Env =
                            [
                                envArgList ?? [],
                                Context.GetWillowContext().ToList(),
                            ],
                        },
                        },
                    },
                    EnvironmentId = containerAppEnvironmentId,
                    WorkloadProfileName = containerOptions.ContainerAppWorkloadProfileName ?? string.Empty,
                    Tags = new Dictionary<string, string>
                    {
                    { "container", containerOptions.Name },
                    { "image", containerOptions.Image },
                    },
                },
                new CustomResourceOptions { Provider = Context.Provider });

            return jobApp;
        }

        public async Task<Job> UpdateJobTrigger<TContainerOptions>(TContainerOptions containerOptions,
                                                             Input<string> resourceGroupName,
                                                             InputList<EnvironmentVarArgs> envArgList,
                                                             string urn,
                                                             IEnumerable<SecretArgs>? appSecrets = null)
            where TContainerOptions : ContainerOptions
        {
            if (containerOptions.TriggerOptions == null)
            {
                throw new Exception("ContainerOptions.TriggerOptions for a job cannot be null.");
            }

            var containerRegistryId = await Context.RequireStampOutput<string>(ResourceOutputs.ContainerRegistryId);
            var containerAppEnvironmentId = await Context.RequireCustomerInstanceOutput<string>(ResourceOutputs.ContainerAppsEnvironmentId);
            var containerRegistryFqdn =
                await Context.RequireStampOutput<string>(ResourceOutputs.ContainerRegistryFullyQualifiedDomainName);
            var (username, password) = await DeploymentContext.GetRegistryCredentials(containerRegistryId);

            var jobConfigurationArgs = new JobConfigurationArgs
            {
                ReplicaTimeout = 300,
                ReplicaRetryLimit = 1,
                TriggerType = containerOptions.TriggerOptions.TriggerType,
                Registries = new RegistryCredentialsArgs
                {
                    Server = containerRegistryFqdn,
                    Username = username,
                    PasswordSecretRef = "registry-secret",
                },
                Secrets = BuildSecretsList(password, appSecrets),
            };

            var scheduleTrigger = GetScheduleTrigger(containerOptions);
            var eventTrigger = GetEventTrigger(containerOptions);

            if (scheduleTrigger != null)
            {
                jobConfigurationArgs.ScheduleTriggerConfig = scheduleTrigger;
            }

            if (eventTrigger != null)
            {
                jobConfigurationArgs.EventTriggerConfig = eventTrigger;
            }

            var jobApp = new Job($"{containerOptions.Name}-job",
                new JobArgs
                {
                    ResourceGroupName = resourceGroupName,
                    Configuration = jobConfigurationArgs,
                    JobName = containerOptions.Name,
                    Identity = new ManagedServiceIdentityArgs
                    {
                        Type = ManagedServiceIdentityType.SystemAssigned,
                    },
                    Template = new JobTemplateArgs
                    {
                        Containers = new[]
                        {
                            new ContainerArgs
                            {
                                Name = containerOptions.Name,
                                Image = $"{containerRegistryFqdn}/{containerOptions.Image}",
                                Resources = new ContainerResourcesArgs
                                {
                                    Cpu = containerOptions.Cpu ?? 0.17,
                                    Memory = containerOptions.Memory ?? "0.5Gi",
                                },
                                Probes = new[]
                                {
                                    containerOptions.StartupProbe.GetProbe(),
                                    containerOptions.ReadinessProbe.GetProbe(),
                                    containerOptions.LivenessProbe.GetProbe(),
                                },
                                Env =
                                [
                                    envArgList ?? [],
                                    Context.GetWillowContext().ToList(),
                                ],
                            },
                        },
                    },
                    EnvironmentId = containerAppEnvironmentId,
                    WorkloadProfileName = containerOptions.ContainerAppWorkloadProfileName ?? string.Empty,
                    Tags = new Dictionary<string, string>
                    {
                        { "container", containerOptions.Name },
                        { "image", containerOptions.Image },
                    },
                },
                new CustomResourceOptions
                {
                    Provider = Context.Provider,
                    ReplaceOnChanges = { "TriggerType", },
                    Urn = new Urn(urn),
                });

            return jobApp;
        }

Consider every 'new' resource to be the whole of managing it. Sometimes there are subsets possible, e.g. managing the Job with IgnoreChanges: TriggerType and then managing Triggers on their own later, but it doesn't look like this resource type supports breaking those out. So when there's an API like this, that is all inline, you're limited to what you can do. You'll have to have the necessary stuff the first pass on the job object. Which you can get by pre-creating the queue and identity then passing them into the job. E.g.

var queue = new Queue("processing-queue", new QueueArgs
{
    ResourceGroupName = resourceGroup.Name,
    NamespaceName = serviceBusNamespace.Name,
    MaxDeliveryCount = 10,
    LockDuration = "PT5M",
    DefaultMessageTimeToLive = "P14D"
});

var managedIdentity = new UserAssignedIdentity("containerapp-identity", new UserAssignedIdentityArgs
{
    ResourceGroupName = resourceGroup.Name
});
var roleAssignment = new RoleAssignment("servicebus-receiver-role", new RoleAssignmentArgs
{
    PrincipalId = managedIdentity.PrincipalId,
    PrincipalType = PrincipalType.ServicePrincipal,
    RoleDefinitionId = "/providers/Microsoft.Authorization/roleDefinitions/4f6d3b9b-027b-4f4c-9142-0e5a2a2247e0", // Azure Service Bus Data Receiver
    Scope = queue.Id
});

// Snipped all properties not relevant to the job configuration
var containerAppJob = new Job("queue-processor-job", new JobArgs
{
    Configuration = new JobConfigurationArgs
    {
        TriggerType = TriggerType.Event,
        EventTriggerConfig = new JobEventTriggerConfigArgs
        {
            Scale = new JobScaleArgs
            {
                Rules =
                {
                    new ScaleRuleArgs
                    {
                        Name = "queue-scaling-rule",
                        Type = "azure-servicebus",
                        Metadata =
                        {
                            { "queueName", queue.Name },
                            { "namespace", serviceBusNamespace.Name },
                            { "messageCount", "5" }
                        },
                        Auth = new List<ScaleRuleAuthArgs>
                        {
                            new ScaleRuleAuthArgs
                            {
                                SecretRef = "connection-string-secret",
                                TriggerParameter = "connection"
                            }
                        }
                    }
                }
            }
        },

        Secrets =
        {
            new SecretArgs
            {
                Name = "connection-string-secret",
                Value = Output.Format(
                    $"Endpoint=sb://{serviceBusNamespace.Name}.servicebus.windows.net/;Authentication=Managed Identity")
            }
        }
    },
    Identity = new ManagedServiceIdentityArgs
    {
        Type = ManagedServiceIdentityType.UserAssigned,
        UserAssignedIdentities =
        {
            { managedIdentity.Id, new object() }
        }
    },
}, new CustomResourceOptions { DependsOn = { roleAssignment } });

Thanks... I was leaning that way after think about it over the weekend. We'll see if we can get the User Assigned Identity to work (had issues with environment variables as we run in multiple subscriptions). Working on that. Thanks again!