https://pulumi.com logo
Title
r

ripe-russia-4239

04/23/2021, 9:28 AM
Morning, I wonder if anyone else has encountered this circular dependency issue with Azure Functions and Event Grid. Our situation is as follows: • We have a Pulumi project for shared resources, like an Event Grid Topic, a CosmosDB account and database, Application Insights, etc. Essentially, anything that is common to a single application's microservices. • Each microservice has its own separate Pulumi project, which deploys an Azure Function app resource but not the Function app binaries (written in C#). • The Function app listens to the Event Grid Topic using the trigger attribute. When the app binaries are deployed, a separate ARM resource is created for each function in the app with an ID like
${functionAppId}/functions/${functionName}
• In order to create an Event Grid Subscription resource with Pulumi/ARM, both the Event Grid Topic and the function resource subscribing to that Topic must already exist So, we have an ordering problem: 1. We deploy the Function App resource with Pulumi, from the microservice's infrastructure pipeline. (This is also where we initially expected to deploy the Event Grid Subscription resources.) 2. We separately deploy the microservice's binaries using the DeployAzureFunction task in Azure DevOps, from the microservice's application pipeline. 3. How/where do we create the Event Grid subscription between the Topic and the deployed function? This is especially a problem for the scenarios of deploying a new function with Event Grid subscription, and deploying a new environment from scratch. Possible solutions we have considered: • Adding the Event Grid subscription from the Function app at startup (e.g. with Pulumi .NET SDK, or Azure ARM REST API call), although my concern is that this will delay the scale-out process of functions • Add subscription to the topic from the Function app's application deployment pipeline (e.g. with Pulumi or Azure CLI), although this sort of blurs the distinction between infrastructure and application pipelines. One further related question: is this infrastructure/application split in pipelines a good (or at least acceptable) practice, or is it an antipattern? Are there any "pipeline design patterns" resources or similar that you can particularly recommend? (We're not using Pulumi's hosted service, so AIUI the CI/CD adviser isn't available sadly.)
t

tall-librarian-49374

04/26/2021, 6:13 AM
I guess I don’t quite understand the issue. What’s the problem with creating the subscription after the Azure Function?
r

ripe-russia-4239

04/26/2021, 4:01 PM
Thanks for the reply @tall-librarian-49374 🙂 The problem is that we need to deploy the Azure Function binaries as well as the application resource, before creating the subscription. The way we've modelled this so far (and implemented in our pipelines) is as two quite separate processes, with separate pipelines: one for our Pulumified infrastructure (deploying an empty shell), and one for our application deployment into that shell. I'm happy to accept that this separated approach is wrong, if that's the case. I guess I'm looking for some direction on good practices around placing Pulumi in our pipelines as much as to solve the issue. If the split we have is not good practice then of course we can fix it by combining the deployment pipelines; if it is accepted practice, then we need to find a way around this issue.
Thinking about again, even with a single pipeline we'd still have to run our Pulumi project twice, unless that project also downloaded and deployed the Function zip package
More research indicates this is a problem with Event Grid itself — the same two-phase deployment is required for ARM deployments also 😞