https://pulumi.com logo
#azure
Title
# azure
g

glamorous-waitress-51149

06/05/2023, 3:47 PM
@clever-sunset-76585 sorry to stalk you, but just came across this issue, did you get past it at all? https://github.com/microsoft/azure-container-apps/issues/525
c

clever-sunset-76585

06/05/2023, 5:11 PM
Hi @glamorous-waitress-51149 ! I did get past the issue but it's not an ideal solution. It works though. I use two flags set as stack config properties to control the order of operations. I use the first flag to add a
TXT
verification record using the verification ID property of the Container Apps environment once it's created. The second flag is used to set the DNS suffix property of the Container Apps environment knowing that the
TXT
record has been added already. Unfortunately, it's a pattern I've found myself employing with a few Azure services when trying to bind a custom domain to them. For instance, API management service and App Service are the other two services where I've had to do this. The problem is that it seems the Azure APIs have validations, as you might have encountered, that only the Azure Portal can honor because of the manual intervention.
Here's a code snippet of what I use with Container Apps. Note that
useCustomDomains
needs to be set first before
setContainerAppsDnsSuffix
can bet set to
true
for any fresh stack. As you might already know this is a problem with fresh stack deployments. Once a stack is Container Apps environment is fully provisioned, you don't need to touch those stack config properties again for that stack.
Copy code
if (!stackConfig.useCustomDomains && stackConfig.setContainerAppsDnsSuffix) {
    throw new Error(
        "Cannot set the DNS suffix cannot for the Container App Environment without also setting useCustomDomains."
    );
}

if (stackConfig.setContainerAppsDnsSuffix) {
    if (!stackConfig.containerAppsCertificate) {
        throw new Error(
            "containerAppsCertificate stack config is required when using custom domains"
        );
    }

    customDomainConfiguration = {
        dnsSuffix: hostedZoneName,
        certificateValue: stackConfig.containerAppsCertificate.apply((c) =>
            Buffer.from(c, "utf-8").toString("base64")
        ),
        certificatePassword: stackConfig.containerAppsCertificatePassword,
    };
}

// The external container apps environment is publicly accessible
// from the internet.
export const externalContainerAppsEnvironment =
    new containerApps.v20221001.ManagedEnvironment("externalContainerAppsEnv", {
        resourceGroupName: resourceGroup.name,
        appLogsConfiguration: {
            destination: "log-analytics",
            logAnalyticsConfiguration: {
                customerId: logAnalyticsWorkspace.customerId,
                sharedKey: logAnalyticsWorkspaceSharedKeys.apply(
                    (k) => k.primarySharedKey!
                ),
            },
        },
        vnetConfiguration: {
            infrastructureSubnetId: containerAppsInfraSubnet.id,
            internal: false,
        },
        customDomainConfiguration,
        tags: getDefaultTags(),
    });

if (stackConfig.useCustomDomains) {
    if (!hostedZoneId) {
        throw new Error("hostedZoneId is required");
    }

    // Add the verification and the CNAME records to the Cloudflare zone.
    const txtVerificationRecord = new cloudflare.Record(
        "txtVerificationRecord",
        {
            name: `asuid.${hostedZoneName}`,
            type: "TXT",
            zoneId: hostedZoneId,
            value: externalContainerAppsEnvironment.customDomainConfiguration.apply(
                (v) => v!.customDomainVerificationId!
            ),
            // 1 is a special value here meaning "automatic".
            ttl: 1,
        }
    );
}
g

glamorous-waitress-51149

06/05/2023, 7:22 PM
Awesome. Thanks. Do I need to do anything for the container app differently?
Weird thing is, is that someone manually configured our env(shock horror) and we just tried to deploy the container app to it and we got this TXT error
Works fine in our staging environment but not prod. No idea why
f

fast-vr-6049

06/05/2023, 7:54 PM
It might be because the verification code that goes into the TXT record is unique to a subscription. If your dev/staging environment is in a different subscription than production (you lucky bastards), then you'll need to manually grab that new verification code
g

glamorous-waitress-51149

06/05/2023, 7:56 PM
Yeah staging container app env is in a different subscription than the prod container app env but they do seem to have separate ids. Need to double check (afk) but they might have the same certificate
c

clever-sunset-76585

06/05/2023, 10:03 PM
@glamorous-waitress-51149 Are you configuring the custom domain at the container app-level instead of at the env-level? Also can you please post the error message you are seeing?
g

glamorous-waitress-51149

06/06/2023, 7:10 AM
We’re doing it at the container app level. Error
azure-native:app:ContainerApp (webhooks-cnx): error: Code="InvalidCustomHostNameValidation" Message="A TXT record pointing from asuid.foo-webhooks.blah.tech to 9A101... was not found."
c

clever-sunset-76585

06/06/2023, 5:43 PM
Ah note how the
TXT
record has the container app's name in it, so it's unique per container app. You'll have to ensure that the
TXT
record gets created before you set the custom domain settings in the ingress object of the container app. What does your
TXT
record resource look like? I have a hunch you might be depending on the verification ID from the container app which is introducing a circular dependency.
g

glamorous-waitress-51149

06/06/2023, 5:47 PM
I think we’ve decided to maybe look at Azure FrontDoor to see if we can get around it and point AFD to the path on the container app 😃
c

clever-sunset-76585

06/06/2023, 5:53 PM
Gotcha. Yeah I suspect I might need to go that route sometime in the future.
b

bored-activity-40468

06/08/2023, 2:54 PM
I had to look but got around this by using ACME provider in a management stack that can be run periodically to create/renew letsencrypt certs. A middle-tier stack contains the managed environment since it takes longer to create/destroy. We're using YARP as the only container app with a public ingress so the stack that deploys yarp has stackrefs to the other two stacks to get all the info required to add the cert to the managed environment.
played with this and its a cool project but I wanted to use pulumi to handle it https://github.com/shibayan/containerapps-acmebot
c

clever-sunset-76585

06/08/2023, 3:22 PM
@bored-activity-40468 that's cool! I was looking at deploying that very container app as well but I ended up going the route of using a GH Actions workflow that runs daily to check if the cert is within an expiration window. The workflow updates the stack config and opens a PR. I source the cert from stack config stored as a secret. To actually renew the cert in the workflow I use https://go-acme.github.io/lego/. The fact that it came with automatic verification for a bunch of DNS providers made it even easier. @glamorous-waitress-51149 by the way, I am curious if you run into a circular dependency issue when adding a custom domain to Azure Front Door as well. Quite a few of their services are plagued by the sort of validation issue that you hit with adding custom domain to a container app.
g

glamorous-waitress-51149

06/08/2023, 3:23 PM
Will let you know 😃
b

bored-activity-40468

07/02/2023, 6:05 PM
@clever-sunset-76585 managed certificates are in preview. I tried them with the new azure native 2 beta. The cert gets created but it blows up using it. Once that’s fixed, it will be handy.
c

clever-sunset-76585

07/02/2023, 6:13 PM
@bored-activity-40468 it's the reason why I haven't switched over from a custom cert to using that just yet. Been watching that issue in their repo 😄
b

bored-activity-40468

09/13/2023, 1:27 AM
Manged certs are working for me now :)
c

clever-sunset-76585

09/13/2023, 1:44 AM
Awesome! I'll give it a shot at the next chance I get. Since then I have transitioned some apps over to the new Workload profiles-based env and used a Front Door profile with a custom domain added to it instead. I still have some apps in the old env that uses the custom domain suffix with a custom cert
l

little-library-54601

09/20/2023, 8:46 PM
Can anyone provide code that enables a custom domain with an Azure managed cert for an app service? I've got all the basics of app services and Azure function apps working. But for our prod stack, we'll be using a custom domain (from a different DNS provider - so the TXT record issue might hit) and an Azure managed cert. I can usually stumble through via the Pulumi docs and looking at the ARM template from a representative Azure resource. But known-working code gives me a lot more confidence. Thanks.
b

bored-activity-40468

10/19/2023, 1:48 AM
I switched back to https://www.pulumi.com/registry/packages/acme/ until it can be done in 1 shot.
3 Views