Following Pulumi examples I added index and 404 html files from the same source folder to the $web container and they upload correctly.

var index_html = new Storage.Blob("index.html", new Storage.BlobArgs
        {
            ResourceGroupName = resourceGroup.Name,
            AccountName = storageAccount.Name,
            ContainerName = staticWebsite.ContainerName,
            Source = new FileAsset("./wwwroot/index.html"),
            ContentType = "text/html",
        });

        var notfound_html = new Storage.Blob("404.html", new Storage.BlobArgs
        {
            ResourceGroupName = resourceGroup.Name,
            AccountName = storageAccount.Name,
            ContainerName = staticWebsite.ContainerName,
            Source = new FileAsset("./wwwroot/404.html"),
            ContentType = "text/html",
        });

Started here - https://github.com/pulumi/examples/tree/master/azure-ts-static-website Pulumi up, all good. Added

const wwwroot = new storage.Blob("wwwroot", {
    resourceGroupName: resourceGroup.name,
    accountName: storageAccount.name,
    containerName: staticWebsite.containerName,
    source: new pulumi.asset.FileArchive(`./wwwroot`),
    contentType: "folder",
});

Returned

error: Error: failed to register new resource index.html [azure-nativestorageBlob]: 2 UNKNOWN: failed to compute asset hash: asset path './wwwroot' is a directory; try using an archive

@tall-librarian-49374 any further insight on this one?