pulumi-community #aws

bumpy-laptop-30846

09/07/2023, 9:46 AM

Hello, does anyone have this issue and know how to solve this? warning: …:awss3/bucket:Bucket$awss3/bucketObjectBucketObject::favicon.ico verification warning: use the aws_s3_object resource instead I am using aws.s3.BucketObject and “@pulumi/aws”: “^6.0.4" in nodejs

gorgeous-lunch-7514

09/08/2023, 1:15 PM

I’m getting the following despite my code seeming sane:

Copy code

aws:s3:BucketObject (index.html):
    warning: urn:pulumi:dev::admin::aws:s3/bucketObject:BucketObject::index.html verification warning: use the aws_s3_object resource instead
    error: aws:s3/bucketObject:BucketObject resource 'index.html' has a problem: Attribute must be a single value, not a map. Examine values at 'index.html.bucket'.

gorgeous-lunch-7514

09/08/2023, 1:16 PM

Copy code

const adminBucketDir = `${__dirname}/../build/`;
const updatedFiles: BucketObject[] = [];
crawlDirectory(adminBucketDir, (filePath: string) => {
  const relativeFilePath = filePath.replace(adminBucketDir + "/", "");
  console.log(relativeFilePath, filePath);
  const contentFile = new aws.s3.BucketObject(
    relativeFilePath,
    {
      key: relativeFilePath,
      acl: "public-read",
      bucket: config.adminBucket,
      contentType: mime.getType(filePath) || undefined,
      source: new pulumi.asset.FileAsset(filePath),
    }
  );
  updatedFiles.push(contentFile);
});

sparse-optician-70334

09/08/2023, 9:17 PM

How does Pulumi on AWS with Databricks interact? https://docs.databricks.com/en/dev-tools/pulumi.html mentions that the workspace must already be deployed/created. But I was hoping to create the databricks worksapces automatically from pulumi. How would this work instead?

ambitious-agent-35343

09/11/2023, 8:47 AM

(Databricks) I am looking to administer the management-account administering the workspaces for users and workspaces. The account-level provider uses password, and this is not great given the sensitivity. Any recommendations? I can think of creating a new user for managing the management account, but was curious what others are doing. Maybe @sparse-optician-70334 from above has taken an insightful approach. Thank you https://www.pulumi.com/registry/packages/databricks/api-docs/user/

import * as pulumi from "@Pulumi Novice/pulumi";

import * as databricks from "@Pulumi Novice/databricks";

// initialize provider at account-level

const mws = new databricks.Provider("mws", {

host: "<https://accounts.cloud.databricks.com>",

accountId: "00000000-0000-0000-0000-000000000000",

username: _var.databricks_account_username,

password: _var.databricks_account_password,

});

const accountUser = new databricks.User("accountUser", {

userName: "<mailto:me@example.com|me@example.com>",

displayName: "Example user",

}, {

provider: databricks.mws,

});

rough-morning-53309

09/11/2023, 9:41 PM

👋 I'm the product manager for providers at Pulumi. I'm currently seeking feedback on the AWS Native provider. If you have experience using it or specific feedback to share, I'd love to hear it! Please reach out to me directly to setup a time to chat. Thanks!

sparse-optician-70334

09/12/2023, 7:25 AM

How can I properly populate the readme? I am following (https://www.pulumi.com/docs/pulumi-cloud/projects-and-stacks/?_ga=2.149016052.318359768.1694094860-180507132.1691596035#stack-readme).

Copy code

import pulumi
from pulumi_aws_native import s3

# Create an AWS resource (S3 Bucket)
bucket = s3.Bucket("my_bucket")

pulumi.export("my_bucket", bucket.id)

with open('./Pulumi.README.md') as f:
    pulumi.export('readme', f.read())

The readme with contents of:

Copy code

- main bucket ${bucket.id}
- my_bucket ${my_bucket}

The file is neatly uploaded. However, the ${} reference to the variable is never resolved and empty. What do I need to change to get the variables to resolve?

sparse-optician-70334

09/12/2023, 8:36 AM

I am trying to create a databricks workspace from pulumi. The steps described here https://docs.databricks.com/en/administration-guide/account-settings-e2/credentials.html#step-3-create-a-credential-configuration-in-databricks have already been ported to pulumi: - create cross account policy - create policy When trying to create the MwsCredentials or MwsStorageConfigurations I run into issues such as: cannot create mws credentials: default auth: cannot configure default credentials This issue also seems to be something for terraform: https://registry.terraform.io/providers/databricks/databricks/latest/docs/data-sources/mws_workspaces If you have a fully automated setup with workspaces created by databricks_mws_workspaces or azurerm_databricks_workspace, please make sure to add depends_on attribute in order to prevent default auth: cannot configure default credentials errors. How can this be solved with Pulumi?

Copy code

import pulumi
from pulumi_aws_native import s3, iam
import json
from pulumi_databricks import MwsWorkspaces, MwsCredentials, MwsStorageConfigurations, MwsNetworks, get_aws_assume_role_policy, get_aws_cross_account_policy, ServicePrincipal, ServicePrincipalSecret
import pulumi_databricks as databricks
from jinja2 import Environment, FileSystemLoader
from pathlib import Path

db_account_id = config.require('db_account_id')

creds = MwsCredentials("my-db-credentials",  credentials_name="my-db-credentials",
# ARN of <https://docs.databricks.com/en/administration-guide/account-settings-e2/credentials.html#step-1-create-a-cross-account-iam-role> (already craeted)
                        role_arn=cross_account_role.arn, account_id=f"{db_account_id}")
storage_config = MwsStorageConfigurations("my-storage-config",
# bucket_databricks_root -> already created
                                           bucket_name=bucket_databricks_root.id,
                                           storage_configuration_name="my-storage-config", 
                                           account_id=f"{db_account_id}")

workspace = MwsWorkspaces("my-workspace", 
     account_id=f"{db_account_id}",
     aws_region=region,
     credentials_id=creds.credentials_id,
     storage_configuration_id=storage_config.storage_configuration_id,
     workspace_name="myname",)

ambitious-agent-35343

09/12/2023, 2:47 PM

This message contains interactive elements.

gorgeous-lunch-7514

09/13/2023, 9:23 AM

Hello comrades 🫡

gorgeous-lunch-7514

09/13/2023, 9:23 AM

Does anyone have a good example of adding AWS WAF or FMS (prefs FMS) to an AWS account via pulumi?

gorgeous-lunch-7514

09/13/2023, 9:23 AM

I’m resorting to reading terraform docs right now

sparse-caravan-37954

09/13/2023, 2:03 PM

Hello #aws, I am getting crazy in trying to use parameters in AWS job definition. A minimal example would look like this

Copy code

container_properties = {
        "jobRoleArn": batch_role.arn,
        "command": [
            'sh', 
            '-c',
            'echo Ref::param1'
        ],
        "image": image_uri,
        "memory": 1024*2,
        "vcpus": 1,
    }


    job_definition = aws.batch.JobDefinition("jobDefinition",
        type="container",
        container_properties=pulumi.Output.json_dumps(container_properties)
        ),
        parameters={
            "param1": "pippo",
        },
    )

When the job is run, however, the output is

Ref::param1

instead of

pippo

. I think the problem is in the fact that the container properties definition contains pulumi Output objects (

batch_role

and

image_uri

are resources defined before in the full code) and therefore I need the

pulumi.Output.json_dumps

story when using it in the job definition, and this is somehow avoiding the parameters to be picked up. Any idea? (edited)

gorgeous-lunch-7514

09/13/2023, 4:01 PM

Hey folks, I’m trying to access my EKS cluster’s OIDC Provider, however it’s showing up null for me. I can see it on the EKS dashboard however though. Any ideas?

Copy code

export const oidcProviderIdentity = cluster.core.oidcProvider?.id;
export const oidcProviderARN = cluster.core.oidcProvider?.arn;

gorgeous-lunch-7514

09/13/2023, 4:01 PM

Cluster was deployed a good while ago

gorgeous-lunch-7514

09/13/2023, 4:02 PM

We’re on k8s 1.27 if it’s any help 😕

gorgeous-lunch-7514

09/13/2023, 4:17 PM

I’m using eksCluster.identities atm, that seems to have something but idk if it’ll work for new deployments

gorgeous-lunch-7514

09/13/2023, 4:17 PM

Found the details I needed there

gorgeous-lunch-7514

09/14/2023, 1:34 PM

For others, I’m having to create an oidcProvider it smells like, this is what I’ve done so far

Copy code

// Get OIDC issuer identity.
const oidcProviderURL = cluster.eksCluster.identities.apply(identities => {
  return identities[0].oidcs[0].issuer.replace(/(^\w+:|^)\/\//, "").replace(/\/$/, "");
});

// Get AWS Account ID.
const awsAccountID = aws.getCallerIdentity().then(identity => identity.accountId);

// Get thumbprint of the OIDC issuer identity using certificate authority data.
const oidcProviderThumbprint = cluster.eksCluster.certificateAuthority.apply(ca => {
  const data = ca.data;
  const ascii = Buffer.from(data, "base64").toString("ascii");
  const thumbprint = crypto.createHash("sha1").update(ascii).digest("hex");
  return thumbprint;
});

// Create an IAM OIDC provider.
const oidcProvider = new aws.iam.OpenIdConnectProvider(`${projectName}-oidc-provider`, {
  clientIdLists: ["<http://sts.amazonaws.com|sts.amazonaws.com>"],
  url: oidcProviderURL.apply(url => `https://${url}`),
  thumbprintLists: [oidcProviderThumbprint],
});

gorgeous-lunch-7514

09/14/2023, 2:13 PM

It doesn’t seem to like sha1, could it be that the certificate from

eks.certAuth

and the certificate for the OIDC provider are different?

gorgeous-lunch-7514

09/14/2023, 2:17 PM

also trying

Copy code

// OpenID config
const oidcConfig = oidcProviderURL.apply(url => {
  return axios.get(`https://${url}/.well-known/openid-configuration`).then(response => response.data);
});

// Get thumbprint of the OIDC issuer identity using certificate authority data.
const oidcProviderThumbprint = oidcConfig.apply(config => {
  const parsedJwksUri = new URL(config.jwks_uri);
  // Use TLS to get hostname's SSL certificate.
  const tlsSocket = tls.connect(443, parsedJwksUri.hostname, { servername: parsedJwksUri.hostname });
  return new Promise((resolve, reject) => {
    tlsSocket.on("secureConnect", () => {
      const certificate = tlsSocket.getPeerCertificate();
      tlsSocket.end();
      console.log(certificate.fingerprint)
      const fingerprint = certificate.fingerprint.replace(/:/g, "").toUpperCase();
      resolve(fingerprint);
    });
    tlsSocket.on("error", (error) => {
      reject(error);
    });
  });
});

bulky-apple-458

09/14/2023, 7:12 PM

hey folks - trying to raise a PR to

pulumi/pulumi-aws-iam

and seeing issues related to misconfigured CI jobs. Let me know if there’s another channel for these kinds of requests 🙏 https://github.com/pulumi/pulumi-aws-iam/pull/17

kind-motorcycle-43615

09/18/2023, 12:27 PM

Hi All 👋 When we create a VPC using

Pulumi.Awsx.Ec2.Vpc

(Pulumi C#), Network ACLs and Route Tables with routes are automatically generated. (Suppose the VPC has

3 availability zones

, and each availability zone has a

public

and a

private

subnet.) I have a couple of questions: 1) How can we add new routes to the route table of the public subnet in availability-zone-1? AND how can we add new rules to the Network ACL of the public subnet in availability-zone-1? 2) Is that possible to access those automatically created resources like Network ACLs, Route Tables and modify them? OR Do we need to go with

Pulumi AWS Classic

package? Your input and guidance are highly valued. TYIA

stale-secretary-86178

09/18/2023, 2:37 PM

Hi, I am using crosswalk components to create infrastructure on AWS. Is it possible to import these components (mainly awsx.ec2.VPC) to another project?

wooden-egg-90698

09/18/2023, 6:55 PM

Hey folks, I've going around a recurring issue with a stack I have with s3 as a backend. Looks like for some reason when performing pulumi up -y from a codebuild pipeline the stack keeps trying to perform a replace over a file stack-name.json.gz to stack-name.json.gz.bak and those files aren´t present in the pulumi folder over in s3. Any ideas why would this happen?

able-machine-72645

09/18/2023, 10:00 PM

Hey all, running into this issue:

Copy code

Diagnostics:
  pulumi:pulumi:Stack (superstate-jon-dev):
    Error: invocation of aws:ec2/getVpc:getVpc returned an error: invoking aws:ec2/getVpc:getVpc: 1 error occurred:
    	* no matching EC2 VPC found
    : Error: invocation of aws:ec2/getVpc:getVpc returned an error: invoking aws:ec2/getVpc:getVpc: 1 error occurred:
    	* no matching EC2 VPC found
        at Object.callback (/snapshot/awsx/node_modules/@pulumi/pulumi/runtime/invoke.js:148:33)
        at Object.onReceiveStatus (/snapshot/awsx/node_modules/@grpc/grpc-js/src/client.ts:338:26)
        at Object.onReceiveStatus (/snapshot/awsx/node_modules/@grpc/grpc-js/src/client-interceptors.ts:426:34)
        at Object.onReceiveStatus (/snapshot/awsx/node_modules/@grpc/grpc-js/src/client-interceptors.ts:389:48)
        at /snapshot/awsx/node_modules/@grpc/grpc-js/src/call-stream.ts:276:24
        at processTicksAndRejections (node:internal/process/task_queues:78:11)

    error: Error: invocation of aws:ec2/getVpc:getVpc returned an error: invoking aws:ec2/getVpc:getVpc: 1 error occurred:
    	* no matching EC2 VPC found


        at Object.callback (/snapshot/awsx/node_modules/@pulumi/pulumi/runtime/invoke.js:148:33)
        at Object.onReceiveStatus (/snapshot/awsx/node_modules/@grpc/grpc-js/src/client.ts:338:26)
        at Object.onReceiveStatus (/snapshot/awsx/node_modules/@grpc/grpc-js/src/client-interceptors.ts:426:34)
        at Object.onReceiveStatus (/snapshot/awsx/node_modules/@grpc/grpc-js/src/client-interceptors.ts:389:48)
        at /snapshot/awsx/node_modules/@grpc/grpc-js/src/call-stream.ts:276:24
        at processTicksAndRejections (node:internal/process/task_queues:78:11)
    error: Running program '/Users/jon/Developer/superstate/devops/index.ts' failed with an unhandled exception:
    <ref *1> Error: failed to register new resource loadbalancer [awsx:lb:ApplicationLoadBalancer]: 2 UNKNOWN: invocation of aws:ec2/getVpc:getVpc returned an error: invoking aws:ec2/getVpc:getVpc: 1 error occurred:
    	* no matching EC2 VPC found


        at Object.registerResource (/Users/jon/Developer/superstate/devops/node_modules/@pulumi/runtime/resource.ts:421:27)
        at new Resource (/Users/jon/Developer/superstate/devops/node_modules/@pulumi/resource.ts:507:13)
        at new ComponentResource (/Users/jon/Developer/superstate/devops/node_modules/@pulumi/resource.ts:1011:9)
        at new ApplicationLoadBalancer (/Users/jon/Developer/superstate/devops/node_modules/@pulumi/lb/applicationLoadBalancer.ts:98:9)
        at Object.<anonymous> (/Users/jon/Developer/superstate/devops/index.ts:142:22)
        at Module._compile (node:internal/modules/cjs/loader:1241:14)
        at Module.m._compile (/Users/jon/Developer/superstate/devops/node_modules/ts-node/src/index.ts:439:23)
        at Module._extensions..js (node:internal/modules/cjs/loader:1295:10)
        at Object.require.extensions.<computed> [as .ts] (/Users/jon/Developer/superstate/devops/node_modules/ts-node/src/index.ts:442:12)
        at Module.load (node:internal/modules/cjs/loader:1091:32) {
      promise: Promise { <rejected> [Circular *1] }
    }

I’m trying to: 1. Create a VPC 2. Create an application load balance in that VPC

Copy code

const vpc = new awsx.ec2.Vpc("superstate-vpc", {});

const loadbalancerSecurityGroup = new aws.ec2.SecurityGroup("loadbalancerSecurityGroup", {
    vpcId: vpc.vpcId,
    ingress: [{
        fromPort: 0,
        toPort: 0,
        protocol: "-1",
        cidrBlocks: ["0.0.0.0/0"],
        ipv6CidrBlocks: ["::/0"],
    }],
    egress: [{
        fromPort: 0,
        toPort: 65535,
        protocol: "TCP",
        cidrBlocks: ["0.0.0.0/0"],
        ipv6CidrBlocks: ["::/0"],
    }],
});

// Create a load balancer to listen for requests and route them to the container.
const loadbalancer = new awsx.lb.ApplicationLoadBalancer("loadbalancer", {
    securityGroups: [ loadbalancerSecurityGroup.id ]
});

Any thoughts on what’s going wrong?

ambitious-salesmen-98185

09/19/2023, 12:12 AM

Any update with the pulumi degraded performance on aws? we've been having issues deploying code for most of today?

ambitious-agent-35343

09/19/2023, 9:53 AM

(Databricks) Now encountering problems even creating a SQL query:

Copy code

databricks:index:SqlQuery (q1):
    error: 1 error occurred:
        * cannot create sql query: Internal Server Error

The query -- I've tried numerous variants, including from docs:

Copy code

export const q1 = new databricks.SqlQuery("q1", {
  dataSourceId: sqlEndpoint.id,
  query: "SELECT 1",
  runAsRole: "owner",
  description: "Query 1",
  name: "q1",
  tags: ["t1", "t2"],
});

CC: @sparse-optician-70334

sparse-teacher-52109

09/19/2023, 3:15 PM

Hey all, anyone knows how to make pulumi deploy my stack to an entirely new AWS account? I created a stack with all resources, based on my current account. The goal was to switch to IaC as a way of life. Now, in order to test, my company set up a new account, and aims to run the stack on it - in order to test we are on par. I worked with the documentation (set a new env variables for access key and secret key of the new account), ran 'pulumi preview' and yet - I get 403, even though I'm sure the credentials I typed are correct Anyone knows how to make the switch?

able-machine-72645

09/19/2023, 8:11 PM

I added an ECS Fargate healthcheck to a new version of a task definition. The health check failed so the new tasks were never healthy, however, my old properly running task definitions were spun down before the new ones got healthy, which would have been downtime if we were live. How can I avoid this scenario? I want my new task to pass health checks before the old tasks are killed. This happened with

pulumi up