pulumi-community #aws

aws

better-agent-48831

04/19/2021, 7:39 AM

Hello, I am trying to create some resource inside and RDS instance. I have one project that creates the RDS Instance itself and a second project where I would like to create the databases / users and other mysql resources. The RDS instance is inside a VPC. I am assuming this is the reason why my simple usage of the

mysql.Provider

is timing out - the endpoint is not accessible publicly. I saw similar question here: https://pulumi-community.slack.com/archives/CRH5ENVDX/p1615361938255800?thread_ts=1615323683.244900&cid=CRH5ENVDX but I am not sure how to set up the "provision machine" and run the mysql setup form there wil pulumi. Is there some concrete example I could look at? I am assuming this is a common scenario

brave-planet-10645

04/19/2021, 7:44 AM

I think that user is referring to the machine that you're running Pulumi on as "a provisioner machine". Basically, you need to either run Pulumi from inside the VPC if it's impossible to interact with the RDS instance from outside. tl;dr The machine running Pulumi needs access to the RDS instance if it's going to interact with it

ambitious-father-68746

04/19/2021, 10:55 AM

Oh how I wish AWS had a mechanism similar to EC2's "user data", but for RDS

agreeable-ram-97887

04/19/2021, 11:22 AM

How does one decrypt the “.encryped_password” output of a UserLoginProfile resource? 🤔

enough-leather-70274

04/19/2021, 11:43 AM

I added

storage_encrypted=True

to an Aurora cluster already managed via pulumi, expecting it would just re-create the cluster. However it tried to "replace" it before deleting the old one (and fell over with an error from AWS). In an attempted workaround, I deleted the instance and cluster in the console, and then tried to delete the instance manally in pulumi, but this fails with an error:

pulumi state delete urn:pulumi:<redacted-cluster-instance>
error: No such resource "urn:pulumi:<redacted-cluster-instance>" exists in the current state

However when I run --show-urns, I can clearly see the state exists. Indeed if I run destroy on the stack, i now get:

error: deleting urn:pulumi:<redacted-cluster-instance>: 1 error occurred:
    	* error deleting Database Instance "<redacted-cluster-instance>": DBInstanceNotFound: DBInstance <redacted> not found.

best-lifeguard-91445

04/19/2021, 3:59 PM

is it possible to associate a Network Load balancer to a ECS container port, and an application load balancer to another ECS container port?

green-musician-49057

04/19/2021, 8:43 PM

EDIT: Solved ✨ -- it was most likely due to AWS api constraints. I'm seeing an issue where the docs may be out of date with the provider, for a Route53 record. https://www.pulumi.com/docs/reference/pkg/aws/route53/record/#properties

* provider.aws: aws_route53_record: join: "ttl": required field is not set

This issue didn't show up with

pulumi preview

, but it did on a

pulumi up

adamant-translator-31969

04/19/2021, 9:35 PM

Hi everyone! I have a problem with getRouteTable method ... I have this error

Your query returned no results. Please change your search criteria and try again

kind-airport-89906

04/20/2021, 3:19 PM

Hi Everyone, i’m having difficulties creating a nodegroup to attach to an ECS cluster. I’ve created a github ticket with more details. https://github.com/pulumi/pulumi-eks/issues/566 I’m getting the following error when running `pulumi up`:

Diagnostics:
  pulumi:pulumi:Stack (proto-pulumi-andrew-dev):
    error: program failed: marshaling properties: awaiting input property cluster: missing one or more required arguments
    exit status 1

Code snippet is in the github issue. Thanks!

powerful-furniture-83753

04/21/2021, 12:12 PM

Hey folks. I'm having some issues trying to access an aurora postgres instance from outside of my VPC (ie, my local machine). The ECS cluster i have running in the VPC can access it without issue. I've got the following setup:

const vpc = new awsx.ec2.Vpc(`vpc`, {});

const rdsSecurityGroup = new aws.ec2.SecurityGroup(`sg`, {
  vpcId: vpc.id,
  ingress: [
    {
      protocol: "tcp",
      fromPort: 5432,
      toPort: 5432,
      cidrBlocks: [vpc.vpc.cidrBlock],
    },
  ],
});

const dbSubnets = new aws.rds.SubnetGroup(`rds-subnet`, {
  subnetIds: vpc.privateSubnetIds,
});

const db = new aws.rds.Cluster(`db`, {
  engineMode: "serverless",
  dbSubnetGroupName: dbSubnets.id,
  vpcSecurityGroupIds: [rdsSecurityGroup.id],
  engine: "aurora-postgresql",
  // .... password/name/db etc
});

And I have manually added an extra inbound rule in AWS on the security group in this code to the created security group to allow access from my IP. What am i missing?

wooden-student-58350

04/21/2021, 2:37 PM

Hi all, I have a Pulumi project using a custom S3 backend on a shared bastion AWS account, but I want each stack to deploy resources on a different “child” AWS account (i.e. dev, staging, and prod each have their own AWS account). How can I configure my project/stacks/env to work with this setup?

Project: infra 				// backend on aws-bastion
 |- Stack: development	    // deploys resources to aws-dev
 |- Stack: staging			// deploys resources to aws-staging
 |- Stack: production		// deploys resources to aws-prod

# Pulumi.yaml
name: infra
runtime: nodejs
backend:
 url: <s3://my-pulumi-backend>

# Pulumi.development.yaml
secretsprovider: <awskms://alias/development/pulumi-secrets-key?region=eu-west-2>
encryptedkey: …
config:
 aws:accessKey: <dev user access key ID>
 aws:allowedAccountIds:
 - <dev AWS account ID>
 aws:region: eu-west-2

# .env
AWS_SECRET_ACCESS_KEY=<development user secret access key>

The problem is that Pulumi only looks for one set of AWS credentials for everything. So if I want my stack to use my dev access key in the program to build resources in my dev AWS account, Pulumi can’t access the backend S3 bucket on the bastion AWS account (and vice versa). I think I could create a custom AWS provider in code based on the stack name, but then wouldn’t I have to manually specify that custom provider on every individual resource in the program? Is there a way to change the default provider on a per-stack basis while still allowing the project to use an S3 backend on a different account?

kind-airport-89906

04/21/2021, 2:40 PM

Hey Everyone, I’m wondering if anyone has successfully created an EKS nodegroup attached to a cluster using the EKS Go module? I’m getting the following somewhat obscure error message:

error: an unhandled error occurred: program exited with non-zero exit code: 1

    error: program failed: marshaling properties: awaiting input property cluster: cannot marshal an input of type pulumi.StringOutput with element type string as a value of type pulumi.StringOutput
    exit status 1

Code Snippet:

_, err = eks.NewManagedNodeGroup(ctx, "managed", &eks.ManagedNodeGroupArgs{
      Cluster: cluster.Core,
      NodeGroupName: pulumi.String("managed-test"),
    })

I get the same error on both the

NewManagedNodeGroup

and

NewNodeGroup

resources. And if i remove the nodegroup resource the rest of the stack works fine

full-artist-27215

04/21/2021, 3:31 PM

After I upgraded to Pulumi 3.0, and using

pulumi_aws

4.0.0, it looks like the DynamoDB Table resource no longer has

hash_key

as an Output. Has anyone else run into this?

breezy-garden-27511

04/21/2021, 4:18 PM

👋 I’m here! What’d I miss?

👋 1

shy-house-53993

04/22/2021, 7:35 AM

Morning (UTC+1)! Here's a Crosswalk question for you. Given that it's supposed to be maximally easy to use for "day one" tasks why doesn't the following joyfully simple code:

import * as aws from '@pulumi/aws';

aws.cloudwatch.onSchedule('log-something', 'rate(5 minutes)', (e) => {
  console.log(`Hello from Scheduler.`);
});

attach the

AWSLambdaBasicExecutionRole

policy by default. Without that basic logging doesn't even function.

bumpy-laptop-30846

04/22/2021, 1:10 PM

Hi, Is there a solution to use

const elb =  aws.elb.getLoadBalancer( { name: elbName });

with elbName being an output. Is there a way around? Otherwise I don’t see how to address my use case, which I don’t develop here…

elegant-pager-5412

04/22/2021, 1:40 PM

How do I create a dead letter queue and assign it to another queue?

elegant-pager-5412

04/22/2021, 1:40 PM

Can’t find an example

elegant-pager-5412

04/22/2021, 2:45 PM

Guys, how can I send an event to a queue from a lambda?

elegant-pager-5412

04/22/2021, 2:46 PM

I have

queueA

and

queueB

, I want that when a message is processed from

queueA

to push a new message to

queueB

elegant-pager-5412

04/22/2021, 3:24 PM

How do I inject values (such as ARN) into a lambda? By default the value is an

Output

abundant-portugal-51177

04/22/2021, 8:55 PM

Hello people. I am wondering if someone can help me enable logging for an AWS API gateway (via code and not the aws console). I havent been able to set the required account setting using the MethodSettings approach I pulled from a terraform example:

abundant-portugal-51177

04/22/2021, 8:56 PM

abundant-portugal-51177

04/22/2021, 8:57 PM

where my API is called called

api

abundant-portugal-51177

04/22/2021, 8:59 PM

updating API Gateway Stage failed: BadRequestException: CloudWatch Logs role ARN must be set in account settings to enable logging

purple-orange-91853

04/22/2021, 9:25 PM

I am trying to set up Pulumi to work with AWS role switching. We utilize several AWS accounts tied back to a single AWS identity account and we don't have users defined in any other account but our identity account. I've got my aws cli configured and I can specify a profile and it connects to the correct role_arn and account. However when trying to leverage this functionality in Pulumi it errors out with either

error configuring Terraform AWS Provider: Error creating AWS session: SharedConfigAssumeRoleError: failed to load assume role for arn:aws:iam::ACCTNUM:role/ROLE, source profile dev_pulumi has no shared credentials

error configuring Terraform AWS Provider: Error creating AWS session: CredentialRequiresARNError: credential type source_profile requires role_arn, profile dev-pulumi

My aws cli config

.aws/credentials
[default]
aws_access_key_id = redacted
aws_secret_access_key = redacted

[prod-main]
aws_access_key_id = redacted
aws_secret_access_key = redacted

[ecr-dev]
aws_access_key_id = redacted
aws_secret_access_key = redacted

[dev-pulumi]
aws_access_key_id = redacted
aws_secret_access_key = redacted

[default]
region = us-east-1

.aws/config
[profile prod]
source_profile = prod-main
role_arn = arn:aws:iam::redacted:role/redacted-TEMP
mfa_serial = arn:aws:iam::redacted:mfa/tony

[profile dev]
source_profile = prod-main
role_arn = arn:aws:iam::redacted:role/redacted-TEMP
mfa_serial = arn:aws:iam::redacted:mfa/tony

[profile eks-dev]
role_arn = arn:aws:iam::redacted:role/redacted-eksRole-role-redacted
source_profile = prod-main
mfa_serial = arn:aws:iam::redacted:mfa/tony

[profile ecr-dev]
source_profile = ecr-dev

[profile dev-pulumi]
source_profile = dev_pulumi
role_arn = arn:aws:iam::redacted:role/redacted-TEMP

I am currently just getting started with Pulumi so I am just following the basic steps outlined here: https://www.pulumi.com/docs/get-started/aws/begin/.

purple-plumber-90981

04/22/2021, 10:22 PM

so i think something is borked in my aws setup. the symptom i am seeing is that

pulumi up

generates the list of stuff to preview but hangs before allowing me to apply….. i will add to thread the debug log of something that seems related and is presenting

[aws-sdk-go] DEBUG: Validate Response iam/GetUser failed, attempt 0/25, error ValidationError: Must specify userName when calling with non-User credentials

sparse-intern-71089

04/23/2021, 10:39 AM

This message was deleted.

elegant-pager-5412

04/23/2021, 12:10 PM

I have a few general questions about AWS using Lambda and SQS: 1. If I have an SQS queue with hundreds of thousands of messages in it with a lambda listening to the queue, how many instances of the same Lambda will AWS spawn? Is there a way I can limit it? 2. If I have a FIFO SQS queue with multiple groups, will each group be processed by a different lambda instance? For example, I have

groupA

and

groupB

, will AWS automatically spawn multiple lambda instances and process

groupA

messages in parallel to

groupB

? 3. For how long will an SQS message processing will be attempted before it gets discarded (or sent to a DLQ) when it fails? 4. Is there a way to send a message that failed to process back to the queue with a delay of few seconds/minutes before it’s being retried? Also, is it possible to get the retry number in the lambda itself? Thanks in advance!

elegant-pager-5412

04/23/2021, 1:40 PM

If my lambda pulls 10 messages as a batch size and only a single record is failing to process. Is there any way to tell lambda to acknowledge the 9 successful messages but only fail who actually failed?