pulumi-community #aws

aws

happy-pencil-64085

10/15/2020, 2:43 PM

I was trying to create an athena named query, but when I go to Athena after doing pulumi up, I don't see my query out there

hundreds-receptionist-31352

10/15/2020, 3:20 PM

Hi , I'm creating a vpc using the module awsx.ec2.Vpc, is there any way to create tags to the routetables that have been created? I see that is possible to add tags to the subnets but don't find the way to add these tags to the routetables

breezy-helmet-47710

10/15/2020, 4:39 PM

Hi Channel. I have been trying to configure an Application Load Balancer to route traffic to target groups and have been unable to set it up successfully. I am able to upload a service and access it through the private IP on my default VPC from an ec2 instance that I have running for my development server, however for some reason the target group that I set up is not able to register the IP:PORT target successfully. I have been trying to set up app for a while now with no success so I would like to verify if my Pulumi setup is correct, or if it is a problem with AWS. I have not been able to find any examples of an application load balancer setup with multiple target groups online so this is the best I could do from the documentation and the basic hello world examples. The code can be found here: https://www.codepile.net/pile/eVpKZPm6. Thanks!

quick-apartment-308

10/15/2020, 5:33 PM

Hi all, I made a minor change to a stack this morning and got this error:

At least one field is expected inside environment

I only found one reference to this error. It's terraform related, but I'm not sure how it's resolved with Pulumi. The stack was working yesterday and the error is still returned after reverting the code. No changes were made to infrastructure code.

gifted-vase-28337

10/15/2020, 10:16 PM

Can anyone comment on their experiences using Control Tower? Currently we have an account factory in pulumi that we can use to create new sandbox accounts. I'm in favor of keeping it in pulumi (vs. CloudFormation templates in the Control Tower account factory). Control Tower doesn't have an API, so it can't be described/reproduced readily using pulumi.

quaint-electrician-41503

10/16/2020, 10:18 PM

I am trying to change my Role Policy Attachemnts but got a Error: DeleteConflict: Cannot delete a policy attached to entities. Can we surround these with a try / catch block?

average-kilobyte-47828

10/17/2020, 9:58 PM

aws and awsx namespaces... I'm defining container registries and there are 2 ecr.Repository classes, one from aws and one from awsx, each taking a subset of possible arguments. From awsx I can get lifecyclepolicy and from aws I can have imageScanningConfiguration and imageTagImmutability but not both. Am I doing something stupid?

blue-morning-55097

10/18/2020, 5:15 PM

hi everyone - i've setup an application version for elastic beanstalk called "default", with fields application, bucket, and key. when i try and deploy however, it says that i am missing a required field. the docs say i should be good, any ideas?

average-kilobyte-47828

10/18/2020, 5:48 PM

Getting an error here and the message looks truncated:

Diagnostics:
  aws:route53:Record (acme-wildcard-record):
    error: aws:route53/record:Record resource 'acme-wildcard-record' has a problem: ConflictsWith
    error: aws:route53/record:Record resource 'acme-wildcard-record' has a problem: ConflictsWith

Untitled

victorious-helmet-11068

10/19/2020, 12:01 PM

hi all. I tried several way to ignore tag: at provider level or at yaml config level. No way. May u suggest something?

victorious-helmet-11068

10/19/2020, 12:01 PM

at the moment I have:

victorious-helmet-11068

10/19/2020, 12:01 PM

Untitled.yaml

little-cartoon-10569

10/19/2020, 8:18 PM

I'm updating the security groups on an EC2 instance I've just created. I'm getting a panic in the Go SDK.

Exception

dazzling-sundown-39670

10/19/2020, 10:02 PM

Any examples with NodeGroups using spotPricing?

crooked-appointment-9302

10/20/2020, 7:24 AM

Hi all. How do I set up a scheduled task in ECS? Can’t find any trace of it in the docs or via Google.

cuddly-dusk-95227

10/20/2020, 11:46 AM

question about EKS; it seems that creating clusters with

eks.NewCluster

results in the cluster name having a random suffix attached. This might be fine for creation, but it also applies to imports, leaving a situation where an existing cluster can never be imported because the source name and the imported resource name is different.

cuddly-dusk-95227

10/20/2020, 11:48 AM

Do you want to perform this update? details
+ pulumi:pulumi:Stack: (create)
    [urn=urn:pulumi:mycluster::eks-cluster::pulumi:pulumi:Stack::eks-cluster-mycluster]
    = aws:eks/cluster:Cluster: (import)
        [id=mycluster]
        [urn=urn:pulumi:mycluster::eks-cluster::aws:eks/cluster:Cluster::mycluster]
      ~ name: "mycluster" => "mycluster-c91b26c"

broad-church-78931

10/20/2020, 11:50 AM

Hi! When people deploy stuff like containers on Fargate ECS etc. with Pulumi, how you generally automate the database migrations and similar that should happen at the same time when the new version is deployed? Does Pulumi have some recommended/supported way of doing this.

calm-greece-42329

10/20/2020, 6:39 PM

is anyone else interested in developing a

@pulumi/awsx

solution to address managing step functions? i’m thinking something like https://github.com/aws/aws-cdk/tree/master/packages/%40aws-cdk/aws-stepfunctions where the library is able to generate the state definition and check for validity, manage roles, and things like that.

astonishing-quill-88807

10/20/2020, 8:12 PM

Does anyone have an example of defining a Fargate deployment in Python? I'm seeing an error with the TaskDefinition resource and setting the

container_definitions

array. It says that the attribute needs to be an array when I set it to a string via

json.dumps

and says it needs to be a single value when I set it to a Python list... Here's my code as it stands right now

sign_and_verify_task = ecs.TaskDefinition(
    f'sign-and-verify-task-{env_suffix}',
    cpu='0.25',
    memory='500',
    network_mode='awsvpc',
    pid_mode='task',
    requires_compatibilities='FARGATE',
    tags=aws_config.merged_tags({'Name': f'sign-and-verify-{env_suffix}'}),
    execution_role_arn=sign_and_verify_task_execution_role.arn,
    family=f'sign-and-verify-task-{env_suffix}',
    container_definitions=[
        {
            'name': 'sign-and-verify',
            'image': f'mitodl/sign-and-verify:{sign_and_verify_config.require("docker_label")}',
            'environment': [
                {'name': 'PORT', 'value': '5000'}
            ],
            'secrets': [
                {'name': 'UNLOCKED_DID', 'valueFrom': unlocked_did_secret.arn}
            ]
        }
    ],
    ipc_mode='task',
)

astonishing-quill-88807

10/20/2020, 8:32 PM

So, the issue was actually that the

requires_compatibilities

attribute is a string in my code and needs to be a list. 🤦‍♂️

salmon-ghost-86211

10/22/2020, 4:25 PM

Here is a trimmed down snippet of creating an EKS cluster and associated nodegroup.

import * as eks from "@pulumi/eks";
const cluster = new eks.Cluster(...)
const clusterNodeGroup1 = cluster.createNodeGroup(...)

I want to adjust

enabledMetrics

and

suspendedProcesses

on the ASG, but I'm not sure how to access it. I think I can get the name from

clusterNodeGroup1.apply(ng => ng.autoScalingGroupName)

. I can also use the autoscaling getGroup function. How do I get an existing object and then update properties?

kind-school-28825

10/23/2020, 6:19 AM

STACK_TRACE:
    Error:
        at Object.debuggablePromise (/Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/node_modules/@pulumi/pulumi/runtime/debuggable.js:69:75)
        at Object.registerResource (/Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/node_modules/@pulumi/pulumi/runtime/resource.js:132:18)
        at new Resource (/Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/node_modules/@pulumi/pulumi/resource.js:211:24)
        at new CustomResource (/Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/node_modules/@pulumi/pulumi/resource.js:303:9)
        at new TaskDefinition (/Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/node_modules/@pulumi/ecs/taskDefinition.ts:203:9)
        at new TaskDefinition (/Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/node_modules/@pulumi/ecs/taskDefinition.ts:80:31)
        at new FargateTaskDefinition (/Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/node_modules/@pulumi/ecs/fargateService.ts:48:9)
        at new FargateService (/Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/node_modules/@pulumi/ecs/fargateService.ts:206:13)
        at Object.<anonymous> (/Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/ecs/index.js:14:22)
        at Module._compile (internal/modules/cjs/loader.js:1133:30)
 
    error: Running program '/Users/qalbaqali/Documents/Personal/rawag-v2/core/infra' failed with an unhandled exception:
    TypeError: Cannot read property 'lastIndexOf' of undefined
        at Object.getImageNameAndTag (/Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/node_modules/@pulumi/utils.ts:25:37)
        at checkRepositoryUrl (/Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/node_modules/@pulumi/docker.ts:190:27)
        at /Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/node_modules/@pulumi/docker.ts:230:5
        at Generator.next (<anonymous>)
        at /Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/node_modules/@pulumi/docker/docker.js:21:71
        at new Promise (<anonymous>)
        at __awaiter (/Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/node_modules/@pulumi/docker/docker.js:17:12)
        at buildAndPushImageWorkerAsync (/Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/node_modules/@pulumi/docker/docker.js:133:12)
        at Object.<anonymous> (/Users/qalbaqali/Documents/Personal/rawag-v2/core/infra/node_modules/@pulumi/docker.ts:172:34)
        at Generator.next (<anonymous>)

Tracking the origin of this error using the stack trace (to my understanding) is because when pulumi is trying to build the image, it's crashing when it's trying to get the Docker image name

Cannot read property 'lastIndexOf' of undefined

which is in the

@pulumi/docker/utils.js

https://github.com/pulumi/pulumi-docker/blob/master/sdk/nodejs/utils.ts#L15 This is all running in github actions CI. Might be related to this github issue: https://github.com/pulumi/pulumi/issues/3626 @faint-table-42725 Deleting all the resources, and creating them again leads to the same error

kind-school-28825

10/23/2020, 6:19 AM

I am trying to push an image to ECR and use the image for a fargate service. My project structure is like so

+ infra/
  ++ ecr/
   +++ index.js <-- ecr.Repository & ecs.Image.fromPath
  ++ ecs/
   +++ index.js <-- ecs.FargateService
  ++ index.js <-- main entry point
+ src/
+ Dockerfile

infra/ecr/index.js

var pulumi = require("@pulumi/pulumi");
var awsx = require("@pulumi/awsx");

var ecrRepo = new awsx.ecr.Repository("core", {
  tags: {
    Name: `core ${pulumi.getStack()} repository`,
  },
});
var ecrRepoUrl = ecrRepo.repository.repositoryUrl;

var dockerImage = awsx.ecs.Image.fromPath(ecrRepo, "../../");

module.exports = { ecrRepoUrl, dockerImage };

infra/ecs/index.js

var awsx = require("@pulumi/awsx");

var { loadBalancerListener, securityGroup, vpc } = require("../network");
var { atlasCluster } = require("../mongodb");
var { dockerImage } = require("../ecr");

var { getEnvironmentVariables } = require("./helpers");

var ecsCluster = new awsx.ecs.Cluster("core-cluster", {
  vpc,
  securityGroups: [securityGroup],
});

var fargateService = new awsx.ecs.FargateService("core-fgs", {
  ecsCluster,
  taskDefinitionArgs: {
    container: {
      image: dockerImage,
      cpu: 202,
      memory: 1024,
      portMappings: [loadBalancerListener],
      healthCheck: {
        command: ["CMD-SHELL", "curl -f <http://localhost:3000/> || exit 1"],
        startPeriod: 10,
        retries: 3,
      },
      environment: [
        ...getEnvironmentVariables(),
        {
          name: "DB_CONNECTION_URL",
          value:
            process.env.DB_CONNECTION_URL ||
            atlasCluster.connectionStrings.standardSrv,
        },
      ],
    },
  },
  desiredCount: 1,
});

module.exports = { ecsCluster, fargateService };

infra/index.js

var { ecrRepoUrl, dockerImage } = require("./ecr");
var { fargateService } = require("./ecs");

module.exports = {
  ecrRepositoryUrl: ecrRepoUrl,
  image: dockerImage.environment,
  fargateService: fargateService.urn,
};

When running pulumi I get the following error

salmon-ghost-86211

10/23/2020, 1:30 PM

I'm going to restate my question from above because it might have been confusing... Using Typescript and

import * as eks from "@pulumi/eks";

but can also import others like aws/awsx. If I create an EKS cluster and then create a node group with the cluster's

createNodeGroup

function, it generates an Auto Scaling Group (ASG) to manage the number of instances in the cluster. I would like to set some properties of the ASG such as enabledMetrics, but I am uncertain how to access the ASG that got created. The node group does have a property named

autoScalingGroupName

, so I should be able to access it as an output but I am not sure how.

adamant-dress-73325

10/23/2020, 5:10 PM

We’re trying to move the db configuration for a service, into pulumi. Our issue is that in order to configure a DB you’ll need to connect to it. When the pulumi client is running on a machine with access, that’s no problem, like through a VPN. But in Github actions CI we want to also deploy, but the runners don’t have access. Now we can seperate this step into a seperate stack like qa-db-setup to be run manually, for example. But I’m wondering if there is a nicer pattern for managing RDS database setup with the pulumi postgres provider, to only run in certain envs but not the others?

crooked-knife-92853

10/26/2020, 1:30 PM

Hi there! I’ve been working with ec2 autoscaling groups recently and noticed a discrepancy between the documentation and implementation of ASG metrics. The docs for

enabledMetrics

have a list of allowed values corresponding to metrics that we can collect for the ASG; however, if we look at the implementation of the Metrics class some are clearly missing (ex:

GroupInServiceCapacity

GroupPendingCapacity

, etc, etc). My question: is there somewhere I can/should report this issue to get it fixed? Like should I create a GH issue? Or may I submit a PR to remedy it? Wondering what you all think of this

adamant-translator-31969

10/26/2020, 1:49 PM

Hi! Could I import cluster with @pulumi/eks library? i tried this but it was not possible

dazzling-sundown-39670

10/26/2020, 5:52 PM

Trying to create a user and a bucket where the user has been granted permission on the bucket but I'm getting this error:

Error putting S3 Grants: InvalidArgument: Invalid id

. I've been looking at this: https://www.pulumi.com/docs/reference/pkg/aws/s3/bucket/#using-acl-policy-grants Any suggestions? Code in comments ➡️

breezy-butcher-78604

10/27/2020, 6:50 AM

I've got an auto-scaling group set up in a pulumi template that also uses a launch configuration in the same template. when i update the launch configuration (eg when the AMI needs to change), it does trigger a subsequent update in the autoscaling group however the instances in that group don't change. i imagine this is due to the underlying terraform provider not supporting it (there's an open issue to resolve it byt triggering an instance refresh). what would be the best way to work around this in Pulumi? basically i want to trigger a new instance refresh whenever the launch configuration changes.

Title

breezy-butcher-78604

10/27/2020, 6:50 AM

for some more context, in CloudFormation, any update to the

AWS::AutoScaling::AutoScalingGroup

resource also orchestrates an instance refresh and waits until it completes (or times out). https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-as-group.html

billowy-army-68599

10/27/2020, 2:38 PM

this isn't exposed as an API option, rolling updates are only supported in CloudFormation. You can work around it by defining a cloudformation template in pulumi

salmon-ghost-86211

10/27/2020, 8:01 PM

The default behavior of just updating a Launch Configuration is that any new instance will use the new launch config. There isn't an instance replacement ever since only the config is updated. I think @billowy-army-68599 is correct in that you would need to use pulumi to automate cloudformation instead if you want that behavior.

breezy-butcher-78604

10/28/2020, 1:43 AM

the instance refresh action is an api endpoint (https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_StartInstanceRefresh.html) but i understand the support for it isn't quite there yet. what i was hoping to get is some suggestions on how I could orchestrate this myself via Pulumi. defining a cloudformation template in pulumi is one option i hadn't considered, so thanks, i'll have a think about that. what I was hoping to try and do is programatically within a template be able to identify if the launch configuration has changed, and if so fire off an instance refresh API call. i assume this might be something doable with

pulumi.dynamic.ResourceProvider

perhaps but i don't have much experience with those yet

am i on the right track?

billowy-army-68599

10/28/2020, 1:53 AM

That could definitely work, and a dynamic provider is indeed the way to go. If you get it working please let me know

👍 1

breezy-butcher-78604

10/29/2020, 7:08 AM

hey @billowy-army-68599 i ended getting something basic working (see attached, let me know if you've got any feedback) however in the end i decided to not pursue this. i discovered a couple of annoying issues with the "Instance Refresh" processes, namely it will terminate instances before creating new instances, and it only ever replaces one instance at a time (meaning replacements are really long for large groups or when instances have long bootstrap times). instead I went with your suggestion - creating a CloudFormation stack with just the autoscaling group in it. It allows me to use the same CloudFormation update functionality im used to while still using all the advantages of Pulumi.

instance-refresh.ts

View count: 1