https://pulumi.com logo
#general
Title
# general
s

steep-beach-52652

11/12/2021, 12:08 PM
getting into this error
Copy code
AttributeError: module 'pulumi_aws.ecr' has no attribute 'get_credentails'
p

prehistoric-activity-61023

11/12/2021, 12:08 PM
typo?
shouldn’t it be
get_credentials
? 🙂
s

steep-beach-52652

11/12/2021, 12:09 PM
let me check
p

prehistoric-activity-61023

11/12/2021, 12:10 PM
not sure if that’s the root cause here but
credentails
is definitely a typo, so let’s rule this out first
s

steep-beach-52652

11/12/2021, 12:10 PM
ya
let me run again
Worked , thanks 😄
p

prehistoric-activity-61023

11/12/2021, 12:12 PM
four eyes see more than two 🙂
s

steep-beach-52652

11/12/2021, 12:13 PM
i think its now failing where im using the docker image name, shall i put some quotes around it
Copy code
container_definitions=json.dumps([{
      File "/usr/local/Cellar/python@3.9/3.9.0_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py", line 231, in dumps
        return _default_encoder.encode(obj)
      File "/usr/local/Cellar/python@3.9/3.9.0_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/encoder.py", line 199, in encode
        chunks = self.iterencode(o, _one_shot=True)
      File "/usr/local/Cellar/python@3.9/3.9.0_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/encoder.py", line 257, in iterencode
        return _iterencode(o, 0)
      File "/usr/local/Cellar/python@3.9/3.9.0_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/encoder.py", line 179, in default
        raise TypeError(f'Object of type {o.__class__.__name__} '
    TypeError: Object of type Output is not JSON serializable
so im follwing thise code to build.push the image then trying to use it inside the ecs task defintion
Copy code
# Build and publish our app's container image:

# ...1) Create a private ECR repository.
repo = aws.ecr.Repository('my-repo')
image_name = repo.repository_url

# ...2) Get registry info (creds and endpoint).
def get_registry_info(rid):
    """Get registry info (creds and endpoint)."""
    creds = aws.ecr.get_credentails(registry_id=rid)
    decoded = base64.b64decode(creds.authorization_token).decode()
    parts = decoded.split(':')
    if len(parts) != 2:
        raise Exception('Invalid credentials')
    return docker.ImageRegistry(creds.proxy_endpoint, parts[0], parts[1])
registry_info = repo.registry_id.apply(get_registry_info)

# ...3) Build and publish the container image.
image = docker.Image('my-app-image',
    build='app',
    image_name=image_name,
    registry=registry_info
)
Ah ok, maybe there is mismatch between quote types
No, its fine
but still that json dumps error
p

prehistoric-activity-61023

11/12/2021, 12:33 PM
is that the full definition of this JSON?
what are the types of
image.image_name
and
app_secret
?
If these things are Outputs (yes, there are, error says it explicitly:
Object of type Output is not JSON serializable
), you will have to use
apply
to construct a JSON.
you will need sth like (assumig that both
image_name
and
app_secret
are pulumi outputs):
Copy code
...
container_definitions=pulumi.Output.all(
   image.image_name, 
   app_secret,
).apply(lambda args: _create_json(image_name=args[0], app_secret=args[1])),
...
Copy code
def _create_json(image_name: str, app_secret: str) -> str:
   return json.dumps({
      ...
      "image": image_name,
      ...
   })
s

steep-beach-52652

11/12/2021, 12:57 PM
so i am creating an image and then need to use the image name inside container definition
if i use a hard coded string it works
if i do str(image.image_name), the error goes away
but im not sure if it will work
pulumi get stuck while applying that change
p

prehistoric-activity-61023

11/12/2021, 1:07 PM
but
image.image_name
is an
Output
class, right?
you can’t use it directly while constructing a string, using
str
function is no-go either
you have to get the value embedded in
Output
with
apply
method or using
pulumi.Output.all
(if you have more than one output)
s

steep-beach-52652

11/12/2021, 1:25 PM
still not working
becuase the result is also of type output
p

prehistoric-activity-61023

11/12/2021, 1:31 PM
can you share the API reference to the object that expects
container_definitions
?
also please share full code snippet when you’re creating this resource
s

steep-beach-52652

11/12/2021, 1:40 PM
yes, im using task defintion
p

prehistoric-activity-61023

11/12/2021, 1:40 PM
container_definitions
is marked as
Optional[pulumi.Input[str]]
so it should deal properly with
Output
type
(input is actually an union:
Input = Union[T, Awaitable[T], 'Output[T]']
)
s

steep-beach-52652

11/12/2021, 1:41 PM
Copy code
from pulumi import export, ResourceOptions
import pulumi_aws as aws
import pulumi_docker as docker
import json
import base64
from pulumi_aws.iam import policy
from pulumi import Output


# Build and publish our app's container image:
repo = aws.ecr.Repository('app-repo')
image_name = repo.repository_url

# Get registry info (creds and endpoint).
def get_registry_info(rid):
    creds = aws.ecr.get_credentials(registry_id=rid)
    decoded = base64.b64decode(creds.authorization_token).decode()
    parts = decoded.split(':')
    if len(parts) != 2:
        raise Exception('Invalid credentials')
    return docker.ImageRegistry(creds.proxy_endpoint, parts[0], parts[1])
registry_info = repo.registry_id.apply(get_registry_info)

# Build and publish the container image.
image = docker.Image('app-image',
    build='app',
    image_name=image_name,
    registry=registry_info
)

## Get Secret from Secret manager
app_secret = json.loads((aws.secretsmanager.get_secret_version("appsecret")).secret_string)["appsecret"]


# Create an ECS cluster to run a container-based service.
cluster = aws.ecs.Cluster('cluster')

# Read back the default VPC and public subnets, which we will use.
default_vpc = aws.ec2.get_vpc(default=True)
default_vpc_subnets = aws.ec2.get_subnet_ids(vpc_id=default_vpc.id)

# Create a SecurityGroup that permits HTTP ingress and unrestricted egress.
group = aws.ec2.SecurityGroup('web-secgrp',
	vpc_id=default_vpc.id,
	description='Enable HTTP access',
	ingress=[aws.ec2.SecurityGroupIngressArgs(
		protocol='tcp',
		from_port=80,
		to_port=80,
		cidr_blocks=['0.0.0.0/0'],
	)],
  	egress=[aws.ec2.SecurityGroupEgressArgs(
		protocol='-1',
		from_port=0,
		to_port=0,
		cidr_blocks=['0.0.0.0/0'],
	)],
)

# Create a load balancer to listen for HTTP traffic on port 80.
alb = aws.lb.LoadBalancer('app-lb',
	security_groups=[group.id],
	subnets=default_vpc_subnets.ids,
)

atg = aws.lb.TargetGroup('app-tg',
	port=80,
	protocol='HTTP',
	target_type='ip',
	vpc_id=default_vpc.id,
)

wl = aws.lb.Listener('web',
	load_balancer_arn=alb.arn,
	port=80,
	default_actions=[aws.lb.ListenerDefaultActionArgs(
		type='forward',
		target_group_arn=atg.arn,
	)],
)

# Create an IAM role that can be used by our service's task.

role = aws.iam.Role('task-exec-role',
	assume_role_policy=json.dumps({
		'Version': '2008-10-17',
		'Statement': [{
			'Sid': '',
			'Effect': 'Allow',
			'Principal': {
				'Service': '<http://ecs-tasks.amazonaws.com|ecs-tasks.amazonaws.com>'
			},
			'Action': 'sts:AssumeRole',
		}]
	}),
)

rpa = aws.iam.RolePolicyAttachment('task-exec-policy',
	role=role.name,
	policy_arn='arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy',
)


# Spin up a load balanced service running our container image.
task_definition = aws.ecs.TaskDefinition('app-task',
    family='fargate-task-definition',
    cpu='256',
    memory='512',
    network_mode='awsvpc',
    requires_compatibilities=['FARGATE'],
    execution_role_arn=role.arn,
    task_role_arn=role.arn,
    container_definitions=json.dumps([{
		'name': 'my-app',
		'image': str(image.image_name),
        'readonlyRootFilesystem': False,
        'environment': [
                {
                    'name': 'appsecret',
                    'value': app_secret
        }],
		'portMappings': [{
			'containerPort': 80,
			'hostPort': 80,
			'protocol': 'tcp'
		}]
	}])
)

service = aws.ecs.Service('app-svc',
	cluster=cluster.arn,
    desired_count=1,
    launch_type='FARGATE',
    task_definition=task_definition.arn,
    enable_execute_command=True,
    network_configuration=aws.ecs.ServiceNetworkConfigurationArgs(
		assign_public_ip=True,
		subnets=default_vpc_subnets.ids,
		security_groups=[group.id],
	),
    load_balancers=[aws.ecs.ServiceLoadBalancerArgs(
		target_group_arn=atg.arn,
		container_name='my-app',
		container_port=80,
	)],
    opts=ResourceOptions(depends_on=[wl]),
)

export('url', alb.dns_name)
export('image_base', image.base_image_name)
export('image_name', image.image_name)
p

prehistoric-activity-61023

11/12/2021, 1:42 PM
Can you paste the improved version based on my comments? That looks like the old one where you were trying to use
str
function.
s

steep-beach-52652

11/12/2021, 1:42 PM
it was the same error, so i change it back to str
p

prehistoric-activity-61023

11/12/2021, 1:42 PM
btw, does
app_secret = json.loads((aws.secretsmanager.get_secret_version("appsecret")).secret_string)["appsecret"]
works as you expect?
s

steep-beach-52652

11/12/2021, 1:43 PM
yes
if i provide a hard coded string like "nginx" to image then everything works
i can even exec into container and see as expected
but i need to run custom image from ecr
p

prehistoric-activity-61023

11/12/2021, 1:43 PM
again, can you show me how you used
pulumi.Outputs.all
so it didn’t work out for you?
s

steep-beach-52652

11/12/2021, 1:43 PM
not dockerhib
p

prehistoric-activity-61023

11/12/2021, 1:44 PM
and I’d like to double check that
app_secret
actually works (not that I don’t trust you but considering problems you describe, it might be worth double checking)
can you export it and see it actually contains the expected value?
Copy code
export('app_secret', app_secret)
s

steep-beach-52652

11/12/2021, 1:45 PM
yes
it works
the new change that im stuck with is using custom image inside task defnition
p

prehistoric-activity-61023

11/12/2021, 1:46 PM
cool
is
app_secret
of type
Output[str]
as well?
s

steep-beach-52652

11/12/2021, 1:47 PM
nope
its a simple string
p

prehistoric-activity-61023

11/12/2021, 1:48 PM
did you run sth like:
Copy code
task_definition = aws.ecs.TaskDefinition('app-task',
    family='fargate-task-definition',
    cpu='256',
    memory='512',
    network_mode='awsvpc',
    requires_compatibilities=['FARGATE'],
    execution_role_arn=role.arn,
    task_role_arn=role.arn,
    container_definitions=image.image_name.apply(lambda image_name: [{
		'name': 'my-app',
		'image': image_name,
        'readonlyRootFilesystem': False,
        'environment': [
                {
                    'name': 'appsecret',
                    'value': app_secret
        }],
		'portMappings': [{
			'containerPort': 80,
			'hostPort': 80,
			'protocol': 'tcp'
		}]
	}])
)
s

steep-beach-52652

11/12/2021, 1:49 PM
no, i did run apply on image name and then tried to use it
p

prehistoric-activity-61023

11/12/2021, 1:49 PM
Oh, I see. Thanks for the clarification.
You have to create the whole container_definition property like that.
s

steep-beach-52652

11/12/2021, 1:50 PM
oh
p

prehistoric-activity-61023

11/12/2021, 1:50 PM
It has to be either: • constant/hardcoded
str
Input[str]
Output[str]
if you want to have a “mix” of
Output[str]
with some additional hardcoded data, you need to construct it using
apply
(or
pulumi.Output.all
/
pulumi.Output.concat
)
s

steep-beach-52652

11/12/2021, 1:52 PM
Copy code
File "/Users/iahmad/sandbox/my-repo/venv/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 107, in _get_list_element_type
        raise AssertionError(f"Unexpected type. Expected 'list' got '{typ}'")
    AssertionError: Unexpected type. Expected 'list' got '<class 'str'>'
    error: an unhandled error occurred: Program exited with non-zero exit code: 1
 
Outputs:
  + image_base: "<http://379821691363.dkr.ecr.us-east-2.amazonaws.com/app-repo-96aa7ae|379821691363.dkr.ecr.us-east-2.amazonaws.com/app-repo-96aa7ae>"
  + image_name: "<http://379821691363.dkr.ecr.us-east-2.amazonaws.com/app-repo-96aa7ae:ee17cc979917a0f3f7a209b0193f12efea536d7b6eedea602bb37bf57768aa23|379821691363.dkr.ecr.us-east-2.amazonaws.com/app-repo-96aa7ae:ee17cc979917a0f3f7a209b0193f12efea536d7b6eedea602bb37bf57768aa23>"
p

prehistoric-activity-61023

11/12/2021, 1:53 PM
arr… I’m stupid
unless you changed my code appropriately
it’s supposed to be a JSON string value
it’s missing
json.dumps
in the lambda function
s

steep-beach-52652

11/12/2021, 1:54 PM
the image nmae?
p

prehistoric-activity-61023

11/12/2021, 1:54 PM
you had it before
Copy code
task_definition = aws.ecs.TaskDefinition('app-task',
    family='fargate-task-definition',
    cpu='256',
    memory='512',
    network_mode='awsvpc',
    requires_compatibilities=['FARGATE'],
    execution_role_arn=role.arn,
    task_role_arn=role.arn,
    container_definitions=image.image_name.apply(lambda image_name: json.dumps([{
		'name': 'my-app',
		'image': image_name,
        'readonlyRootFilesystem': False,
        'environment': [
                {
                    'name': 'appsecret',
                    'value': app_secret
        }],
		'portMappings': [{
			'containerPort': 80,
			'hostPort': 80,
			'protocol': 'tcp'
		}]
	}]))
)
s

steep-beach-52652

11/12/2021, 1:55 PM
ah , i see
no python error but pulumi up stucks as before
not going forward
Copy code
Resources:
    ~ 1 to update
    +-1 to replace
    2 changes. 10 unchanged

Do you want to perform this update? yes
Updating (dev)

View Live: <https://app.pulumi.com/iahmad-khan/aws-py-fargate/dev/updates/6>

   Type  Name  Status
how to run it in debug mode
ok let me try debug mode
this is working , thanks
🙌 1
352 Views