salmon-motherboard-78006
10/31/2022, 7:45 PMaws:mwaa:Environment (dev-aqua-airflow):
error: 1 error occurred:
* error creating MWAA Environment: ValidationException: Failed to assume role arn:aws:iam::<account_id>:role/dev-airflow-execution-role. This could be due to the role's trust policy. Please ensure your role is assumable by '<http://airflow-env.amazonaws.com|airflow-env.amazonaws.com>' Service Principal and try again.
And this is what my execution role looks like:
mwaa_execution_role = aws.iam.Role(f"{stack}-airflow-execution-role",
name=f"{stack}-airflow-execution-role",
assume_role_policy=json.dumps({
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": [
"<http://airflow.amazonaws.com|airflow.amazonaws.com>",
"<http://airflow-env.amazonaws.com|airflow-env.amazonaws.com>"
],
},
"Effect": "Allow",
},
]
}))
I then decided to create the S3 bucket, role and policies before creating the MWAA Environment and this is the error I’m getting now:
aws:mwaa:Environment (dev-aqua-airflow):
error: 1 error occurred:
* creating urn:pulumi:dev::data-ml-airflow::aws:mwaa/environment:Environment::dev-aqua-airflow: 1 error occurred:
* error waiting for MWAA Environment (dev-aqua-airflow-dd6bc3e) creation: unexpected state 'CREATE_FAILED', wanted target 'AVAILABLE'. last error: %!s(<nil>)
When I go to the AWS Console, this is the error I see:
Error code
INCORRECT_CONFIGURATION
Message
You may need to check the execution role permissions policy for your environment, and that each of the VPC networking components required by the environment are configured to allow traffic. Troubleshooting: <https://docs.aws.amazon.com/mwaa/latest/userguide/troubleshooting.html>
And this is my Pulumi MWAA code:
airflow_env = aws.mwaa.Environment(f"{stack}-aqua-airflow",
dag_s3_path="dags/",
execution_role_arn=mwaa_execution_role.arn,
airflow_version='2.2.2',
kms_key=mwaa_kms_key.arn,
logging_configuration=aws.mwaa.EnvironmentLoggingConfigurationArgs(
dag_processing_logs=aws.mwaa.EnvironmentLoggingConfigurationDagProcessingLogsArgs(
enabled=True,
log_level="DEBUG",
),
scheduler_logs=aws.mwaa.EnvironmentLoggingConfigurationSchedulerLogsArgs(
enabled=True,
log_level="INFO",
),
task_logs=aws.mwaa.EnvironmentLoggingConfigurationTaskLogsArgs(
enabled=True,
log_level="WARNING",
),
webserver_logs=aws.mwaa.EnvironmentLoggingConfigurationWebserverLogsArgs(
enabled=True,
log_level="ERROR",
),
worker_logs=aws.mwaa.EnvironmentLoggingConfigurationWorkerLogsArgs(
enabled=True,
log_level="CRITICAL",
),
),
network_configuration=aws.mwaa.EnvironmentNetworkConfigurationArgs(
security_group_ids=[vpc["vpcDefaultSecurityGroupID"]],
subnet_ids=[vpc["privateSubnetsIDs"][0], vpc["privateSubnetsIDs"][1]],
),
source_bucket_arn=airflow_dags_bucket.arn,
tags={
"Environment": f"{stack}",
},
opts=ResourceOptions(
depends_on=[mwaa_execution_role, mwaa_kms_key, airflow_dags_bucket]))
Any idea what I’m doing incorrectly?
I’m trying to look into this:
That your Amazon VPC is configured to allow network traffic between the different AWS resources used by your Amazon MWAA environment, as defined in About networking on Amazon MWAA. For example, your VPC security group must either allow all traffic in a self-referencing rule, or optionally specify the port range for HTTPS port range 443 and a TCP port range 5432.
{
"Effect": "Allow",
"Principal": {
"Service": "<http://logs.us-west-2.amazonaws.com|logs.us-west-2.amazonaws.com>"
},
"Action": [
"kms:Encrypt*",
"kms:Decrypt*",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:Describe*"
],
"Resource": "*",
"Condition": {
"ArnLike": {
"kms:EncryptionContext:aws:logs:arn": "arn:aws:logs:us-west-2:*:*"
}
}
}