Have anyone been able to create an aws.codepipeline.Pipeline that have no triggers at all on the Sou...
c

Claes Jonsson

12 months ago
Have anyone been able to create an aws.codepipeline.Pipeline that have no triggers at all on the Source actions? I am trying to create a pipeline, with Sources connected to a self-managed GitLab, and thus using a CodeConnection. According to AWS documentation, I should be able to specify
"DetectChanges": "false"
in the
configuration
section of the source action, when using a provider of type
CodeStarSourceConnection
. However, there seem to be no way to avoid creating an active push trigger, which I DO NOT want. The pipeline will be triggered from an CloudWatch event bus, via an aws.cloudwatch.EventTarget. After the pipeline is created, I can remove the push trigger from using the AWS console, and they do not come back on subsequent
pulumi up
updates, but introducing a manual step that can easily be forgotten, defeats the purpose of IaC. Neither the AWS documentation, nor the Pulumi documentation are very helpful on the topic, except that the AWS docs says that
"DetectChanges": "false"
should do it. It does not, it seems. Speaking of triggering a pipeline via an aws.cloudwatch.EventTarget, has anyone been able to pass along event data to the pipeline, or to the aws.codebuild.Project(s) in the pipeline? Both pulumi AI and ChatGPT et.al. suggests using an input transform in the aws.cloudwatch.EventTarget and the data should be possible to pick up as environment variables like this
INSTANCE_ID="<instance_id>"
, but that just sets
INSTANCE_ID
to
<instance_id>
. In code it might be similar to this, with some suggestions by Pulumi AI (tried all version of suggestions, none works as I would like):
gitlab_host = aws.codestarconnections.Host(
    "GitLabHost",
    opts=pulumi.ResourceOptions(retain_on_delete=True),
    name="GitLabHost",
    provider_type="GitLabSelfManaged",
    provider_endpoint="<https://gitlab.example.com>",
    vpc_configuration={
        "securityGroupIds": [security_group.id],
        "subnetIds": [subnet1.id, subnet2.id, subnet3.id],
        "vpcId": vpc.id,
    })

gitlab_connection = aws.codestarconnections.Connection(
    "GitLabConnection",
    opts=pulumi.ResourceOptions(retain_on_delete=True),
    name="GitLabConnection",
    host_arn=gitlab_host.arn,
)

build_project = aws.codebuild.Project(
    "BuildProject",
    name="BuildProject",
    service_role=code_build_service_role.arn,
    environment={
        "computeType": "BUILD_GENERAL1_SMALL",
        "image": latest_image_version,
        "type": "LINUX_CONTAINER",
        "image_pull_credentials_type": "CODEBUILD",
        "environment_variables": [
            {
                "name": "INSTANCE_ID",
                "value": "<instanceId>",   # <--- Suggested by Pulumi AI, does not work
                "type": "PLAINTEXT",
            },
        ],
    },
    artifacts={
        "type": "CODEPIPELINE",
    },
    logs_config={
        "cloudwatch_logs": {
            "group_name": "BuildProject",
            "stream_name": "",
        },
        "s3_logs": {
            "status": "ENABLED",
            "location": codebuild_bucket.id.apply(lambda id: f"{id}/build-logs/service-infra"),
        },
    },
    source={
        "type": "CODEPIPELINE",
    },
    source_version="main",
    vpc_config={
        "security_group_ids": [security_group.id],
        "subnets": [subnet1.id, subnet2.id, subnet3.id],
        "vpc_id": vpc.id,
    },
)


pipeline = aws.codepipeline.Pipeline(
    "Pipeline",
    name="Pipeline",
    role_arn=pipeline_service_role.arn,
    pipeline_type="V2",
    execution_mode="QUEUED",
    artifact_stores=[
        {
            "location": codebuild_bucket.id,
            "type": "S3",
        }
    ],
    stages=[
        {
            "name": "Sources",
            "actions": [
                {
                    "name": "GitLabSource",
                    "category": "Source",
                    "owner": "AWS",
                    "provider": "CodeStarSourceConnection",
                    "version": "1",
                    "output_artifacts": ["GitLabSource"],
                    "configuration": {
                        "ConnectionArn": gitlab_connection.arn,
                        "FullRepositoryId": "my-group/my-project",
                        "BranchName": "main",
                        "OutputArtifactFormat": "CODE_ZIP",
                        "DetectChanges": "false",
                    },
                }
            ],
        }, {
            "name": "Build",
            "actions": [{
                "name": "Build",
                "category": "Build",
                "run_order": 1,
                "owner": "AWS",
                "provider": "CodeBuild",
                "version": "1",
                "input_artifacts": ["GitLabSource"],
                "output_artifacts": ["BuildOutput"],
                "configuration": {
                    "ProjectName": build_project.name,
                    "EnvironmentVariables": json.dumps([
                        {
                            "name": "INSTANCE_ID",
                            "value": "<instanceId>",   # <--- Also suggested by Pulumi AI, does not work
                            "type": "PLAINTEXT",
                        },
                    ]),
                },
            }],
        }
    ],
)

rule = aws.cloudwatch.EventRule(
    "EventRule",
    name="EventRule",
    event_pattern=json.dumps({
        "source": ["my.source"],
        "detail-type": ["my.detail-type"],
    }),
    state="ENABLED",
    event_bus_name=event_bus.name)

aws.cloudwatch.EventTarget(
    "PipelineTarget",
    arn=pipeline.arn,
    role_arn=service_role.arn,
    event_bus_name=event_bus.name,
    rule=rule.name,
    input_transformer={
        "input_paths": {
            "instance_id": "$.detail.instance-id",
            "state": "$.detail.state"
        },
        "input_template": """{
            "instanceId": <instance_id>,
            "state": <state>
        }"""
    }
)
with a
buildspec.yml
a like this:
version: 0.2

run-as: root

env:
  variables:
    INSTANCE_ID: "<instanceId>"   # <--- Suggested by Pulumi AI, does not work

phases:

  build:
    commands:
      - echo "Build for instance id $INSTANCE_ID" # <--- prints 'Build for instance id <instanceId>'
I am running into more issues around using Azure Managed Identities. I have a VM with a system assi...
j

Jonas Dorsch

over 1 year ago
I am running into more issues around using Azure Managed Identities. I have a VM with a system assigned managed identity from which i can easily sign into az cli and create secrets using some code like this:
az login --identity
az keyvault secret set --name <secret_name> --vault-name <vault_name>
on the keyvault, the managed identity has been given reader and key vault secrets officer. (Keyv is set to RBAC). Since i can run above statements from the associated VMs with success, I assume access is working. Yet, enabling pulumi to utilize the managed identity always returns the following error:
error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client '<client obj id>' with object id <client obj id>' does not have authorization to perform action 'Microsoft.KeyVault/vaults/secrets/write' over scope '/subscriptions/<subscription>/resourceGroups/<resource group>/providers/Microsoft.KeyVault/vaults/<vault_name>/secrets/<secret name>' or the scope is invalid. If access was recently granted, please refresh your credentials."
I checked all the object ids and paths a million time by now but can not find any inconsistencies. pulumis python sdk is utilized and the following variables are set as env_vars for LocalWorkspaceOptions:
...
'ARM_USE_MSI': true
'ARM_SUBSCRIPTION_ID': <subscription>
...
Can anyone point me to what I am missing here?