https://pulumi.com logo
Join Slack
Powered by
This message was deleted.
# python
s
This message was deleted.
g
Due to the async nature of Outputs, I don't think you can set and then use environment variables like this.
You could potentially use the set environment variables from within the .apply() after you immediately set them. But setting them in the apply and then attempting to use them outside won't work I believe.
👍 1
How are you wanting to use those values once they are available? In some custom code to connect to the database?
b
It is for connecting to a GCP CloudSQL Instance via pulumi-postgresql module and specifically the Provider class:
Copy code
postgre_provider = pulumi_postgresql.Provider(
        'cloud_pgsql_main_1_provider',
        host=cloud_pgsql_main_1.public_ip_address,
        opts=ResourceOptions(
            depends_on=[cloud_pgsql_main_1, service_user,
                        service_user_certificate],
        ),
        username=service_user.name,
        password=service_user.password,
        port=5432,
        sslmode='verify-ca',
        expected_version='11',
    )

    for role in ROLES_TO_CREATE:
        role_object = pulumi_postgresql.Role(
            'postge_user_' + role.replace('-', '_'),
            login=True,
            name=role,
            opts=ResourceOptions(
                depends_on=[cloud_pgsql_main_1, service_user,
                            service_user_certificate, postgre_provider],
                protect=False,
                provider=postgre_provider,
            ),
            password=CLOUDSQL_POSTGRE_APPLICATION_USERS_AND_PASSWORDS.get(
                role,
                password_generator.random_password_gen(to_base64_string=False)),
        )
Since I want to use HTTPS, I need to provide environmental variables for that, as it uses the libpq-ssl https://www.postgresql.org/docs/current/libpq-ssl.html
m
Modifying the OS environment in a concurrent/asynchronous process is inherently risky since os.environ is a singleton across threads/tasks, and multiple tasks may be competing to change it. IMO you should capture the environment variables you need at startup (before overlapping/asynchronous activity begins), then pass the captured or computed values to your async tasks, then explicitly pass them down to operations as variables rather than through the environment. If the library you are using requires per-instance values to be passed through environment variables, it may not be safe to use it within an asynchronous task, unless appropriate locks are placed around updating and using the os environment. One thing that is safe to do is to create a new dict representing the environment you wish to run your operation in, then spawn a child process (with subprocess.check_call, etc) passing the new dict to the child process with 
env=
. You could do that within apply()
In the case of libpq-ssl, all of the parameters that can be passed as environment variables can also be passed as connection parameters https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-PARAMKEYWORDS . However, the terraform provider for postgresql (from which the pulumi provider is derived) does not seem to expose these parameters, which is why you are stuck passing environment variables around. One solution that may or may not work for you is to create a Connection Service File that contains all of the parameter settings, then create the connection using the logical service name: https://www.postgresql.org/docs/current/libpq-pgservice.html
b
Hello, @many-garden-84306. Thanks a lot for the explanation! "However, the terraform provider for postgresql (from which the pulumi provider is derived) does not seem to expose these parameters" - exactly, there is no other way. In addition, I don't know if connection service file is going to solve the deal, as I create a SslCert object during the same runtime. Nevertheless, I will definitely take a look at that. Thank you once again. Yeah, I agree that using env variables is not the best approach, but it is better to use an encrypted connection to the instance to create the users in 🙂 and there were no other option as far as I understood
@many-garden-84306, @white-balloon-205 I don't know why, but
first 
pulumi.Output.all
works perfectly, but second one doesn't. It seems like the values from 
cloud_sql.postgre_sql
cannot be parsed. When I change the second output to `
Copy code
# pulumi.Output.all(
#     network.vpc.ha_vpn_gw_hq.vpn_interfaces[0]['ip_address'],
# ).apply(pulumi_output_renderer_postgre)
it works, so once again, I think that the values from the classes
Copy code
from pulumi_gcp.sql import DatabaseInstance, SslCert, User
cannot be derived. Please help 🙂
Copy code
import os
import pulumi
import network.vpc
from cloud_api import apis
import cloud_sql.postgre_sql
# from cloud_sql import postgre_sql
from service_scripts.mandrill_send import send_message


# Variables for FortiGate Configuration
def pulumi_output_renderer_fortigate(args_to_render) -> None:
    print('I am called, 0000000000000')
    print(args_to_render)

    from service_scripts.template_files import template_config

    jinja2_variables_dict_fortigate = {
        'phase1_interface_1_name': 'vpn-to-pulumi-1',
        'phase1_interface_2_name': 'vpn-to-pulumi-2',
        'phase1_interface_1_comment': 'GCP HA-VPN Interface0',
        'phase1_interface_2_comment': 'GCP HA-VPN Interface1',
        'ha_vpn_if_0_ip': args_to_render[0],
        'ha_vpn_if_1_ip': args_to_render[1],
        'fgt_vpn_if_1_ip': network.vpc.hq_bgp_peer_interface_1_ip,
        'fgt_vpn_if_2_ip': network.vpc.hq_bgp_peer_interface_2_ip,
        'neighbor_ip_1':
            network.vpc.vpn_router_to_hq_1_interface_1_ip.split('/', 1)[0],
        'neighbor_ip_2':
            network.vpc.vpn_router_to_hq_2_interface_1_ip.split('/', 1)[0],
        'remote_asn_1': network.vpc.vpn_router_to_hq_1_bgp_asn,
        'remote_asn_2': network.vpc.vpn_router_to_hq_2_bgp_asn,
    }

    template_config(
        './out_files/fgt_vpn_config.conf',
        './fortigate_vpn_config_template/fgt_vpn_config.j2',
        jinja2_variables_dict_fortigate,
        render_in_place=False,
        write_to_stdout=False,
    )


# libpq_config variables for database connections
def pulumi_output_renderer_postgre(args_to_render) -> None:
    print('I am called, 111111111111')
    print(args_to_render)

    from service_scripts.template_files import template_config

    # jinja2_variables_dict_postgre = {
    #     'postgre_host': args_to_render[0],
    #     'postgre_port': '5432',
    #     'postgre_service_user': args_to_render[1],
    #     'postgre_service_user_password': args_to_render[2],
    #     'postgre_sslmode': 'verify-ca',
    #     'postgre_sslcert': args_to_render[3],
    #     'postgre_sslkey': args_to_render[4],
    #     'postgre_sslrootcert': args_to_render[5],
    # }

    jinja2_variables_dict_postgre = {
        'postgre_host': '1',
        'postgre_port': '5432',
        'postgre_service_user': '2',
        'postgre_service_user_password': '23',
        'postgre_sslmode': 'verify-ca',
        'postgre_sslcert': '5',
        'postgre_sslkey': '66',
        'postgre_sslrootcert': '22',
    }

    template_config(
        f'{os.environ["HOME"]}/.pg_service.conf',
        './postgre_templates/libpq_config.j2',
        jinja2_variables_dict_postgre,
        render_in_place=False,
        write_to_stdout=False,
    )


def apply_and_send(instance_name: str, data_to_send: dict):

    from service_scripts.template_files import template_config

    # Templating Jinja2 Template for Postgre CloudSQL users
    cloudsql_postgre_users_out_file = './out_files/postgre_users_data.txt'

    template_config(
        cloudsql_postgre_users_out_file,
        './service_objects/password_email_template.j2',
        {
            'postgre_users': data_to_send
        },
        render_in_place=False,
        write_to_stdout=False,
    )

    send_message(
        # Send a message via Mandrill API
        f"Postgre CloudSQL users for "
        f"'{instance_name}'"
        f" instance in "
        f"'{pulumi.get_project()}' project",
        None,
        attachment_present=True,
        attachments=[
            {'content': cloudsql_postgre_users_out_file,
             'name': 'postgre_cloudsql_users.html',
             'type': 'text/html',
             },
        ],
    )


pulumi.Output.all(
    network.vpc.ha_vpn_gw_hq.vpn_interfaces[0]['ip_address'],
    network.vpc.ha_vpn_gw_hq.vpn_interfaces[1]['ip_address'],
).apply(pulumi_output_renderer_fortigate)

pulumi.Output.all(
    cloud_sql.postgre_sql.cloud_pgsql_main_1.ip_addresses,
    cloud_sql.postgre_sql.service_user.name,
    cloud_sql.postgre_sql.service_user.password,
    cloud_sql.postgre_sql.service_user_certificate.cert,
    cloud_sql.postgre_sql.service_user_certificate.private_key,
    cloud_sql.postgre_sql.service_user_certificate.server_ca_cert
).apply(pulumi_output_renderer_postgre)
Open in Slack
PreviousNext
Powered by