https://pulumi.com
Join Slack
Menu
Home
Archive
Hi all, I've got an issue when using the docker.Provider connecting to a AWS EC2 instance that I'm a...
# general
f
Hi all, I've got an issue when using the docker.Provider connecting to a AWS EC2 instance that I'm also creating in the same stack. I have declared the EC2 instance resources as a dependency of the docker.Provider resource, but when I create my stack the docker.Provider() is attempting to connect to my EC2 VM before it has been created. Is there any other way to express the dependency? Even using 
pulumi preview
will timeout because the docker Provider is assuming it can access the target host.
d
How are you setting it as a dependency?
The code snippet of how you initialise the provider is preferred
f
Via the depends_on parameter of the ResourceOptions when creating the docker.Provider resource. (I'm using Python)
Just a second, fighting Slacks formatting widget...
a
If you're trying to set the host of the docker provider to the not yet created VM that obviously won't work but please share your code for better context.
f
Copy code
admin_docker = docker.Provider(
        "docker-admin",
        host=admin_server.public_ip.apply(
            lambda l: f"<ssh://ec2-user>@{l}"
        ),
        
        opts=pulumi.ResourceOptions(
            parent=admin_server,
            depends_on=[admin_server]
        )
    )
d
And 
admin_server
is an ec2 resource?
f
Yes, essentially
Copy code
admin_server = aws.ec2.Instance(...)
a
What version of the docker provider are you using?
f
Copy code
pulumi_docker      4.5.4
d
There's a note that if you're using an elastic IP, the public_ip property won't be correct. Is this the case?
And just to check, does docker connect on subsequent runs?
f
When I originally added this code, it worked because I added to a stack where admin-server had already been created.
When I start a fresh stack, I run this into ordering issue
a
Is Pulumi not respecting 
depends_on
- that is is the Docker provider created before the ec2 creation? Or is Pulumi respecting the dependency but the ec2 resource has been created but the vm is not available for connection yet?
f
Pulumi is not respecting the 
depends_on
constraint, correct
admin_server
has not been created at all at this point
d
Olafur, the latter would be my guess. Ec2 created, but not yet actually ready
a
What version of Pulumi are you running?
f
v3.32.1
a
Pulumi is currently at 
3.122.0
- try updating
f
Good point, I had forgotten I hadn't ugpraded it on this machine. Let's try that first...
No, no difference on 3.122.0
a
Try adding this to your stack configuration:
Copy code
config:
  pulumi:disable-default-providers:
    - docker
Just a thought - In case the provider is being initialized implicitly
f
In my case I'm explicitly creating a docker Provider object and passing it into anything that uses it (like docker.Image, docker.Network, etc). I've tried that, no difference.
d
Just to say again, ec2 isn't necessarily ready/connectable after the ec2 resource is created
f
I assure you, at this point the ec2 resource has not been created by Pulumi at all.
There are no EC2 instances created by the stack at all at this point.
a
What is the error output you get? Have you analyzed the outputs when running the operation with 
--logflow -v*=*10
?
f
The docker Provider is timing out trying to open an SSH connection to the host.
a
And which host is it trying to connect to? Does the output reveal that?
f
Ah, wait a second. My example code is incorrect, I had elided it slightly. The actual code is more like this:
Copy code
admin_docker = docker.Provider(
        "docker-admin",
        host=admin_eip.public_ip.apply(
            lambda l: f"<ssh://ec2-user>@{l}"
        ),
        
        opts=pulumi.ResourceOptions(
            parent=admin_server,
            depends_on=[admin_server]
        )
    )
So in this case, the elastic IP had been created, but the EC2 instance had not been created.
And the elastic IP had not been associated with a server, but the elastic IP did exist, so the lambda could complete
It's not waiting for anything else in the 
depends_on
constraint though.
d
Ohhh. OK, this is pulumi getting confused with parent + depends_on being used together
Sorry I should have caught that when you posted originally
Because the provider is a child of the server, it won't actually wait for the server to finish because it can't finish until the children are done. Depends_on effectively gets ignored in this case
f
I'll try removing parent and see if that changes things.
d
There's an issue somewhere on github about adding warnings/errors when this is detected
f
I wonder if there is a race here, since the 
admin_server.public_ip
will change once the EIP is associated with it.
a
This might be due to this issue: https://github.com/pulumi/pulumi/issues/7012 If that's the case you would probably have to initialize the docker provider within an apply callback as a workaround.
d
The note about that is pulumi will just present the old ip before the eip is attached
Hopefully removing 'parent' will at least get the ec2 instance created. You might need to run the stack twice if the provider times out on trying to connect again
f
I see what you mean re apply(), I'll have a think on that.
a
Yes, that's not optimal for sure but it seems until #7012 is resolved, resource dependencies on Provider is not possible. Another approach which would probably require a consecutive 
pulumi up
operation would be to initialize the docker provider in a callback to 
admin_server.instance_state
output, preferrably when it returns "`running`"
Or split this into two stacks...
f
I think if I can ensure the host parameter takes a Promise which won't complete until the admin server is created, it should defer the startup of the provider.
Thanks guys, I'll subscribe to issue 7012.
d
Issue 7012 seems unrelated, it's to do with when you end up with a Resource that's an Output
f
Perhaps I need to depend on an output of the admin_server, rather than just the admin server itself?
d
Depending on the server is correct. Did removing it as a parent help?
f
No, removing either the parent or the depends_on made no difference there.
d
A little trick you can do is reference the server ID while you're accessing the EIP, which will prevent the host value resolving until both resources are created. 
host=pulumi.Output.all(admin_eip.public_ip, admin_server.id).apply(lambda args: f"{<ssh://ec2-user@{args[0]}")|ssh://ec2-user@{args[0]}")>
f
On my way to pickup the kids from school, I was thinking a similar thing! Good tip
5 Views
Open in Slack
PreviousNext
Powered by

Pulumi Community

No matter how you like to participate in developer communities, Pulumi wants to meet you there. If you want to meet other Pulumi users to share use-cases and best practices, contribute code or documentation, see us at an event, or just tell a story about something cool you did with Pulumi, you are part of our community.

Powered by