https://pulumi.com logo
q

quaint-teacher-33292

04/16/2019, 11:58 AM
Hey all, I'm liking Pulumi a lot so far, but is there a way to make it work with existing infrastructure? I know that `import`/`adopt` is not supported, but is there a workaround to make it happen if I'm careful? In my case, I don't want to re-create a Cloud SQL instance, or a GKE cluster.
t

tall-librarian-49374

04/16/2019, 12:16 PM
q

quaint-teacher-33292

04/16/2019, 12:22 PM
This looks very promising, is there a way to convert this to a managed resource? Maybe with
stack export
on the configuration using get() and then
stack import
on a configuration that defines the resource? Does that make sense?
t

tall-librarian-49374

04/16/2019, 12:42 PM
How would the code definition of such resource appear?
s

shy-army-45455

04/16/2019, 12:52 PM
Are you saying you want Pulumi to automatically generate code for an existing infrastructure and then import it as a stack?
q

quaint-teacher-33292

04/16/2019, 12:58 PM
I can write this code by hand, that's not the issue. I want to link the code I write to an existing resource, so that Pulumi can manage it without recreating on first run. Does that make sense?
t

tall-librarian-49374

04/16/2019, 1:01 PM
I would guess that if you are careful with defining the code,
pulumi refresh
should get you where you want to be
💡 1
q

quaint-teacher-33292

04/16/2019, 1:09 PM
I thought about doing this, but couldn't figure out what was needed to refresh to notice the existing resource. Do you know? I'm trying
Copy code
export const database = new gcp.sql.DatabaseInstance('test-database-asd', {
  databaseVersion: 'POSTGRES_11',
  name: 'test-database-asd',
  settings: {
    diskAutoresize: false,
    diskSize: 10,
    diskType: 'PD_HDD',
    tier: 'db-f1-micro'
  }
});
AFAIK the
name
parameter defines the name used in the API, the same one that can be seen on GCP. I already have a Cloud SQL instance with that name, in the same zone, but it is not being discovered by
pulumi refresh
. Is there something else that needs to match up? I know there are some internal IDs being passed around, do I need to generate one of those?
I think I figured it out
s

shy-army-45455

04/16/2019, 1:14 PM
Have you tried creating the same type of resource through Pulumi and seeing what names/ids are set on the cloud side?
q

quaint-teacher-33292

04/16/2019, 1:15 PM
I used
gcp.sql.DatabaseInstance.get('test-database-asd', 'test-database-asd');
, then ran
pulumi up
to fetch this data into the internal cache. Then replaced that code with a definition that matches the state of the resource in GCP. Then ran
pulumi refresh
and it went fine, unchanged. Then to test, I increased diskSize and ran
pulumi up
. It correctly diffed the state before & after, and is recommending a
replace
operation.
replace
sounds destructive, but since it has no real data I will see what it actually does
Looks like I spoke too soon, or it was trying to create a replacement with the same name:
Copy code
Updating (staging):

     Type                         Name               Status                 Info
     pulumi:pulumi:Stack          estetio-staging    **failed**             1 error
 >   └─ gcp:sql:DatabaseInstance  test-database-asd  **reading failed**     [diff: -__defaults,databaseVersion,masterInstanceName,name,project,region,settings]; 1 error
 
Diagnostics:
  pulumi:pulumi:Stack (estetio-staging):
    error: update failed
 
  gcp:sql:DatabaseInstance (test-database-asd):
    error: Plan apply failed: Error, failed to create instance test-database-asd: googleapi: Error 409: The Cloud SQL instance already exists., instanceAlreadyExists
@shy-army-45455 Yes, the name on GCP was the one I gave to Pulumi + some 6 or 8 character hash
But if I specify a name in DatabaseInstanceArgs, the 2nd argument to the constructor, it will use that one for GCP
s

shy-army-45455

04/16/2019, 1:24 PM
Yeah, I was wondering if the random characters were getting in the way
Not sure what the problem is if that's not it, seems possible though
If you can get it figured out, it would probably be worthwhile to document it for other users, probably a fairly common use case
q

quaint-teacher-33292

04/16/2019, 1:32 PM
I feel like I'm really close, I ended up editing Pulumi's json file for my stack manually, but it still wants to recreate my database, even when there are no differences.
Copy code
pulumi preview --diff
Previewing update (staging):
  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:staging::estetio::pulumi:pulumi:Stack::estetio-staging]
    ++gcp:sql/databaseInstance:DatabaseInstance: (create-replacement)
        [id=test-database-asd]
        [urn=urn:pulumi:staging::estetio::gcp:sql/databaseInstance:DatabaseInstance::test-database-asd]
        databaseVersion: "POSTGRES_11"
        name           : "test-database-asd"
        settings       : {
            diskAutoresize : true
            diskSize       : 10
            diskType       : "PD_HDD"
            pricingPlan    : "PER_USE"
            replicationType: "SYNCHRONOUS"
            tier           : "db-f1-micro"
        }
    +-gcp:sql/databaseInstance:DatabaseInstance: (replace)
        [id=test-database-asd]
        [urn=urn:pulumi:staging::estetio::gcp:sql/databaseInstance:DatabaseInstance::test-database-asd]
        databaseVersion: "POSTGRES_11"
        name           : "test-database-asd"
        settings       : {
            diskAutoresize : true
            diskSize       : 10
            diskType       : "PD_HDD"
            pricingPlan    : "PER_USE"
            replicationType: "SYNCHRONOUS"
            tier           : "db-f1-micro"
        }
    <<gcp:sql/databaseInstance:DatabaseInstance: (discard-replaced)
        [id=test-database-asd]
        [urn=urn:pulumi:staging::estetio::gcp:sql/databaseInstance:DatabaseInstance::test-database-asd]
        databaseVersion: "POSTGRES_11"
        name           : "test-database-asd"
        settings       : {
            diskAutoresize : true
            diskSize       : 10
            diskType       : "PD_HDD"
            pricingPlan    : "PER_USE"
            replicationType: "SYNCHRONOUS"
            tier           : "db-f1-micro"
        }
Resources:
    +-1 to replace
    1 unchanged
Permalink: file:///home/kuba/.pulumi/stacks/staging.json
Can't post a snippet in a Slack thread, sorry
but even when there are no changes to reconcile, it's trying to recreate it 😞
I will let it replace my DB, just to see what the end result would be. Maybe that will give my ideas as to what's missing, because the diff is not telling me that
s

shy-army-45455

04/16/2019, 1:36 PM
I'm not verse enough on Pulumi's internals to give a useful response, but it seems like its some kind of internal ID that's not matching up?
q

quaint-teacher-33292

04/16/2019, 1:36 PM
That would be my guess as well
Hmm, the only difference I see is a new
__meta
field in the stack JSON, and it happens to be hidden from the CLI diffs. Perhaps that's the one
🤔 1
g

gentle-diamond-70147

04/16/2019, 2:50 PM
You essentially export your stack, add the resources's type, urn, etc. to the stack, re-improt it and
refresh
. Then write the code to get the refresh details to "match up".
Note: we will be making this a first-class feature relatively soon - https://github.com/pulumi/pulumi/issues/1662.
👍 1
q

quaint-teacher-33292

04/16/2019, 2:58 PM
I just figured it out as you sent these messages. I only have to remove the
"external": true
part of the JSON and it is reconciled properly
@gentle-diamond-70147 does the
import
operation take care of dropping this "external" field, or is it different entirely?
g

gentle-diamond-70147

04/16/2019, 3:08 PM
The import operation described in that issue?
q

quaint-teacher-33292

04/16/2019, 3:10 PM
I assume you meant
pulumi stack import
, right?
g

gentle-diamond-70147

04/16/2019, 3:14 PM
Oh, sorry. TBH, I'm not sure if the
stack import
command removes that.
q

quaint-teacher-33292

04/16/2019, 3:31 PM
I think my solution is the simplest and least laborious: I define my existing resources with
get(...)
functions, run
pulumi up
. Edit the stack json directly (or export -> modify -> import probably works too) to remove
"external":  true
, and that's it. No
refresh
is needed, although the first time you update this resource, the schema will change slightly, some keys move around etc. But it's not destructive. Thank you for the help, I will note down this solution in the import/adopt issue on GH, so others are aware.
👍 1
t

tall-librarian-49374

04/16/2019, 3:33 PM
But you still had to replace
get(...)
with a constructor at some point?
q

quaint-teacher-33292

04/16/2019, 3:34 PM
yes, forgot to mention it, it should happen directly before or after removing the "external" attribute
and I was wrong, after re-testing with and without refresh, it's indeed better to do it
👍 2