https://pulumi.com logo
#general
Title
# general
g

glamorous-jelly-86558

10/20/2021, 2:35 AM
Identify all non-Pulumi managed resources in an AWS Account?
Our DevOps team is somewhat new to Pulumi and occasionally find it easier to manually create certain resources in the AWS console to speed up the iteration cycle. Most of the time they also manually remove these resources when they are done. Sometimes, though, there will be a leftover resource that shouldn't exist. So is there any way to do a diff on an AWS account and see what resources exist that are not managed by Pulumi?
l

little-cartoon-10569

10/20/2021, 2:35 AM
No. There is no way, even in AWS, to list all resources across all services.
g

glamorous-jelly-86558

10/20/2021, 2:36 AM
Ahhh... bummer. If given some type of scope, like databases or cidr blocks or s3 buckets... would it be any more feasible?
l

little-cartoon-10569

10/20/2021, 2:37 AM
Yes, but you'd have to write the code yourself, using the AWS SDK.
I know it sounds trite, but if "losing" resources is a genuine issue, the correct solution is to remove console access and programmatic access from all members of this team.
g

glamorous-jelly-86558

10/20/2021, 2:38 AM
Great... so the general code for that would be... 1. Use Pulumi to output the managed resources... 2. Use the AWS SDK to enumerate all resources of given types... 3. Manually write some diffing layer ... does that sound about right?
l

little-cartoon-10569

10/20/2021, 2:38 AM
If there isn't reviewed, accepted and merged code that created the resource, then the resource won't exist.
Yes that would work. It wouldn't be too hard, since you've got
pulumi state --show-urns
to start with.
👍 1
g

glamorous-jelly-86558

10/20/2021, 2:39 AM
Thanks a bunch!
👍 1
b

billowy-army-68599

10/20/2021, 2:46 AM
@glamorous-jelly-86558 you could scrape the API with the AWS SDK and then use
import
to import the resources
but I agree with tenwit, having the ability to create resources in the console is going to create chaos down the line unfortunately, I've been there myself
g

glamorous-jelly-86558

10/20/2021, 5:39 AM
For the record: I agree. We have some locked down environments that everything needs to be reviewed and deployed through CI and IaC. But we have some mixed mode environments and I was mainly curious if it was possible.
It could also be helpful as a tool for identifying if there was a breach and if there were any resources created by a malicious attacker.
b

bumpy-grass-54508

10/20/2021, 1:50 PM
It isn't airtight but you could also make sure your IaC / CD use default tags on any resource that takes tags - include things like ManagedBy=Pulumi and some build ID or git sha sort of thing. And mix in some AWS Config tagging policies could get you part of the way there. not immune to malicious spoofing and not every resource has tags, but makes browsing the console easier to tell what is and isn't pulumi
m

millions-umbrella-34765

10/20/2021, 3:55 PM
@glamorous-jelly-86558 You may want to look at https://steampipe.io/ as a tool to query for AWS resources.
🙌 1
l

little-cartoon-10569

10/20/2021, 8:54 PM
It doesn't cover all resources (yet) but it looks like a big step in the right direction...
g

glamorous-jelly-86558

10/20/2021, 9:22 PM
This community is amazing.
🕺 1
l

little-cartoon-10569

10/20/2021, 9:25 PM
I'm playing with the the cloud control API right now. It's nowhere near fit for solving this problem right now. Loads of stuff isn't supported.
For example: there is support for listing NACLs, but no support for listing subnets. And there is no support of any kind for EC2 instances....
b

billowy-army-68599

10/21/2021, 1:22 AM
@little-cartoon-10569 the AWS native provider is driven by the cloud control API. It'll add more resources as time goes on
l

little-cartoon-10569

10/21/2021, 2:51 AM
Yep I'm following it 🙂 The same AWS API can help solve this problem, eventually. Though afaik there will never be a "discover all resources" API. You'll need to script "discover all resources of type X" for all values of X.
Fortunately, there's another API for discovering all values of X (cloudformation).
However, it doesn't seem to include the "list" feature. You can limit to types that are FULLY_MUTABLE or IMMUTABLE, but neither of those 100% overlaps with "listable".
6 Views