https://pulumi.com logo
Join Slack
Powered by
Identify all non-Pulumi managed resources in an AW...
# general
g
Identify all non-Pulumi managed resources in an AWS Account?
Our DevOps team is somewhat new to Pulumi and occasionally find it easier to manually create certain resources in the AWS console to speed up the iteration cycle. Most of the time they also manually remove these resources when they are done. Sometimes, though, there will be a leftover resource that shouldn't exist. So is there any way to do a diff on an AWS account and see what resources exist that are not managed by Pulumi?
l
No. There is no way, even in AWS, to list all resources across all services.
g
Ahhh... bummer. If given some type of scope, like databases or cidr blocks or s3 buckets... would it be any more feasible?
l
Yes, but you'd have to write the code yourself, using the AWS SDK.
I know it sounds trite, but if "losing" resources is a genuine issue, the correct solution is to remove console access and programmatic access from all members of this team.
g
Great... so the general code for that would be... 1. Use Pulumi to output the managed resources... 2. Use the AWS SDK to enumerate all resources of given types... 3. Manually write some diffing layer ... does that sound about right?
l
If there isn't reviewed, accepted and merged code that created the resource, then the resource won't exist.
Yes that would work. It wouldn't be too hard, since you've got 
pulumi state --show-urns
to start with.
👍 1
g
Thanks a bunch!
👍 1
b
@glamorous-jelly-86558 you could scrape the API with the AWS SDK and then use 
import
to import the resources
but I agree with tenwit, having the ability to create resources in the console is going to create chaos down the line unfortunately, I've been there myself
g
For the record: I agree. We have some locked down environments that everything needs to be reviewed and deployed through CI and IaC. But we have some mixed mode environments and I was mainly curious if it was possible.
It could also be helpful as a tool for identifying if there was a breach and if there were any resources created by a malicious attacker.
b
It isn't airtight but you could also make sure your IaC / CD use default tags on any resource that takes tags - include things like ManagedBy=Pulumi and some build ID or git sha sort of thing. And mix in some AWS Config tagging policies could get you part of the way there. not immune to malicious spoofing and not every resource has tags, but makes browsing the console easier to tell what is and isn't pulumi
m
@glamorous-jelly-86558 You may want to look at https://steampipe.io/ as a tool to query for AWS resources.
🙌 1
l
It doesn't cover all resources (yet) but it looks like a big step in the right direction...
g
This community is amazing.
🕺 1
l
I'm playing with the the cloud control API right now. It's nowhere near fit for solving this problem right now. Loads of stuff isn't supported.
For example: there is support for listing NACLs, but no support for listing subnets. And there is no support of any kind for EC2 instances....
b
@little-cartoon-10569 the AWS native provider is driven by the cloud control API. It'll add more resources as time goes on
l
Yep I'm following it 🙂 The same AWS API can help solve this problem, eventually. Though afaik there will never be a "discover all resources" API. You'll need to script "discover all resources of type X" for all values of X.
Fortunately, there's another API for discovering all values of X (cloudformation).
However, it doesn't seem to include the "list" feature. You can limit to types that are FULLY_MUTABLE or IMMUTABLE, but neither of those 100% overlaps with "listable".
48 Views
Open in Slack
PreviousNext
Powered by