This message was deleted.

sparse-intern-71089

10/06/2021, 5:36 PM

This message was deleted.

billowy-army-68599

10/06/2021, 5:37 PM

hmm, how long is it taking?

tall-nightfall-10556

10/06/2021, 5:38 PM

it's about 1sec per file.

tall-nightfall-10556

10/06/2021, 5:38 PM

so for a directory with ~500 files, it's about 10 minutes

tall-nightfall-10556

10/06/2021, 5:38 PM

vs 2-3 sec with aws sync

billowy-army-68599

10/06/2021, 5:40 PM

i think it's because of the design.

aws sync

is just a splat, whereas for each object in Pulumi we have to upload, store the result of that update in the state/checkpoint, which is going to have an impact

billowy-army-68599

10/06/2021, 5:40 PM

some users have switched to a dynamic provider to manage objects instead

tall-nightfall-10556

10/06/2021, 5:41 PM

is there an example of the dynamic provider approach?

billowy-army-68599

10/06/2021, 5:41 PM

https://github.com/sevenwestmedia-labs/pulumi-components/tree/main/libs/throw-files-in-s3

billowy-army-68599

10/06/2021, 5:41 PM

knew you were gonna ask so went looking 😄

tall-nightfall-10556

10/06/2021, 5:41 PM

Or what I was hoping was to seed the state with sync and then do an import of the objects with refresh so it could handle deltas from that point.

billowy-army-68599

10/06/2021, 5:42 PM

the caveat here is that it won't manage the objects on delete, it has no delete in the crud

billowy-army-68599

10/06/2021, 5:42 PM

i'm not sure if you could seed the state with sync, you'd have to import every object into the state which is going to take just as long I suspect 😞

Open in Slack

Previous Next

Pulumi Community

No matter how you like to participate in developer communities, Pulumi wants to meet you there. If you want to meet other Pulumi users to share use-cases and best practices, contribute code or documentation, see us at an event, or just tell a story about something cool you did with Pulumi, you are part of our community.