https://pulumi.com logo
Title
t

thousands-art-55778

03/06/2023, 1:56 PM
Hello everyone! I’m trying to setup a
SNS -> Kinesis Firehose -> S3
using Pulumi. I was able to do it to save JSON files. Now I want to save data in Parquet. Do we have any documentation/example code of how to do it? Any material is appreciated! 😄
e

echoing-dinner-19531

03/06/2023, 2:12 PM
Looks like there's some settings for that in the FirehoseDeliveryStream class: https://www.pulumi.com/registry/packages/aws/api-docs/kinesis/firehosedeliverystream/#parquet_ser_de_python
t

thousands-art-55778

03/06/2023, 2:31 PM
I’ve seen it! I was hoping that we could have some source ready, similar to what we have for the
Extended S3 Destination with dynamic partitioning
. And then I would only have to change the references (glue table, tuning of configurations). Although I’m familiarized with this configuration in the AWS Console, the IaC is a bit confusing. (but I’m pretty sure I can figure out 🙂)
e

echoing-dinner-19531

03/06/2023, 2:32 PM
I don't think there's any examples of this already but it does look likes its just a case of setting up a few more options on the resource. Auto-complete in the IDE might be easier to read than the docs page though with the long names used here!
t

thousands-art-55778

03/06/2023, 2:33 PM
I agree! Thanks for the support. Autocomplete is working perfectly.