https://pulumi.com logo
Powered by
c

clean-car-30047

01/30/2019, 8:58 PM
I'd like to use Pulumi to prototype a workflow system using SQS queues. In a lambda that reads off of one queue, I would like to write an SQS message to another queue. How can I write to an SQS queue from within a lambda? Note: not interested in using a step function.
t

tall-librarian-49374

01/30/2019, 9:02 PM
Call aws sdk?
c

clean-car-30047

01/30/2019, 9:07 PM
Haha, well that will work if nothing else. I was hoping there was a simpler way to work with Pulumi constructs. Particularly with the abstracted concepts in Pulumi's cloud-aws library.
t

tall-librarian-49374

01/30/2019, 9:08 PM
Copy code
import * as aws from "@pulumi/aws";
bucket.onObjectCreated("postToSlack", async (e) => {
  // direct access to the aws-sdk through `aws.sdk`.
  const sqs = new aws.sdk.SQS();
  sqs.sendMessage(/*...*/);
});
(that's s3 bucket but whatever)
c

clean-car-30047

01/30/2019, 9:09 PM
Thanks Mikhail! That's exactly what I was looking for
t

tall-librarian-49374

01/30/2019, 9:10 PM
Well, good question about the cloud library
I think I asked it several months ago and there was nothing extra. But maybe things changed?
cc @lemon-spoon-91807
c

clean-car-30047

01/30/2019, 9:36 PM
And @tall-librarian-49374 the method you described isn't working for me, as the pulumi/aws library isn't meant to be referenced from within a lambda function (in my case). Here's the end of the error. Happy to post the rest if helpful.
Copy code
Module './node_modules\\@pulumi\\aws\\index.js' is a 'deployment only' module. In general these cannot be captured inside a 'run time' function.
    error: an unhandled error occurred: Program exited with non-zero exit code: 1
t

tall-librarian-49374

01/30/2019, 9:43 PM
Last code block
and the previeww image 🙂
c

clever-sunset-76585

01/30/2019, 9:57 PM
As @tall-librarian-49374 pointed out, that’s how I have done it too. It’s pretty straightforward. I sent mine in batches so I used 
sendMessageBatch
from the SQS client.
Copy code
// The `aws-sdk` is a transient dependency of pulumi-aws.
import { SQS } from 'aws-sdk`;
try {
  const outputSQS = new SQS();
  await outputSQS.sendMessageBatch({
    Entries: [{Id: "someUniqueIDForThisMessage", MessageBody: myMessageBody}],
    QueueUrl: outputSQSURL
  }).promise();
} catch (err) {
  console.log("Error sending a batch", err);
}
Although, I do think that creating an SQS client from the 
aws.sdk
object should have serialized the right imports in your Lambda. @lemon-spoon-91807 would be able to comment on that.
c

clean-car-30047

01/30/2019, 10:10 PM
Thanks @clever-sunset-76585. Here's what I am trying to do:
Copy code
import * as aws from '@pulumi/aws';
import * as serverless from '@pulumi/aws-serverless';

const stage1Queue = new aws.sqs.Queue('stage-1', { visibilityTimeoutSeconds: 360 });
const stage2Queue = new aws.sqs.Queue('stage-2', { visibilityTimeoutSeconds: 360 });

serverless.queue.subscribe('stage-1', stage1Queue, async (e: any) => {
    console.log('Starting stage 1...');
    // Do some things
    // ...

    // Send to next stage
    const sqs = new aws.sdk.SQS();
    sqs.sendMessage({
        QueueUrl: stage2Queue.id.get(),
        MessageBody: '...'
    });
}, { batchSize: 1 });

exports = {
    stage1QueueURL: stage2Queue.id,
    stage2QueueURL: stage2Queue.id
};
It looks like pulumi/aws-serverless does not allow sqs.sendMessage to be called. I will continue trying other ways to approach this.
l

lemon-spoon-91807

01/30/2019, 10:12 PM
How can I write to an SQS queue from within a lambda?
c

clever-sunset-76585

01/30/2019, 10:12 PM
Have you tried to import the 
SQS
client directly from the 
aws-sdk
as in my code snippet?
l

lemon-spoon-91807

01/30/2019, 10:12 PM
i would just use the normal aws sdk for this
note: you can get the aws sdk directly either by doing what @clever-sunset-76585 mentions, or by doing: 
aws.sdk
if 
aws
is your imported 
@pulumi/aws
We even show the example here:
Copy code
bucket.onObjectCreated("postToSlack", async (e) => {
  // direct access to the aws-sdk through `aws.sdk`.
  const sqs = new aws.sdk.SQS();
  sqs.sendMessage(/*...*/);
});
Oh, slack just caught up, and i see now that that is mentioend above
nice.
c

clean-car-30047

01/30/2019, 10:14 PM
@lemon-spoon-91807 Thanks for jumping in! Please see the code I am trying to work with, just above ^ This is using new aws.sdk.SQS() / sqs.sendMessage, just as in your example. But it is failing from within the serverless.queue.subscribe context
l

lemon-spoon-91807

01/30/2019, 10:15 PM
Can you send a full example of what's in your lambda?
c

clean-car-30047

01/30/2019, 10:15 PM
That is the full example (hope you can see message that has the code in question.
l

lemon-spoon-91807

01/30/2019, 10:16 PM
as a temp workaround, can you simply do a 
await import('aws-sdk')
t

tall-librarian-49374

01/30/2019, 10:23 PM
Isn't 
aws-serverless
sorta legacy?
l

lemon-spoon-91807

02/03/2019, 1:48 AM
@tall-librarian-49374 Yes. I showed John how to do this without aws-servlerless
Powered by