https://pulumi.com logo
Powered by
m

microscopic-florist-22719

11/30/2018, 12:54 AM
as a workaround, you can do 
new aws.s3.Bucket('prod-us-west-1', {}, { id: 'sdp-tsm-prod-uw1-artifacts', provider: uw1Provider })
e

early-musician-41645

11/30/2018, 1:12 AM
Would this create a new resource, or get an existing one?
m

microscopic-florist-22719

11/30/2018, 1:38 AM
Get an existing one
This is what 
get
is doing under the covers. The 
id
property in the resource options causes a read rather than a create
e

early-musician-41645

11/30/2018, 4:51 AM
I tried that out and it did indeed get further along, but then blew up on the S3 notification config:
Copy code
Diagnostics:
  aws:s3:BucketNotification (sdp-tsm-s3-artifact-indexer-onNewObject):
    error: Plan apply failed: Error putting S3 notification configuration: BucketRegionError: incorrect region, the bucket is not in 'us-west-2' region
        status code: 301, request id: , host id:
Is the provider not feeding through down to all other resources?
It did manage to create the other resources, though:
Copy code
Updating (s3-object-indexer-dev):

     Type                                Name                                              Status                  Info
 +   pulumi:pulumi:Stack                 s3-object-indexer-s3-object-indexer-dev           created
 +   ├─ pulumi:providers:aws             us-west-1-aws-provider                            created
 >-  ├─ aws:s3:Bucket                    prod-us-west-1                                    read
 +   │  └─ aws:s3:BucketNotification     sdp-tsm-s3-artifact-indexer-onNewObject           **creating failed**     1 error
 +   └─ aws:s3:BucketEventSubscription   sdp-tsm-s3-artifact-indexer-onNewObject           created
 +      ├─ aws:iam:Role                  sdp-tsm-s3-artifact-indexer-onNewObject           created
 +      ├─ aws:iam:RolePolicyAttachment  sdp-tsm-s3-artifact-indexer-onNewObject-32be53a2  created
 +      ├─ aws:lambda:Function           sdp-tsm-s3-artifact-indexer-onNewObject           created
 +      └─ aws:lambda:Permission         sdp-tsm-s3-artifact-indexer-onNewObject           created
m

microscopic-florist-22719

11/30/2018, 4:52 AM
You’ll need to pass that provider to all of the resources that must be in that region, yes
e

early-musician-41645

11/30/2018, 5:00 AM
How do I do that on this?
Copy code
// Trigger a Lamda function when something is added
bucket.onObjectCreated("sdp-tsm-s3-artifact-indexer-onNewObject", onObjectCreatedLambda);
my VS code intellisense isn't showing me a route to specifying the provider
m

microscopic-florist-22719

11/30/2018, 5:50 PM
oh, that’s interesting
Looks like we need an extension point there as well.
cc @lemon-spoon-91807 @white-balloon-205
l

lemon-spoon-91807

11/30/2018, 5:53 PM
Will have to look when I get in. But doesn't onObjectCreated take a ResourceOptions opts?
They all take an optional opts argument. I'm not super familiar with providing providers. However, wouldn't that be how you passed that along here @microscopic-florist-22719?
m

microscopic-florist-22719

11/30/2018, 5:56 PM
Oh! Nice.
Then yes, that’s how you would pass it in.
Admittedly I did not look at the code myself—sorry about that.
l

lemon-spoon-91807

11/30/2018, 6:23 PM
Not a problem. In the office in 10. Will follow-up then!
e

early-musician-41645

11/30/2018, 6:36 PM
I tried this:
Copy code
bucket.onObjectCreated("sdp-tsm-s3-artifact-indexer-onNewObject", onObjectCreatedLambda, {}, { provider: uw1Provider });
And got this:
Copy code
error: TSError: ⨯ Unable to compile TypeScript:
    index.ts(29,96): error TS2345: Argument of type '{ provider: Provider; }' is not assignable to parameter of type 'ResourceOptions'.
      Object literal may only specify known properties, and 'provider' does not exist in type 'ResourceOptions'.
Are the 
ResourceOptions
different per method?
l

lemon-spoon-91807

11/30/2018, 6:59 PM
Looking 🙂
I need @microscopic-florist-22719 To weigh in. Pat, what's the expected coding pattern for providing this extra information? The confusing part for me is that there are three different ResourceOptions types. It's unclear which should be used in this scenario given that we're both creating a component, and we're creating custom resources.
m

microscopic-florist-22719

11/30/2018, 7:05 PM
I think that these should take ComponentResourceArgs in that case.
l

lemon-spoon-91807

11/30/2018, 7:05 PM
actually... i see we're only making a component resource directly. that component resource then creates custom resource children.
agreed.
@early-musician-41645 Simplest thing to do for now is 
.onObjectCreate(..., <pulumi.ComponentResourceArgs>{ provider: Provider })
unfortunate to have to do the cast. but it should work
you can also extract that to a local if you need to use it in several places. i.e.:
const usw1Args: pulumi.ComponentResourceOptions = { provider: uw1Provider }
then pass 
usw1Args
to calls like this.
I'm discussing now with Pat the best way for the pulumi code to be fixed up to make this simpler on your end
Ok: https://github.com/pulumi/pulumi-aws/pull/386 has gone in and will make things nicer.
We're also going to eb discussing; https://github.com/pulumi/pulumi-aws/pull/387
which woudl ideally make it so you don't even need to pass that arg in as long as the bucket is in the same region that you want the subscription resource to be created in
e

early-musician-41645

11/30/2018, 10:08 PM
Here's the result:
Copy code
bucket.onObjectCreated("sdp-tsm-s3-artifact-indexer-onNewObject", onObjectCreatedLambda, <pulumi.ComponentResourceArgs>{ provider: uw1Provider });
Copy code
error: TSError: ⨯ Unable to compile TypeScript:
    index.ts(29,98): error TS2694: Namespace '"/home/tsi/eshamay/git/mustang/sdp-mustang-terraform/pulumi/s3-object-indexer/node_modules/@pulumi/pulumi/index"' has no exported member 'ComponentResourceArgs'.
It seems that some classes/methods take 
ResourceOptions
for provider, some take 
ComponentResourceArgs
, and in some cases I have to do a type cast, etc. It's a bit confusing. Either way, still stuck because I can't pass the provider into the 
onObjectCreated
method
l

lemon-spoon-91807

11/30/2018, 10:12 PM
it should be ComponentResourceOptions 🙂
sorry if i wrote 'args'.
don't always have a compiler in front of me to check my work 🙂
e

early-musician-41645

11/30/2018, 10:13 PM
still doesn't work:
Copy code
index.ts(29,98): error TS2694: Namespace '"/home/tsi/eshamay/git/mustang/sdp-mustang-terraform/pulumi/s3-object-indexer/node_modules/@pulumi/pulumi/index"' has no exported member 'CompontentResourceOptions'.
l

lemon-spoon-91807

11/30/2018, 10:13 PM
no 't' in Compontent
just Component
e

early-musician-41645

11/30/2018, 10:14 PM
my bad
still doesn't work:
Copy code
index.ts(29,90): error TS2559: Type 'ComponentResourceOptions' has no properties in common with type 'ObjectCreatedSubscriptionArgs'.
l

lemon-spoon-91807

11/30/2018, 10:14 PM
intellisense should help you hre
here
right, this is the last arg to pass along
ok. so full example (again with no compiler in front of me to verify) should be something like
e

early-musician-41645

11/30/2018, 10:15 PM
Copy code
error: Error: Explicit providers may not be used with component resources
l

lemon-spoon-91807

11/30/2018, 10:15 PM
.onObjectCreated(name, handler, args, opts) i ebelieve
so, we're trying to pass along the 'opts' argument
e

early-musician-41645

11/30/2018, 10:16 PM
Here's the code so far:
Copy code
bucket.onObjectCreated("sdp-tsm-s3-artifact-indexer-onNewObject", onObjectCreatedLambda, {}, <pulumi.ComponentResourceOptions>{ provider: uw1Provider });
Here's the current error copied from above:
Copy code
error: Error: Explicit providers may not be used with component resources
l

lemon-spoon-91807

11/30/2018, 10:17 PM
@microscopic-florist-22719 as the best way to solve this. As this is a component and not a custom-resource, you need to pass 'providers'... but i'm not certain the right form that should have
@microscopic-florist-22719 can you help out here?
m

microscopic-florist-22719

11/30/2018, 10:17 PM
{ providers: { aws: uw1Provider } }
give that a go
l

lemon-spoon-91807

11/30/2018, 10:19 PM
i'm walking back to the office now
e

early-musician-41645

11/30/2018, 10:19 PM
that works, thanks. I haven't followed the issues, but it would be nice to have a standard format for this.
Also, now that all the code errors are resolved there's a deeper error indicating that passing the provider had no effect:
Copy code
error: Plan apply failed: Error putting S3 notification configuration: BucketRegionError: incorrect region, the bucket is not in 'us-west-2' region
        status code: 301, request id: , host id:
The code generating that error is this:
Copy code
// @ts-check
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Lambda handlers that processes the object creation and removal events in S3 and adds corresponding
// items to a DynamoDB table for indexing the objects.
let onObjectCreatedLambda = function(event: any, context: any, callback: any) {
    let record = event.Records[0];
    console.log("==== New object created");
    console.log("Event: "+JSON.stringify(event));
    console.log("Event Type: "+record.eventName);
    console.log("Region: "+record.awsRegion);
    console.log("Bucket: "+record.bucket.name);
    console.log("Object: "+JSON.stringify(record.s3.object));
}

// Fetch all the buckets that need the lambda attached

// Fetch an existing bucket
let uw1Provider = new aws.Provider("us-west-1-aws-provider", { region: aws.USWest1Region });
const bucket = new aws.s3.Bucket('prod-us-west-1', {}, { id: 'sdp-tsm-prod-uw1-artifacts', provider: uw1Provider })

// Create a new bucket
//const bucket = new aws.s3.Bucket("eshamay-test-bucket");

const bucketName = bucket.id;

// Trigger a Lamda function when something is added
bucket.onObjectCreated("sdp-tsm-s3-artifact-indexer-onNewObject", onObjectCreatedLambda, {}, <pulumi.ComponentResourceOptions>{ providers: { aws: uw1Provider } });

// Export the bucket name.
exports.bucketName = bucketName;
l

lemon-spoon-91807

11/30/2018, 10:30 PM
To narrow things down, are you getting that if you remove all the lambda+objCreated code?
In other words, if you just try to make the bucket does that work?
e

early-musician-41645

11/30/2018, 10:35 PM
works fine without that 
onObjectCreated
line
with it things fail
Copy code
Updating (s3-object-indexer-dev):

     Type                                Name                                              Status                  Info
     pulumi:pulumi:Stack                 s3-object-indexer-s3-object-indexer-dev
 >-  ├─ aws:s3:Bucket                    prod-us-west-1                                    read
 +   │  └─ aws:s3:BucketNotification     sdp-tsm-s3-artifact-indexer-onNewObject           **creating failed**     1 error
 +   └─ aws:s3:BucketEventSubscription   sdp-tsm-s3-artifact-indexer-onNewObject           created
 +      ├─ aws:iam:Role                  sdp-tsm-s3-artifact-indexer-onNewObject           created
 +      ├─ aws:iam:RolePolicyAttachment  sdp-tsm-s3-artifact-indexer-onNewObject-32be53a2  created
 +      ├─ aws:lambda:Function           sdp-tsm-s3-artifact-indexer-onNewObject           created
 +      └─ aws:lambda:Permission         sdp-tsm-s3-artifact-indexer-onNewObject           created

Diagnostics:
  aws:s3:BucketNotification (sdp-tsm-s3-artifact-indexer-onNewObject):
    error: Plan apply failed: Error putting S3 notification configuration: BucketRegionError: incorrect region, the bucket is not in 'us-west-2' region
        status code: 301, request id: , host id:
l

lemon-spoon-91807

11/30/2018, 10:36 PM
Great. thanks for the extra info. investigating now
I'll need to talk to @microscopic-florist-22719 about this.
e

early-musician-41645

11/30/2018, 10:38 PM
okay, ping me later on if you have a workaround
l

lemon-spoon-91807

11/30/2018, 10:58 PM
Hey Eric. We have a handle on teh problem, and are going to try to make a quick fix on our end. We'll then try to cut a new release for you as soon as possible.
Unfortunately, i don't think there's a workaround you can apply locally. We're also going to open an issue to track some more medium work around improving the design here to prevent these problems in the future.
e

early-musician-41645

11/30/2018, 11:08 PM
👍
l

lemon-spoon-91807

11/30/2018, 11:08 PM
On a good note: we just learned a bunch and we think we can make this better in the medium term.
Before proceeding though, would you be willing to try editing your local node_modules version:
node_modules/@pulumi/aws/s3/s3Mixins.js
one sec.
Actually, we need to make two cahnges. so it's not totally simple to have you try this out locally.
e

early-musician-41645

11/30/2018, 11:42 PM
happy to try, let me know if you have a build to try out
l

lemon-spoon-91807

11/30/2018, 11:43 PM
Ok. Could you make this change locally:
you'll be editing: 
node_modules/@pulumi/aws/s3/s3Mixins.js
maybe back up the contents first to s3Mixins.bak
then replace contents with:
e

early-musician-41645

11/30/2018, 11:56 PM
give me 15 mins, got another thread going on
l

lemon-spoon-91807

11/30/2018, 11:57 PM
No worries 🙂
e

early-musician-41645

12/01/2018, 6:10 PM
@lemon-spoon-91807 That code worked to fix the bucket notification issue. Thanks! I've now got the lambda and notification working and just verified it's all wired up correctly.
Let me know when you cut a new build and I'll upgrade my node packages to match
2 Views
Powered by