https://pulumi.com logo
Join Slack
Powered by
The new CloudWatch Dashboard stuff added to awsx i...
# general
f
The new CloudWatch Dashboard stuff added to awsx in 0.18.2 looks nice! Will there be support for getting 
.metrics
from a 
FargateService
?
It'd be nice to have some examples here https://github.com/pulumi/pulumi-awsx/blob/master/nodejs/examples/dashboards/index.ts that show how to create a Metric widget if the resource you're using doesn't have a 
.metrics
property.
w
cc @lemon-spoon-91807
l
Makes sense. Could you give an example of something that is missing a '.metrics' property that you would like?
i tried to hit the major areas
f
A Fargate service exposes CPU and Memory metrics.
l
riht
so that's def available today 🙂
that's in awsx.ecs.metrics
if it's a metric i missed, you can always make it by doing: 
new awsx.cloudwatch.Metric({ args })
which is also how you'd make them for your own metrics. though i def agree we should doc/example this well 🙂
f
Oh I read the example wrong! I thought it was pulling the metrics directory from a FargateService instance. I see that property now. Will try it out.
l
right! note: we're publshing 0.18.3 in teh next 30 minutes
i originally considered having these be on the isntance. but it turned out to be less nice than i originally thought
so now it's an aspect of the package
f
I like this interface more than directly on the instance.
l
note: for ecs, iv'e tried to make it quite simple. so you can just do 
awsx.ecs.metrics.memoryUtilization({ service })
which is actually nicer than using the raw AWS api
since htat requires you to pass along info about the cluster and the service. but we can infer that for you from just the service.
f
Oh, so much better. Switching to this method will probably remove ~100 lines of ugly dashboard code.
l
it's the same for us
i would def love to hear your thoughts on usability/flexibility of this system
f
One thing that is a common pattern for me is to create graphs like this:
Copy code
new awsx.cloudwatch.LineGraphMetricWidget({
    title: "Metrics Streams CPU",
    metrics: [
        awsx.ecs.metrics.cpuUtilization({
            cluster: dashboardArgs.metricsStreamsService.cluster,
            service: dashboardArgs.metricsStreamsService.service,
            statistic: "Minimum",
            label: "min"
        }),
        awsx.ecs.metrics.cpuUtilization({
            cluster: dashboardArgs.metricsStreamsService.cluster,
            service: dashboardArgs.metricsStreamsService.service,
            statistic: "Maximum",
            label: "max"
        })
    ]
})
One of those would be created per service, resulting in a lot of copy/paste. Not sure if this is something that could be handled by this API.
l
pulumi is code! 😄
write a 
for
-loop over your services
✔️ 1
you can also simplify:
Copy code
awsx.ecs.metrics.cpuUtilization({
            cluster: dashboardArgs.metricsStreamsService.cluster,
            service: dashboardArgs.metricsStreamsService.service,
into:
Copy code
awsx.ecs.metrics.cpuUtilization({
            service: dashboardArgs.metricsStreamsService,
f
Oh, cool!
l
cluster is optional if 'service' is passed an awsx.ecs.Service (not an aws.ecs.Service)
because awsx.ecs.Service points-to its Cluster
let me know if the dashboard/widgets api makes sense
👍 1
also, based onhte above, it sounds like you're using Fargate and not EC2?
f
Right
l
If so, then this won't apply, but i did a bunch of work to make it easier to work with AutoScalingGroups
f
How would I pass Outputs to the 
widgets
? For example:
Copy code
function fargateServiceCpuMemGraphs(service: awsx.ecs.FargateService) {
    return service.service.name.apply(serviceName => {
        return [
            new awsx.cloudwatch.LineGraphMetricWidget({
                title: `${serviceName} CPU`,
                metrics: [
                    awsx.ecs.metrics.cpuUtilization({
                        service: service,
                        statistic: "Minimum",
                        label: "min"
                    }),
                    awsx.ecs.metrics.cpuUtilization({
                        service: service,
                        statistic: "Maximum",
                        label: "max"
                    })
                ]
            }),
            new awsx.cloudwatch.LineGraphMetricWidget({
                title: `${serviceName} Memory`,
                metrics: [
                    awsx.ecs.metrics.memoryUtilization({
                        service: service,
                        statistic: "Minimum",
                        label: "min"
                    }),
                    awsx.ecs.metrics.memoryUtilization({
                        service: service,
                        statistic: "Maximum",
                        label: "max"
                    })
                ]
            })
        ];

    });
}
new awsx.cloudwatch.Dashboard("cw-dashboard-" + env, {
    name: `vm-scaler-${env}`,
    widgets: pulumi.all([
        fargateServiceCpuMemGraphs(dashboardArgs.workloadMetricsWorkerService),
        fargateServiceCpuMemGraphs(dashboardArgs.workloadMetricsWorkerService)
    ]).apply(function ([graphs1, graphs2]) {
        return graphs1.concat(graphs2);
    })
});
l
much easier:
f
Changing the function to this works:
Copy code
function fargateServiceCpuMemGraphs(service: awsx.ecs.FargateService) {
    const serviceName = service.service.name;
    return [
        new awsx.cloudwatch.LineGraphMetricWidget({
            title: serviceName.apply(n => `${n} CPU`),
            metrics: [
                awsx.ecs.metrics.cpuUtilization({
                    service: service,
                    statistic: "Minimum",
                    label: "min"
                }),
                awsx.ecs.metrics.cpuUtilization({
                    service: service,
                    statistic: "Maximum",
                    label: "max"
                })
            ]
        }),
        new awsx.cloudwatch.LineGraphMetricWidget({
            title: serviceName.apply(n => `${n} Memory`),
            metrics: [
                awsx.ecs.metrics.memoryUtilization({
                    service: service,
                    statistic: "Minimum",
                    label: "min"
                }),
                awsx.ecs.metrics.memoryUtilization({
                    service: service,
                    statistic: "Maximum",
                    label: "max"
                })
            ]
        })
    ];
}
l
Copy code
title: pulumi.interpolate `${service.name} CPU`,
f
Oh, interesting. Is that new?
l
for a few versions now 🙂
service: service,
is also redundant in JS. but it's a personal pref. you an say: 
service
f
`pulumi.interpolate(
${serviceName} CPU
),` gives 
Argument of type 'string' is not assignable to parameter of type 'TemplateStringsArray'
. Coming from Clojure, I prefer the explicit syntax 🙂
l
right, you write it as: title:
Copy code
pulumi.interpolate`${service.name} CPU`
that's how JS/TS does interpolations
if you do
Copy code
foo(`${...} bar`)
then the inner
Copy code
`${...} bar`
is just a normal interpolation
that is reoslved, converted to string, then passed to 'foo'
to pass 
foo
the actual template pieces and args, you do
Copy code
foo`${...} bar`
note: your approach of
Copy code
serviceName.apply(n => `${n} CPU`),
is also fine
it's effectively what 
interpolate
is doing
f
Oh gotcha, that was new syntax to me. Thanks for the explanation 🙂
l
just briefer (and nicer if you have multiple values you want to go 'in the holes')
i.e.
Copy code
pulumi.interpolate`${cluster.name} - ${service.name}`
much nicer than
Copy code
pulumi.all([cluster.name, service.name]).apply(([cn, sn]) => `${sn} - {cn}`)
f
Curious if an API like this would be supported for the Dashboard:
Copy code
widgetMatrix: [[r1c1 r1c2, ..., r1cN], [r2c1, r2c2, ... r2cN]]
Makes specifying rows & cols pretty clean.
l
great point. right now you can almost do that, but you'd have to specify the "new RowWidget" yourself
but def should support the above
so you can do: 
widgets: [new RowWidget(r1c1, r1c2, ..., r1cN), new RowWidet(r2c1, r2c2, ... r2cN)]
so almost as clean. but i agree that the array-of-arrays should be supported
f
I also suggest a RowWidget ctor that does not take var args.
l
how come?
f
Because I may want to pass a list I created to the RowWidget without needing to do this https://stackoverflow.com/questions/1606797/use-of-apply-with-new-operator-is-this-possible
l
you can do this: 
new RowWidget(...list)
🙂
✨ 1
brb
but i can def add support for just taking an iterable directly
f
Copy code
function rows(rs: awsx.cloudwatch.Widget[][]) {
    return rs.map(cols => new awsx.cloudwatch.RowWidget(...cols))
}
😄
l
there we go 🙂
f
Getting somewhat unexpected rows in the CloudWatch UI. I have a 
RowWidget
with 5 
LineGraphMetricWidget
and then several more row widgets with 2 
LineGraphMetricWidget
each. Example:
Copy code
[[r1c1, r1c2, r1c3, r1c4, r1c5]
 [r2c1, r2c2]
 [r3c1, r3c2]]
But in the CloudWatch UI, it renders like this:
Copy code
[[r1c1, r1c2, r1c3, r1c4]
 [r1c5, r2c1]
 [r2c2, r3c1]
 [r3c2]]
I would've expected this:
Copy code
[[r1c1, r1c2, r1c3, r1c4]
 [r1c5]
 [r2c1, r2c2]
 [r3c1, r3c2]]
l
cloudwatch has a 24 column limit
are you going past that?
f
Don't know. I didn't set any column width on the charts so it's using the default.
l
so the critical parts are:
[Dashboard]s are represented by a grid of columns 24 wide, with an unlimited number of rows.
and
The width of the widget in grid units (in a 24-column grid). The default is 6.
Represents a horizontal sequence of [Widget]s in the [Dashboard]. Widgets are laid out horizontally in the grid until it would go past the max width of 24 columns. When that happens, the widgets will wrap to the next available grid row.
so, basically, you have 5 items, each 6 'grid units' wide
so it's going up to 30 grid-width, but cloudwatch dashboards are 24 width max
so we wrapped
f
Right - wrapping is ok. The way it wrapped seems wrong.
l
oh... i'm sorry
yes
can you show me your code you wrote?
f
Copy code
import * as awsx from "@pulumi/awsx";
import * as pulumi from "@pulumi/pulumi";

interface DashArgs {
    metricsStreamsService: awsx.ecs.FargateService
    aggregateStreamsService: awsx.ecs.FargateService
    modelExecutorService: awsx.ecs.FargateService
    modelUpdaterService: awsx.ecs.FargateService
    workloadMetricsWorkerService: awsx.ecs.FargateService
    appIonsService: awsx.ecs.FargateService,
    appIonsLb: awsx.elasticloadbalancingv2.ApplicationLoadBalancer
}

function fargateServiceCpuMemGraphs(service: awsx.ecs.FargateService) {
    const serviceName = service.service.name;
    return [
        new awsx.cloudwatch.LineGraphMetricWidget({
            title: pulumi.interpolate`${serviceName} CPU`,
            metrics: [
                awsx.ecs.metrics.cpuUtilization({
                    service: service,
                    statistic: "Minimum",
                    label: "min"
                }),
                awsx.ecs.metrics.cpuUtilization({
                    service: service,
                    statistic: "Maximum",
                    label: "max"
                })
            ]
        }),
        new awsx.cloudwatch.LineGraphMetricWidget({
            title: pulumi.interpolate`${serviceName} Memory`,
            metrics: [
                awsx.ecs.metrics.memoryUtilization({
                    service: service,
                    statistic: "Minimum",
                    label: "min"
                }),
                awsx.ecs.metrics.memoryUtilization({
                    service: service,
                    statistic: "Maximum",
                    label: "max"
                })
            ]
        })
    ];
}

function rows(rs: awsx.cloudwatch.Widget[][]) {
    return rs.map(cols => new awsx.cloudwatch.RowWidget(...cols))
}

export function createDashboard(env: string, dashboardArgs: DashArgs) {
    // dashboardArgs.appIonsService.service.loadBalancers
    new awsx.cloudwatch.Dashboard("cw-dashboard-" + env, {
        name: `vm-scaler-${env}`,
        widgets: rows([
            fargateServiceCpuMemGraphs(dashboardArgs.appIonsService)
                .concat([
                    new awsx.cloudwatch.LineGraphMetricWidget({
                        title: "HTTP Target Response Time",
                        metrics: [awsx.elasticloadbalancingv2.metrics.application.targetResponseTime(dashboardArgs.appIonsLb)]
                    }),
                    new awsx.cloudwatch.LineGraphMetricWidget({
                        title: "HTTP Requests",
                        metrics: [awsx.elasticloadbalancingv2.metrics.application.requestCount(dashboardArgs.appIonsLb)]
                    }),
                    new awsx.cloudwatch.LineGraphMetricWidget({
                        title: "HTTP 5XXs",
                        metrics: [awsx.elasticloadbalancingv2.metrics.application.httpCodeTarget5XXCount(dashboardArgs.appIonsLb)]
                    })]),
            fargateServiceCpuMemGraphs(dashboardArgs.workloadMetricsWorkerService),
            fargateServiceCpuMemGraphs(dashboardArgs.metricsStreamsService),
            fargateServiceCpuMemGraphs(dashboardArgs.aggregateStreamsService),
            fargateServiceCpuMemGraphs(dashboardArgs.modelExecutorService),
            fargateServiceCpuMemGraphs(dashboardArgs.modelUpdaterService)
        ])
    });
}
l
i'm somewhat amazed this compiled
something is wonky afaict
oh wait, maybe not
was confused by all the parentheses
f
Is the code style bad? I write Lisp so I'm not super familiar with TS code style... 🙃
l
not at all
i'm trying to find any fault with it, but it all seems reasonable
may just be a bug in the core lib...
but surprising to me because i tested cases like these
let me see if i can repro locally
f
Ok
l
ok made a test running now
ok strange. i condensed this down to
DM'ed you so i could post snippets
one. can we check the dashboard json you ended up producing?
(it can be found either in the pulumi stack page. or by going to hte actual cloudwatch dashboard
For example, the first 5 resources are at 0,0 6,0 12,0 18,0 and 0,6 as expected
the next two are at 0,12, and 6,12 as expected
So i repro the issue
we make the right dimensions, but the json on cloudwatch itself is off
going to check the forums
The crazy thing is that if you just literally make a new dashboard and paste in teh code pulumi produces, AWS overrides it
don't have permission to post in the forum. will try tomorrow once my forum account activates
i'm hoping this is some sort of stranggggggggggge bug in AWS
f
That's wild haha.
9 Views
Open in Slack
PreviousNext
Powered by