🔰 Newcomers new to AWS CDK, please do have a look at my earlier articles one after the other on this collection.
If in case missed my earlier article, do discover it with the under hyperlinks.
🔁 Authentic earlier publish at 🔗 Dev Publish
🔁 Reposted the earlier publish at 🔗 dev to @aravindvcyber
Additionally, we’ve began to develop an open supply undertaking which we’d be utilizing to mess around with refracting the structure in addition to study CDK stuff on the identical time we are going to present one thing helpful for our group. Discover extra about this mentioned within the article under.
🔁 Authentic undertaking publish at 🔗 Dev Publish
🔁 Reposted undertaking publish at 🔗 dev to @aravindvcyber
Cross Account sendMessage 🍡
Earlier in our article, we’ve seen how one can use customized Eventbridge and SQS by configuring an occasion rule and goal which shifts the messages to the sqs queue and prolonged the identical to distant stacks as properly. Now allow us to make another addition to our stack by retrieving the dlq messages from the distant stack throughout areas to our processor area.
Authentic publish at 🔗 Dev Publish
Reposted at 🔗 dev to @aravindvcyber
To begin with we might be first discussing how one can begin polling the messages from the dlq utilizing a lambda processor.
Earlier than that allow us arrange a lambda layer that may have our exterior dependencies obligatory for logging and monitoring.
export const generateLayerVersion = (
scope: Assemble,
layerName: string,
props: Partial<LayerVersion>
): LayerVersion => {
return new LayerVersion(scope, layerName, {
...defaultLayerProps,
code: Code.fromAsset(be part of(__dirname, "..", "layers", layerName)),
...props,
});
};
const powertoolsSDK = generateLayerVersion(this, "powertoolsSDK", {});
exportOutput(this, "powertoolsSDKArn", powertoolsSDK.layerVersionArn);
Lambda processor definition 🪴
Right here yow will discover the definition of the lambda operate which might be used to ballot messages from dlq and push to SNS matter.
const failedMessageAggregator = new Perform(
this,
"failedMessageAggregator",
{
code: Code.fromAsset("dist/lambda/failed-message-aggregator"),
handler: "failed-message-aggregator.handler",
...commonLambdaProps,
functionName: "failedMessageAggregator",
layers: [powertoolsSDK],
atmosphere: {
TOPIC_ARN: remoteStackEventTargetDlqSns.topicArn,
TZ: config.get("timeZone"),
LOCALE: config.get("locale"),
},
}
);
failedMessageAggregator.applyRemovalPolicy(RemovalPolicy.DESTROY);
Lambda handler code 🌷
The total and newest code ought to be discovered within the git hub repo under.
class Lambda implements LambdaInterface {
@tracer.captureMethod()
non-public async processSQSRecord (rec: SQSRecord) {
logger.information("Fetching DLQ message:", {rec});
const params: PublishInput = {
Message: rec.physique,
Topic: "Forwarding occasion message to SNS matter",
TopicArn: course of.env.TOPIC_ARN,
};
const snsResult: PublishResponse = await sns.publish(params).promise();
logger.information("Success", { params, snsResult });
}
public async handler(occasion: SQSEvent) {
attempt {
await Promise.all(
occasion.Information.map(async (rec: SQSRecord) => {
await this.processSQSRecord(rec);
})
);
return {
statusCode: 200,
headers: { "Content material-Sort": "textual content/json" },
physique: {
EventsReceived: [...event.Records].size,
},
};
} catch (error) {
logger.error("Error", { error });
return {
statusCode: 400,
headers: { "Content material-Sort": "textual content/json" },
physique: {
EventsReceived: [...event.Records].size,
Error: error
},
};
}
};
}
Occasion Supply mapping DLQ to lambda 🌳
Right here we are going to map the distant dlq to set off the lambda which we’ve constructed above.
failedMessageAggregator.addEventSource(
new SqsEventSource(remoteStackEventTargetDlq.queue, {
batchSize: 10,
maxBatchingWindow: Length.seconds(20),
})
);
SNS matter to push to subscribers 🦚
This matter might be used to obtain messages from the lambda and push into related subscriber channels. Right here we are going to subscribe this to widespread dlq within the processor stack.
const remoteStackEventTargetDlqSns = new Subject(
this,
"remoteStackEventTargetDlqSns",
{
displayName: "remoteStackEventTargetDlqSns",
topicName: "remoteStackEventTargetDlqSns",
}
);
remoteStackEventTargetDlqSns.applyRemovalPolicy(RemovalPolicy.DESTROY);
exportOutput(
this,
"remoteStackEventTargetDlqSnsArn",
remoteStackEventTargetDlqSns.topicArn
);
Granting entry to lambda to Ship Message 🍋
Now might be grant entry to the lambda operate to ship messages because the producer.
remoteStackEventTargetDlqSns.grantPublish(failedMessageAggregator);
Two-way handshake to hyperlink SNS to SQS 🥬
Almost about sns and sqs in numerous account it’s important to arrange the two-way handshake for this there should be two actions allowed one at every finish.
-
sns:Subscribein distant matter -
sqs:SendMessagein client queue (subscriber)
Distant stack configurations
Granting entry to processor account to subscribe
Right here we might be granting entry to processor account sources to subscribe to this matter as follows.
remoteStackEventTargetDlqSns.addToResourcePolicy(
new PolicyStatement({
sid: "Cross Account Entry to subscribe",
impact: Impact.ALLOW,
principals: [new AccountPrincipal(targetAccount)],
actions: ["sns:Subscribe"],
sources: [remoteStackEventTargetDlqSns.topicArn],
})
);
Processor stack configurations 🏝️
remoteAccounts.map((account: string) => {
remoteRegions.map((area: string) => {
// Right here we might be including the reference and the subscription
});
});
Referencing to the distant matter
Within the processor stack, we might be getting the reference to the related subjects as follows.
const remoteStackEventTargetDlqSns = Subject.fromTopicArn(
this,
`remoteStackEventTargetDlqSns-${area}-${account}`,
`arn:aws:sns:${area}:${account}:remoteStackEventTargetDlqSns`
);
Subscribing to the distant matter
Right here we might be subscribing to the processor area dlq to obtain the messages from the distant area SNS matter as follows.
Notice it’s extremely beneficial to subscribe from the buyer stack in order that the subscription will get auto-confirmed, else there might be one other affirmation step chances are you’ll must do from the console or affirmation message to try this your self.
const subProps: SqsSubscriptionProps = {
rawMessageDelivery: true,
};
remoteStackEventTargetDlqSns.addSubscription(
new aws_sns_subscriptions.SqsSubscription(
stackEventTargetDlq.queue,
subProps
)
);
The above subscription setup from the processor stack additionally grants the sqs:SendMessage implicitly whereas the subscription is created.
Conclusion ⛲
With this method similar to how we pooled the distant cfn occasions to a standard occasion bridge throughout areas and accounts, we’re additionally in a position to get the distant dlq occasions to a standard dlq. These messages in dlq may be inspected with out switching to a different area or account, which the maintainer does not have any entry.
This might be extraordinarily helpful while you construct comparable event-driven options.
We might be speaking about extra comparable engineering ideas as we refactor and refine the occasion forwarder undertaking. Hold following for comparable posts on engineering with IaC primarily utilizing AWS CDK and Serverless.
Additionally, be happy to contribute to the progress of the under answer together with your feedback, and points, perhaps you too can do a pr for those who really feel it could actually assist our group.
🔁 Authentic undertaking publish at 🔗 Dev Publish
🔁 Reposted undertaking publish at 🔗 dev to @aravindvcyber
⏭ We’ve got our subsequent article in serverless and IaC, do take a look at
🎉 Thanks for supporting! 🙏
Could be nice for those who wish to ☕ Purchase Me a Espresso, to assist enhance my efforts 😍.
🔁 Authentic publish at 🔗 Dev Publish
🔁 Reposted at 🔗 dev to @aravindvcyber
