Saturday, September 10, 2022
HomeWordPress Development๐Ÿฌ AWS CDK 101 -๐Ÿ  Ship message throughout accounts utilizing SNS matter...

๐Ÿฌ AWS CDK 101 -๐Ÿ  Ship message throughout accounts utilizing SNS matter and SQS


๐Ÿ”ฐ Newcomers new to AWS CDK, please do have a look at my earlier articles one after the other on this collection.

If in case missed my earlier article, do discover it with the under hyperlinks.

๐Ÿ” Authentic earlier publish at ๐Ÿ”— Dev Publish

๐Ÿ” Reposted the earlier publish at ๐Ÿ”— dev to @aravindvcyber

Additionally, weโ€™ve began to develop an open supply undertaking which weโ€™d be utilizing to mess around with refracting the structure in addition to study CDK stuff on the identical time we are going to present one thing helpful for our group. Discover extra about this mentioned within the article under.

arch

๐Ÿ” Authentic undertaking publish at ๐Ÿ”— Dev Publish

๐Ÿ” Reposted undertaking publish at ๐Ÿ”— dev to @aravindvcyber

event-forwarder Github repo



Cross Account sendMessage ๐Ÿก

Earlier in our article, weโ€™ve seen how one can use customized Eventbridge and SQS by configuring an occasion rule and goal which shifts the messages to the sqs queue and prolonged the identical to distant stacks as properly. Now allow us to make another addition to our stack by retrieving the dlq messages from the distant stack throughout areas to our processor area.

Authentic publish at ๐Ÿ”— Dev Publish

Reposted at ๐Ÿ”— dev to @aravindvcyber

cross messages

To begin with we might be first discussing how one can begin polling the messages from the dlq utilizing a lambda processor.

Earlier than that allow us arrange a lambda layer that may have our exterior dependencies obligatory for logging and monitoring.

export const generateLayerVersion = (
  scope: Assemble,
  layerName: string,
  props: Partial<LayerVersion>
): LayerVersion => {
  return new LayerVersion(scope, layerName, {
    ...defaultLayerProps,
    code: Code.fromAsset(be part of(__dirname, "..", "layers", layerName)),
    ...props,
  });
};

const powertoolsSDK = generateLayerVersion(this, "powertoolsSDK", {});

exportOutput(this, "powertoolsSDKArn", powertoolsSDK.layerVersionArn);

Enter fullscreen mode

Exit fullscreen mode



Lambda processor definition ๐Ÿชด

Right here yow will discover the definition of the lambda operate which might be used to ballot messages from dlq and push to SNS matter.

 const failedMessageAggregator = new Perform(
      this,
      "failedMessageAggregator",
      {
        code: Code.fromAsset("dist/lambda/failed-message-aggregator"),
        handler: "failed-message-aggregator.handler",
        ...commonLambdaProps,
        functionName: "failedMessageAggregator",
        layers: [powertoolsSDK],
        atmosphere: {
          TOPIC_ARN: remoteStackEventTargetDlqSns.topicArn,
          TZ: config.get("timeZone"),
          LOCALE: config.get("locale"),
        },
      }
    );

failedMessageAggregator.applyRemovalPolicy(RemovalPolicy.DESTROY);
Enter fullscreen mode

Exit fullscreen mode

lambda def



Lambda handler code ๐ŸŒท

The total and newest code ought to be discovered within the git hub repo under.

failed-message-aggregator.ts

class Lambda implements LambdaInterface {

  @tracer.captureMethod()
  non-public async processSQSRecord (rec: SQSRecord)  {
    logger.information("Fetching DLQ message:", {rec});
    const params: PublishInput = {
      Message: rec.physique,
      Topic: "Forwarding occasion message to SNS matter",
      TopicArn: course of.env.TOPIC_ARN,
    };
    const snsResult: PublishResponse = await sns.publish(params).promise();
    logger.information("Success", { params, snsResult });
  }

  public async handler(occasion: SQSEvent) {
    attempt {
      await Promise.all(
        occasion.Information.map(async (rec: SQSRecord) => {
          await this.processSQSRecord(rec);
        })
      );
      return {
        statusCode: 200,
        headers: { "Content material-Sort": "textual content/json" },
        physique: {
          EventsReceived: [...event.Records].size,
        },
      };
    } catch (error) {
      logger.error("Error", { error });
      return {
        statusCode: 400,
        headers: { "Content material-Sort": "textual content/json" },
        physique: {
          EventsReceived: [...event.Records].size,
          Error: error
        },
      };
    }
  };

}
Enter fullscreen mode

Exit fullscreen mode



Occasion Supply mapping DLQ to lambda ๐ŸŒณ

Right here we are going to map the distant dlq to set off the lambda which weโ€™ve constructed above.

failedMessageAggregator.addEventSource(
      new SqsEventSource(remoteStackEventTargetDlq.queue, {
        batchSize: 10,
        maxBatchingWindow: Length.seconds(20),
      })
);
Enter fullscreen mode

Exit fullscreen mode

lambda trigger

trigger info



SNS matter to push to subscribers ๐Ÿฆš

This matter might be used to obtain messages from the lambda and push into related subscriber channels. Right here we are going to subscribe this to widespread dlq within the processor stack.

const remoteStackEventTargetDlqSns = new Subject(
      this,
      "remoteStackEventTargetDlqSns",
      {
        displayName: "remoteStackEventTargetDlqSns",
        topicName: "remoteStackEventTargetDlqSns",
      }
);

remoteStackEventTargetDlqSns.applyRemovalPolicy(RemovalPolicy.DESTROY);

exportOutput(
      this,
      "remoteStackEventTargetDlqSnsArn",
      remoteStackEventTargetDlqSns.topicArn
);
Enter fullscreen mode

Exit fullscreen mode



Granting entry to lambda to Ship Message ๐Ÿ‹

Now might be grant entry to the lambda operate to ship messages because the producer.


remoteStackEventTargetDlqSns.grantPublish(failedMessageAggregator);

Enter fullscreen mode

Exit fullscreen mode

sns-lambda-sqs



Two-way handshake to hyperlink SNS to SQS ๐Ÿฅฌ

Almost about sns and sqs in numerous account itโ€™s important to arrange the two-way handshake for this there should be two actions allowed one at every finish.

  • sns:Subscribe in distant matter
  • sqs:SendMessage in client queue (subscriber)



Distant stack configurations



Granting entry to processor account to subscribe

Right here we might be granting entry to processor account sources to subscribe to this matter as follows.

remoteStackEventTargetDlqSns.addToResourcePolicy(
      new PolicyStatement({
        sid: "Cross Account Entry to subscribe",
        impact: Impact.ALLOW,
        principals: [new AccountPrincipal(targetAccount)],
        actions: ["sns:Subscribe"],
        sources: [remoteStackEventTargetDlqSns.topicArn],
      })
);
Enter fullscreen mode

Exit fullscreen mode



Processor stack configurations ๐Ÿ๏ธ


remoteAccounts.map((account: string) => {
      remoteRegions.map((area: string) => {

        // Right here we might be including the reference and the subscription
    });
});
Enter fullscreen mode

Exit fullscreen mode



Referencing to the distant matter

Within the processor stack, we might be getting the reference to the related subjects as follows.

const remoteStackEventTargetDlqSns = Subject.fromTopicArn(
    this,
    `remoteStackEventTargetDlqSns-${area}-${account}`,
    `arn:aws:sns:${area}:${account}:remoteStackEventTargetDlqSns`
);

Enter fullscreen mode

Exit fullscreen mode



Subscribing to the distant matter

Right here we might be subscribing to the processor area dlq to obtain the messages from the distant area SNS matter as follows.

Notice itโ€™s extremely beneficial to subscribe from the buyer stack in order that the subscription will get auto-confirmed, else there might be one other affirmation step chances are youโ€™ll must do from the console or affirmation message to try this your self.

const subProps: SqsSubscriptionProps = {
          rawMessageDelivery: true,
};

remoteStackEventTargetDlqSns.addSubscription(
    new aws_sns_subscriptions.SqsSubscription(
      stackEventTargetDlq.queue,
      subProps
    )
);
Enter fullscreen mode

Exit fullscreen mode

The above subscription setup from the processor stack additionally grants the sqs:SendMessage implicitly whereas the subscription is created.

topic sub

subscription details



Conclusion โ›ฒ

With this method similar to how we pooled the distant cfn occasions to a standard occasion bridge throughout areas and accounts, weโ€™re additionally in a position to get the distant dlq occasions to a standard dlq. These messages in dlq may be inspected with out switching to a different area or account, which the maintainer does not have any entry.

This might be extraordinarily helpful while you construct comparable event-driven options.

We might be speaking about extra comparable engineering ideas as we refactor and refine the occasion forwarder undertaking. Hold following for comparable posts on engineering with IaC primarily utilizing AWS CDK and Serverless.

Additionally, be happy to contribute to the progress of the under answer together with your feedback, and points, perhaps you too can do a pr for those who really feel it could actually assist our group.

event-forwarder Github repo

Arch Diag

๐Ÿ” Authentic undertaking publish at ๐Ÿ”— Dev Publish

๐Ÿ” Reposted undertaking publish at ๐Ÿ”— dev to @aravindvcyber

โญ Weโ€™ve got our subsequent article in serverless and IaC, do take a look at

๐ŸŽ‰ Thanks for supporting! ๐Ÿ™

Could be nice for those who wish to โ˜• Purchase Me a Espresso, to assist enhance my efforts ๐Ÿ˜.

Buy Me a Coffee at ko-fi.com

๐Ÿ” Authentic publish at ๐Ÿ”— Dev Publish

๐Ÿ” Reposted at ๐Ÿ”— dev to @aravindvcyber

RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

- Advertisment -
Google search engine

Most Popular

Recent Comments