Amazon DynamoDB is THE database for serverless AWS functions. Its HTTP-based connection mannequin makes integrating with serverless computing companies like AWS Lambda simple. One of many extra capabilities of Amazon DynamoDB is the world tables, which lets you run duplicate situations of the database in different areas.
On this weblog submit, I’ll contact on Amazon DynamoDB world tables within the context of the AWS CDK framework – there are a few nuances with large implications you ought to be conscious of.
Allow us to dive in.
Conventional manner of making world tables
There are two methods I am conscious of to create world tables utilizing AWS CDK, with certainly one of them being, in my humble opinion, a lot better for the maintainability of your software.
Allow us to begin with the “conventional” manner of making the Amazon DynamoDB world tables through the AWS CDK – through the use of the aws_dynamodb.Desk assemble and specifying the replicationRegions property.
const globalTable = new dynamodb.Desk(this, "Desk", {
partitionKey: { identify: "id", kind: dynamodb.AttributeType.STRING },
replicationRegions: ["us-east-1", "us-east-2", "us-west-2"],
billingMode: dynamodb.BillingMode.PROVISIONED
});
Right here I’ve outlined a desk with three replication areas. When you deploy this code snippet within the AWS DynamoDB net console, you will notice that the desk has three replicas – one for every area.
On the floor, all appears to be like good and works as anticipated. However as quickly as you peek “below the curtains” of the assemble, you will notice one thing sudden – that deploying the “Desk” created an AWS CloudFormation customized useful resource alongside the DynamoDB useful resource.
The issue with the customized useful resource
You’ll find the code the customized useful resource in query makes use of right here.
The customized useful resource is answerable for creating the duplicate tables. It makes use of the AWS SDK updateTable API and offers the suitable worth for the ReplicaUpdates property. On the floor, it doesn’t sound unhealthy, however by creating the replicas through the AWS SDK name, we successfully detach these sources out of your AWS CloudFormation template and AWS CDK code.
The implications of this structure are monumental from the maintainability perspective. Listed here are the 2 circumstances of confusion and toil I needed to cope with a number of instances when working world tables deployed through the replicationRegions property.
The DeletionPolicy attribute
If you’re operating a manufacturing workload, it’s thought-about good follow (I might argue a must have) to specify the DeletionPolicy (in AWS CDK, the identify is removalPolicy) of RETAIN for sources which might be stateful and maintain manufacturing knowledge.
When you delete the CloudFormation stack, the CloudFormation will not delete the sources after we set the DeletionPolicy to RETAIN. As you may think about, this will prevent from a catastrophe.
Normally, in AWS CDK, that is achieved through the use of the applyRemovalPolicy methodology on a given assemble.
const globalTable = new cdk.aws_dynamodb.Desk(this, "Desk", {
// ...
});
globalTable.applyRemovalPolicy(cdk.RemovalPolicy.RETAIN);
The issue is that replicas created through the customized useful resource won’t inherit that removalPolicy. This setting will solely apply to the “root” desk.
So, If I had been to delete a CloudFormation stack, the CloudFormation will protect the “root” desk however delete the customized useful resource. Because the customized useful resource triggered with a Delete occasion, it will difficulty the Delete motion for all of the duplicate tables. And with that, your entire duplicate tables are gone.
To verify the Delete occasion by no means will get to the customized useful resource the AWS CDK makes use of, you need to set the DeletionPolicy on the customized useful resource itself. That’s not really easy as you need to “discover” the customized useful resource node within the AWS CDK tree and override its DeletionPolicy attribute. Right here is how I might do it.
const globalTable = new cdk.aws_dynamodb.Desk(this, id, {
// ...
});
/**
* This solely applies to the "root desk" and NOT THE REPLICAS!
*/
globalTable.applyRemovalPolicy(removalPolicy);
/**
* Be certain we don't take away the customized useful resource
*/
const customReplicaResource = globalTable.node.youngsters.discover(baby => {
return (baby as any).useful resource?.cfnResourceType === "Customized::DynamoDBReplica";
}) as cdk.CfnResource;
customReplicaResource.applyRemovalPolicy(removalPolicy);
Enabling point-in-time restoration on the duplicate tables
For manufacturing utilization, enabling point-in-time restoration (PITR) is arguably a should. It’s not solely useful in data-corruption eventualities but additionally for catastrophe restoration. In CDK, making use of the PITR on a worldwide desk (created through the replicationRegions property) may initially appear simple.
const globalTable = new cdk.aws_dynamodb.Desk(this, id, {
pointInTimeRecovery: true
// ...
});
The issue is, equally to the DeletionPolicy now we have checked out earlier, that the pointInTimeRecovery property solely applies to the “root” desk. It’s going to NOT be “forwarded” to the duplicate tables. An auditing instrument will catch that for you within the best-case situation. Within the worst, you won’t be able to get better your duplicate tables if one thing goes fallacious.
All hope shouldn’t be misplaced, although. You can additionally use a workaround to make sure that the duplicate tables have the identical PITR setting as your “root” desk. This includes calling updateContinuousBackups your self through the AwsCustomResource assemble for every area the place the duplicate resides.
const replicationRegions = ["eu-center-1", "eu-west-1"];
const globalTable = new cdk.aws_dynamodb.Desk(this, id, {
pointInTimeRecovery: true,
replicationRegions
// ...
});
for (const replicationRegion of replicationRegions) {
new cdk.custom_resources.AwsCustomResource(this, id, {
onUpdate: {
service: "DynamoDB",
motion: "updateContinuousBackups",
parameters: {
TableName: globalTable.tableName,
PointInTimeRecoverySpecification: {
PointInTimeRecoveryEnabled: true
}
},
area: replicationRegion,
physicalResourceId: cdk.custom_resources.PhysicalResourceId.of(id)
},
coverage: cdk.custom_resources.AwsCustomResourcePolicy.fromSdkCalls({
sources: [globalTable.tableArn]
})
});
}
A special manner of making world tables
As a substitute of utilizing the aws_dynamodb.Desk assemble, think about using the aws_dynamodb.CfnGlobalTable. To the perfect of my information, this useful resource was made accessible to us in Might 2021, and I might extremely encourage you for it to be your default, even when you don’t anticipate utilizing duplicate tables within the fast future in your structure.
If you wish to dive deeper into technical particulars concerning the useful resource to which
CfnGlobalTablecorresponds, look no additional than this glorious weblog submit.
I advocate for the CfnGlobalTable utilization as a result of the gotchas talked about within the earlier part don’t apply to this assemble. For instance, to specify the DeletionPolicy, which might apply to all tables, even the replicas, all it is advisable to do is to make use of the applyRemovalPolicy methodology accessible on the assemble.
const globalTable = new cdk.aws_dynamodb.CfnGlobalTable(this, id, {
// ...
});
globalTable.applyRemovalPolicy(cdk.RemovalPolicy.RETAIN);
What concerning the PITR? To allow it on all of the tables (replicas and the “root” desk) don’t must fiddle with AWS SDK calls by a customized useful resource (or another manner). You possibly can set the PITR for the “root” desk and the duplicate tables.
const globalTable = new cdk.aws_dynamodb.CfnGlobalTable(this, id, {
// ...
replicas: [
{
region: "eu-center-1",
pointInTimeRecoverySpecification: {
pointInTimeRecoveryEnabled: true
}
},
// Let us say that, the root table lives in "eu-west-1"
{
region: "eu-west-1",
pointInTimeRecoverySpecification: {
pointInTimeRecoveryEnabled: true
}
}
]
});
One other vital purpose for utilizing the CfnGlobalTable is that this useful resource makes use of a more recent model of Amazon DynamoDB world tables. You possibly can examine totally different variations of worldwide tables right here.
Migration
I am not but able to publish a migration story right here (I am nonetheless evaluating totally different avenues to take action), so please control my subsequent article. If you wish to carry out migration now, I might base the steps vital on this and this article.
Closing phrases
I hope that the knowledge on this article will prevent some toil and frustrations I needed to undergo whereas working with world tables created through the replicationRegions property in CDK.
As I discussed, the CfnGlobalTable is objectively higher fitted to the job. Sure, you may lose among the niceties of the L2 (L3?) CDK assemble, however utilizing the newer model of the Amazon DynamoDB world tables can pay dividends in the long term.
For extra related content material, please contemplate following me on Twitter – @wm_matuszewski.
Thanks in your time.
