Utilizing AWS Lambda, DynamoDB and API Gateway
This weblog submit covers the best way to construct a Serverless URL shortener software utilizing Go. It leverages AWS Lambda for enterprise logic, DynamoDB for persistence and API Gateway to supply the HTTP endpoints to entry and use the appliance. The pattern software offered on this weblog is a trimmed down model of bit.ly or different options you’ll have used or encountered.
It is structured as follows:
- I’ll begin off with a fast introduction and dive into the best way to deploy attempt the answer.
- After that, I’ll concentrate on the code itself. This can cowl:
- The half which is used to put in writing the infrastructure (utilizing Go bindings for AWS CDK)
- And in addition the core enterprise logic which comprises the Lambda perform (utilizing Lambda Go assist) in addition to the DynamoDB operations (utilizing the DynamoDB Go SDK)
On this weblog, you’ll be taught:
- Find out how to use the DynamoDB Go SDK (v2) to execute CRUD operations resembling
PutItem,GetItem,UpdateItemandDeleteItem - Find out how to use AWS CDK Go bindings to deploy a Serverless software to create and handle a
DynamoDBdesk, Lambda features, API Gateway and different elements as nicely.
When you deploy the appliance, it is possible for you to to create brief codes for URLs utilizing the endpoint uncovered by the API Gateway and in addition entry them.
# create brief code for a URL (e.g. https://abhirockzz.github.io/)
curl -i -X POST -d 'https://abhirockzz.github.io/' -H 'Content material-Sort: textual content/plain' $URL_SHORTENER_APP_URL
# entry URL by way of brief code
curl -i $URL_SHORTENER_APP_URL/<short-code>
Lets get began – Deploy the Serverless software
Earlier than you proceed, be sure you have the Go programming language (v1.16 or greater) and AWS CDK put in.
Clone the undertaking and alter to the appropriate listing:
git clone https://github.com/abhirockzz/serverless-url-shortener-golang
cd cdk
To begin the deployment…
.. all you’ll do is run a single command (cdk deploy), and watch for a bit. You will note a (lengthy) record of assets that can be created and might want to present your affirmation to proceed.
Don’t be concerned, within the subsequent part I’ll clarify what’s occurring.
cdk deploy
# output
Bundling asset ServerlessURLShortenerStack1/create-url-function/Code/Stage...
Bundling asset ServerlessURLShortenerStack1/access-url-function/Code/Stage...
Bundling asset ServerlessURLShortenerStack1/update-url-status-function/Code/Stage...
Bundling asset ServerlessURLShortenerStack1/delete-url-function/Code/Stage...
✨ Synthesis time: 10.28s
This deployment will make probably delicate modifications based on your present safety approval degree (--require-approval broadening).
Please affirm you propose to make the next modifications:
.......
Do you want to deploy these modifications (y/n)?
This can begin creating the AWS assets required for our software.
If you wish to see the AWS CloudFormation template which can be used behind the scenes, run
cdk synthand verify thecdk.outfolder
You possibly can preserve monitor of the progress within the terminal or navigate to AWS console: CloudFormation > Stacks > ServerlessURLShortenerStack
As soon as all of the assets are created, you’ll be able to check out the appliance. It’s best to have:
- 4 Lambda features (and associated assets)
- A DynamoDB desk
- An API Gateway (in addition to routes, integrations)
- together with a couple of others (like IAM roles and many others.)
Earlier than you proceed, get the API Gateway endpoint that you’ll want to make use of. It is out there within the stack output (within the terminal or the Outputs tab within the AWS CloudFormation console in your Stack):
Shorten some URLs!
Begin by producing brief codes for a couple of URLs
# export the API Gateway endpoint
export URL_SHORTENER_APP_URL=<exchange with apigw endpoint above>
# for instance:
export URL_SHORTENER_APP_URL=https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/
# invoke the endpoint to create brief codes
curl -i -X POST -d 'https://abhirockzz.github.io/' -H 'Content material-Sort: textual content/plain' $URL_SHORTENER_APP_URL
curl -i -X POST -d 'https://dzone.com/customers/456870/abhirockzz.html' -H 'Content material-Sort: textual content/plain' $URL_SHORTENER_APP_URL
curl -i -X POST -d 'https://abhishek1987.medium.com/' -H 'Content material-Sort: textual content/plain' $URL_SHORTENER_APP_URL
To generate a brief code, you must go the unique URL within the payload physique as a part of a HTTP POST request (for e.g. https://abhishek1987.medium.com/)
‘Content material-Sort: textual content/plain’ is vital, in any other case API Gateway will do
base64encoding of your payload
If all goes nicely, you need to get a HTTP 201 together with the brief code within the HTTP response (as a JSON payload).
HTTP/2 201
date: Fri, 15 Jul 2022 13:03:20 GMT
content-type: textual content/plain; charset=utf-8
content-length: 25
apigw-requestid: VTzPsgmSoAMESdA=
{"short_code":"1ee3ad1b"}
To substantiate, verify the DynamoDB desk.
Discover an lively attribute there? Extra on this quickly
Entry the URL utilizing the brief code
With companies like bit.ly and many others. you sometimes create brief hyperlinks in your URLs and share them with the world. We are going to do one thing comparable. Now that you’ve the brief code generated, you’ll be able to share the hyperlink (it is probably not a brief hyperlink like bit.ly however that is okay for now!) with others and as soon as they entry it, they’d see the unique URL.
The entry hyperlink can have the next format – <URL_SHORTENER_APP_URL>/<generated brief code> for e.g. https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/1ee3ad1b
If you happen to navigate to the hyperlink utilizing a browser, you can be mechanically redirected to the unique URL that you simply had specified. To see what is going on on, attempt the identical with curl:
curl -i $URL_SHORTENER_APP_URL/<brief code>
# instance
curl -i https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/0e1785b1
That is merely an HTTP GET request. If all goes nicely, you need to get an HTTP 302 response (StatusFound) and the URL re-direction occurs because of the the Location HTTP header which comprises the unique URL.
HTTP/2 302
date: Fri, 15 Jul 2022 13:08:54 GMT
content-length: 0
location: https://abhirockzz.github.io/
apigw-requestid: VT0D1hNLIAMES8w=
How about utilizing a brief code that doesn’t exist?
Set the standing
You possibly can allow and disable the brief codes. The unique URL will solely be accessible if the affiliation is in lively state.
To disable a brief code:
curl -i -X PUT -d '{"lively": false}' -H 'Content material-Sort: software/json' $URL_SHORTENER_APP_URL/<brief code>
# instance
curl -i -X PUT -d '{"lively": false}' -H 'Content material-Sort: software/json' https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/1ee3ad1b
That is an HTTP PUT request with a JSON payload that specifies the standing (false on this case refers to disable) together with the brief code which is a path parameter to the API Gateway endpoint. If all works nicely, you need to see a HTTP 204 (No Content material) response:
HTTP/2 204
date: Fri, 15 Jul 2022 13:15:41 GMT
apigw-requestid: VT1Digy8IAMEVHw=
Test the DynamoDB report – the lively attribute will need to have switched to false.
As an train, attempt the next:
- entry the URL by way of the identical brief code now and verify the response.
- entry an invalid brief code i.e. that doesn’t exist
- allow a disabled URL (use
{"lively": true})
Okay, thus far now we have coated all operations, besides delete. Lets attempt that and wrap up the CRUD!
Delete
curl -i -X DELETE $URL_SHORTENER_APP_URL/<brief code>
# instance
curl -i -X DELETE https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/1ee3ad1b
Nothing too stunning. We use a HTTP DELETE together with the brief code. Similar to in case of replace, you need to get a HTTP 204 response:
HTTP/2 204
date: Fri, 15 Jul 2022 13:23:36 GMT
apigw-requestid: VT2NzgjnIAMEVKA=
However this time in fact, the DynamoDB report ought to have been deleted – affirm the identical.
What occurs whenever you attempt to delete a brief code that doesn’t exist?
Do not forget to scrub up!
When you’re carried out, to delete all of the companies, merely use:
cdk destroy
Alright, now that you have truly seen “what” the appliance does, let’s transfer on to the “how”.
We are going to begin with the AWS CDK code and discover the way it does all of the heavy lifting behind to setup the infrastructure for our Serverless URL shortener service.
With AWS CDK, Infrastructure-IS-code!
You possibly can try the code in this GitHub repo. I’ll stroll you thru the keys elements of the NewServerlessURLShortenerStack perform which defines the workhorse of our CDK software.
I’ve omitted a number of the code for brevity
We begin by making a DynamoDB desk. A main secret’s all that is required with a purpose to do this – on this case shortcode (we do not have vary/kind key on this instance)
dynamoDBTable := awsdynamodb.NewTable(stack, jsii.String("url-shortener-dynamodb-table"),
&awsdynamodb.TableProps{
PartitionKey: &awsdynamodb.Attribute{
Identify: jsii.String(shortCodeDynamoDBAttributeName),
Sort: awsdynamodb.AttributeType_STRING}})
Then, we create an API Gateway (HTTP API) with only one line of code!
urlShortenerAPI := awscdkapigatewayv2alpha.NewHttpApi(stack, jsii.String("url-shortner-http-api"), nil)
We transfer on to the primary Lambda perform that creates brief codes for URLs. Discover that we use an experimental module awscdklambdagoalpha (right here is the steady model on the time of writing). In case your Go undertaking is structured in a selected method (particulars right here) and also you specify its path utilizing Entry, it would mechanically maintain constructing, packaging and deploying your Lambda perform! Not unhealthy in any respect!
Along with Native bundling (as used on this instance), Docker based mostly builds are additionally supported.
createURLFunction := awscdklambdagoalpha.NewGoFunction(stack, jsii.String("create-url-function"),
&awscdklambdagoalpha.GoFunctionProps{
Runtime: awslambda.Runtime_GO_1_X(),
Surroundings: funcEnvVar,
Entry: jsii.String(createShortURLFunctionDirectory)})
dynamoDBTable.GrantWriteData(createURLFunction)
Lastly, we add the final little bit of plumbing by making a Lambda-HTTP API integration (discover how the Lambda perform variable createURLFunction is referenced) and including a path to the HTTP API we had created – this in flip refers back to the Lambda integration.
createFunctionIntg := awscdkapigatewayv2integrationsalpha.NewHttpLambdaIntegration(jsii.String("create-function-integration"), createURLFunction, nil)
urlShortenerAPI.AddRoutes(&awscdkapigatewayv2alpha.AddRoutesOptions{
Path: jsii.String("https://dev.to/"),
Strategies: &[]awscdkapigatewayv2alpha.HttpMethod{awscdkapigatewayv2alpha.HttpMethod_POST},
Integration: createFunctionIntg})
This was only for one perform – now we have three extra remaining. The nice half is that the template for all these are comparable i.e.
- create the perform
- grant permission for DynamoDB
- wire it up with API Gateway (with the proper HTTP technique i.e.
POST,PUT,DELETE)
So I can’t repeat it over right here. Be at liberty to grok by way of the remainder of the code.
Now that you simply perceive the magic behind the “one-click” infrastructure setup, let’s transfer on to the core logic of the appliance.
URL shortener Lambda perform and DynamoDB logic
There are 4 totally different features, all of that are of their respective folders and all of them have a couple of issues in frequent in the way in which they function:
- They do dome preliminary processing – course of the payload, or extract the trail parameter from the URL and many others.
- Invoke a frequent database layer – to execute the CRUD performance (extra on this quickly)
- Deal with errors as applicable and return response
With that data, it must be simple to comply with alongside the code.
As earlier than, some elements of the code have been omitted for brevity
Create perform
func handler(ctx context.Context, req occasions.APIGatewayV2HTTPRequest) (occasions.APIGatewayV2HTTPResponse, error) {
url := req.Physique
shortCode, err := db.SaveURL(url)
if err != nil {//..deal with error}
response := Response{ShortCode: shortCode}
respBytes, err := json.Marshal(response)
if err != nil {//..deal with error}
return occasions.APIGatewayV2HTTPResponse{StatusCode: http.StatusCreated, Physique: string(respBytes)}, nil
}
This perform begins by studying the payload of the HTTP request physique – it is a string which has the URL for which the brief code is being created. It invokes the database layer to attempt to save this report to DynamoDB and handles errors. Lastly, it returns a JSON response with the brief code.
Right here is the perform that truly interacts with DynamoDB to get the job carried out.
func SaveURL(longurl string) (string, error) {
shortCode := uuid.New().String()[:8]
merchandise := make(map[string]sorts.AttributeValue)
merchandise[longURLDynamoDBAttributeName] = &sorts.AttributeValueMemberS{Worth: longurl}
merchandise[shortCodeDynamoDBAttributeName] = &sorts.AttributeValueMemberS{Worth: shortCode}
merchandise[activeDynamoDBAttributeName] = &sorts.AttributeValueMemberBOOL{Worth: true}
_, err := shopper.PutItem(context.Background(), &dynamodb.PutItemInput{
TableName: aws.String(desk),
Merchandise: merchandise})
if err != nil {//..deal with error}
return shortCode, nil
}
For the needs of this pattern app, the brief code is created by producing a UUID and trimming out the final 8 digits. It is easy to exchange this with one other approach – all that issues is that you simply generate a singular string that may work as a brief code. Then, all of it about calling the PutItem API with the required knowledge.
Entry the URL
func handler(ctx context.Context, req occasions.APIGatewayV2HTTPRequest) (occasions.APIGatewayV2HTTPResponse, error) {
shortCode := req.PathParameters[pathParameterName]
longurl, err := db.GetLongURL(shortCode)
if err != nil {//..deal with error}
return occasions.APIGatewayV2HTTPResponse{StatusCode: http.StatusFound, Headers: map[string]string{locationHeader: longurl}}, nil
}
When somebody accesses the brief hyperlink (as demonstrated within the earlier part), the brief code is handed in as a path parameter e.g. http://<api gw url>/<brief code>. the database layer is invoked to get the corresponding URL from DynamoDB desk (errors are dealt with as wanted). Lastly, the response is returned to the person whereby the standing code is 302 and the URL is handed within the Location header. That is what re-directs you to the unique URL whenever you enter the brief code (within the browser)
Right here is the DynamoDB name:
func GetLongURL(shortCode string) (string, error) {
op, err := shopper.GetItem(context.Background(), &dynamodb.GetItemInput{
TableName: aws.String(desk),
Key: map[string]sorts.AttributeValue{
shortCodeDynamoDBAttributeName: &sorts.AttributeValueMemberS{Worth: shortCode}}})
if err != nil {//..deal with error}
if op.Merchandise == nil {
return "", ErrUrlNotFound
}
activeAV := op.Merchandise[activeDynamoDBAttributeName]
lively := activeAV.(*sorts.AttributeValueMemberBOOL).Worth
if !lively {
return "", ErrUrlNotActive
}
longurlAV := op.Merchandise[longURLDynamoDBAttributeName]
longurl := longurlAV.(*sorts.AttributeValueMemberS).Worth
return longurl, nil
}
Step one is to make use of GetItem API to get the DynamoDB report containing URL and standing equivalent to the brief code. If the merchandise object within the response is nil, we are able to make certain that a report with that brief code does not exist – we return a customized error which may be useful for our perform which might then return an applicable response to the caller of the API (e.g. a HTTP 404). We additionally verify the standing (lively or not) and return an error if lively is about to false. If all is nicely, the URL is returned to the caller.
Replace standing
func handler(ctx context.Context, req occasions.APIGatewayV2HTTPRequest) (occasions.APIGatewayV2HTTPResponse, error) {
var payload Payload
reqBody := req.Physique
err := json.Unmarshal([]byte(reqBody), &payload)
if err != nil {//..deal with error}
shortCode := req.PathParameters[pathParameterName]
err = db.Replace(shortCode, payload.Energetic)
if err != nil {//..deal with error}
return occasions.APIGatewayV2HTTPResponse{StatusCode: http.StatusNoContent}, nil
}
Step one is to marshal the HTTP request payload which is a JSON e.g. {"lively": false} after which get the brief code from the trail parameter. The database layer is invoked to replace the standing and deal with errors.
func Replace(shortCode string, standing bool) error {
replace := expression.Set(expression.Identify(activeDynamoDBAttributeName), expression.Worth(standing))
updateExpression, _ := expression.NewBuilder().WithUpdate(replace).Construct()
situation := expression.AttributeExists(expression.Identify(shortCodeDynamoDBAttributeName))
conditionExpression, _ := expression.NewBuilder().WithCondition(situation).Construct()
_, err := shopper.UpdateItem(context.Background(), &dynamodb.UpdateItemInput{
TableName: aws.String(desk),
Key: map[string]sorts.AttributeValue{
shortCodeDynamoDBAttributeName: &sorts.AttributeValueMemberS{Worth: shortCode}},
UpdateExpression: updateExpression.Replace(),
ExpressionAttributeNames: updateExpression.Names(),
ExpressionAttributeValues: updateExpression.Values(),
ConditionExpression: conditionExpression.Situation(),
})
if err != nil && strings.Comprises(err.Error(), "ConditionalCheckFailedException") {
return ErrUrlNotFound
}
return err
}
The UpdateItem API name takes care of fixing the standing. It is pretty easy aside from the all these expressions that you simply want – particularly when you’re new to the idea. The primary one (necessary) is the replace expression the place you specify the attribute you must set (lively on this case) and its worth. The second makes certain that you’re updating the standing for a brief code that truly exists within the desk. That is vital since, in any other case the UpdateItem API name will insert a new merchandise (we do not need that!). As a substitute of rolling out the expressions by hand, we use the expressions package deal.
Delete brief code
func handler(ctx context.Context, req occasions.APIGatewayV2HTTPRequest) (occasions.APIGatewayV2HTTPResponse, error) {
shortCode := req.PathParameters[pathParameterName]
err := db.Delete(shortCode)
if err != nil {//..deal with error}
return occasions.APIGatewayV2HTTPResponse{StatusCode: http.StatusNoContent}, nil
}
The delete handler is not any totally different. After the brief code to be deleted is extracted from the trail parameter, the database layer is invoked to take away it from the DynamoDB desk. The consequence returned to the person is both an HTTP 204 (on success) or the error.
func Delete(shortCode string) error {
situation := expression.AttributeExists(expression.Identify(shortCodeDynamoDBAttributeName))
conditionExpression, _ := expression.NewBuilder().WithCondition(situation).Construct()
_, err := shopper.DeleteItem(context.Background(), &dynamodb.DeleteItemInput{
TableName: aws.String(desk),
Key: map[string]sorts.AttributeValue{
shortCodeDynamoDBAttributeName: &sorts.AttributeValueMemberS{Worth: shortCode}},
ConditionExpression: conditionExpression.Situation(),
ExpressionAttributeNames: conditionExpression.Names(),
ExpressionAttributeValues: conditionExpression.Values()})
if err != nil && strings.Comprises(err.Error(), "ConditionalCheckFailedException") {
return ErrUrlNotFound
}
return err
}
Similar to UpdateItem API, the DeleteItem API additionally takes in a situation expression. If there isn’t any report within the DynamoDB desk with the given brief code, an error is returned. In any other case, the report is deleted.
That completes the code stroll by way of!
Wrap up
On this weblog submit you learnt the best way to use DynamoDB Go SDK utilizing a URL Shortener pattern software. You additionally built-in it with AWS Lambda and API Gateway to construct a Serverless answer whose infrastructure was additionally outlined utilizing precise code (versus yaml, JSON and many others.), because of the Go assist in AWS CDK.
