AWS S3 or Amazon S3 is a storage system from Amazon to retailer and retrieve information from wherever on the internet. It’s a extremely scalable, dependable, quick, cheap information storage system from Amazon. It offers straightforward to make use of developer package to retailer and retrieve information.
On this tutorial you’ll learn to add bulk information to S3 bucket aws golang. We’ll use the Go AWS SDK to add information to AWS S3.
The AWS SDK for Go requires Go 1.15 or later variations. You possibly can view your present model of Go by working the next command.
go model
For details about putting in or upgrading your model of Go, see https://golang.org/doc/set up
Earlier than you should utilize the AWS SDK for Go V2, you need to have an Amazon account. See How do I create and activate a brand new AWS account? for particulars
Obtain the AWS Golang SDK for utilizing in your codebase utilizing under command
go get github.com/aws/aws-sdk-go/aws
We’ll cowl this tutorial step-by-step to add bulk information to AWS S3 Bucket server utilizing Golang with instance code.
Load require packages
We’ll create primary.go file and import needed packages we have to add the information into AWS S3 Bucket.
package deal primary
import (
"bytes"
"fmt"
"log"
"web/http"
"os"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
)
Customized struct for receiver
We might be utilizing a Session struct so we are able to use identical it as receiver, and we’ll move this receiver to our primary add operate,
By way of utilizing a customized struct we’re eliminating the duplication from our code. We’ll be utilizing solely single session for our all bulk information to add into s3 bucket aws golang.
kind Session struct {
S3Session *session.Session
}
Arrange primary configuration and AWS session
We’ll setup configuration utilizing AWS S3 REGION and SECRET ID and SECRET KEY and create single AWS session to add a number of information to AWS S3. Then we’ll name methodology add() and move AWS session occasion and file particulars to add file to AWS S3 server.
func primary() {
paths := []string{"", ""}
credential := credentials.NewStaticCredentials(
os.Getenv("SECRET_ID"),
os.Getenv("SECRET_KEY"),
"",
)
awsConfig := aws.Config{
Area: aws.String(os.Getenv("REGION")),
Credentials: credential,
}
s, err := session.NewSession(&awsConfig)
if err != nil {
log.Println("did not create S3 session:", err.Error())
return
}
se := Session{s}
err = se.add(paths)
if err != nil {
log.Println(err.Error())
return
}
}
Create Add Technique to Add Bulk Recordsdata to AWS S3
We’ll create methodology add() to add information to AWS S3 server. We’ll open the file one after the other and retailer into buffer after which put the file to AWS S3 utilizing PutObject() methodology from S3. We can even specify choices within the PutObjectInput when importing the file. We can even allow AES256 encryption on information utilizing ServerSideEncryption choices of s3.PutObjectInput struct.
func (s Session) add(paths []string) error {
for _, path := vary paths {
upFile, err := os.Open(path)
if err != nil {
log.Printf("failed %s, error: %v", path, err.Error())
proceed
}
defer upFile.Shut()
upFileInfo, err := upFile.Stat()
if err != nil {
log.Printf("did not get stat %s, error: %v", path, err.Error())
proceed
}
var fileSize int64 = upFileInfo.Measurement()
fileBuffer := make([]byte, fileSize)
upFile.Learn(fileBuffer)
// importing
_, err = s3.New(s.S3Session).PutObject(&s3.PutObjectInput{
Bucket: aws.String(os.Getenv("BUCKET_NAME")),
Key: aws.String(path),
ACL: aws.String("public-read"), // could possibly be personal if you'd like it to be entry by solely approved customers
Physique: bytes.NewReader(fileBuffer),
ContentLength: aws.Int64(int64(fileSize)),
ContentType: aws.String(http.DetectContentType(fileBuffer)),
ContentDisposition: aws.String("attachment"),
ServerSideEncryption: aws.String("AES256"),
StorageClass: aws.String("INTELLIGENT_TIERING"),
})
if err != nil {
log.Printf("did not add %s, error: %v", path, err.Error())
proceed
}
url := "https://%s.s3-%s.amazonaws.com/%s"
url = fmt.Sprintf(url, os.Getenv("BUCKET_NAME"), os.Getenv("REGION"), path)
fmt.Printf("Uploaded File Url %sn", url)
}
return nil
}
you’ll be able to checkout extra article like this on my private weblog
