Risk Modeling for Containers. Safety concerns for containers… | by Teri Radichel | Cloud Safety | Jul, 2022

July 25, 2022

1

Safety concerns for containers utilized by batch jobs

This can be a continuation of my collection of posts on Batch Jobs for Cybersecurity and Cyber Safety Metrics.

I reply plenty of requires IANS Analysis about Kubernetes, AKS, EKS, GKE, Docker, containers, and the whole lot associated to it. For instance, how do you safe your deployment pipeline? What about secrets and techniques, encryption, and community safety? What about APIs that run on containers or “serverless”? Networking? Everytime you use a brand new expertise or service you want to perceive the way it works, what assaults exist associated to that expertise, and what safety controls can be found to you to safe that expertise.

Stepping by all of the issues I would like to consider with an AWS Batch job is an extended record. I’m going to begin with one side of it — the container itself. I’ll contact on some associated infrastructure points, however I’ll dive extra into these once we really arrange an AWS Batch job.

Want extra assist? In order for you me to overview your cloud and utility structure and make suggestions, schedule a name with IANS. For non-IANS prospects contact me on LinkedIn for a penetration check or safety evaluation of your cloud setting and purposes. If you're simply following alongside and wish to learn what I write right here it helps when folks create a membership and comply with me on Medium as that is just about the one approach I get earnings from sharing this info. That earnings permits me to put in writing extra typically. In any other case, I am completely happy to assist out and writing about what pursuits me as time permits. Hopefully it pursuits you too and helps a little bit. :)

There are quite a few methods to make use of containers and lots of safety concerns which differ relying on the precise structure you utilize. On this weblog submit, I’m going to begin by considering by among the concerns for a container used as a part of a batch job. We’ll want to have a look at all the varied layers of safety that apply to our batch job however it begins with the container and the way it’s configured.

This weblog submit is absolutely me considering free type in regards to the issues I’ll want to handle once I create my batch job and I’ll in all probability write about plenty of it in future posts. It’s not meant to be an entire record however it’s a place to begin. I additionally share some tales beneath that you simply would possibly wish to find out about in case you’re working in a cloud setting associated to among the controls if you’re questioning if they’re overkill.

We’re excited about this container in relation to AWS Batch. If you happen to’re working a container on Kubernetes, you’ve a complete lot extra to consider on the subject of safety controls. In actual fact, I simply helped somebody on a name associated to batch jobs working on Kubernetes. If you happen to choose to make use of AWS Batch, a few of these issues are offloaded to AWS by means of the shared safety mannequin within the cloud, however not all.

Which controls belong to you? I wish to say that in case you’re utilizing a cloud platform, in case you can see it, contact it, and alter it, you doubtless personal it. You will must be sure to safe it correctly. Nonetheless, it is best to actually overview the authorized agreements you've with the cloud supplier at the beginning, adopted by the documentation, which isn't essentially legally binding. Discuss to a lawyer about that.

Right here’s what we find out about how AWS secures AWS Batch and what you are able to do in your aspect to additional safe your batch jobs:

Once we’re excited about the containers we’re utilizing we are going to wish to be sure that we comply with all of the container finest practices accessible to us although the CIS benchmarks. In my current presentation on the IANS summit in Dallas I talked about CIS protection. It’s an excellent start line, however not sufficient. We’ll additionally to grasp vendor steerage and all of the methods wherein containers could also be attacked to attempt to make sure we will stop these assaults.

Networking:

You may configure networking inside a container, however that isn’t that useful if an attacker can get into the container and alter it as soon as they’re in there. We’ll wish to think about that menace.
If we will run a batch job in a personal community that may assist be sure that attackers can’t get to containers from the surface or ship visitors to and from the Web.
Nonetheless, even when we will run in a personal community, the containers doubtless want to speak with DNS servers can be utilized for information exfiltration or command and management. That’s a subject I wish to discover additional if time permits. Different protocols like ICMP have been used equally as you’ll be able to examine in my paper on the Goal breach written in 2014.
A container in a personal community working in a cloud setting is trusting that the cloud setting is safe. What if cloud insiders receive entry the container? You would possibly wish to overview this state of affairs together with your cloud supplier to see how they deal with their insider safety.

How a lot entry does your cloud supplier have? I used to be at an OWASP meetup in Seattle. An individual began speaking to me however didn't ask what I do for a dwelling (train and check cloud safety). He began speaking to me about how he was happy with the structure and the thought put into the safety of that individual cloud platform. Then he went on to inform me that sadly, they did not assume by the flexibility for their very own staff to bypass safety controls and had some points with that, however they have been engaged on it. I requested him what he went and he defined a couple of of the problems. Ask your cloud supplier how they stop insiders from accessing your information for every cloud service you utilize. Higher but, get it in writing, in a contract.

In addition to insiders, what if an attacker have been capable of breach the cloud platform and get onto the cloud supplier community with out their information? How safe would your batch jobs be in that case? We’ll wish to assume by that and attempt to assemble an structure that protects us as a lot as doable in that state of affairs. Networking is a key a part of any structure that helps you block and spot attackers — no matter what the identification zealots let you know. You want each. Particularly since stolen or abused credentials is your primary danger within the cloud.

Attackers within the cloud networks. Not too long ago, whereas utilizing Azure in preparation for a category, I arrange very stringent networking utilizing varied service controls accessible for storage. In some unspecified time in the future, I bought an error message stating that I couldn't entry my very own storage as a result of an IP beginning with 20.x.x.x was not allowed to entry the information. Odd. That isn't my IP handle. That was an IP handle within the Microsoft IP vary.I submitted the error message to Azure assist considering possibly certainly one of their providers had a bug. I discovered many bugs whereas making an attempt to make use of Azure in that final run by my class supplies and whereas testing new providers. The message I bought again finally from Azure assist stated one thing very obscure about an inner service they could not inform me about discovering an incident which they have been addressing. A safety incident? I do not know. I do not use Azure for something production-related so I simply made a observe of it and moved on. But when it was a safety incident and I had simply arrange my storage with no networking controls, I'd haven't identified that was taking place. If it was a safety incident, that simply goes to point out you that you simply should not blindly check the cloud networks on which you use. Even when it wasn't a safety incident, one thing unusual was taking place as a result of my entry was being blocked as a result of entry by an IP handle aside from my very own by the networking restrictions I put in place on the service stage. At a minimal I helped Microsoft discover a bug of some form. I used to be too busy at that second to look into it additional.

Backside line: Thoughts your networking. If we have been working Kubernetes or docker in another setting I’d be speaking to you about different choices however on this case we’ll be taking a look at what AWS gives when it comes to networking controls for AWS Batch and what we will do on the container itself.

Secrets and techniques and Encryption:

Will we have to entry any consumer names, passwords, AWS credentials or in any other case from our batch job? We’ll wish to assume by how we shield these secrets and techniques.
If we’re deploying a batch job from dev to QA to manufacturing we wish to use completely different secrets and techniques in every setting. I may not get to that matter right here for some time or in any respect as a result of I’ve plenty of floor to cowl. However you’ll be able to name me up and ask me at IANS Analysis.
I wrote in regards to the encryption fallacy in my final guide. If we’re counting on encryption, however an attacker obtains credentials and is ready to execute the encryption and decryption instructions within the cloud, then our encryption is just about ineffective. We’ll wish to architect our safety with that chance in thoughts.

On condition that I’ve already completed some testing and a POC of utilizing MFA with a container, I do know that we’ll in all probability must leverage some AWS credentials to imagine a task that requires MFA. Except I can consider a greater answer, we’ll must assume by how we retailer these credentials and what occurs with the MFA code we wish to move by to run our container and finally our batch job. I’ve some concepts how to do that however I’m not but certain they’ll work. And people concepts result in a complete new stage of complexity and safety concerns as a result of we’ll find yourself passing round an MFA code.

One step at a time…let’s simply begin with how we will shield secrets and techniques on our container. I’ll handle all these different issues if and once we can really get MFA working.

IAM ~ Id and Entry Administration

Credentials are your primary drawback within the cloud as a result of many experiences reminiscent of these I reference in my IANS displays. The following one I’ll be presenting at will likely be LA in October if all goes in accordance with plan:

IAM concerns:

How can we arrange minimal permissions for our batch jobs?
How can we introduce segregation of duties to attenuate danger? I’m an enormous fan of segregation and will likely be masking some concepts for that I’ve been taking part in round with these days on AWS.
As I’ve already talked about I’m curious about utilizing MFA to kick off a job. Perhaps somebody at AWS is already fixing this for me. 🙂 #awswishlist
How can we guarantee there is no such thing as a path to escalation that enables somebody who can assume a task in a batch job to do one thing nefarious they wouldn’t in any other case be allowed to do? Unsure if I’ll get to this explicit matter on this collection because it’s extra relevant to a big group than to me in my very own account, however if you’re at a big group it’s one thing you’ll want to think about.
If an attacker can receive entry to a number working containers or entry by a container to the host, the attacker can doubtlessly leverage the metadata service to acquire credentials. We’ll need to check out if and the way that is likely to be doable and how you can stop it.
How a lot might an attacker entry if she or he obtains entry to a single batch job? We’ll wish to take into consideration the blast radius, a time period I describe in my cloud safety guide talked about on the backside of this submit, together with the important thing issues that may have an effect on and enhance it.

Container configuration concerns:

The configuration of the container itself and the operation of containers might additionally pose a safety menace, past storing secrets and techniques and accessing cloud information through IAM roles. A few of these tasks belong to AWS however should fall to us. Listed below are just some concerns we’ll want to consider:

Is the container working as root?
May somebody receive entry to PID1? (I will likely be utilizing Amazon Linux.)
We’ll wish to be sure that directories can’t be mapped to root or something with increased permissions.
No entry to docker socket or any of the controls for docker or the underlying orchestration system.
Guarantee no secrets and techniques can be found as a result of approach the container will get constructed or the underlying layers.
Restrict what an attacker can do as soon as they get onto a dwell container.
Integrity of our container code after it goes although the construct course of into manufacturing.

These are just some of our concerns. Many books and sources exist on container safety if you wish to dive a lot deeper than I’ve simply gone however we’ll have a look at this a bit extra in upcoming posts.

Software, container, and infrastructure code concerns:

With regards to the applying working on the container, the code utilized by the applying might enable an attacker to acquire entry to issues inside the container or the community. This can be a complete further layer past the container itself and among the issues I check net purposes for once I carry out cloud penetration checks, together with the whole lot else listed right here.

Safety of all our deployment code used for the container, the applying code working on it, and the cloud infrastructure.
Rogue third-party libraries.
Injection assaults of any form.
New CVEs found within the the libraries utilized by our containers (vulnerability administration).
SSRF, DNS Rebinding, open redirects, and different assaults that might enable an attacker to acquire entry to sources on non-public networks even with out direct entry.
In my subsequent submit I’ll write about how an utility working within the cloud might be leveraged to acquire AWS credentials. That might be a difficulty for our batch jobs and we’ll want to pay attention to the likelihood and defend in opposition to it. Purposes can be utilized as C2 channels to entry the underlying host, along with the assaults within the final bullet that may present entry to AWS credentials.

Something that you simply open up your community to is likely to be a supply that an attacker can ship instructions over and you’ll assume it’s regular, anticipated visitors. Any sort of injection flaw could result in a way for an attacker to acquire or leverage credentials. Any technique of leveraging credentials on an utility would possibly enable an attacker to acquire entry to information in our cloud setting. We’ll wish to take into consideration how we will shield shore up the code in our batch jobs to forestall assaults.

You can begin with this collection of weblog posts I hope to show right into a guide, which began out with me making an attempt to refresh AWS credentials in Python and led to a dialogue on injection in a later submit:

This isn’t an entire record of the safety points you could want to think about in relation to your batch job. It’s going to rely in your programming language, code, what libraries you’re utilizing, and what your utility does. The record would change in case you have been utilizing one thing aside from AWS Batch. However this can allow you to perceive among the concerns you’ll have when it comes to menace modeling to your batch job. This record gives some subjects for my upcoming posts the place I’ll write about how you can handle a few of these points.

Teri Radichel

If you happen to appreciated this story please clap and comply with:

Medium: Teri Radichel or Electronic mail Record: Teri Radichel
Twitter: @teriradichel or @2ndSightLab
Requests providers through LinkedIn: Teri Radichel or IANS Analysis