← 返回列表

Cloud Security Breaches and Vulnerabilities: 2021 in Review

单站 Christophe Tafani-Dereeper 作者 christophetd 发布 Wed, 22 Dec 2021 13:41:42 +0000 入库 2026-11-18T05:03:30

As 2021 fades away, we look back on cloud data breaches and vulnerabilities that were publicly disclosed this year.

Last updated: March 14th, 2022.

Introduction

Data about cloud security incidents in the wild is scarce, and often lacks details on tactics, techniques and procedures (TTPs) used by attackers. Breached organizations often don’t disclose the specifics publicly. Available data suffers from survivorship bias; we know about attacks that were detected, not advanced compromises that flew under the radar.

In this post, we’ll analyze cloud security incidents and vulnerabilities that were publicly disclosed in 2021. We’ll focus on the ones related to the usage of cloud providers, and we won’t cover vulnerabilities of the cloud providers themselves. Scott Piper has done a great job for these with his repository “csp_security_mistakes”.

We’ll include in particular:

Don’t expect new, ground-breaking attack techniques. Read this post asking yourself: are my cloud security controls covering these basics? This is a strongly AWS-focused post, because there is even less data for incidents involving other cloud providers.

Disclaimer: I am currently employed by Datadog, who is also a cloud security vendor. This is not a corporate or sponsored post.

Trends From 2021

Static Credentials Remain the Major Initial Access Vector

Unsurprisingly, a lot of attacks start with the compromise of static, long-lived credentials. Long-lived credentials are a nightmare for security practitioners, because they end up getting leaked in a log file, monitoring tool, build log, stack trace… Assuming your organization has 10 static credentials, each with a 0.01% risk of getting leaked every day, there’s a 52% probability that at least one of them gets leaked within 2 years!

Data Breaches

Here are a few cases of actual data breaches experienced by an organization, due to static, long-lived credentials.

Codecov attackers added a malicious command to the install script hosted in a GCS bucket.

Kaspersky phishing e-mail

Sample phishing e-mail sent using Kaspersky’s AWS SES.

Vulnerabilities

We also know about several occurrences of misplaced, vulnerable static credentials. While we have no evidence confirming they led to actual data breaches, the problem is the same.

Popular Android applications packaged with AWS credentials. Picture fromBeVigil.

Exposed Apache Airflow instances leaking AWS credentials to unauthenticated attackers. Picture fromIntezer

Sample SNS topic an attacker could access after having compromised publicly accessible, static AWS credentials.

Malware

The first public report of malware stealing AWS credentials dates back to August 2020, and is attributed to the TeamTNT group. It would send the contents of .aws/credentials to a malicious server. As of 2021, we have evidence that TeamTNT now also targets static GCP and Docker credentials (source).

More recently, we have seen attackers exploiting the Log4Shell vulnerability to steal AWS credentials from environment variables, through both remote code execution, and DNS exfiltration (source, source).

It seems that TeamTNT is the only publicly discussed threat actor actively targeting AWS credentials. As of December 2021, I am not aware of any other well-known one. MITRE ATT&CK isn’t either (see this google search). It’s likely that more will come, since AWS credentials can be a high-value low-hanging fruit after compromising a machine.

Public S3 Buckets

You knew this was coming. I compiled below a non-comprehensive list of publicly disclosed data breaches in 2021 involving a misconfigured S3 bucket. Again, it only includes publicly disclosed data breaches in 2021. We can reasonably imagine there were many more.

| Company | Description | Data leaked | Number of data subjects impacted |
| Hobby Lobby | US-based crafts store | Names, e-mails, physical address, source code of company application | > 300k |
| Decathlon Spain | French sporting goods retailer | Names, e-mails, phone numbers, authentication tokens | >7.8k |
| 86 US cities | US cities using the “PeopleGIS” service | Tax reports, passport copies, etc. | Undisclosed |
| Pixlr | Online photo editing tool | E-mail, hashed passwords, country | 1.9M |
| Acquirely | Australian marketing company | Names, e-mails, DoB, physical addresses, phone numbers | > 200k |
| MobiKwik | Indian digital payment provider | KYC data (including passport copies), transaction logs, e-mail addresses, phone numbers | 3.5M (KYC data) around 100M (rest) |
| Senior Advisor | Comparator for senior care services | Names, e-mails, phone numbers | > 3M |
| TeamResourcing | Recruiting company (formerly “TeamBMS” and “FastTrack Reflex Recruitment”) | Resumes, passport copies, names, e-mails, physical addresses | >20k |
| İnova Yönetim | Law firm | Court cases and associated PII | >15k (15k court cases) |
| Cosmolog Kozmetik | Beauty products company | Names, physical addresses, purchased items | >576k |
| Premier Diagnostics | COVID-19 testing company | Passport / ID card copies, names, DoB, | 52k |
| PaleoHacks | Paleo diet app | Names, e-mails, hashed passwords, DoB, employer | 70k |
| Phlebotomy Training Specialists | Medical training company | Names, DoB, pictures, physical addresses, ID cards, resumes, diplomas, phone numbers | Between 27k and 50k |
| CallX | Telemarketing company | Chat transcripts, names, physical addresses, phone numbers | Between 10k and 100k |
| Prisma Promotora | Enterprise software provider | Names, e-mails, DoB, background checks, physical addresses, ID cards, debit card information | > 10k |
| Sennheiser | Audio hardware manufacturer | Names, physical addresses, phone numbers, | > 28k |
| Fleek | Now-defunct Snapshat alternative | Images shared by users of the app | Undisclosed |
| Ghana’s National Service Scheme | Ghana public organization | ID cards, employment records | >100k |
| BabyChakra | Pregnancy and parenting platform | Child pictures, medical test results, medical prescriptions, full names, phone numbers, physical addresses | >100k |
| SEGA Europe | Video games and entertainment company | AWS / MailChimp / Steam API keys, | Undisclosed |
| D.W. Morgan | Transportation and logistics | Full names, phone numbers, physical addresses, order details, invoices, pictures of shipments, contracts | |

I also found a reference to an open Azure Blob Storage bucket of GSI Immobilier, a French estate agency, leaking PII of around 1000 customers.

Stolen Instance Credentials Through SSRF Vulnerabilities

Four and a half years ago, I wrote a blog post entitled “Abusing the AWS metadata service using SSRF vulnerabilities” – before I even started working in security. Unfortunately, using SSRF vulnerabilities to access the AWS metadata service is still a thing, mostly due to insecure defaults and a lack of communication from AWS.  Why is it that in soon-to-be 2022, newly launched instances still default to using the insecure version of the instance metadata service? (IMDSv1)

While we don’t have illustrations of security incidents leveraging this technique in 2021, several disclosed bug bounty reports confirm this is still a thing:

It’s also the first year we see a threat actor, TeamTNT, stealing temporary credentials through the instance metadata service after compromising a machine (source). This may indicate increased automation from attackers, since stolen credentials are only valid for a few hours.

Preventing and Detecting These Common Misconfigurations

Now, we know what went wrong. Let’s see how we can make things better, and remediate the root cause of these breaches and vulnerabilities.

Getting Rid of Static, Long-Lived Credentials

The advice below generally boils down to: Do not use IAM users unless you have no other choice.

For Humans

Authenticate your human users through AWS SSO (it’s free), or IAM role federation inside an account. There is no good reason for humans to use IAM users. IAM users are terrible not only because they have static long-lived credentials, but also because they’re not linked to a centralized identity provider. I recommend blocking the creation of IAM users using an organization-wide Service Control Policy (SCP). If you do need IAM users, you want to make sure only a limited set of people can create them, ideally in a dedicated AWS account. From my experience, many people unfamiliar with AWS create IAM users with the incorrect belief that they are required to use the CLI or AWS SDKs locally on their development machines.

Note that AWS SSO fully supports the AWS CLI, AWS SDKs, and aws-vault. Speaking of which: use aws-vault, which makes sure to always encrypt your credentials on disk.

For Applications

Your applications accessing AWS services should ideally be running in AWS. When this is the case, use the native AWS constructs to give them an identity with the right permissions and temporary credentials: EC2 instance roles, Lambda execution roles, EKS IAM roles for service accounts.

In some cases, applications accessing AWS don’t run in AWS. In that case, something like Hashicorp Vault is great if your applications have another way to bootstrap their identities to Vault. For instance, Vault makes it easy to authenticate an application running in Kubernetes and generate temporary AWS credentials on the fly. If your application runs outside AWS, and on a platform that doesn’t give it a verifiable “workload identity”, you’ll have to use static credentials to authenticate to Vault, which only shifts the problem. But at least, it allows you to centralize all your AWS credentials in Vault, and ensure your applications only use temporary AWS credentials.

Note that you can also leverage Vault for humans. For instance, it works great for mapping Okta/AD groups to AWS roles.

For SaaS Solutions

Many SaaS solutions integrate with your AWS accounts. These should not hold static AWS credentials. Instead, you should create a dedicated IAM role they can assume from their own AWS account, using a randomly generated and secret ExternalID to avoid the confused deputy problem.

See AWS guidance on that, and this great piece of research by Praetorian. A good example is how Datadog integrates with their customers’ AWS accounts (disclaimer: my current employer).

Scanning for Credentials in Places They Shouldn’t Be

Several solutions exist to scan for secrets in source code, locally (e.g. in a pre-commit hook) or in CI/CD. Popular open-source and free solutions are detect-secrets, gitleaks and truffleHog (who also has a commercial edition). GitGuardian’s ggshield is free for up to 25 users.

From my experience, it’s also easy to overlook static credentials ending up in Docker images. Here’s an example of a Dockerfile that looks benign at first glance (at least to me), but leaks access keys in the image layers:

FROM amazon/aws-cli:2.4.6
ARG AWS_ACCESS_KEY_ID
ARG AWS_SECRET_ACCESS_KEY
RUN aws s3 cp s3://my-app/artifact.jar /app.jar
CMD ["java", "-jar", "/app.jar"]

Sample Dockerfile leaking AWS credentials in the image layers. In this case, one should use a multi-stage build copying the application artifact to a second-stage build.

Interestingly, though, the only open-source container scanning tool that supports scanning for secrets in a Docker image seems to be SecretScanner by DeepFence. Several VCS offerings now ship with secret scanning, including Gitlab (free) and GitHub (free for public repositories). GitGuardian also supports scanning Docker images for secrets. It works great and has tons of VCS and CI/CD integrations. Plus, it’s a French company!

GitGuardian identifying AWS credentials in the layers of a Docker image. It evens validates the credentials against AWS, to reduce false positives.

GitGuardian identifying AWS credentials in the layers of a Docker image. It evens validates the credentials against AWS, to reduce false positives.

Avoiding Misconfigured S3 Buckets, Once and for All

Many resources are available to secure S3 buckets. Still, I want to compile below a few actions you should take to secure your S3 buckets, and a few more advanced hardening tips if you’re storing particularly sensitive data in S3.

Walk Before You Run

Let’s start with the basics.

Hardening When It’s Worth It

Then, let’s look at potential hardening measures.

user/christophe IS authorized to call action s3:GetObject for resource my-aws-cloudtrail-logs/*
role/DB_InstanceRole IS authorized to call action s3:GetObject for resource my-aws-cloudtrail-logs/*
user/dany CAN call action s3:GetObject for resource my-aws-cloudtrail-logs/* THRU role/DB_InstanceRole
user/dany can call ssm:SendCommand to access an EC2 instance with access to role/DB_InstanceRole
role/DB_InstanceRole IS authorized to call action s3:GetObject for resource my-aws-cloudtrail-logs/*

Securing the Instance Metadata Service

In July 2019, Capital One suffered a well-known breach. Attackers exploited a SSRF vulnerability, and used it to steal credentials of an EC2 instance with a privileged role. A few months after, AWS released IMDSv2, a new version of the instance metadata service making it much harder to exploit.

However, AWS chose to keep insecure defaults, so you must explicitly enforce it. The best way to do this is to use an organization-wide SCP to require the use of IMDSv2. When using features such as autoscaling, make sure your launch templates are configured to enable it.  In any case, do have a look at metabadger and the awesome blog post that the Latacora team put together.

If you use AWS EKS, you should also block egress pod traffic to the instance metadata service and use the “IAM Roles for Service Accounts” feature instead. See my detailed blog post on the topic.

Finally, note that GuardDuty has two finding types (InstanceCredentialExfiltration.InsideAWS and InstanceCredentialExfiltration.OutsideAWS) flagging the usage of instance credentials outside AWS or in another AWS account.

Conclusion

While 2021 was an unusual year in many respects, we can’t say it was for techniques used by attackers to breach cloud environments. Some cynics might even argue they are boring, or ask “what is this, 2017?”. To me, this reaffirms the need to get the basics right, and the importance of pragmatic security, guardrails and secure defaults.

It’s likely 2022 will see data breaches caused by similar weaknesses. We’ll probably see malware authors and threat actors picking up on similar techniques as TeamTNT, adding cloud-specific logic to their arsenal. Attackers might also start using more advanced persistence techniques – up to now, we’ve hardly seen anything more advanced than creating IAM users or malicious AMIs.

Thanks for reading, let’s continue the discussion on Hacker News or Twitter!

References and Acknowledgements

Thank you to Rami McCarthy for his thorough review. He actively helped improve this blog post, and his great presentation “Learning from AWS customer security incidents” was inspirational to me in my AWS security journey. Don’t miss his great Cloud Security Orienteering post and talk at the DEFCON29 Cloud Village.

I also like to recommend the following great resources:

Updates

原文链接:https://blog.christophetd.fr/cloud-security-breaches-and-vulnerabilities-2021-in-review/