Introduction
During a recent cloud breakout test on an AWS environment, several significant vulnerabilities were uncovered, shedding light on potential areas of improvement for cloud security that organisations can easily overlook. Contextually a breakout test assumes that the attacker already has some kind of foothold on a system which they will seek to exploit or ‘breakout’ of to obtain further privileged access or lateral movement within an environment.
Below are some of the more intriguing findings, which highlight often-overlooked risks in cloud-deployed systems. Be sure to address these in your own environment and perform regular penetration testing to ensure there are no hidden attack surfaces. Stay tuned as we explore more case studies in future.
Root Access via IMDS Metadata
One of the most critical vulnerabilities discovered was the ability to gain root access via the Instance Metadata Service (IMDS) on an AWS VM instance.
IMDS is a service provided by cloud providers like AWS which supplies metadata to running instances. This metadata can be used to configure, manage and obtain information about the running instance. An attacker gaining a foothold on a server instance can query the IMDS by default and some of this data can be considered sensitive depending how it is configured, including:
· IAM role credentials: These credentials provide temporary security access for the instance's IAM role, including access keys and session tokens. By default, these credentials are designed to grant the necessary permissions for the instance to perform its intended functions. However, if these credentials are misconfigured or have overly permissive policies applied, they can lead to an elevated role and potential to compromise the environment.
· User Data: The user data is arbitrary and can contain scripts, configuration details, or other information that is passed to the instance at launch. This means its sensitivity can vary widely.
· Network information: This includes details about the instance's network configuration, such as private IP addresses and security group memberships which can be useful to an attacker who has breached the system.
· Tags: Similar to user data, tags may be configured with overly sensitive information.
In this case sensitive data was improperly placed in the user data metadata, which the tester accessed to find root account credentials, leading to immediate and trivial privilege escalation and further compromise of the instance. This was a severe security lapse that resulted in what we in cybersecurity colloquially call “Game Over”.
To mitigate such risks, it is crucial to ensure that sensitive information is never placed in user data. It is also recommended to use IMDSv2, which requires session-based authentication and provides enhanced security against such attacks. This vulnerability underscores the need for strict policies governing metadata accessibility and highlights the risk of storing sensitive information such as root credentials within reach without adequate security measures.
History Data Exposure
Another trivially exploited vulnerability stemmed from the discovery of a ‘bash_history’ file accessible as a standard user. This file, often overlooked in security audits, contained credentials that were used for setting up a web application. Using these credentials, it was possible to log in as an admin granting the full sysadmin role on the public facing web site.
This incident highlights the critical need to be mindful of sensitive data that may be entered on the command line. Training users and clearing history files regularly or during logout following administrative sessions and implementing strict access controls that limit who can view or access these files is a simple yet effective way to minimise the risk of credential leakage on server instances.
CouchDB Privileged Access
During the test, an Erlang instance was identified that was configured with ports typically used by RabbitMQ and CouchDB. Erlang is a runtime environment and programming language where Erlang code is executed and typically deployed for its concurrency, fault tolerance, and distributed computing capabilities. Initially, nothing seemed amiss as the ports were not responsive during a routine check. However, a deeper investigation revealed a more nuanced issue. From a non-privileged user perspective on a server, the CouchDB port was accessible but only from the localhost. This setup indicated that the service was not exposed externally, contrary to initial assumptions when querying the Erlang configuration.
Further monitoring of processes (which can be conducted by any user regardless of their permission level) uncovered a process that makes requests to a Nagios monitoring remote plugin by passing CouchDB credentials in the command line. These credentials were then used to connect to CouchDB locally where it was possible to query all data stored within. This database included comprehensive monitoring data, posing a significant risk for information disclosure.
Preventive Measures and Best Practices for Cloud Security
To mitigate the risks uncovered during cloud breakout tests, organisations must implement comprehensive security measures tailored to cloud environments. Here are some recommendations:
Securing IMDS Access
· Restrict IMDS Access: Implement strict access controls on the Instance Metadata Service (IMDS) and consider enforcing IMDSv2 for session-oriented request mechanisms, reducing the risk of unauthorised access.
· Avoid Storing Sensitive Data: Ensure sensitive data, particularly credentials, is not stored directly within instance metadata. Use secure mechanisms like AWS Secrets Manager or encrypted storage solutions.
· Audit And Monitoring: Regularly audit metadata access logs and set up alerts for unusual access patterns to detect and mitigate potential exposures early.
Managing History Data Risks
· Automate History Cleaning: Implement scripts to clear history files at the end of sessions or upon logout, minimizing the risk of leaving sensitive commands.
· Access Control: Restrict access to history files, ensuring they are only readable by authorised users and not exposed to other local users or services.
· User Education: Train users on avoiding sensitive command entry directly into the command line, promoting the use of environment variables or secure input methods.
Protecting CouchDB And Internal Services
· Localhost Restrictions: For services like CouchDB that do not need to be accessed externally, ensure they are bound only to localhost. This limits exposure to the outside world and reduces the attack surface.
· Use Firewalls: Implement firewall rules that restrict access to sensitive ports, ensuring that only authorised services or users can interact with specific applications.
· Process Monitoring: Review the commands that processes make when running to identify sensitive data such as credentials being used and implement alternative ways to send them securely where possible. Regularly monitor and audit system processes to detect unexpected behaviour or unauthorised process executions. Implementing tools like SELinux can help in enforcing strict process-level security policies.
Credential Management
· Centralised Systems: Use of centralised systems for credential management that offer dynamic secrets, rotation, and tight access controls to mitigate risks associated with hard-coded credentials. Systems within AWS include Secrets Manager, and Systems Manager Parameter Store. IAM whilst primarily used for managing user access to AWS also has capabilities to manage access keys.
· Separate Accounts and Encrypted Storage: When processes send credentials ensuring the process runs with a separate account and accesses credentials from an encrypted file location can help prevent trivial access to sensitive data.
Continuous Security Assessment
· Regular Penetration Testing: Schedule regular penetration tests to identify and fix vulnerabilities. Cloud environments should be tested as rigorously as traditional IT infrastructure.
· Security Training: Continually train technical staff on the latest cloud security threats and mitigation techniques. Knowledge is a critical defence mechanism.
· Configuration and Compliance Audits: Conduct regular compliance audits to ensure that security policies are being adhered to and are effective in protecting against known threats.
Implementing these measures requires a proactive approach to security, focusing not just on protective measures but also on detection and rapid response. Cloud security is a dynamic field, and staying ahead requires constant vigilance and adaptation to new threats.
Why Conduct Breakout Testing?
Breakout testing offers a critical method for assessing security by identifying previously unknown attack surfaces and simulating potential actions an attacker might take after gaining a basic foothold in a server or environment. Unlike traditional penetration testing, which typically examines the environment from an unauthenticated perspective, breakout testing assumes a breach has already occurred. This approach provides a more comprehensive evaluation of security by emulating threats from both external attackers and malicious insiders. It focuses on infrastructure components and underlying service configurations that are often overlooked, thus highlighting vulnerabilities that could leave a company exposed.
This testing method identifies potential issues that might not be apparent during integrated system testing. By expanding the approach to testing, organisations can further reduce potential attack surfaces and validate the performance, security, and reliability of their cloud services.
Benefits for the Organisation
· Comprehensive Security: Identifying and addressing security issues in this context helps in creating a more secure cloud environment. This increases the detection and mitigation of vulnerabilities before they can be exploited.
· Better Threat Detection and Prevention: Testing components from a breakout perspective helps identify security vulnerabilities that might otherwise be overlooked. This enables the organisation to implement robust security measures at each level of the cloud architecture.
· Real-World Simulation: By simulating real-world attack scenarios, this helps organisations to understand how attackers might exploit vulnerabilities across multiple components.
· Identification of Complex Attack Paths: This approach aids in identification of complex attack paths that may span multiple components, revealing potential security weaknesses that may not be apparent when testing individual components in isolation or from an external only perspective.
Categories