Announcing ‘Cirrus’ – New Opensource Tool to Combat Google Cloud Incident Response Challenges

Background

• Today Sygnia released ‘Cirrus’, an opensource tool designed to streamline Google Cloud forensic evidence collection. Together with the information in Sygnia’s ‘Incident Response in Google Cloud’ blog series, the Cirrus tool can help organizations overcome these challenges.
• The increased use of Google Cloud and of cloud-related incidents has highlighted a lack of research and documentation available to security teams during investigations.
• There are three major challenges associated with conducting incident response, threat hunting, and posture assessments in Google Cloud: Knowledge, Access and Collection, and Effective Analysis.
• This blog is part of the Google Cloud blog series published by Sygnia with the intention of sharing accessible knowledge about Google Cloud forensics and easing investigators’ work.

Introduction

In this blog Sygnia introduces the ‘Cirrus’ opensource tool, and demonstrate how it can be used to simplify incident response activities, and improve an organization’s security posture. This blog shows how Cirrus can be used to streamline environment access and evidence collection in investigations that involve both Google Workspace and GCP.

Google is known as a challenging environment for incident response teams; Sygnia published two blogs that serve as an introduction to incident response and threat hunting in Google Cloud – which includes Google Workspace and Google Cloud Platform (GCP). The first – Google Cloud Infrastructure – introduced readers to the Google Cloud ecosystem, outlining how Google Workspace and GCP work together, and share the same identity management infrastructure. The second – Google Cloud Forensic Artifacts – explored the forensic artifacts that investigators can use to handle various types of activities, such as incident response, threat hunting, and posture assessments.

This blog serves not only as an overview of the ‘Cirrus’ tool but marks the conclusion of the introduction series, and paves the way for more advanced content in the near future.

Incident Response Challenges in Google Cloud

There are three primary challenges when conducting incident response, threat hunting or posture assessments in a Google Cloud ecosystem:

1. Knowledge
2. Access & Collection
3. Analysis

Challenge #1: Knowledge

Lack of knowledge can be a serious obstacle in incident response – as in any cybersecurity field. Simply gaining the relevant access to Google Cloud is a complex process that requires familiarity with the authentication mechanisms, application integration procedures, and the overall Google Cloud environment.

Although Google Cloud has a 9% market share, making it a widely-used tool, there is a scarcity of resources and information that address incident responders’ needs. Such information includes the connection between Google Workspace and GCP, their shared identity management, available and valuable forensics artifacts, and ways to investigate Google Cloud cases.

For these reasons, we created an introductory series of Google blogs – of which this is the third in the series. These blogs are intended to contribute knowledge to the cybersecurity community, and to manage the challenges of working with Google Cloud.

Challenge #2: Access & Collection

• Gaining access to start incident response operations in Google Cloud can be a complex process. This initial effort requires knowledge of targeted evidence sources, the platforms and services where they reside, and the necessary permissions for access. Due to the dynamic nature of incident response, data collection must be efficient, flexible, and scalable, in order to ensure timely response. In addition to collecting data that resides in separate logical locations (i.e., GCP vs Google Workspace), it can be necessary to access user data such as mailbox settings and email messages. An organization that has not purchased services specifically for this purpose has limited options for gaining visibility into user data. To address this challenge, a solution is needed that provides access throughout the Google Cloud ecosystem, supports scalable delegation of authority, and allows for programmatic access.
• Collecting forensic artifacts plays a vital role during the initial phases of incident response. The main challenges associated with collection are flexibility, and scalability. Although the web consoles or Google Cloud CLI (GCloud) are user-friendly, their real-world application is limited. GCloud’s Admin console can be useful for collecting data in Google Workspace or Cloud Identity, but its implementation is limited compared to the use of APIs for data collection. For example, the exporting of log events is restricted to a maximum of 100,000 rows, and must be done separately for each log. When dealing with a large volume of log data, this makes the collection inefficient and may hinder incident response workflows. While using the Admin Console to observe data associated with users, groups, and organizational units is convenient, it is not scalable for more complex inquiries. In addition, the ability to obtain application-specific user settings and data is limited or unavailable in most Google subscription levels. For example, GCloud’s ‘Email Log Search’ option provides only email metadata information and does not include information relating to content or attachments. There are also limitations when extracting GCP data through GCloud commands or the Cloud console. If a company has not already aggregated log data to a centralized location, it is impossible to export historical events from individual projects, folders, and the organization in an efficient and scalable manner. A lack of programmatic access also increases the difficulty of examining basic GCP environment configuration (e.g., IAM role bindings and service configuration), requiring navigation of the Cloud console or scripting GCloud commands.

One of the options that Google offers to centralize the data scattered throughout different locations is through forwarding Google Workspace logs to GCP, using the ‘Share data with Google Cloud services’ option. However, this solution does not fulfill security teams’ needs, as it forwards only specific selected Google Workspace logs and is applicable only to live-stream data, leaving us with the same problem of centralizing all the data in one place.

Challenge #3: Effective Analysis

Public cloud environments are complex and constantly evolving, making it difficult for security teams to handle incidents in Google Cloud. When compared to its market-dominant competitors (e.g., AWS and Azure), there is a lack of publicly-accessible knowledge and resources for incident response and threat hunting in GCloud – especially in terms of how to utilize logs and configurations to identify suspicious or anomalous activity. Effective security operations can be aided by acquiring knowledge of concepts like privilege escalation through impersonation, high-risk API calls, and misconfiguration in ‘crown jewel’ services. Although the concepts of log analysis and architecture review are not new, their implementation in Google Cloud can be different from the implementation in other cloud providers.

Subsequent blogs in this series will delve deeper into the analysis of Google Cloud logs and configurations.

Solutions

In order to handle incident response more effectively in Google Cloud, it is vital to overcome the obstacles relating to Knowledge, Access and Collection, and Analysis. Having identified these challenges, Sygnia team focused the efforts on two subjects:

• Knowledge and Analysis Strategies – develop Sygnia blog posts and webinars for a better understanding of Google Cloud.
• Collection – release Cirrus, an open-source collection tool that is flexible and simple to use.

Exploring Sygnia’s Cloud Evidence Collection Tool: Cirrus

Sygnia developed the Cirrus Python-based tool to assist and improve incident response and threat hunting operations in Google Cloud. Cirrus consists of two individual components:

• Assistant: automates access to Google Cloud environments for evidence collection, and removes access once operations have concluded. This script fulfills the required prerequisites for the second component: the Collector script.
• Collector: collects logs, configurations, and user data on-demand from Google Workspace and GCP.

Capabilities Highlights – Why to Use Cirrus

The main capabilities of Cirrus include the following:

• Aggregate logs and configurations from different Google Cloud components.
• Access user-specific data in Gmail.
• Automate access prerequisites in preparation for evidence collection.
• Obtain significant insights to improve security posture.
• Provide an intuitive and efficient method of collecting specific, or all available logs.

The collection capabilities of Cirrus are summarized in the following table:

Additionally, all of the tool’s output is preserved in its original JSON formatting, so that it may be used for further investigation by any analysis platform or tool, such as Splunk, Elastic, or the jq JSON processor.

More information about Cirrus capabilities can be found in the GitHub page: https://github.com/SygniaLabs/Cirrus

Methodology – When to Use Cirrus

Cirrus can be used in various types of projects to collect forensic logs, configurations, and data from Google Cloud. The setup phase takes only a few minutes, and can be initiated either prior to, or at the start of a forensic investigation.

Once a project starts, evidence should be collected based on the project’s type and goal. Following are several guidelines on how to use Cirrus:

Access & Collection – Incident Response

• When an incident is detected – whether it is indicated by a Google Alert Center alert, or any source that suggests Google Cloud is involved – logs should be the main priority to focus on.
  • If the incident is identified in Google Workspace, collect the logs relevant to the incident based on the module table available in the Cirrus tool’s README file, or collect all the available logs.
  • If the incident involves GCP,collect the logs fromtherelevant resource – for example projects, folders, or organization. If possible, it is recommended to collect logs from specific resources, to minimize the potentially high volume of entries in GCP logs.
• If the investigation lead suggests that emails may be involved, Cirrus can be used to collect emails from a suspected mailbox, or to use queries to search for suspicious emails across all mailboxes, if applicable. This capability is important, given that Gmail visibility may be limited by subscription level.

Access & Collection – Threat Hunting

• Define a scope for the threat hunting activity and decide on main threat hunting analytics.
• Based on the defined scope, use Cirrustocollect all the available logs and configurations from Google Workspace and/or GCP. If Gmail is in scope, collect mailbox configurations for relevant users, or query Gmail messages using Cirrus.

Access & Collection – Posture Assessment

• Risk assessments require inspecting problematic configurationsthat may be exploited by threat actors, internal threats, or mistakes made by over-privileged users. At the start of a risk assessment, define the project’s scope, including what configurations should be inspected.
• Once the scope is decided, use Cirrusto collect identity management, Gmail, and GCP configuration data for specific users or for all users.

Analysis – General Investigation

Following any of the above types of project, continue with the following steps after initial evidence collection:

• Once the evidence files are collected by Cirrus in JSON format, it is recommended to index them to any SIEM platform before analyzing. For best results, index logs and configurations separately.
• During the investigation, if any additional logs, email data, or configurations from different sources are required, use Cirrus again to collect them instantly. It is important to remember that Google Workspace and GCP are linked, and therefore it may be necessary to collect evidence from both platforms. Further demonstration of this concept can be found in the case analyses that will be shared in future blogs.

How Does Cirrus Work?

In this section, we cover how Cirrus utilizes Google APIs and GCloud commands to gain access to and acquire the relevant evidence from Google Cloud.

Accessing Google Cloud

Cirrus utilizes service accounts to access Google Cloud, providing a flexible solution for incident response at scale. Any service account can make authorized API calls by authenticating as itself, or as any user account, through Google’s domain-wide delegation.

The process for gaining access to Google Cloud is detailed in the following diagram:

The Cirrus tool creates a separate project and service account in GCP, to enable the clear separation of responsibilities and access controls, and to provide visibility over incident response activities. This approach contributes to making security operations more efficient, effective, and secure.

Collection at Scale

Cirrus employs Google API client libraries to programmatically collect logs, configurations, and data in both Google Workspace (or Cloud Identity) and GCP. By interacting with Google APIs, in conjunction with domain-wide delegation of authority, Cirrus can bypass any log export restrictions and access data that is not readily available in the web consoles. By making API requests on behalf of a user account, the service account can gather all the relevant evidence, including user-specific data such as mailbox settings, email threads, and email attachments. The use of APIs provides a programmatic way to access data, which can increase the speed and efficiency of acquiring forensic evidence, as well as providing more flexibility and control over the data collected.

When using APIs, we can gather data that is otherwise only made available to the Admin console by purchasing additional services (such as Google Vault), or a higher subscription tier.

Necessary Prerequisites

To use Cirrus collection, the following prerequisites are required:

• A service account key file in JSON format, as downloaded after using Cirrus Assistant in Google Cloud Shell. Detailed usage explanation can be found in the Cirrus Assistant  README file.
• The email address of a Super Admin user,for example: admin@example.com. If a Super Admin user cannot be used, choose a dedicated user with the appropriate privileges during collection. The requirements for the dedicated user can be found in Appendix #1: Necessary Permissions for Utilizing Cirrus.
• If required, the id of any specific GCP or Workspace resource.

Conclusion and Case Studies – Incident Response with Cirrus

The purpose of this blog – along with Sygnia’s previous blogs on the subject and the open-source tool Cirrus – is to help organizations overcome incident response challenges in Google Cloud. In this blog, we began by exploring the three categories of challenges that organizations face, and then introduced Cirrus, a forensic collection tool for Google Cloud that streamlines evidence collection and provides insights to improve security posture.

You can access the Cirrus open-source tool here: https://github.com/SygniaLabs/Cirrus. We encourage you to share with us your thoughts about the scripts, to ask questions, and contribute to Cirrus!

In order to gain a better understanding of Cirrus functionality, and the value added to organizations that makes use of this tool, it is recommended to combine the reading of this blog with a review of case studies, which will be published in the near future.

The case studies are based on real attacks that were investigated by Sygnia’s IR team, while leveraging Cirrus to efficiently collect evidence and explore analysis strategies.

Appendix #1: Necessary Permissions for Utilizing Cirrus

If you decide not to use a Super Admin account for the Cirrus delegation process, or this account cannot be used, use a dedicated user with the following permissions, based on the desired actions from Cirrus:

Category	Permission	Short Summary
Admin Directory	admin.directory.user.readonly	View users’ basic profile information
Admin Directory	admin.directory.domain.readonly	View domain’s basic profile information
Admin Directory	admin.directory.user.security	Manage security settings for users
Admin Directory	admin.directory.device.chromeos.readonly	View Chrome OS devices
Admin Directory	admin.directory.customer.readonly	View customers’ basic profile information
Admin Directory	admin.directory.group.readonly	View groups’ basic profile information
Admin Directory	admin.directory.device.mobile.readonly	Retrieves details of mobile devices
Admin Directory	admin.directory.orgunit.readonly	View organizational units’ basic profile information
Admin Directory	admin.directory.rolemanagement.readonly	View role assignments and privileges
Log Events	admin.reports.audit.readonly	View audit activity logs
Log Events	admin.reports.usage.readonly	View usage activity logs
Gmail	gmail.readonly	View emails and settings in Gmail

Contributors: Itay Shohat, Wesley Guerra, Shani Adir

Google Cloud forensics blog series

• Incident Response in Google Cloud: Foundations
• Incident Response in Google Cloud: Forensic Artifacts
• Announcing ‘Cirrus’ – New Opensource Tool to Combat Google Cloud Incident Response Challenges
• Proof of Concept: Overcoming Google Cloud Incident Response Issues with ‘Cirrus’

This advisory and any information or recommendation contained herein has been prepared for general informational purposes and is not intended to be used as a substitute for professional consultation on facts and circumstances specific to any entity. While we have made attempts to ensure the information contained herein has been obtained from reliable sources and to perform rigorous analysis, this advisory is based on initial rapid study, and needs to be treated accordingly. Sygnia is not responsible for any errors or omissions, or for the results obtained from the use of this Advisory. This Advisory is provided on an as-is basis, and without warranties of any kind.