Terraform-gcp-onboarding

This Terraform module provides an easy way to configure Aqua Security’s CSPM and agentless solutions on Google Cloud Platform (GCP).

It creates the necessary resources, such as service accounts, roles, and permissions, to enable seamless integration with Aqua’s platform.

---

Table of Contents

• Pre-requisites
• Usage
• Examples
• Providing Project ID List
• Excluding Projects Using Regex
• Using Dedicated Project
• Using Existing Network

Pre-requisites

Before using this module, ensure that you have the following:

• Terraform version 1.6.4 or later.
• gcloud CLI installed and configured.
• Python 3+ installed.
• Aqua Security account API credentials.

Usage

1. Leverage the Aqua platform to generate the local variables required by the module.
2. Important: Replace aqua_api_key and aqua_api_secret with your generated API credentials.
3. Run terraform init to initialize the module.
4. Run terraform apply to create the resources.

Examples

Onboarding a Single Project using a dedicated project

Here's an example of how to onboard a single project while using a dedicated project to host the infrastructure:

# Defining local variables
locals {
  region                 = "us-central1"                                 # Google Cloud region to use
  dedicated              = true                                          # Whether to create a dedicated project for Aqua resources
  type                   = "single"                                      # Type of deployment (single project or organization)
  org_name               = "my-org-name"                                 # Google Cloud Organization name
  aqua_tenant_id         = "12345"                                       # Aqua tenant ID
  project_id             = "project_id"                                  # Google Cloud project ID (existing project to be onboarded)
  aqua_aws_account_id    = "123456789101"                                # Aqua AWS account ID
  aqua_bucket_name       = "generic-bucket-name"                         # Aqua bucket name
  aqua_configuration_id  = "234e3cea-d84a-4b9e-bb36-92518e6a5772"        # Aqua configuration ID
  aqua_cspm_group_id     = 123456                                        # Aqua CSPM group ID
  aqua_custom_labels     = { label = "true" }                            # Additional custom labels to apply to Aqua resources
  aqua_api_key           = "REPLACE_ME"                                  # Replace with generated aqua API key
  aqua_api_secret        = "REPLACE_ME"                                  # Replace with generated aqua API secret
  aqua_autoconnect_url   = "https://example-aqua-autoconnect-url.com"    # Aqua Autoconnect API URL
  aqua_volscan_api_token = "REPLACE_ME"                                  # Replace with Aqua Volume Scanning API token
  aqua_volscan_api_url   = "https://example-aqua-volscan-api-url.com"    # Aqua Volume Scanning API URL
  dedicated_project_id   = "aqua-agentless-${local.tenant_id}-${local.org_hash}"
  labels                 = merge(local.aqua_custom_labels, { "aqua-agentless-scanner" = "true" }) # Combined labels for Aqua resources
  org_hash               = substr(sha1(local.org_name), 0, 6)                                     # Hashed organization name (first 6 characters)
}

################################

# Defining the root google provider
provider "google" {
  project        = local.project_id # Existing project to be onboarded
  region         = local.region
  default_labels = local.labels
}

# Creating a dedicated project for Aqua resources
module "aqua_gcp_dedicated_project" {
  source          = "aquasecurity/onboarding/gcp//modules/dedicated_project"
  org_name        = local.org_name
  type            = local.type
  project_id      = local.dedicated_project_id
  root_project_id = local.project_id
  labels          = local.labels
}

################################

# Defining the dedicated google provider
provider "google" {
  alias          = "dedicated"
  project        = module.aqua_gcp_dedicated_project.project_id
  region         = local.region
  default_labels = local.labels
}

# Creating onboarding resources on the dedicated project
module "aqua_gcp_onboarding" {
  source = "aquasecurity/onboarding/gcp"
  providers = {
    google.onboarding = google.dedicated # Using the dedicated project provider
  }
  type                   = local.type
  project_id             = module.aqua_gcp_dedicated_project.project_id # Dedicated project for Aqua resources
  region                 = local.region
  dedicated_project      = local.dedicated
  org_name               = local.org_name
  aqua_tenant_id         = local.aqua_tenant_id
  aqua_aws_account_id    = local.aqua_aws_account_id
  aqua_bucket_name       = local.aqua_bucket_name
  aqua_volscan_api_token = local.aqua_volscan_api_token
  aqua_volscan_api_url   = local.aqua_volscan_api_url
  depends_on             = [module.aqua_gcp_dedicated_project]
}

################################

## Onboarding the existing project and attaching it to the dedicated project
module "aqua_gcp_project_attachment" {
  source = "aquasecurity/onboarding/gcp//modules/project_attachment"
  providers = {
    google = google # Using the root project provider
  }
  aqua_api_key                                  = local.aqua_api_key
  aqua_api_secret                               = local.aqua_api_secret
  aqua_autoconnect_url                          = local.aqua_autoconnect_url
  aqua_bucket_name                              = local.aqua_bucket_name
  aqua_configuration_id                         = local.aqua_configuration_id
  aqua_cspm_group_id                            = local.aqua_cspm_group_id
  type                                          = local.type
  org_name                                      = local.org_name
  project_id                                    = local.project_id # Existing project to be onboarded
  dedicated_project                             = local.dedicated
  labels                                        = local.aqua_custom_labels
  onboarding_create_role_id                     = module.aqua_gcp_onboarding.create_role_id                     # Referencing outputs from the onboarding module
  onboarding_service_account_email              = module.aqua_gcp_onboarding.service_account_email              # Referencing outputs from the onboarding module
  onboarding_workload_identity_pool_id          = module.aqua_gcp_onboarding.workload_identity_pool_id          # Referencing outputs from the onboarding module
  onboarding_workload_identity_pool_provider_id = module.aqua_gcp_onboarding.workload_identity_pool_provider_id # Referencing outputs from the onboarding module
  onboarding_project_number                     = module.aqua_gcp_onboarding.project_number                     # Referencing outputs from the onboarding module
  depends_on                                    = [module.aqua_gcp_onboarding]
}

output "onboarding_status" {
  value = module.aqua_gcp_project_attachment.onboarding_status
}

---

Onboarding an Organization using a dedicated project

Here's an example of how to onboard an organization while using a dedicated project to host the infrastructure:

# Defining local variables
locals {
  region                 = "us-central1"                               # Google Cloud region to use
  dedicated              = true                                        # Whether to create a dedicated project for Aqua resources
  type                   = "organization"                              # Type of deployment (single project or organization)
  org_name               = "my-org-name"                               # Google Cloud Organization name
  aqua_tenant_id         = "12345"                                     # Aqua tenant ID
  billing_account_id     = "012A3B-4567CD-8EFGH9"                      # Google Cloud billing account ID
  aqua_aws_account_id    = "123456789101"                              # Aqua AWS account ID
  aqua_bucket_name       = "generic-bucket-name"                       # Aqua bucket name
  aqua_configuration_id  = "234e3cea-d84a-4b9e-bb36-92518e6a5772"      # Aqua configuration ID
  aqua_cspm_group_id     = 123456                                      # Aqua CSPM group ID
  aqua_custom_labels     = { label = "true" }                          # Additional custom labels to apply to Aqua resources
  aqua_api_key           = "<REPLACE_ME>"                              # Replace with generated aqua API key
  aqua_api_secret        = "<REPLACE_ME>"                              # Replace with generated aqua API secret
  aqua_autoconnect_url   = "https://example-aqua-autoconnect-url.com"  # Aqua Autoconnect API URL
  aqua_volscan_api_token = "<REPLACE_ME>"                              # Replace with Aqua Volume Scanning API token
  aqua_volscan_api_url   = "https://example-aqua-volscan-api-url.com"  # Aqua Volume Scanning API URL
  dedicated_project_id   = "aqua-agentless-${local.aqua_tenant_id}-${local.org_hash}"
  project_id             = "my-project-id"                             # Google Cloud project ID used to run the Cloud Asset query to fetch all project IDs and create CSPM IAM resources (Cloud Asset API must be enabled)
  projects_list          = module.aqua_gcp_org_projects.filtered_projects 
  labels                 = merge(local.aqua_custom_labels, { "aqua-agentless-scanner" = "true" }) # Combined labels for Aqua resources
  org_hash               = substr(sha1(local.org_name), 0, 6)                                     # Hashed organization name (first 6 characters)
}

################################

# Defining the root google provider
provider "google" {
  region         = local.region
  default_labels = local.labels
}

# Getting google organization ID
data "google_organization" "org" {
  domain = local.org_name
}

################################

# Defining the org_projects google provider to fetch all projects ids
provider "google" {
  alias                 = "org_projects"
  region                = local.region
  default_labels        = local.labels
  user_project_override = true
  billing_project       = local.project_id
  project               = local.project_id
}

# Fetching all active projects ids
module "aqua_gcp_org_projects" {
  source   = "../../modules/org_projects"
  providers = {
    google = google.org_projects
  }
  org_name = local.org_name
}

################################

# Creating a dedicated project
module "aqua_gcp_dedicated_project" {
  source             = "../../modules/dedicated_project"
  org_name           = local.org_name
  project_id         = local.dedicated_project_id
  type               = local.type
  billing_account_id = local.billing_account_id
  labels             = local.labels
}

################################

# Defining the dedicated google provider
provider "google" {
  alias          = "dedicated"
  project        = module.aqua_gcp_dedicated_project.project_id # Dedicated project for Aqua resources
  region         = local.region
  default_labels = local.labels
}

# Creating discovery and scanning resources on the dedicated project
module "aqua_gcp_onboarding" {
  source = "../../"
  providers = {
    google.onboarding = google.dedicated
  }
  type                   = local.type
  project_id             = module.aqua_gcp_dedicated_project.project_id
  dedicated_project      = local.dedicated
  region                 = local.region
  org_name               = local.org_name
  aqua_tenant_id         = local.aqua_tenant_id
  aqua_aws_account_id    = local.aqua_aws_account_id
  aqua_bucket_name       = local.aqua_bucket_name
  aqua_volscan_api_token = local.aqua_volscan_api_token
  aqua_volscan_api_url   = local.aqua_volscan_api_url
  depends_on             = [module.aqua_gcp_dedicated_project]
}

################################

## Iterating over all project and attaching them to the dedicated project
module "aqua_gcp_projects_attachment" {
  source = "../../modules/project_attachment"
  providers = {
    google = google # Using the root project provider
  }
  for_each                                      = toset(local.projects_list)
  aqua_api_key                                  = local.aqua_api_key
  type                                          = local.type
  aqua_api_secret                               = local.aqua_api_secret
  aqua_autoconnect_url                          = local.aqua_autoconnect_url
  aqua_bucket_name                              = local.aqua_bucket_name
  aqua_configuration_id                         = local.aqua_configuration_id
  aqua_cspm_group_id                            = local.aqua_cspm_group_id
  org_name                                      = local.org_name
  project_id                                    = each.value # Referencing each project from given project id list 
  dedicated_project                             = local.dedicated
  labels                                        = local.aqua_custom_labels
  onboarding_create_role_id                     = module.aqua_gcp_onboarding.create_role_id                       # Referencing outputs from the onboarding module
  onboarding_cspm_service_account_key           = module.aqua_gcp_onboarding.cspm_service_account_key             # Referencing outputs from the onboarding module
  onboarding_service_account_email              = module.aqua_gcp_onboarding.service_account_email               # Referencing outputs from the onboarding module
  onboarding_workload_identity_pool_id          = module.aqua_gcp_onboarding.workload_identity_pool_id           # Referencing outputs from the onboarding module
  onboarding_workload_identity_pool_provider_id = module.aqua_gcp_onboarding.workload_identity_pool_provider_id  # Referencing outputs from the onboarding module
  onboarding_project_number                     = module.aqua_gcp_onboarding.project_number                      # Referencing outputs from the onboarding module
  depends_on                                    = [module.aqua_gcp_onboarding]
}

output "onboarding_status" {
  value = {
    for project_id, attachment_instance in module.aqua_gcp_projects_attachment :
    project_id => attachment_instance.onboarding_status
  }
}

For more examples and use cases, please refer to the examples folder in the repository.

Providing Project ID List

By default we fetch all active projects and use that project list, but you can also provide your own list of project IDs by populating the projects_list local. To accommodate this, ensure to remove the module.aqua_gcp_org_projects and then replace the local projects_list with your list.

locals {
projects_list = [
  "my-project-id-1",
  "my-project-id-2",
  // Add more project IDs as needed
]
}

Excluding Projects Using Regex

You can exclude specific projects from getting onboarded by using regular expressions.

To exclude projects by id, add the variable projects_ids_exclude="regex1, regex2, regex3" to the module aqua_gcp_org_projects.

To exclude projects by name, add the variable projects_names_exclude="regex1, regex2, regex3" to the module aqua_gcp_org_projects.

Here are some examples of traditional exclusions following the instructions above:

1. Exclude Projects Starting with test-:

   • Regex: ^test-.*$
   • Description: This regex pattern matches GCP project names that start with test-.

2. Exclude Projects Ending with -test:

   • Regex: ^.*-test$
   • Description: This regex pattern matches GCP project names that end with -test.

3. Exclude Projects which include test anywhere:

   • Regex: .*test.*
   • Description: This regex pattern matches GCP project names containing the word test anywhere in the name.

Using an Existing Dedicated Project

If you have an existing dedicated project that you want to use to host Aqua Security resources, you can import it into the Terraform configuration.

To do so, use the following Terraform import command:

terraform import module.aqua_gcp_dedicated_project.google_project.project <dedicated_project_id>

Replace <dedicated_project_id> with the ID of your existing dedicated project.

It's important to note that the dedicated project ID should follow the naming convention "aqua-agentless-${local.tenant_id}-${local.org_hash}", where local.org_hash is calculated as:

org_hash = substr(sha1(<org_name>), 0, 6)

You can also check for the naming convention using the bash command:

#!/bin/bash

# Replace with your Aqua tenant ID
TENANT_ID="<your_tenant_id>"

# Replace with your organization name
ORG_NAME="<your_org_name>"

# Calculate the org_hash
ORG_HASH=$(echo -n "${ORG_NAME}" | shasum -a 1 | awk '{ print $1 }' | cut -c1-6)

# Print the dedicated project ID naming convention
echo "aqua-agentless-${TENANT_ID}-${ORG_HASH}"                                    

For example, if your Aqua tenant ID is 12345 and the first six characters of the SHA1 hash of your organization name are 12a456, the dedicated project ID should be aqua-agentless-12345-12a456.

Using Existing Network and Firewall

If you prefer to use an existing network and firewall instead of creating new ones, you can do so by setting create_network = false in the onboarding module input variables. In this case, you will need to create, prior to onboarding, network and firewall resources with the following naming convention:

Dedicated project:

• Firewall: <project_id>-rules-aqua-aas
• Network: <project_id>-network

Same project:

• Firewall: <project_id>-rules-<aqua_tenant_id>aqua-aas
• Network: <project_id>-network-<aqua_tenant_id>

When using a dedicated project, the <project_id> should follow the format "aqua-agentless-${local.tenant_id}-${local.org_hash}" as mentioned above.

Requirements

Name	Version
terraform	>= 1.6.4
external	~> 2.3.3
google	~> 5.30.0
http	~> 3.4.2
random	~> 3.6.0

Providers

Name	Version
google	~> 5.30.0

Modules

Name	Source	Version
onboarding	./modules/onboarding	n/a

Resources

Name	Type
google_organization.organization	data source
google_project.project	data source

Inputs

Name	Description	Type	Default	Required
aqua_aws_account_id	Aqua AWS Account ID	string	n/a	yes
aqua_bucket_name	Aqua Bucket Name	string	n/a	yes
aqua_tenant_id	Aqua Tenant ID	string	n/a	yes
aqua_volscan_api_token	Aqua Volume Scanning API Token	string	n/a	yes
aqua_volscan_api_url	Aqua volume scanning API URL	string	n/a	yes
create_network	Toggle to create network resources	bool	true	no
create_role_name	The name of the role to be created for Aqua	string	"AquaAutoConnectAgentlessRole"	no
cspm_role_name	The name of the role used for CSPM	string	"AquaAutoConnectCSPMRole"	no
dedicated_project	Indicates whether dedicated project is enabled	bool	true	no
delete_role_name	The name of the role used for deleting Aqua resources	string	"AutoConnectDeleteRole"	no
identity_pool_name	Name of the identity pool. If not provided, the default value is set to 'aqua-agentless-pool-<aqua_tenant_id>' in the 'identity_pool_name' local	string	null	no
identity_pool_provider_name	Name of the identity pool provider. If not provided, the default value is set to 'agentless-provider-<aqua_tenant_id>' in the 'identity_pool_provider_name' local	string	null	no
org_name	Google Cloud Organization name	string	n/a	yes
project_id	Google Cloud Onboarding Project ID	string	n/a	yes
region	Google Cloud Main Deployment Region	string	n/a	yes
service_account_name	Name of the service account. If not provided, the default value is set to 'aqua-agentless-sa-<aqua_tenant_id>' in the 'service_account_name' local	string	null	no
show_outputs	Whether to show outputs after deployment	bool	false	no
sink_name	Name of the sink. If not provided, the default value is set to '<project_id>-sink' in the 'sink_name' local	string	null	no
topic_name	Name of the topic. If not provided, the default value is set to '<project_id>-topic' in the 'topic_name' local	string	null	no
trigger_name	Name of the trigger. If not provided, the default value is set to '<project_id>-trigger' in the 'trigger_name' local	string	null	no
type	The type of onboarding. Valid values are 'single' or 'organization' onboarding types	string	n/a	yes
workflow_name	Name of the workflow. If not provided, the default value is set to '<project_id>-workflow' in the 'workflow_name' local	string	null	no

Outputs

Name	Description
create_role_id	Create role ID
create_role_name	Create role name
create_role_permissions	Permissions of the created role
cspm_role_id	CSPM role ID
cspm_role_name	CSPM role name
cspm_role_permissions	Permissions of the CSPM role
cspm_service_account_email	CSPM Service account email
cspm_service_account_id	CSPM Service account ID
cspm_service_account_key	CSPM Service account key
cspm_service_account_name	CSPM Service account name
delete_role_name	Delete role name
delete_role_permissions	Permissions of the deleted role
eventarc_trigger_destination_workflow	Destination workflow for the eventarc trigger
eventarc_trigger_name	Eventarc trigger name
firewall_name	Firewall name
network_name	Network name
org_id	Google Cloud Organization ID
org_name	Google Cloud Organization name
project_api_services	API services enabled in the project
project_id	Google Cloud Project ID
project_number	Google Cloud Project number
pubsub_topic_name	Pubsub topic name
region	Google Cloud Region
service_account_email	Service account email
service_account_id	Service account ID
service_account_name	Service account name
sink_name	Sink name
workflow_name	Workflow name
workload_identity_pool_id	Workload identity pool ID
workload_identity_pool_provider_id	Workload identity pool provider ID
workload_identity_pool_provider_id_aws_account_id	Workload identity pool provider AWS account ID