Google Secret Manager for Enterprise: Architecture, Implementation, and Security Hardening

Overview

Google Secret Manager is Google Cloud's managed secrets vault for API keys, database passwords, certificates, tokens, and other sensitive configuration values. Its core purpose is to centralize secret storage, provide versioned retrieval, and enforce access through Google Cloud IAM, audit logging, and optional customer-managed encryption keys.

Enterprises adopt it to replace hardcoded credentials, reduce operational overhead compared with self-hosted vaults, and standardize secret access across GKE, Cloud Run, Compute Engine, and CI/CD pipelines. It is especially effective in organizations already using Google Cloud because it aligns with native IAM, Cloud Audit Logs, VPC Service Controls, and organization policy controls.

Architecture

Core components

• Secret: Logical container such as prod-db-password.
• Secret version: Immutable value revisions with states like enabled or disabled.
• IAM policies: Access control at project or secret level using roles such as roles/secretmanager.secretAccessor.
• Cloud KMS integration: Optional CMEK for encryption control.
• Audit logs: Admin Activity and Data Access logs for creation, access, and policy changes.

Deployment models

• Centralized secrets project: A dedicated security project hosts secrets and grants cross-project access to workloads.
• Per-application project: Each application team manages its own secrets for stronger tenancy boundaries.
• Hybrid enterprise model: Shared platform secrets are centralized, while app-specific secrets stay in workload projects.

Data flow

1. A platform engineer creates a secret and adds version 1.
2. IAM grants a workload identity or service account read access.
3. A runtime such as Cloud Run or GKE retrieves latest or a pinned version.
4. Access events are written to Cloud Audit Logs.
5. Rotation creates a new version, and workloads switch by alias or explicit version reference.

Implementation Guide

1. Enable the API and set project context.

gcloud config set project enterprise-prod-001
gcloud services enable secretmanager.googleapis.com cloudkms.googleapis.com

1. Create a dedicated service account for an application.

gcloud iam service-accounts create app-runtime-sa --display-name="App Runtime SA"

1. Create a secret with automatic replication.

echo -n 'S3cureP@ssw0rd!' | gcloud secrets create prod-db-password --data-file=- --replication-policy="automatic"

1. Grant least-privilege access.

gcloud secrets add-iam-policy-binding prod-db-password --member="serviceAccount:app-runtime-sa@enterprise-prod-001.iam.gserviceaccount.com" --role="roles/secretmanager.secretAccessor"

1. Retrieve the secret securely.

gcloud secrets versions access latest --secret=prod-db-password

1. Create a CMEK key and a secret protected by KMS.

gcloud kms keyrings create sec-ring --location=global
gcloud kms keys create sm-key --location=global --keyring=sec-ring --purpose=encryption
echo -n 'api-token-123' | gcloud secrets create third-party-api-token --data-file=- --replication-policy="automatic" --kms-key-name="projects/enterprise-prod-001/locations/global/keyRings/sec-ring/cryptoKeys/sm-key"

1. Example Cloud Run deployment consuming a secret as an environment variable.

gcloud run deploy payments-api --image=gcr.io/enterprise-prod-001/payments:2026-08-14 --service-account=app-runtime-sa@enterprise-prod-001.iam.gserviceaccount.com --update-secrets=DB_PASSWORD=prod-db-password:latest --region=europe-west1

Code Examples

# Rotate a secret by adding a new version and disabling the old one
echo -n 'N3wS3cureP@ssw0rd!' | gcloud secrets versions add prod-db-password --data-file=-
gcloud secrets versions disable 1 --secret=prod-db-password

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: payments-api
spec:
  template:
    spec:
      serviceAccountName: app-runtime-sa@enterprise-prod-001.iam.gserviceaccount.com
      containers:
      - image: gcr.io/enterprise-prod-001/payments:2026-08-14
        env:
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: prod-db-password
              key: latest

from google.cloud import secretmanager
client = secretmanager.SecretManagerServiceClient()
name = "projects/enterprise-prod-001/secrets/prod-db-password/versions/latest"
response = client.access_secret_version(request={"name": name})
secret_value = response.payload.data.decode("UTF-8")
print("Secret length:", len(secret_value))

Security Hardening

• Use least privilege: Grant roles/secretmanager.secretAccessor only to workload identities that need read access.
• Prefer secret-level IAM for high-value credentials instead of broad project-level grants.
• Use CMEK when separation of duties or key revocation control is required.
• Enable Data Access audit logs to capture secret reads for regulated environments.
• Use VPC Service Controls to reduce exfiltration risk from trusted perimeters.
• Pin versions for sensitive rollouts when deterministic deployment behavior matters.
• Rotate regularly and disable old versions after validation.
• Avoid writing secrets to logs by masking environment output and preventing debug dumps.

Comparison

Troubleshooting

1. Permission denied

Log sample:

ERROR: (gcloud.secrets.versions.access) PERMISSION_DENIED: Permission 'secretmanager.versions.access' denied for resource 'projects/enterprise-prod-001/secrets/prod-db-password/versions/latest'

Fix: Verify the caller identity and grant roles/secretmanager.secretAccessor on the specific secret or project.

2. API not enabled

Log sample:

google.api_core.exceptions.PermissionDenied: 403 Secret Manager API has not been used in project 123456789 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/secretmanager.googleapis.com/overview?project=123456789

Fix: Run gcloud services enable secretmanager.googleapis.com in the target project and retry after propagation.

3. Secret version disabled

Log sample:

FAILED_PRECONDITION: Secret Version [projects/enterprise-prod-001/secrets/prod-db-password/versions/1] is disabled.

Fix: Access latest if appropriate, or re-enable the required version with gcloud secrets versions enable 1 --secret=prod-db-password.

Best Practices

Do

• Centralize naming such as env-app-purpose, for example prod-payments-db-password.
• Use workload identity instead of long-lived service account keys.
• Separate duties by letting security teams manage KMS keys and platform teams manage secret lifecycle.
• Automate rotation through CI/CD or Cloud Scheduler plus Cloud Functions.

Don't

• Do not store secrets in source control, even encrypted files, when native secret retrieval is available.
• Do not grant Owner or Editor just to solve access issues.
• Do not inject secrets into build logs with commands like echo $DB_PASSWORD.
• Do not rely only on latest for all production systems; pin versions during controlled releases.