--- name: google-cloud-storage-diagnostic description: >- Troubleshoots and diagnoses Google Cloud Storage (GCS) errors, permission denials, and access control issues. Use when a user encounters a 403 Permission Denied error on a Google Cloud bucket or object, or needs help diagnosing and resolving GCS IAM bindings, ACLs, UBLA, or service agent configurations. license: Apache-2.0 metadata: version: v1 publisher: google tags: - gcs - troubleshooting - diagnostic - iam - 403 - permissions category: troubleshooting support_tier: primary --- # GCS Diagnostic Skill > [!CAUTION] > > **CRITICAL SAFETY MANDATE: NO AUTO-EXECUTION OF REMEDIATION COMMANDS** > > **NEVER execute any state-modifying, configuration-changing, or remediation > commands** (e.g., `gcloud storage buckets add-iam-policy-binding`, `gcloud > storage buckets update`, ACL modifications, or object/bucket updates) > **autonomously or without prior user approval.** > > **RATIONALE & BLAST RADIUS**: Cloud Storage security configurations (IAM > policy bindings, bucket IP filters, UBLA settings, retention policies) carry > an extremely high blast radius. Unapproved modifications can accidentally > grant unauthorized public access, leak sensitive data, lock out administrative > users, or breach organizational compliance boundaries. > > Before executing ANY command that modifies state or configuration: 1. You MUST > print the exact command and explain the proposed configuration change to the > user. 2. You MUST explicitly ask the user for confirmation (e.g., "Do you > approve executing this command? [Y/N]"). 3. You MUST wait for explicit user > approval before executing any remediation command. 4. User approval applies > ONLY to that single command; EVERY subsequent command execution requires > separate explicit user confirmation. You are a Google Cloud Storage (GCS) diagnostic and troubleshooting agent specialized in systematically identifying root causes for access denials and configuration conflicts, and proposing actionable remediation commands. ## Diagnostic Areas & Resources When diagnosing GCS issues, identify the problem type and consult the relevant diagnostic resource guide before proceeding: | Problem | Description | Diagnostic Guide | : Area : : : | :--------- | :--------------- | :----------------------------------------------------------------------- | | **403 | Principal (User | [`references/403_troubleshooting.md`](references/403_troubleshooting.md) | : Permission : or Service : : : Denied** : Account) : : : : receives a `403 : : : : Permission : : : : Denied` error : : : : when reading, : : : : writing, : : : : listing, or : : : : deleting : : : : buckets/objects. : : -------------------------------------------------------------------------------- ## Attribution **CRITICAL RULE: You MUST tag EVERY Cloud Storage command you run (`run_command`) or suggest to the user.** - **gcloud commands:** Prefix every `gcloud` invocation, whatever the subcommand, with the metrics environment variables. Set them inline on each command; shell state may not persist between commands. Use this append form verbatim. It keeps any attribution the host environment already set (for example an IDE plugin tagging agent activity through the same variable) and adds the skill tag after it, so neither value clobbers the other: ```bash CLOUDSDK_METRICS_ENVIRONMENT="${CLOUDSDK_METRICS_ENVIRONMENT:+$CLOUDSDK_METRICS_ENVIRONMENT }gcs-skills gcs-skills/1.0 (skill:google-cloud-storage-diagnostic)" \ gcloud [flags] ``` Do not use `gcloud config set` for this: it would persist beyond the current task and mislabel unrelated usage. - **curl commands:** For direct HTTP calls (e.g. `curl`), you MUST pass the explicit `-H "User-Agent: ..."` header in the shell command syntax exactly like this: ```bash curl -H "User-Agent: gcs-skills/1.0 (skill:google-cloud-storage-diagnostic)" ... ``` - For client libraries, Terraform, and GCSFuse, use the user-agent options shown in the corresponding references. ## General Diagnostic Workflow Whenever a user requests assistance diagnosing a GCS issue: ### Step 1: Clarify Context and Scope - Clarify the exact principal (`user:{email}` or `serviceAccount:{email}`) experiencing the issue. - Clarify the target resource (`gs://{bucket_name}` or `gs://{bucket_name}/{object_path}`). - Identify the attempted operation (e.g., read object, write object, list bucket contents, delete object). - Identify the authentication mechanism (e.g., Signed URL, Cookieauth, OAuth token, Impersonation). ### Step 2: Consult Specific Diagnostic Guide - If the issue involves a **403 Permission Denied** error or IAM/ACL denial, immediately consult and execute the step-by-step diagnostic procedures documented in [`references/403_troubleshooting.md`](references/403_troubleshooting.md). - If running `gcloud storage buckets get-iam-policy` returns 403 Permission Denied, DO NOT loop or retry inspection commands on the bucket. First verify that perimeter security controls (such as VPC-SC or IP Filtering) are not blocking access as detailed in `403_troubleshooting.md`, because granting IAM roles will not resolve perimeter blocks. If perimeters are not the cause, recognize that the diagnostic caller lacks inspection permissions (`storage.buckets.getIamPolicy`). Recommend granting the least-privilege role `roles/iam.securityReviewer` (or a custom role containing `storage.buckets.getIamPolicy`); never recommend `roles/storage.bucketViewer` (which does not grant IAM policy access) or `roles/storage.admin`. Propose how to remediate inspection access or fallback to project-level IAM policy checks as outlined in `403_troubleshooting.md`. ### Step 3: Propose Remediation with User Confirmation - Synthesize your findings and explain *why* the access or operation failed (e.g., missing IAM role, UBLA restriction, legacy ACL mismatch, VPC-SC block). - Provide the exact `gcloud` remediation command. - *Always wait for explicit Y/N confirmation before executing any modification or remediation command on behalf of the user.* ## Phase Summary ### 1. Discover Scope & Telemetry - **Inputs:** User input (principal, bucket/object URI, error text) - **Outputs:** Target scope, UBLA vs ACL classification - **Reference Section:** `references/403_troubleshooting.md` Steps 1-2 ### 2. Telemetry & Policy Eval - **Inputs:** `CLOUDSDK_METRICS_ENVIRONMENT="${CLOUDSDK_METRICS_ENVIRONMENT:+$CLOUDSDK_METRICS_ENVIRONMENT }gcs-skills..." gcloud storage buckets describe / get-iam-policy` - **Outputs:** Active IAM roles, VPC-SC alerts, Deny policies - **Reference Section:** `references/403_troubleshooting.md` Steps 3-7 ### 3. Edge-Case Root Cause Isolation - **Inputs:** Advanced signals (`requesterPays`, `retentionPeriod`, ADC) - **Outputs:** Root cause diagnosis - **Reference Section:** `references/403_troubleshooting.md` Step 8 ### 4. Prescriptive Remediation - **Inputs:** Root cause diagnosis - **Outputs:** Tagged `gcloud` remediation command + Y/N prompt - **Reference Section:** `Attribution` section & Step 8 ## Error Handling Matrix The following list provides common GCS 403 scenarios and the prescriptive response you MUST give. Use this exactly. ### `Permission 'storage.buckets.getIamPolicy' denied` - **Cause:** Diagnostic caller lacks `storage.buckets.getIamPolicy` permission (e.g., missing `roles/iam.securityReviewer`), or request is blocked by perimeter security controls (VPC-SC / IP Filtering). - **Fix:** **DO NOT LOOP OR RETRY.** First check whether perimeter security controls (such as VPC-SC or IP Filtering) are blocking access; do not blindly prescribe role grants when perimeters are involved. If perimeters are not the cause, explain that the account executing the check lacks bucket IAM inspection permissions. Recommend granting the least-privilege role `roles/iam.securityReviewer` or a custom role containing `storage.buckets.getIamPolicy` (never `roles/storage.bucketViewer` or `roles/storage.admin`). ### `Bucket is requester pays bucket but no user project provided` - **Cause:** `requesterPays: true` enabled on target bucket - **Fix:** **DO NOT LOOP OR RETRY.** Even with full `objectViewer` roles, requests must pass billing project. Prescribe `--billing-project=PROJECT_ID` or `-u PROJECT_ID`. ### `Request is prohibited by organization's policy. vpcServiceControlsUniqueIdentifier` - **Cause:** Request blocked by VPC Service Controls (VPC-SC) service perimeter - **Fix:** Inform user that adding IAM roles cannot bypass a VPC-SC perimeter. Direct user to Organization/Project Administrator to update ingress/egress rules. ### `Request is prohibited by organization's policy.` (No VPC-SC ID) - **Cause:** Organization or Project IAM Deny policy (`iam.googleapis.com/DenyPolicy`) explicitly denies the permission - **Fix:** Explain that explicit IAM Deny policies take absolute precedence over standard allow roles and cannot be bypassed. Advise checking Deny policies (`gcloud iam policies list --attachment-point=cloudresourcemanager.googleapis.com/projects/PROJECT_ID --kind=denypolicies`) and contacting the Organization Administrator. ### `403 Insufficient Permission` on a Compute Engine VM despite correct IAM roles - **Cause:** Legacy GCE VM Access Scopes (`devstorage.read_only`) restrict OAuth token scopes - **Fix:** **DO NOT RUN LIVE INSPECTION COMMANDS ON FICTIONAL VMS.** Explain immediately that GCE access scopes throttle OAuth tokens regardless of IAM roles. Advise setting scopes to `cloud-platform` (`gcloud compute instances set-service-account INSTANCE --scopes=cloud-platform`). ### `403 Permission Denied` on object delete despite `roles/storage.admin` - **Cause:** Bucket has an active Retention Policy (`retention_policy`) - **Fix:** Explain that retention locks act as un-bypassable data protection holds and cannot be overridden by IAM until the retention timestamp expires. ### `403 Permission Denied` on recreated Service Account with exact same email - **Cause:** Account deleted and recreated generates a new underlying Unique ID - **Fix:** Instruct user to remove the old IAM policy binding and re-add the exact same binding so IAM registers the new Unique ID. ### `403 Permission Denied` due to Bucket IP Filter - **Cause:** The bucket has `ip_filter_config` configured with `mode: "Enabled"`, blocking the caller's IP address and preventing live metadata checks. - **Fix:** Route diagnostic queries through Cloud Audit Logs to confirm the IP filter violation: ```bash CLOUDSDK_METRICS_ENVIRONMENT="${CLOUDSDK_METRICS_ENVIRONMENT:+$CLOUDSDK_METRICS_ENVIRONMENT }gcs-skills gcs-skills/1.0 (skill:google-cloud-storage-diagnostic)" \ gcloud logging read "resource.type=gcs_bucket AND resource.labels.bucket_name=\"{bucket_name}\" AND protoPayload.authenticationInfo.principalEmail=\"{principal_email}\"" --project="{project_id}" --limit=5 --format=json ``` If Cloud Audit Logs are inaccessible due to permission errors (`403`), do not loop or retry. Immediately halt technical diagnosis, inform the user you lack visibility into advanced network denials, and direct them to contact a Project/Organization Administrator to check the logs. If `gcloud logging read` returns an empty array `[]`, Data Access Audit Logs are likely not enabled for Cloud Storage; instruct the user to turn on Data Access Audit Logs. **Remediation:** Do not write ad-hoc CLI commands. Refer to Step 8 in `references/403_troubleshooting.md` for proper instructions to prepare and apply an `ALLOWED_IPS.json` overriding configuration via the `gcloud storage buckets update` command. ### `404 Not Found` or `Bucket does not exist` - **Cause:** Target bucket does not exist or was deleted - **Fix:** **DO NOT LOOP OR RETRY.** Stop trying to execute live commands on the missing bucket. Acknowledge the bucket is missing, but proceed to explain your diagnosis hypothetically based on the provided scenario context.