Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: 403 Forbidden Google Storage Error in GBFS pipeline #835

Open
cka-y opened this issue Nov 27, 2024 · 0 comments
Open

Fix: 403 Forbidden Google Storage Error in GBFS pipeline #835

cka-y opened this issue Nov 27, 2024 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@cka-y
Copy link
Contributor

cka-y commented Nov 27, 2024

Describe the Bug

A total of 297 GBFS feeds are affected by a 403 Forbidden error when attempting to upload files to Google Cloud Storage.

Example Error:

Error processing GBFS files: 403 POST https://storage.googleapis.com/upload/storage/v1/b/mobilitydata-gbfs-snapshots-prod/o?uploadType=multipart: {
  "error": {
    "code": 403,
    "message": "functions-service-account@mobility-feeds-prod.iam.gserviceaccount.com does not have storage.objects.delete access to the Google Cloud Storage object.",
    "errors": [
      {
        "message": "functions-service-account@mobility-feeds-prod.iam.gserviceaccount.com does not have storage.objects.delete access to the Google Cloud Storage object.",
        "domain": "global",
        "reason": "forbidden"
      }
    ]
  }
}
: ('Request failed with status code', 403, 'Expected one of', <HTTPStatus.OK: 200>)

Affected Feeds (Sample Stable IDs):

['gbfs-nextbike_ng', 'gbfs-nextbike_tv', 'gbfs-bcycle_greenbikeslc', 'gbfs-nextbike_ea', 'gbfs-nextbike_xc', 'gbfs-bcycle_redding', 'gbfs-donkey_ge', ...]


Steps to Reproduce

  1. Run the pipeline with any of the affected feeds listed above.
  2. Observe the 403 Forbidden error when processing GBFS files.

Expected Results

The files should be processed successfully without encountering a 403 Forbidden error.


Actual Results

The pipeline fails due to a lack of necessary storage.objects.delete permissions for the service account:
functions-service-account@mobility-feeds-prod.iam.gserviceaccount.com.


Additional Notes

The issue appears to be related to insufficient permissions on the Google Cloud Storage bucket. The service account lacks the necessary storage.objects.delete access to manage objects in the bucket.

Recommendation: Investigate and update the service account permissions to include storage.objects.delete for the affected bucket.


@cka-y cka-y added the bug Something isn't working label Nov 27, 2024
@cka-y cka-y self-assigned this Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant