Internal Logs: Receiving "Request had insufficient authentication scopes" When Attempting to Export to Google Calendar

[conzty01]

Describe the bug
When attempting to export a schedule to Google Calendar after a period of time has past since authorizing RADSA, the UI will indicate that the application is working, however nothing will happen. Looking at the logs, the following error appears:

[2020-12-10 12:12:38.302] INFO in _internal: 127.0.0.1 - - [10/Dec/2020 12:12:38] "GET /api/exportToGCal?monthNum=1&year=2021 HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/home/conzty01/.envs/raScheduler/lib/python3.6/site-packages/flask/app.py", line 2301, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/conzty01/.envs/raScheduler/lib/python3.6/site-packages/flask/app.py", line 2287, in wsgi_app
    response = self.handle_exception(e)
  File "/home/conzty01/.envs/raScheduler/lib/python3.6/site-packages/flask/app.py", line 1733, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/conzty01/.envs/raScheduler/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/home/conzty01/.envs/raScheduler/lib/python3.6/site-packages/flask/app.py", line 2284, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/conzty01/.envs/raScheduler/lib/python3.6/site-packages/flask/app.py", line 1807, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/conzty01/.envs/raScheduler/lib/python3.6/site-packages/flask/app.py", line 1710, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/conzty01/.envs/raScheduler/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/home/conzty01/.envs/raScheduler/lib/python3.6/site-packages/flask/app.py", line 1805, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/conzty01/.envs/raScheduler/lib/python3.6/site-packages/flask/app.py", line 1791, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/conzty01/.envs/raScheduler/lib/python3.6/site-packages/flask_login/utils.py", line 261, in decorated_view
    return func(*args, **kwargs)
  File "/home/conzty01/Dropbox/cs330/RAScheduler/scheduleServer.py", line 2157, in exportToGCal
    status = gCalInterface.exportScheduleToGoogleCalendar(token, gCalId, regSched + breakSched)
  File "/home/conzty01/Dropbox/cs330/RAScheduler/gCalIntegration.py", line 216, in exportScheduleToGoogleCalendar
    calendarId = self.createGoogleCalendar(client_creds)
  File "/home/conzty01/Dropbox/cs330/RAScheduler/gCalIntegration.py", line 180, in createGoogleCalendar
    created_calendar = service.calendars().insert(body=newCalBody).execute()
  File "/home/conzty01/.envs/raScheduler/lib/python3.6/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/home/conzty01/.envs/raScheduler/lib/python3.6/site-packages/googleapiclient/http.py", line 840, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://www.googleapis.com/calendar/v3/calendars?alt=json returned "Request had insufficient authentication scopes.">

To Reproduce

1. Steps to reproduce the behavior:
2. Connect a Google Calendar account to a Res Hall via the Manage Hall page.
3. Wait for a period of time (roughly 24 hours in this example).
4. Navigate to the Edit Schedule page.
5. Click the "Export Schedule" button and click "Export".
6. Observe that the cursor indicates that the application is loading.
7. Observe the above error message in the logs.

Expected behavior
When RADSA requests permissions to access a user's calendar, we request offline access which should allow us to refresh the token if it expires without re-prompting the user. The desired behavior is to have the user authorize once and let us re-authorize when needed.

Desktop (please complete the following information):

• OS: Ubuntu
• Browser: Firefox
• Version: v20.12.9

Additional context
The branch I am on does bump Cryptography from 3.0 to 3.2, however I have noticed this behavior without this bump. At the time, I assumed the issue was due to a legacy token as I had just corrected a spelling error when requesting the user access.

[conzty01]

While testing, I was able to confirm that after reconnecting the Google Account, I do not experience this issue. That being said, before I reconnected the account, I did attempt to add more scopes to the gCalIntegratinator to see if I was missing something. After doing so, I retested and saw the same error so it appears that we might not be missing scopes, but rather that our authorization expires? However, I have confirmed that we are requesting offline access so we should be able to refresh our user credentials. In order to confirm this I will need to wait until this latest authentication attempt expires so that I can step through the debugger and see if we are able to refresh our token.

[conzty01]

Troubleshooting this again this morning.

I have confirmed that the credentials we pull from the DB have expired and do contain a refresh token to be able to refresh the user credentials.

I have also confirmed that the following scopes are associated with the credentials which are the scopes that are requested by the gCalIntegratinator.

During my testing, I was able to reproduce this behavior and I feel that I have a decent understanding of what is occurring. This is really only an issue if the user has deleted the Secondary Google Calendar we created prior to exporting. If the user does not delete the calendar, then we can add events to it as expected.

The execution flow that brought us to the error is as follows:

1. In the editSched page, the user opts to export the schedule to Google Calendar.
2. In the exportToGCal method, credentials are loaded from the DB. These credentials have expired but also contain a refresh tooken. The Google Calendar Id for the Res Hall is also loaded in.
3. The exportToGCal method calls the gCalInterface.exportScheduleToGoogleCalendar method passing it the user credentials (client_creds), the Google Calendar Id (calendarId) and the schedule to be imported (schedule).
4. The gCalInterface.exportScheduleToGoogleCalendar method checks to see if the user credentials are valid.
5. Since the credentials are not valid, the integration refreshes them.
6. The gCalInterface.exportScheduleToGoogleCalendar method checks with Google Calendar to see if the client_creds has the Google Calendar associated with calendarId listed in its calendar list.
7. If the Google Calendar is not listed in the calendar list, then the interface attempts to create a new Secondary Google Calendar by calling self.createGoogleCalendar() passing it the client_creds.
8. The createGoogleCalendar method then checks the validity of the credentials and builds a new service.
9. The createGoogleCalendar method then attempts to execute the following: service.calendars().insert(body=newCalBody).execute() which errors out with the Traceback defined above.

During this troubleshooting, I can see that the credentials have the necessary scopes attached to them all the way up until the service.calendars().insert(body=newCalBody).execute() line is executed.

At this time, I am unsure as to why reconnecting the Google Calendar Account and generating a new Secondary Calendar does not run into this issue. I will be exploring these items more below.

[conzty01]

Did some more troubleshooting and found that when reconnecting a Google Account and following the above steps, I am unable to recreate this issue. However, after reconnecting the Google Account and stepping through the code, I was able to find the following scopes set to the service that was used to create the Secondary Google Calendar:

These differ that what was associated with the user's credentials from what I could tell, however I was not able to check what the scopes for the service are while the issue is still occurring. As such, I will need to check this again after the credentials have expired to see if there is a difference.

If there is a difference, then it seems that something might be taking place either in the storing of the credentials in the DB, or in the creation of the service via the build() function. So far, I have not been able to find anything online related to this.

Generally speaking, it appears we would only run into this if an upper staff member for a given hall removes the Google Calendar we create from their calendarList. In these instances, it is possible for us to simply have the Hall Director reconnect the Google Account and we can rebuild the calendar, however this does feel like an inelegant solution and I would much prefer to try to recreate the calendar for ourselves with no user intervention.

[conzty01]

After waiting for the token to expire and testing again, I am able to reproduce the behavior once more. This time, I checked the authentication scopes associated with services that are built by the exportScheduleToGoogleCalendar method and the createGoogleCalendar method, and in both instances, the scopes were the same as the ones found above.

[conzty01]

After waiting for the token to expire and testing again, I am able to reproduce the behavior once more. This time, I checked the authentication scopes associated with services that are built by the exportScheduleToGoogleCalendar method and the createGoogleCalendar method, and in both instances, the scopes were the same as the ones found above.

I have created a question on Stack Overflow here to ask for some community assistance. Otherwise another step might be to reach out to Google with this issue.