SharePoint API Authentication Updates

[TylerMatteo]

Let's use this discussion to capture details related to the work we're undertaking to update how our backend applications authenticate to SharePoint's APIs.

Context

This issue was brought to our attention when a member of the CRM team informed @TylerMatteo that they had to make updates to their code to accommodate new Sharepoint client/secret keys. Based on answer to initial questions sent to that team, it sounds like these changes are due to Microsoft sunsetting support for OAuth 1 and requiring 2 instead. However, as a part of these changes, that team also switched to using secrets generated in "Azure App Registration" and using the "Microsoft Graph API".

We will be starting this work with a research to investigate the scope and impact of these changes. In particular, we need to identify whether or not we need to use a different set of APIs and update our backend code accordingly. Whatever changes we have to make will likely have to be made in this repo as well as in labs-applicant-portal, as both of these backends make use of Sharepoint APIs for viewing and/or uploading documents.

Step 1

The next step for researching the scope and impact of these updates will be to hold a meeting where we can ask the CRM team some more questions about these changes. Here is a rough script of questions we should ask:

• We should ask them to elaborate on why this change necessitated them moving to the MS Graph API and new NuGet packages. It sounds like the underlying catalyst for their change was moving from OAuth 1 to 2, but my hunch is that necessitated changes APIs/packages entirely. Based on some poking around in our code, it looks like we're already using OAuth 2 in our code.
• We should ask if they know which APIs, if any, are deprecated as a part of this change and what the new equivalents would be. They may not know which endpoints are being used because they mentioned changing NuGet packages, which suggests they're using SDKs instead of calling APIs directly like we do.
• We should ask them how they got new client/secret keys created. Rotating these keys every so often is a normal thing for us, and we usually get them from this team, but it sounds like their new keys are from something new called "Azure App Registration". We'll need key pairs for UAT to develop and eventually for Production, so we need to understand how to create those ourselves, if we can, or who to reach out to and through what channel.
• We should leave time to ask how their team came to know that they would need to make changes to their code in the first place - it's great that they informed us when they did, but really we should find out about this sort of thing from the same primary source that they did, whether that is some sort of Microsoft rep or another communication channel.

Relevant Code

Most of the relevant existing code for this issue is found in the sharepoint service. This service is used by other services and controllers but encapsulates the code calling Sharepoint-related APIs.

Note that there is also a sharepoint service in labs-applicant-portal

It looks like we call an API accounts.accesscontrol.windows.net to get an auth token. The endpoint we call here contains /OAuth/2, suggesting we're already using OAuth v2.

We then use that token to call endpoints on https://nyco365.sharepoint.com/sites/<site id>/_api/web/, passing the token as an Authorization Bearer token header. We use these endpoints to view/list/upload documents to Sharepoint. In Sharepoint parlance, these files fall under "lists", which fall under "sites". We have one site for files that correspond to CRM's UAT2 environments, and another for production files.

It's unclear if we will have to move away from either of these APIs and, if so, what the closest new equivalents are. My best guess as to what those might be are probably something under Microsoft's "Graph" APIs. It has endpoints for doing stuff with files in Sharepoint, but the endpoints are different and likely have different response/request schemas.

[TylerMatteo]

I did some digging into Microsoft's documentation for Sharepoint APIs. I haven't found any references to that accounts.accesscontrol.windows.net URL we're hitting to get the token. I think this is the intro page for the API endpoints we're hitting that start with https://nyco365.sharepoint.com/:

https://learn.microsoft.com/en-us/sharepoint/dev/sp-add-ins/get-to-know-the-sharepoint-rest-service?tabs=http

This, and the rest of the documentation under the "SharePoint REST API v1" header in the left-hand navigation, seem to look similar to the endpoints we're calling.

If we do have to change APIs, we may have to effectively move to what they call "SharePoint REST API v2". Interestingly, that documentation then points us toward's Microsoft Graph REST APIs. It includes this link to the documentation for those APIs. It seems like they effectively saying that the SharePoint portion of Graph API 1.0 is effectively a wrapper around SharePoint's 2.0 APIs.