Option to make APIs / S3 uploads "always update" the solution

[allquixotic]

Thinking about recent problems I've been having, I think the problems would be best resolved by enabling a mode in SSOEx where any API calls into the SSOEX API, or -- in S3 mode -- any bucket data changes -- cause the current state in the DynamoDB solution to be overwritten unconditionally.

For example, let's say we have the following state:

• In SSO live: Permission set doesn't exist (say a user deleted it).
• In Dynamo: Permission set exists because it was originally created via SSOEx (through S3) and SSOEx isn't aware that the live state includes the permset being deleted.
• In S3: Permission set has been "re-uploaded", overwriting an identical copy of the file from before, with intent to restore the permset to live.

I would expect that SSO live would include the latest permset data from S3 after the S3 bucket PutObject occurs, even if the data in the permset is the same as what DynamoDB has.

More confusingly, even if I change the content of the permset, such as bumping a version tag, SSOEx still refuses to create the permset again in live, once it has been deleted.

As currently implemented, the solution is extremely fragile and cannot handle any changes made to SSO configuration outside of SSOEx.

I want to arrive at a "Latest Change Wins" model where the possible sources of change include S3 updates or manual management console updates (in any sequencing/order).

Analogy... If we think of the "live" SSO state as a mutable (editable) Wikipedia article, and there are 3 people editing the page, and they all click save... the latest person to hit save gets their changes published. This is the model I want.

[leelalagudu]

@allquixotic , agree 100% on the solution reliance on DDB causing it to go out of sync. When we originally designed the solution, we were banking on the premise that all changes would be made through the solution interface. However, we've come to realise that this premise is neither always correct (nor) a safe assumption to make.

In order to handle this and the other issues we've seen , we're working on a refactoring of the flows for permission set management.

We will stop using solution peristence (i.e. DDB) for lookup of (a) whether the permission set exists and (b) to understand how it looks like, to determine delta with the incoming permission set object. Instead, we will look up SSO directly for both (a) and (b). Additionally, we will update our S3 persistence with the lookup result every time a permisison set operation is triggered. As a side note, solution's S3 persistence is always populated irrespective of the interface enabled, to facilitate user's switching their config from API to S3 interface modes and viceversa multiple times for permissionsets.

We've been putting off our nightlyRun feature for quite some time now, so our plan is to first push that PR in, and then add this refactor on top of the nightlyRun.

Do let us know if this works in line with what your expectations are.

[allquixotic]

Getting rid of solution persistence (DynamoDB) sounds awesome to me. I fully agree that we should remove this additional layer of data storage, as it just creates additional headaches of data that needs to be kept in sync. Querying the live source of truth is much more reliable.