Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use new event_bus_producer_config #33458

Merged
merged 7 commits into from
Oct 16, 2023

Conversation

rgraber
Copy link
Contributor

@rgraber rgraber commented Oct 10, 2023

This PR replaces #32874 and #33269

Description

Update edx-platform to use the new event bus producer configuration format specified in [email protected]. In order to better control the rollout, we are doing an expand-contract. For this PR, all manual sends in the lms and cms will first check to see if the new configuration is present for the given event/topic combination. If the event/topic is enabled in the new configuration, the manual send will not occur. Otherwise the logic for the manual send will continue as before.

How we use the new config is a little complex in order to account for a few use cases:

  1. Multiple events being enabled/disabled as a group. This is useful for multiple events that go on a single lifecycle topic. To accomplish that, we add the config to derived_settings so the enabled flags of the relevant events can be set according to the value of a single feature flag.
  2. Enabling/disabling a single known event from yml. To accomplish this, we add the config to KEYS_WITH_MERGED_VALUES and call merge_producer_configs from openedx-events to combine the base config and the yml config. This means that operators don't need to copy over the whole configuration from edx-platform to enable/disable a single event while keeping everything else the same as the base config.

Supporting information

Roll out plan: https://openedx.atlassian.net/wiki/spaces/AC/pages/3886120980/Rollout+plan+for+publish-via-config

GH isssue: edx/edx-arch-experiments#381

Testing instructions

We can test by firing all the relevant events on stage and making sure nothing is duplicated or lost.

Tested locally to the best of my ability by trying to fire CMS events with combinations of both the old ENABLE_SEND_XBLOCK_EVENTS_OVER_BUS flag and the new config. I was unable to figure out how to fire the LMS events locally but am reasonably confident they will behave the same.

Deadline

"None" if there's no rush, or provide a specific date or event (and reason) if there is one.

@rgraber rgraber force-pushed the rsgraber/20231004-upgrade-openedx-events-to-9 branch 2 times, most recently from 41335a1 to 405780d Compare October 11, 2023 15:02
@rgraber rgraber changed the title fixup!: extract feat: use new event_bus_producer_config Oct 11, 2023
@rgraber rgraber marked this pull request as ready for review October 11, 2023 16:13
@rgraber rgraber requested a review from a team as a code owner October 11, 2023 16:13
@rgraber rgraber force-pushed the rsgraber/20231004-upgrade-openedx-events-to-9 branch from 50739ee to fd482c6 Compare October 11, 2023 16:33
Copy link
Contributor

@robrap robrap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry - I did not really get a chance to review - but wanted to drop some quick thoughts. Don't let me be a blocker, and feel free to use our typical review request process.

cms/djangoapps/contentstore/signals/handlers.py Outdated Show resolved Hide resolved

Parameters
signal (OpenEdxPublicSignal): The signal being sent to the event bus
topic (string): The topic to which the signal is being sent
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you document which topic names include or exclude the prefix, and ensure that everything is consistent?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[fixed]

Copy link
Contributor

@timmc-edx timmc-edx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some initial questions.

lms/envs/common.py Outdated Show resolved Hide resolved
lms/envs/common.py Outdated Show resolved Hide resolved
lms/envs/common.py Show resolved Hide resolved
lms/envs/common.py Outdated Show resolved Hide resolved
cms/envs/common.py Outdated Show resolved Hide resolved
cms/envs/common.py Show resolved Hide resolved
@rgraber rgraber force-pushed the rsgraber/20231004-upgrade-openedx-events-to-9 branch from 6e8c0c8 to 39deed0 Compare October 12, 2023 17:56
lms/envs/common.py Outdated Show resolved Hide resolved
@rgraber rgraber force-pushed the rsgraber/20231004-upgrade-openedx-events-to-9 branch from 2828379 to 44d7cc9 Compare October 13, 2023 12:52
@rgraber rgraber force-pushed the rsgraber/20231004-upgrade-openedx-events-to-9 branch from 17966e8 to b5c74f8 Compare October 16, 2023 13:19
@rgraber rgraber requested a review from timmc-edx October 16, 2023 13:19
@rgraber rgraber merged commit 80a25bc into master Oct 16, 2023
61 checks passed
@rgraber rgraber deleted the rsgraber/20231004-upgrade-openedx-events-to-9 branch October 16, 2023 18:07
@edx-pipeline-bot
Copy link
Contributor

2U Release Notice: This PR has been deployed to the edX staging environment in preparation for a release to production.

@edx-pipeline-bot
Copy link
Contributor

2U Release Notice: This PR has been deployed to the edX production environment.

1 similar comment
@edx-pipeline-bot
Copy link
Contributor

2U Release Notice: This PR has been deployed to the edX production environment.

@edx-pipeline-bot
Copy link
Contributor

2U Release Notice: This PR has been deployed to the edX staging environment in preparation for a release to production.

@edx-pipeline-bot
Copy link
Contributor

2U Release Notice: This PR has been deployed to the edX production environment.

1 similar comment
@edx-pipeline-bot
Copy link
Contributor

2U Release Notice: This PR has been deployed to the edX production environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants