feat: make extract_translations to use i18n_tool

[shadinaif]

feat: make extract_translations to use i18n_tool

This PR prepares the repository to comply with openedx-translations by doing the following:

• Move local translations from locale to conf/locale directory. This is the default used one in openedx-translations
• Create a symbolic link from locale to conf/locale for backward compatibility
• Use i18_tool extract to extract transactions
• Avoid using i18_tool validate that requires po files metadata to be filled. This is because we'll not be using these metadata anymore when we completely move to OEP-58 ... edx-i18n-tools>=1.3.0 fixes the issue

• Tested on local devstack palm to ensure that everything renders fine
• Tested on local devstack palm to ensure that local translation still works

References

This pull request is part of the FC-0012 project which is sparked by the Translation Infrastructure update OEP-58.

Check the links above for full information about the overall project.

Internalization is being rearchitected in Open edX Python, XBlock, Micro-frontend, and other projects. There are a number of immediately visible changes:

• Remove source and language translations from the repositories, hence no .json, .po or .mo files will be committed into the repos.
• Add standardized make extract_translations in all repositories
• Push user-facing messages strings into openedx/openedx-translations.
• Integrate root repositories with openedx/openedx-atlas to pull translations on build/deploy time

Breaking Changes

One of the primary goals of the project is to avoid breaking changes. If you noticed any suspicious code, please raise your concern. But before that, please know the strategy we're following to avoid breaking changes

For XBlocks:

• The standard translation path must be conf/locale. Therefore, we are creating a link from conf/locale to translations so Atlas can find the path without disrupting the current way of having translations locally within the XBlocks
• openedx-translations will have a related PR that adds the XBlock to the pipeline. This will not affect the current way of managing/updating translations
• At the end of the project, a clear change log will be added and all translations will be handled by Atlas. Thus, the local translation will be removed from the repo within the version bump
• A notification for the community will be published to ensure that everyone knows why translations directories are removed from all repos

[openedx-webhooks]

Thanks for the pull request, @shadinaif! Please note that it may take us up to several weeks or months to complete a review and merge your PR.

Feel free to add as much of the following information to the ticket as you can:

• supporting documentation
• Open edX discussion forum threads
• timeline information ("this must be merged by XX date", and why that is)
• partner information ("this is a course on edx.org")
• any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Please let us know once your PR is ready for our review and all tests are green.

[mphilbrick211]

@e0d would you mind running tests for this one? It's for FC-0012.

[e0d]

Tests run

…

On Thu, Apr 20, 2023, 4:44 PM Michelle Philbrick ***@***.***> wrote: @e0d <https://github.com/e0d> would you mind running tests for this one? It's for FC-0012. — Reply to this email directly, view it on GitHub <#613 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJWEAQBMV3VY4JQOFRVGTDXCGN4HANCNFSM6AAAAAAXFUKOYE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[OmarIthawi]

@shadinaif I was assuming that this repo only needed a symlink (ln -s) for the conf/locale directory.

I'm not sure if Makefile needs to be updated at all.

[shadinaif]

I've reverted Makefile changes @OmarIthawi . But still not convinced, unless I got something wrong regarding the proposal. I thought we must eliminate all use of direct django-admin calls and use i18n_tool instead

Thank you for msgcat suggestion, but it replaces the text too and the error is still being raised

So, I believe now we have four options:

• option1: Keep Makefile untouched because I understood the requirements the wrong way
• option2: Update the current flow to use extract_translations_for_ci until we remove local translations. And add a new extract_translations to be used by openedx-translations
• option3: Fix i18n_tool because we need those headers filled correctly when openedx-translations extracts translations
• option4: Keep this Makefile file untouched and fix all other Makefile to use django-admin instead of i18n_tool because we need the correct headers but we don't want to fix i18n_tool. edit: this option will not fix the header for new files, and will not fix the header after we remove all local translations!

Sorry if I made it looks like a mess, but I'm sure it's not

[OmarIthawi]

we must eliminate all use of direct django-admin calls and use i18n_tool instead

@shadinaif I'm not sure if that's a requirement. Nothing in the OEP-58 document says that. The requirements I'm aware of is to have make extract_translations work reliably in a standard way and paths should be standardized to roughly conf/locale/**/{django,djangojs}.po for python projects and src/i18n/transifex_input.json for MFEs.

option1: Keep Makefile untouched because I understood the requirements the wrong way

I think it's just that. In my opinion, depending on the context, we can use either django-admin directly or i18n_tools when needed. i18n_tools is very useful for the edx-platform repo.

Let's confirm with others first.

[e0d]

@shadinaif this PR is still marked as draft, let us know when it is ready to review.

[shadinaif]

Note: I'll be working on this tomorrow

[shadinaif]

@OmarIthawi @brian-smith-tcril please review. Also, I didn't test it on devstack because I think it'll consume a lot of time. We actually didn't change anything significant in the repo. Or must I?

[shadinaif]

@e0d @hurtstotouchfire this part of FC-0012 as in the description. please review. Also, I didn't test it on devstack because I think it'll consume a lot of time. We actually didn't change anything significant in the repo. Or must I?

[OmarIthawi]

@e0d @hurtstotouchfire this part of FC-0012 as in the description. please review. Also, I didn't test it on devstack because I think it'll consume a lot of time. We actually didn't change anything significant in the repo. Or must I?

As far as I know, all PRs should be tested before merging. It takes time, but testing is part of the FC-0012 project.

[brian-smith-tcril]

@shadinaif the PR description still shows you haven't tested this. Is that accurate and this still needs testing, or is that inaccurate and this is ready for review?

[OmarIthawi]

@hurtstotouchfire we've been asked to inform you about the change to this repository.

Please check this pull request and let us know if you have any questions.

cc: @brian-smith-tcril @shadinaif

[OmarIthawi]

@brian-smith-tcril @hurtstotouchfire Would you mind reviewing this and merging it?

We're aiming to make a backward-compatible pull request.

[OmarIthawi]

Tests run

[OmarIthawi]

@mphilbrick211 @e0d another test run for this FC-0012 pull request please.

[shadinaif]

I've rebased on master, please run tests, and maybe merge @mphilbrick211 @e0d

cc: @OmarIthawi

[OmarIthawi]

Thanks Shadi.

@mphilbrick211 is it possible to enable me and/or other Core Contributors to approve workflow runs?

[OmarIthawi]

@e0d @brian-smith-tcril @hurtstotouchfire please approve the GitHub Actions workflow and kindly review and merge the PR.

It's a been a while since this PR has been opened.

[OmarIthawi]

LGTM. Thanks @shadinaif!

[hurtstotouchfire]

Hi all, I was under the impression that we would just be informed, and that we had no blocking role, so we have not been following closely. @e0d / @brian-smith-tcril Do you want us to handle this? From my perspective you were good to merge this on your own whenever you want.

[brian-smith-tcril]

I've completed my review of this. @hurtstotouchfire I'm pinging you since you're listed as "please inform" on the spreadsheet. I plan on merging this by EOD today. If you have any objections please let me know. Thanks!

[OmarIthawi]

Thanks @hurtstotouchfire and @brian-smith-tcril for triaging this.

It's only been practically ready for merge two weeks ago, before that it's been blocked by other work on i18n-tools. I've got confused and said "it's been open for a while".

Thanks again!!

[brian-smith-tcril]

@shadinaif it looks like this needs another rebase before it can be merged. As soon as it's up to date with the base branch I'll merge it.

[openedx-webhooks]

@shadinaif 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future.