address PR feedback

DEV_FAQ.md

@@ -41,7 +41,7 @@ It happens, try restarting.
 > I'm getting TemplateSyntaxErrors related to Webpack.
 
 Please run `yarn dev`. If this build is failing, and you didn't make any changes to JavaScript files on your branch,
-please raise this as an issue on `#gtd-web-team`.
+please raise this as an issue.
 
 ## Generating Sample Data
 

docs/js-guide/amd-to-esm.rst

@@ -10,14 +10,15 @@ See the `Historical Background on Module Patterns
 <https://github.com/dimagi/commcare-hq/blob/master/docs/js-guide/module-history.rst>`__
 for a more detailed discussion of module types. As a quick refresher, here are some definitions:
 
-AMD (Asynchronous Module Definition)
-    The legacy module type used for older JavaScript modules on HQ.
-    This was the only module type compatible with RequireJS, our first JavaScript bundler.
-    It is still needed as a format for modules required by No-Bundler pages.
+Modified AMD (Asynchronous Module Definition)
+    The legacy module type used for older JavaScript modules on HQ, identified by having an ``hqDefine``
+    statement near the top of the file. AMD was the only module type compatible with RequireJS,
+    our first JavaScript bundler. It is still needed as a format for modules required by No-Bundler pages.
 
 ESM (ES Modules)
     The newest module type with updated powerful import and export syntax. This is the module
     format that you will see referenced by documentation in modern javascript frameworks.
+    This is quickly identified by the ``import`` statements at the top used for including dependencies.
 
 The different types of modules you will encounter are:
 
@@ -28,7 +29,7 @@ Entry Point Modules
     page-specific code is needed to render that page / entry point.
 
 Dependency Modules
-    These are modules that are never referenced in a bundler template tag and are only
+    These are modules that are never referenced by ``webpack_main`` and are only
     in the list of dependencies for other modules. Often these modules are used as utility modules
     or a way to organize JavaScript for a page that is very front-end heavy.
 
@@ -79,9 +80,13 @@ Instead, please first
 Dependency Modules
 ~~~~~~~~~~~~~~~~~~
 
-If this module is a dependency of any modules that are ``requirejs_main`` entry points, then this module is not
-eligible for migration. If a module's syntax is updated when it's still required by RequireJS modules, then
-it will result in a RequireJS build failure on deploy.
+If this module is a dependency of any modules that are ``requirejs_main`` entry points,
+then this module is not eligible for migration. If a module's syntax is updated when it's still
+required by RequireJS modules, then it will result in a RequireJS build failure on deploy.
+
+You can check the status of a dependency module's RequireJS usage by looking at the
+`Bootstrap 3 <https://www.commcarehq.org/static/build.b3.txt>`__ and
+`Bootstrap 5 <https://www.commcarehq.org/static/build.b5.txt>`__ module list.
 
 If this module is referenced by any ``hqImport`` calls (for instance ``hqImport('hqwebapp/js/my_module')``),
 then this module is NOT yet eligible, and must continue using the older AMD-style syntax until
@@ -97,13 +102,24 @@ If the AMD-style module looks a bit different than the syntax above--for instanc
 `migrated to use a JS Bundler <https://github.com/dimagi/commcare-hq/blob/master/docs/js-guide/migrating.rst>`__.
 
 
-Step 2: Update Surrounding Module Structure
--------------------------------------------
+Step 2: Update the Module Syntax
+--------------------------------
+
+Key Points
+~~~~~~~~~~
 
-ESM no longer needs to define the module name within the module itself. Instead, Webpack (our bundler) is configured
-to know how to reference this module by its filename and relative path within an application.
+-   ESM no longer needs to define the module name within the module itself. Instead, Webpack (our bundler) is configured
+    to know how to reference this module by its filename and relative path within an application.
+-   By default, you can use the same dependency names with the ``import`` syntax. If the ``import`` statement results
+    in a Webpack Build error, look at ``webpack.common.js`` because it might require an alias. If you still have
+    a problem, check ``requirejs_config.js``, because there might have been an alias defined there that hasn't
+    been added to ``webpack.common.js``.
 
-You can start this by changing:
+
+Example Structural Change
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is a rough example of what the changes will look like:
 
 ::
 
@@ -127,36 +143,6 @@ You can start this by changing:
 
 to
 
-::
-
-    [
-        'jquery',
-        'knockout',
-        'underscore',
-        'hqwebapp/js/initial_page_data',
-        'hqwebapp/js/assert_properties',
-        'hqwebapp/js/bootstrap5/knockout_bindings.ko',
-        'commcarehq',
-    ] (
-        $,
-        ko,
-        _,
-        initialPageData,
-        assertProperties
-    )
-    // additionally, the indentation for the module-specific code can be updated
-    ...
-
-Step 3: Update Dependency Imports
----------------------------------
-
-Common ``yarn`` dependencies can now be referenced by their NPM name (or the alias defined in
-``webpack/webpack.common.js``.
-
-The same can be done with named internal dependencies, or referenced internal dependencies.
-
-The final module dependency structure will look something like:
-
 ::
 
     import "commcarehq";  // Note: moved to top
@@ -170,7 +156,7 @@ The final module dependency structure will look something like:
     import initialPageData from "hqwebapp/js/initial_page_data";
     import assertProperties from "hqwebapp/js/assert_properties";
 
-    // referenced internal dependencies:
+    // unnamed internal dependencies:
     import "hqwebapp/js/bootstrap3/knockout_bindings.ko";
 
     // module specific code...
@@ -179,7 +165,9 @@ The final module dependency structure will look something like:
 Note that ``import "commcarehq";`` has been moved to the top of the file. The ordering is
 for consistency purposes, but it's important that either ``import "commcarehq";`` or
 ``import "commcarehq_b3";`` (for Bootstrap 3 / ``webpack_main_b3``) is present in the list
-of imports for Webpack Entry Point modules.
+of imports for Webpack Entry Point modules. If this import is not present in an entry point,
+then site-wide navigation, notifications, modals, and other global widgets will not
+work on that page.
 
 Remember, an Entry Point is any module that is included directly on a page using the
 ``webpack_main`` or ``webpack_main_b3`` template tags.

docs/js-guide/code-organization.rst

@@ -25,6 +25,3 @@ which we structure as follows:
 
 To develop with javascript locally, make sure you run ``yarn dev`` and
 restart ``yarn dev`` whenever you add a new Webpack Entry Point.
-
-Please review the next section for a more detailed discussion of Static Files
-and the use of JavaScript bundlers, like Webpack.

docs/js-guide/dependencies.rst

@@ -5,12 +5,20 @@ Front-end dependencies are managed using ``yarn`` and are defined in ``package.j
 root of the ``commcare-hq`` repository.
 
 Most JavaScript on HQ is included on a page via a JavaScript bundle.
-These bundles are created by a JavaScript bundler. The bundler is given a
-list of "entry points" (or pages), builds a dependency graph of modules to determine what
-code is needed for a page, and combines related code into bundles.
+These bundles are created by Webpack. Webpack is given a list of "entry points"
+(or pages) and builds a dependency graph of modules to determine what
+code is needed for a page, combining related code into bundles.
 These bundles are split along ``vendor`` (npm modules),
 ``common`` (all of hq), and application (like ``hqwebapp`` or ``domain``).
 
+For legacy code, RequireJS creates bundles around Django applications.
+These bundles are created during deploy with the ``build_requirejs`` management
+command, and are typically not created during development. Here
+are the production bundles split along Bootstrap versions for RequireJS:
+
+- `Bootstrap 3 <https://www.commcarehq.org/static/build.b3.txt>`__
+- `Bootstrap 5 <https://www.commcarehq.org/static/build.b5.txt>`__
+
 By bundling code, we can make fewer round-trip requests to fetch all of a page's JavaScript.
 Additionally, the bundler minifies each bundle to reduce its overall size. You can learn
 more about bundlers in `the Static Files Overview
@@ -104,8 +112,9 @@ Then in your HTML page:
    {% webpack_main 'prototype/js/combined_example' %}
 
 The exception to the above is if your page inherits from a legacy page that
-doesn't use a JavaScript bundler. This is rare, but one example would be adding a
-new page to app manager that inherits from ``managed_app.html``.
+doesn't use a JavaScript bundler, like reports and app manager. This is rare,
+but one example would be adding a new page to app manager that inherits
+from ``managed_app.html``.
 
 
 Why is old code formatted differently?
@@ -120,7 +129,7 @@ once these entry points `are migrated to Webpack
 
 However, be careful when migrating modified AMD modules that aren't entry points, as some of these modules,
 like ``hqwebapp/js/initial_page_data``, are still being referenced by pages not using a JavaScript bundler.
-These pages still require this modified AMD approach until they transition to using a bundler.
+These pages still require this modified AMD approach until they transition to using Webpack.
 
 We will cover what common modified AMD modules look like in this section, but you can read more
 about this choice of module format in the `Historical Background on Module Patterns
@@ -167,12 +176,11 @@ instead relying on globals like ``ko`` (for Knockout.js) in the example below.
    });
 
 
-How do I know whether or not I’m working with Webpack or RequireJS?
--------------------------------------------------------------------
+How do I know whether I’m working with Webpack or RequireJS?
+------------------------------------------------------------
 
 You are likely working with either Webpack or RequireJS, as most of HQ has been migrated to use a bundler.
-However, several major areas have **not** been migrated: app manager,
-reports, and web apps.
+However, two major areas have **not** been migrated: app manager and reports.
 
 The easiest way to determine if a page is using either Webpack or RequireJS is to
 open the JavaScript console on that page and type ``window.USE_WEBPACK``, which will return
@@ -198,7 +206,8 @@ ESM can quickly be identified by scanning the file for ``import`` statements lik
 How do I add a new internal module or external dependency to an existing page?
 ------------------------------------------------------------------------------
 
-Webpack supports multiple module formats, with ES Modules (ESM) being the preferred format. New modules should be written in the ESM format.
+Webpack supports multiple module formats, with ES Modules (ESM) being the preferred format.
+New modules should be written in the ESM format.
 
 That being said, a lot of legacy code on HQ is written in a modified AMD format.
 If you are adding a lot of new code to such a module, it is recommended that you
@@ -238,7 +247,7 @@ Modified AMD (previously "RequireJS")
     RequireJS is being replaced by Webpack. You should NOT create NEW modules with this style.
 
 To use your new module/dependency, add it your module’s ``hqDefine`` list of dependencies.
-If the new dependency will be directly referenced in the body of the odule, also add a
+If the new dependency will be directly referenced in the body of the module, also add a
 parameter to the ``hqDefine`` callback:
 
 ::

docs/js-guide/requirejs-to-webpack.rst

@@ -5,30 +5,12 @@ RequireJS `reached its end of life <https://github.com/requirejs/requirejs/issue
 in September of 2020. Since then, it has been increasingly difficult to work with as more modern libraries
 no longer provide AMD modules (the preferred module format of RequireJS), and use modern JavaScript
 syntax that is no longer compatible with RequireJS's build process. We decided to use Webpack as our
-replacement for RequireJS in September of 2024. We expect the migration to be brief and straightforward
-with the following challenges to highlight:
-
-1. The use of ``stripe`` will need to be updated to use the module from ``npm``, and we will need update
-    our front-end to use Stripe's credit card widgets library.
-2. Shimmed dependencies in requirejs that do no yet have an analogous ``exports-loader`` statement in
-    ``webpack.common.js`` will need a statement added and usage tested. This should be straightforward, but
-    there might be a couple challenging shims in this process.
-3. Tests will also need to be moved to using Webpack instead of RequireJS. The difference with webpack is that
-    the ``commcarehq`` or ``commcarehq_b3`` common module is always imported in the entry point. In RequireJS
-    this main module was included in ``hqwebapp/partials/requirejs.html``. However, Webpack can't reliably build
-    bundles with this global bundle separated from the entry point. This shouldn't cause issues for tests, but it might.
+replacement for RequireJS in September of 2024. We expect the migration to be brief and straightforward.
 
 
 Overview of the Process
 -----------------------
 
-Migrations should happen in pull-request "chunks", meaning that pull requests should contain migrations for
-all (or most) entry points within a single application or a set of closely related applications. Additionally,
-commits should be made separately for each entry point migration.
-
-If new ``exports-loader`` statements are added, it is recommended to test the changes on staging to ensure
-the functionality is maintained between production and staging.
-
 The following steps outline the migration process on a per entry point basis.
 
 As a reminder, Entry Points are modules that are included directly on a page using a bundler template tag,
@@ -42,61 +24,21 @@ that it knows what bundle of javascript dependencies and page-specific code is n
 See `the Static Files Overview <https://github.com/dimagi/commcare-hq/blob/master/docs/js-guide/static-files.rst>`__
 for a more detailed explanation.
 
-Step 1: Identify the Entry Point and Details
---------------------------------------------
-
-First, identify the entry point you would like to migrate. Additionally, we need to determine if the entry
-point is split along the Bootstrap 3 or Bootstrap 5 RequireJS build.
-
-Bootstrap 5 Entry Points
-~~~~~~~~~~~~~~~~~~~~~~~~
-
-As an example, let's migrate ``case_importer/js/main``.
-
-We should find a ``requirejs_main_b5`` template tag referencing this entry point inside a template.
+.. note::
 
-The following template tag is present in ``case_importer/excel_config.html``:
-
-::
-
-    {% requirejs_main_b5 "case_importer/js/main" %}
-
-Since the ``requirejs_main_b5`` template tag is being used, we know this entry point is part of the Bootstrap 5
-build. We also do not need to be concerned that this file might be undergoing a Bootstrap 5 migration.
-
-We can now proceed to Step 2.
-
-Bootstrap 3 Entry Points
-~~~~~~~~~~~~~~~~~~~~~~~~
+    If new ``exports-loader`` statements are added, it is recommended to test the changes on staging to ensure
+    the functionality is maintained between production and staging.
 
-As an example, let's migrate ``domain/js/my_project_settings``.
 
-We should find a ``requirejs_main`` template tag referencing this entry point inside a template.
-
-The following template tag is preset in ``domain/admin/my_project_settings.html``:
-
-::
-
-    {% requirejs_main "domain/js/my_project_settings" %}
-
-Since the ``requirejs_main`` template tag is being used, we know this entry point has not been migrated
-from Bootstrap 3 to 5. Since we are currently undergoing a migration from Bootstrap 3 to 5, it is important
-to establish that this page is not actively undergoing a migration.
-
-Please see the `Bootstrap Migration Status List
-<https://docs.google.com/spreadsheets/d/1tkSXR643Da-fp6a-uYPa5dYs5if4W2LqtvUJs3IfUKs/edit?gid=0#gid=0>`__
-to see if the application housing that entry point is undergoing a migration. If unsure, please
-raise a question to ``#gtd-dev``.
-
-If you are able to determine that this entry point is NOT actively undergoing a Bootstrap 3 to 5 migration,
-then please proceed to Step 2.
-
-
-Step 2: Update the Template Tag and Add Global ``commcarehq`` Dependency
+Step 1: Update the Template Tag and Add Global ``commcarehq`` Dependency
 ------------------------------------------------------------------------
 
+First, find either the ``requirejs_main`` tag (Bootstrap 3 pages) or the ``requirejs_main_b5`` tag
+(Bootstrap 5 pages) that references the entry point you want to migrate.
+
 The migration of an entry point from RequireJS to Webpack will involve updating the template tag
 used to define the entry point and then adding the ``commcarehq`` global dependency to the list of dependencies.
+Two examples are below.
 
 Bootstrap 5 Entry Points
 ~~~~~~~~~~~~~~~~~~~~~~~~
@@ -157,7 +99,7 @@ Then, in the file itself, we add the ``commcarehq_b3`` dependency to the list of
         ...
 
 
-Step 3: Verify Webpack Build
+Step 2: Verify Webpack Build
 ----------------------------
 
 The next step is to ensure that the Webpack build succeeds with the newly-added
@@ -181,15 +123,10 @@ please see the troubleshooting guide below. If there is no help there, please re
 the lead developers in charge of this migration for assistance. Please add troubleshooting
 guidance afterward.
 
-Once the build succeeds, please commit all tho changes for that entry point, with
-a commit message that might look like the following
-
-::
-
-    migrated case_importer/js/main to webpack
+Once the build succeeds, please commit all the changes for that entry point.
 
 
-Step 4: Verify Page Loads Without JavaScript Errors
+Step 3: Verify Page Loads Without JavaScript Errors
 ---------------------------------------------------
 
 The final step is to ensure that the page with the Entry Point loads without
@@ -206,9 +143,3 @@ existing patterns of ``exports-loader`` statements for dependencies that were sh
 in ``requirejs_config`` similarly to the dependency you are having issues with now.
 
 Please add any additional guidance here as the migration continues.
-
-
-Troubleshooting Guide
----------------------
-
-TBD -- Please add any additional guidance here as the migration continues.

docs/js-guide/static-files.rst

@@ -20,9 +20,6 @@ to using Webpack as part of the `JS Bundler Migration
 Image files generally stay as-is. The only "dynamic" images
 come from file attachments in our database.
 
-As of this writing, we don't compile our JavaScript from a higher level scripting
-language, although Webpack offers ways to do this easily.
-
 Due to their static natures, the primary objective when working with static files is
 to make as few requests as possible to them during page load. We aim to combine
 static files into one larger, minified file whenever possible.
@@ -34,7 +31,8 @@ Why use a javascript bundler?
 -----------------------------
 
 We use a bundler for our javascript files to reduce the amount of separate
-javascript ``script`` tags needed for any page load. A bundler, like Webpack, not only combines the necessary javascript, it minifies the javascript to reduce
+network requests tags needed for any page load. A bundler, like Webpack,
+not only combines the necessary javascript, it minifies the javascript to reduce
 its overall size. Additionally, Webpack employs code splitting to split commonly referenced
 "chunks" of code into separate collective bundles:
 
@@ -83,7 +81,8 @@ Developing on No-Bundle pages
 
 There are still some very old sections of the codebase that are not under the jurisdiction of a JavaScript bundler.
 These pages can be developed without needing ``yarn dev`` to run in the background. However, you should pay special
-attention to your ``localsettings`` setup for Django Compressor, which is explained in the Compression
+attention to your ``localsettings`` setup for Django Compressor, which is explained in the `Compression
+<#compression>`__
 section below.
 
 Developing with RequireJS page