RFC-022: HMR

[bvanjoi]

HMR Design and Implementation

HMR, Hot Module Replacement, is a feature in front-end engineering that can greatly improve the development experience by re-rendering modified code in the browser without re-freshing the page.

This article will go through the process and technical details of HMR.

Basic concepts and processes

First, a brief overview of the HMR data passing process.

                file change
WatchFileSystem -----------> Bundler
                                 | rebuild
                                 | 
                                 | 
                                 \/
Client(browser) <----------- Server(locally)
                 webSocket

For some running front-end project.

1. when the developer has modified some files, the filesystem can listen for changedFiles, and then pass this file information to the build tool;
2. the Bundler re-executes the build process and relies on the changedFiles to get the updated module product.
3. the local development server sends the updated module product to the browser and finally passes it to Runtime to determine if it can be replaced.

WatchFileSystem

In the file listening section, it is necessary to take into account the circumstances under which the build logic will be retriggered, looking step by step at.

• The simplest way to handle this is to listen to the compiler.context directory in its entirety and re-build any files in it when they change.
• A more refined solution would be to store the files involved in the build link in some form, and then control the list of files to listen to with configurations such as ignored, additionFiles, etc., and then rebuild when the files change.

It is also important to consider the types of file changes, which are divided into four categories here.

• meta data: The metadata of the file. For example, file modification times, user group read/write permissions, etc., which are considered to be rebuilt according to the internal design of the build tool.
• modify: Modify the file. In general, modifying a file requires a rebuild, perhaps with the help of some pre-analysis optimisation: for example, modifying a comment section in a file does not affect the build product, so the process can be terminated.
• create: Creates a file. If you create a file that is used in the build process (which can be determined by the output of the last build), you will need to rebuild it.
• remove: Removes the file. If the file you are removing is in the build process, it needs to be built.

Rebuild in Bundler

TODO: increment build or only build changed files?

When you get the changed files, the build tool needs to rebuild them. Usually, to increase the speed of execution, Bundler will update in incremental builds, i.e. it will only build those modules that are affected, eventually getting the current build product and using that to compare with the previous product and get the modules that need to be updated.

In webpack, the compilation.hooks.processAssets stage looks for updateChunkIds, removedChunkIds and removedModules based on the hash value and generates a hot-update. json; also, generate hot-update.js based on the updated module.
After compiler.hooks.done the browser requests hotEmitter to do a check() based on the hash received, which is actually hotCheck in HotModuleReplacement.runtime, in which it sends the updated module code to the browser. sends the updated module code (hot-update.js) and update file (hot-update.json) to the browser, and finally, the runtime in the browser executes hotApply to update the module.

Server

There are two main functions involved in the Server.

• local server: sends product information to the browser client to enable development previews. It does the following.
  • Injecting the appropriate reload function at startup, choosing between hot reload and refresh reload depending on the build result and configuration items.
  • Enables the watch function of the build tool based on configuration items, routing, etc.
• websocket: full-duplex communication between the browser and the local development server to transfer the data required by hmr.

Runtime in Browser

In addition to getting the socket code for the updated file contents, the Runtime section also needs to handle the rebuilt chunk, which is divided into the following steps.

1. find all expired modules upwards based on the updated module.

function getAffectedModules(id) {
  const outDated = {
    dependencies: {},
    modules = []
  }
  const queue = [id];
  while (queue.length) {
    const frontedId = queue.pop();
    const module = MODULE_CACHE[frontedId];
    if (!preCheck(module)) { // such as `selfDeclined`
      return {
        type: 'rejected'
      }
    }
    for (const parentId of module.parents) {
      const parentModule = MODULE_CACHE[parentId];
      if (isAccepted(parentModule.hot, moduleId)) { // defined in project
        add(outDated.dependencies, parentId, moduleId);
      } else {
        delete(outDated.dependencies, parentId)
        add(outData.modules, parentId)
        queue.push(parentId)
      }
    }
  }
  return {
    type: 'accepted',
    outDated
  };
}

2. Extending the above process to all updated modules.

const outDated = {
  dependencies: {},
  modules = []
}
const newModule = {};

for (const moduleId in allChangedModules) {
  const affected = getAffectedModules(moduleId);
  switch (affected.type) {
    case 'affected':
      add(newModule, moduleId, MODULE[moduleId])
      add(outDated, affected.outDated);
      break;
    default:
      fullReload()
  }
}

3. Remove references associated with expired modules.

function dispose(outDate) {
  for (const id of oudDate.modules) {
    const module = MODULE_CACHE[id];
    delete MODULE_CACHE[id];
    delete outDate.dependencies[id];
    for (const childId of module) {
      const child = MODULE_CACHE[childId];
      deleteReference(child.parent, id);
    }
  }

  for (const id in outDate.dependencies) {
    const module = MODULE_CACHE[id];
    for (const childId of module) {
      const child = MODULE_CACHE[childId];
      deleteReference(child.parent, id);
    } 
  }
}
dispose(outDate);

4. Add the updated module.

function apply(newModule, outDate) {
  for (const id of newModule) {
    MODULE_CACHE[id] = newModule[id];
  }
  const callbacks = [];
  for (const module in outDate.dependencies) {
    const children = outDate.dependencies[module];
    for (const child of children) {
      const acceptCallback = getAcceptCallback(module.hot, child);
      if (acceptCallback) {
        callbacks.push(acceptCallback)
      }
    }
  }
  for (const callback of callbacks) {
    callback.call()
  }
}

apply(newModule);

Test

What needs to be tested.

• Corresponding functional tests, such as whether the module meets expectations based on diffs before and after the build; this functionality can be tested using unit tests.
• Verify that the page is refreshed (log or debug location.reload) and that the updated page meets expectations; this function needs to be checked for success using a round robin, and marked as a failure if the round robin is consumed and still does not meet expectations.

Other

• react, vue, css
• import.meta.hot

[hardfist]

get ChangedFile may become a problem if watcher is not reliable(watch-bug), if changedFile is not reliable, then we have to do full rebuild with cache, if so module build cache is crucial to HMR performance.

[hardfist]

How to be compatible with https://github.com/webpack/webpack-dev-middleware is also important here since we need to be compatible with webpack-builder in the future.

[hardfist]

seems missing webpack prior art explanation