Force waiting for automated merge

README.md

@@ -36,10 +36,10 @@ are satisfied:
   text displayed by GitHub for these so called "mergeable" PRs varies
   depending on whether the changes are approved and whether the PR
   branch is out of date with its target branch.
-* All the _required_ checks have succeeded on the PR branch:
-  * If _all_ checks have succeeded, then GitHub says "All checks have
-    passed" next to a green check mark:
-    ![](./docs/images/all_passed.png)
+* All the _required_ checks have succeeded on the PR branch.
+  * _all_ checks, except for the special-purpose "automated merge status"
+    test (see below) have succeeded:
+    ![](./docs/images/automated_status_pending.png)
   * If any _optional_ checks have failed, then GitHub will show "Some
     checks were not successful" message:
     ![](./docs/images/required_passed.png)
@@ -90,6 +90,12 @@ algorithm:
 If the bot is killed while executing the above algorithm, it will resume
 from the beginning of the unfinished step.
 
+The above algorithm can be divided into three merging phases:
+
+1. pre-staging: step 1
+2. staging: step 2 and 3
+3. merged: step 4 and 5
+
 While the bot is merging a pull request, it does not touch other PRs. If
 the bot receives a GitHub event while merging a pull request, then the
 bot re-examines all PRs _after_ it is done merging the current PR. This
@@ -112,6 +118,73 @@ If a PR processing step fails for PR-specific reasons (e.g., a CI test
 failure), then the bot moves on to the next pull request, labeling the
 failed PR if/as needed (see below for PR labels).
 
+## Automated merge status
+
+The bot adds this required status automatically to PR and staging commit.
+During PR lifecycle the status is "pending" or "failure", thus preventing
+the manual merge button on the GitHub PR page from becoming green. Anubis
+uses this mechanism to forbid manual merges, which result in wrong commit
+messages and missed staging checks. The bot satisfies this check just before
+merging (for the staging commit) and just after merging (for PR).
+
+Depending on the current PR merging phase, an encountered problem or event
+(such as approval), the automated status is assigned a status (failure, pending, or
+success) and is supplied with a description, consisting of the problem-specific
+PR label and message, detailing it.
+
+Two tables below outlines possible states and messages for both commits.
+
+PR HEAD commit:
+
+*state* | *label* | *message* | *phase* |
+--- | --- | --- | ---
+failure | - | GitHub will not be able to merge | pre-staging
+failure | - | failed PR tests | pre-staging
+failure | M-failed-staging-other | an unexpected error during staging some time ago | pre-staging
+failure | M-failed-staging-checks | staged commit tests failed some time ago (3) | pre-staging
+failure | M-failed-description | invalid commit message | pre-staging
+pending | - | waiting on WIP | pre-staging
+pending | - | waiting for approval | pre-staging
+pending | - | waiting for objections | pre-staging
+pending | - | waiting for PR tests | pre-staging
+pending | - | waiting for another staged PR | pre-staging
+pending | - | staged at SHA | staging
+success | M-merged | - | merged
+
+Staging commit:
+
+*state* | *label* | *message* | *phase* |
+--- | --- | --- | ---
+failure | - | failed PR tests | staging
+failure | M-failed-staging-other | an unexpected error during staging | staging
+failure | M-failed-staging-checks | staged commit tests have failed | staging
+failure | M-failed-description | invalid commit message | staging
+failure | - | staged commit tests are stale (1) | staging
+pending | - | waiting on WIP | staging
+pending | - | waiting for approval | staging
+pending | - | waiting for objections | staging
+pending | - | waiting for PR tests | staging
+pending | - | waiting for the dry-run mode to end | staging
+pending | - | waiting for staging-only mode to end | staging
+pending | - | waiting for staging tests (...) (2)| staging
+success | - | will be merged as SHA | staging
+success | M-merged | - | merged
+
+(1): Staged commit tests become stale when either the PR state changes (e.g.,
+the author commits new code) or the staging_branch is unexpectedly modified.
+The status lets the developer know why Anubis ignored (failed, unfinished, or
+successful) staging tests.
+
+(2): The (...) in the table above denotes specific information about the number
+of tests succeeded/pending/failed so far, e.g.:
+![](./docs/images/automated_status_details.png)
+
+(3): This state means that the bot will ignore this PR (even if the unterlying
+problem has been fixed) until the M-ffailed-staging-checks is manually removed.
+Please also see the description of M-failed-staging-checks below.
+
+In some cases (usually errors), the message is prefixed with a label, e.g.:
+![](./docs/images/automated_status_problem.png)
 
 ## Pull request labels
 
@@ -140,14 +213,6 @@ request state:
   mode (see `config::staged_run` and `config::guarded_run`). The bot
   removes this label when either the PR was successfully merged or its
   staging results are no longer fresh/applicable.
-* `M-abandoned-staging-checks`: Anubis discovered a stale _staging commit_.
-  This usually happens when either the PR state changes (e.g., the author
-  commits new code) or the staging_branch is unexpectedly modified. After
-  labeling the PR, the bot ignores any failed or unfinished tests associated
-  with that stale commit. This label was added so that the developer observing
-  the PR page on GitHub knows why Anubis ignored (failed, unfinished, or
-  successful) staging tests. The bot removes this (usually short-lived) label
-  after creating a fresh staging commit for the PR.
 * `M-failed-staging-checks`: Essentially duplicates GitHub "red x" mark
   for the _staging commit_. The bot does not attempt to merge this PR
   again until a human decides that this problem is resolved and removes
@@ -296,8 +361,9 @@ All configuration fields are required.
 *sufficient_approvals* | The minimal number of core developers required for a PR to be merged fast (i.e., without waiting for `config::voting_delay_max`) | 2
 *voting_delay_min*| The minimum merging age of a PR. Younger PRs are not merged, regardless of the number of votes. The PR age string should comply with [timestring](https://github.com/mike182uk/timestring) parser. | "2d"
 *voting_delay_max* | The maximum merging age of a PR that has fewer than `config::sufficient_approvals` votes. The PR age string should comply with [timestring](https://github.com/mike182uk/timestring) parser. | "10d"
-*staging_checks*| The expected number of CI tests executed against the staging branch. | 2
+*staging_checks*| The expected number of checks executed against the staging branch. This includes CI checks and bot-generated checks. | 2
 *approval_url*| The URL associated with an approval status test description. | ""
+*automated_merge_url*| The URL associated with an automated merge status test description. | ""
 *logger_params* | A JSON-formatted parameter list for the [Bunyan](https://github.com/trentm/node-bunyan) logging library [constructor](https://github.com/trentm/node-bunyan#constructor-api). | <pre>{<br>    "name": "anubis",<br>    "streams": [ ... ]<br>}</pre>
 
 TODO: Merge all three "mutually exclusive" boolean `*_run` options into one `run_mode` option accepting on of four mode names, including "production". Document individual string values in a separate table (here).

src/Config.js

@@ -32,6 +32,7 @@ class ConfigOptions {
         this._stagingChecks = conf.staging_checks;
         this._loggerParams = conf.logger_params;
         this._approvalUrl = conf.approval_url;
+        this._automatedMergeUrl = conf.automated_merge_url;
 
         // unused
         this._githubUserNoreplyEmail = null;
@@ -81,6 +82,12 @@ class ConfigOptions {
     votingDelayMax() { return this._votingDelayMax; }
     votingDelayMin() { return this._votingDelayMin; }
     stagingChecks() { return this._stagingChecks; }
+    // the number of staged commit checks supplied by CIs
+    stagingChecksCI() { return this._stagingChecks - this.derivativeRequiredChecks() - this.copiedRequiredChecks(); }
+    // the number of 'internal' checks
+    derivativeRequiredChecks() { return this.manageApprovalStatus() + this.manageAutomatedMergeStatus(); }
+    // the number of checks copied from PR commit to staged commit
+    copiedRequiredChecks() { return 1; }
     loggerParams() { return this._loggerParams; }
 
     // an unexpected error occurred outside the "staged" phase
@@ -99,8 +106,6 @@ class ConfigOptions {
     failedDescriptionLabel() { return "M-failed-description"; }
     // allows target branch update in 'guarded_run' mode
     clearedForMergeLabel() { return "M-cleared-for-merge"; }
-    // whether the PR was abandoned due to a stale staged commit
-    abandonedStagingChecksLabel() { return "M-abandoned-staging-checks"; }
 
     // an URL of the description of the approval test status
     approvalUrl() { return this._approvalUrl; }
@@ -112,6 +117,15 @@ class ConfigOptions {
     approvalContext() { return "PR approval"; }
 
     copiedDescriptionSuffix() { return " (copied from PR by Anubis)"; }
+
+    // an URL of the description of the automated merge test status
+    automatedMergeStatusUrl() { return this._automatedMergeUrl; }
+
+    // the 'context name' of the automated merge test status
+    automatedMergeStatusContext() { return "Automated merge"; }
+
+    // whether the bot will create the automated merge test statuses for PR and staged commit
+    manageAutomatedMergeStatus() { return this.automatedMergeStatusUrl().length > 0; }
 }
 
 const configFile = process.argv.length > 2 ? process.argv[2] : './config.json';