[Proposal] OpenSearch Decision Making Process (core and/or top-level)

[nknize]

Is your feature request related to a problem? Please describe

I realize this may be a bit controversial and spark a lot of opinions but an important topic to discuss civilly nonetheless. I imagine this will get moved to another repo. I put it here in the core repo to get the eyes and thoughts from many of the day to day core developers that experience a lot of deadlock on core progress and key decisions that affect many downstream plugins. In no way does this devalue the thoughts and opinions of any downstream "first class" OpenSearch plugin. All are welcome to contribute their thoughts and opinions.

Quick disclaimer: These are my (@nknize) personal thoughts and views on codifying an official decision making process to promote community progress on making technical and non-technical decisions on the project. They in no way reflect the official opinions of the Leadership Committee or any other stakeholder of the project. It is heavily adopted from The ASF Decision Making docs and Xen Project Governance documents as these have been some of the most enjoyable and productive projects I've had the privileged to collaborate with. Enjoy, discuss, converse, or ignore...

---

Decision making is critical to ensure progress of any group lead project. Without a clear well defined decision making process, that is agreed to by existing and adhered to by future members and maintainers of the project, the members will deadlock on decisions and inevitably stall or slow the advancement of the project or, worse, dissolve into chaos and fail without structure.

This document serves to propose an official decision making process for the OpenSearch Project. It defines the categories of decisions, types of decision making models, and selects appropriate models for the different categories to formally define how the decision makers on the project will make decisions in different areas of the project.

Describe the solution you'd like

Categories of Decisions

Because OpenSearch consists of a large bundle of team projects with different purposes, missions, and implementation level details, decision making is split into two fundamental categories: top level project decisions and team level (per-repository) decisions.

Top level project decisions

Top level project decisions are those that would typically fall in the purview of an organizational governing body. In a formal organization (e.g., non-profit, political, charitable, public, private), top level decisions are carried out by a board of directors and well defined in the by-laws which are created during the formation of the organization.

Since OpenSearch is not a formal organization it has not created or codified any formal processes for top level decisions (e.g., bylaws or operating agreements) nor has it defined the types of top level decisions under the purview of the project. As an AWS Vendor Controlled Open Source project, and absent membership to any neutral software foundation (e.g., Apache Software Foundation, Linux Foundation), AWS bears all the risk of the project, thus necessitating OpenSearch to follow the top level legal and security governance dictated by AWS with some freedom to create community lead operational processes on its own. The OpenSearch Leadership Committee has been formed to help answer the “who makes these decisions” question and guide the leadership of the project in the best interest of its community constituents.

The following categories (along with explicitly listed decisions) are identified as top level decision topics that fall under the purview of the formal Leadership Committee.

1. Governance and Leadership / Leadership Committee Nominations -
   1. proposing new or changing existing governance process docs (e.g., maintainer responsibilities, nomination process, decision making process)
      1. defining and assigning leadership roles (e.g., GitHub Admin, per repository leads / chairpersons, slack admins)
      2. adding new and/or changing existing project principles / tenets
   2. creating, graduating, or archiving sub-projects in the organization
2. Bundle releases - major project releases, go/no-go decisions, and versioning decisions
   1. initiating major or patch release voting threads
   2. formal go/no-go decision process
   3. versioning decisions (e.g., SemVer application / enforcement, technical versioning mechanisms)

Team Level (sub project / per repository) decisions

Team level decisions are those that fall in the purview of the individual teams / repositories. These decisions are carried out by the Maintainers of the sub project which are defined in the MAINTAINERS.md file of each repository. While some Implementation details are shared across projects (e.g., bundled versions, core APIs, SDKs, developer operations and tooling) there are many team level decisions that need to be made during the day to day developer operations (devops).

The following three categories of decisions are identified as those made on a daily basis and may vary team to team:

1. Procedural
   1. Defining additional repo-specific labels
   2. review policies (e.g., review then commit vs lazy consensus)
   3. backlog or roadmap grooming
   4. overall repository hygiene processes (e.g., when to fail or block PRs)
2. Releases and Maintainer Nominations
   1. voting on major, minor, bug releases outside the bundle
   2. individual release processes and implementation details outside the top level project process
   3. new maintainer nominations for the specific project
3. Code, Documentation, project build script modifications
   1. specific changes to the source code.
   2. specific changes to individual project docs (e.g., javadocs, markdowns)
   3. specific changes to build scripts and/or automation

Definitions

Prior to defining the decision making model proposed herin, the following terms are defined to ensure no ambiguity in their usage (leveraged from The ASF, Xen Project, and other general governing / legal practices):

1. Binding - comments, votes, justifications, or input made by active (non-emeritus) members of the project.
2. Community - a group of individuals working toward a common objective.
3. Consensus Approval - This is a decision making model that has at least three binding +1 votes and 0 vetos.
4. Contributor - An individual that makes any improvement, proposal, suggestion, or offering to the project with or without commit privileges. This includes, but is not limited to, source code, documentation, bug reports, proposals, use case definitions, or requests for changes.
5. Emeritus - An individual that is no longer active in the project but entitled to the rights and privileges of the position. This status indicates an interest in the project but has no intent in participating in the project. Emeritus members have no voting rights until their maintainer status is reinstated.
6. Impasse - a proposal or decision in which no progress is possible.
7. Lazy Consensus - Adopted from The Apache Software Foundation this is defined as “a decision-making policy which assumes general consent if no responses are posted within a defined period.”
8. Maintainer - a project member that has the privilege to make direct changes to any file in the project repository(ies).
9. Majority Approval - A decision making model that relies on more binding +1 votes than -1 votes.
10. Member - Any individual that has interest and elevated privileges on the project.
11. Quorum - A quorum is the minimum number of votes required before a proposal or decision can be declared as passed, failed, or impasse.
12. Quorum Majority Approval - A decision making model that requires a majority approval of quorum votes.
13. Review then commit - Also known as “RTC”, this is a policy that requires all changes receive a review and formal approval before being committed to the organization. RTC is a modified form of Lazy Consensus where at least one binding +1 votes are required and 0 vetoes.
14. Veto / Valid Veto - The process of blocking a proposed change to the project. Each member shall be entitled to one veto per proposed change. All vetoes must include a valid technical justification and any veto without a justification is considered invalid and non-binding. Vetoes may not be overridden or voted down and only cease to exist when the vetoing member withdraws their veto. In some instances vetoes may only be considered binding with an alternative proposal.
15. Vote - The expression of a positive or negative opinion on a topic in a formal decision making process. Each member (except emeritus members) shall be entitled to one vote.

Decision Making Models

There are many different types of Group Decision Making Methodologies and Models. Performing a comprehensive trade study with literature review would be quite extensive, elaborate, and overly burdensome, thus is left outside the scope of this proposal. Instead this proposal takes a “progress not perfection” approach and recommends following some of the best practices identified across existing software foundations, along with initial decision making processes already established in existing OpenSearch policies (e.g., Maintainer Nomination).

The following three types of decision making models proposed for OpenSearch are defined above and provided below. Their definitions are restated for completeness with some further clarification as applied to the project. Following their definitions are the proposal’s recommendation on which model should be applied to each top-level and team-level category.

1. Quorum Majority Approval - A decision making model that requires a majority approval of quorum votes. For OpenSearch Project a Quorum shall be met with no less that one-third the total number of maintainers/members.
2. Consensus Approval - This is a decision making model that has at least three binding +1 votes and 0 valid vetoes. At least one valid veto fails the vote. Less than three +1 votes results in an impasse until three +1 votes are received. Not all members are required to cast a vote.
3. Lazy Consensus - Adopted from The Apache Software Foundation, and used on many other foundations (e.g., Xen Project), “a decision-making policy which assumes general consent if no responses are posted within a defined period.” A modified form of Lazy consensus is Review then Commit which requires at least one +1 binding vote and 0 vetoes.
   1. Lazy consensus requires no +1 binding votes, thus consensus can be reached through silence. This is often used to prevent endlessly stalling changes and deadlocking progress. Under lazy consensus it is common policy to require a notice be posted prior to committing any change without explicit approval. The notice shall include a time window informing members when a change is intended to be committed. An example is provided below:

"This change addresses issue #XYZABC; I have not received any explicit approval or feed back in X days. If there are no objections, I am going to assume lazy consensus and merge the change in three days."

Proposed Decision Making Process

Again following the “progress not perfection” and “bias for action” tenets, the proposal recommends adopting from the best practices outlined by several existing foundations. As such, the following proposes the decision making models be used:

Top Level Project

• Governance Establishment and Changes - Quorum Majority Approval
• Bundle Releases - Consensus Approval

Team Level Per-Repository

• Procedural - Quorum Majority Approval
• Code Modifications - Lazy Consensus
• Releases - Consensus Approval

Related component

Other

Describe alternatives you've considered

Continue as-is w/o any decision making process - continues to lead to stalled PRs, issues, etc.
Adopt a pre-defined foundation process

Additional context

No response

[dblock]

Does this belong in https://github.com/opensearch-project/.github?

[reta]

@nknize Thanks for finding the time to put this writing up, do you see any particular model appealing to you more than others? I personally have pretty good experience with ASF governance (PMCs) and I think it works pretty well, however that would require some changes to the governance model we have now.

[nknize]

@nknize Thanks for finding the time to put this writing up, do you see any particular model appealing to you more than others? I personally have pretty good experience with ASF governance (PMCs) and I think it works pretty well, however that would require some changes to the governance model we have now.

I realized I hadn't included the important part. I updated the text to include my proposed models in each of the decision categories and topics. The proposal doesn't require any changes to the LC model we have today. It only defines the process / criteria needed to make a decision.

[peternied]

[Triage - attendees 1 2 3]
@nknize Thanks for filing, moving this to .github repository