Merge concepts and design into single document

docs/architecture.md

@@ -7,7 +7,8 @@ functions as commands. In the future, an API server may be built from
 the library as well. Arlon adds CRDs (custom resource definitions) for several
 custom resources such as ClusterRegistration and Profile.
 
-## Management cluster
+## Components 
+### Management cluster
 
 The management cluster is a Kubernetes cluster hosting all the components
 needed by Arlon, including:
@@ -22,7 +23,7 @@ needed by Arlon, including:
 The user is responsible for supplying the management cluster, and to have
 access to a kubeconfig granting administrator permissions on the cluster.
 
-## Controller
+### Controller
 
 The Arlon controller observes and responds to changes in `clusterregistration`
 custom resources. The Arlon library creates a `clusterregistration` at the
@@ -31,15 +32,15 @@ causing the controller to wait for the cluster's kubeconfig
 to become available, at which point it registers the cluster with ArgoCD to
 enable manifests described by bundles to be deployed to the cluster.
 
-## Library
+### Library
 
 The Arlon library is a Go module that contains the functions that communicate
 with the Management Cluster to manipulate the Arlon state (bundles, profiles, clusterspecs)
 and transforms them into git directory structures to drive ArgoCD's gitops engine. Initially, the
 library is exposed via a CLI utility. In the future, it may also be embodied
 into a server an exposed via a network API.
 
-## Workspace repository
+### Workspace Repository
 
 As mentioned earlier, Arlon creates and maintains directory structures in a git
 repository to drive ArgoCD *sync* operations.
@@ -51,7 +52,7 @@ register the workspace registry in ArgoCD before referencing it from Arlon data
 Starting from release v0.9.0, Arlon now includes two commands to help with managing various git repository URLs. With these commands in place, the `--repo-url` flag in commands requiring a hosted git repository is no longer needed.
 A more detailed explanation is given in the next [section](#repo-aliases).
 
-### Repo Aliases
+#### Repo Aliases
 
 A repo(repository) alias allows an Arlon user to register a GitHub repository with ArgoCD and store a local configuration file on their system that can be referenced by the CLI to then determine a repository URL and fetch its credentials when needed.
 All commands that require a repository, support a `--repo-url` flag also support a `repo-alias` flag to specify an alias instead of an alias, such commands will consider the "default" alias to be used when no `--repo-alias` and no `--repo-url` flags are given.
@@ -83,11 +84,11 @@ The structure of the file is as shown:
 
 On running `arlon git unregister ALIAS`, it removes that entry from the configuration file. However, it does NOT remove the repository from `argocd`. When the "default" alias is deleted, we also clear the "default" entry from the JSON file.
 
-#### Examples
+##### Examples
 
 Given below are some examples for registering and unregistering a repository.
 
-##### Registering Repositories
+###### Registering Repositories
 
 Registering a repository requires the repository link, the GitHub username(`--user`), and a personal access token(`--password`).
 When the `--password` flag isn't provided at the command line, the CLI will prompt for a password(this is the recommended approach).
@@ -105,7 +106,7 @@ arlon git register https://github.com/GhUser/manifests --user GhUser --password
 arlon git register https://github.com/GhUser/prod-manifests --user GhUser --alias prod --password $GH_PAT
 ```
 
-##### Unregistering Repositories
+###### Unregistering Repositories
 
 Unregistering an alias only requires a positional argument: the repository alias.
 

docs/concepts.md

@@ -1,14 +1,20 @@
 # Concepts
 
+## Understanding the Fundamentals
+
+It is important to understand the fundamentals of the underlying technology that Arlon is built on, before you start making use of Arlon. 
+We recommend a good understanding of core concepts around [Docker](https://www.docker.com/), containerization, [Kubernetes](https://kubernetes.io/), GitOps and Continuous Delivery to understand the fundamentals behind Arlon. 
+We also recommend having a basic understanding of application templating technologies such as [Helm](https://helm.sh) and [Kustomize](https://kustomize.io) that are commonly used with Kubernetes. 
+
 ## Management cluster
 
-This Kubernetes cluster hosts the following components:
+Before you can use arlon, you need a Kubernetes cluster to host the following components:
 
 - ArgoCD
 - Arlon
 - Cluster management stacks e.g. Cluster API and/or Crossplane
 
-The Arlon state and controllers reside in the arlon namespace.
+The Arlon state and controllers reside in the arlon namespace on this cluster. 
 
 ## Configuration bundle
 
@@ -17,6 +23,15 @@ produce a set of Kubernetes manifests via a *tool*. This closely follows ArgoCD'
 definition of *tool types*. Consequently, the list of supported bundle
 types mirrors ArgoCD's supported set of manifest-producing tools.
 Each bundle is defined using a Kubernetes ConfigMap resource in the arlon namespace.
+Additionally, a bundle can embed the data itself ("static bundle"), or contain a reference
+to the data ("dynamic bundle"). A reference can be a URL, GitHub location, or Helm repo location.
+The current list of supported bundle types is:
+
+- manifest_inline: a single manifest yaml file embedded in the resource
+- manifest_ref: a reference to a single manifest yaml file
+- dir_inline: an embedded tarball that expands to a directory of YAML files
+- helm_inline: an embedded Helm chart package
+- helm_ref: an external reference to a Helm chart
 
 ### Static bundle
 
@@ -41,6 +56,18 @@ by ArgoCD, including plain YAML, Helm and Kustomize.
 When the user updates a dynamic bundle in git, all clusters consuming that bundle
 (through a profile specified at cluster creation time) will acquire the change.
 
+### Bundle purpose
+
+Bundles can specify an optional *purpose* to help classify and organize them.
+In the future, Arlon may order bundle installation by purpose order (for e.g.
+install bundles with purpose=*networking* before others) but that is not the
+case today. The currently *suggested* purpose values are:
+
+- networking
+- add-on
+- data-service
+- application
+
 ### Other properties
 
 A bundle can also have a comma-separated list of tags, and a description.
@@ -51,6 +78,15 @@ Tags can be useful for classifying bundles, for e.g. by type
 
 A profile expresses a desired configuration for a Kubernetes cluster.
 It is just a set of references to bundles (static, dynamic, or a combination).
+A profile is composed of:
+
+- An optional clusterspec. If specified, it allows the profile
+  to be used to create new clusters.
+  If absent, the profile can only be applied to existing clusters.
+- A list of bundles specifying the configuration to apply onto the cluster
+  once it is operational
+- An optional list of `values.yaml` settings for any Helm Chart type bundle
+  in the bundle list
 A profile can be static or dynamic.
 
 ### Static profile
@@ -81,13 +117,11 @@ or acquire new bundles in real time.
 
 ## Cluster
 
-An Arlon cluster, also known as workload cluster, is a Kubernetes cluster
+An Arlon cluster, also called a 'workload cluster', is a Kubernetes cluster
 that Arlon creates and manages via a git directory structure stored in
 the workspace repository.
 
-(Under construction)
-
-## Cluster spec
+## Cluster Specification
 
 A cluster spec contains desired settings when creating a new cluster.
 They currently include:
@@ -99,6 +133,33 @@ They currently include:
 - The initial (worker) node count
 - The Kubernetes version
 
-## Base cluster
+## Base Cluster 
+
+NOTE: The 'Base Cluster' is a new and evolved way of specifying cluster configuration. It will become the default mechanism for specifying cluster configuration starting version 0.10.0 of Arlon
+
+A Base Cluster allows you to create workload clusters from a manifest file stored in a git repository. The manifest
+typically contains multiple related resources that together define an arbitrarily complex cluster.
+If you make subsequent changes to the Base Cluster manifest, workload clusters originally created from it will automatically acquire the changes.
+The Base Cluster defines:
+
+- A predefined list of Cluster API objects: Cluster, Machines, Machine Deployments, etc. to be deployed in the current namespace
+- The specific infrastructure provider to be used (e.g aws)
+- Kubernetes version
+- Cluster nodepool type that need to be used for creating the cluster manifest (e.g eks, eks-managedmachinepool)
+
+To know more about 'Base Cluster', read about it [here](./baseclusters.md)
+
+## Cluster Chart
+
+The cluster chart is a Helm chart that creates (and optionally applies) the manifests necessary to create a cluster and deploy desired configurations and applications to it as a part of cluster creation, the following resources are created: The profile's Cluster Specification, bundle list and other settings are used to generate values for the cluster chart, and the chart is deployed as a Helm release into the *arlon* namespace in the management cluster.
+
+Here is a summary of the kinds of resources generated and deployed by the chart:
 
-To know more about basecluster (Arlon gen2 clusters), read it [here](./baseclusters.md)
+- A unique namespace with a name based on the cluster's name. All subsequent
+  resources below are created inside that namespace.
+- The stack-specific resources to create the cluster (for e.g. Cluster API resources)
+- A ClusterRegistration to automatically register the cluster with ArgoCD
+- A GitRepoDir to automatically create a git repo and/or directory to host a copy
+  of the expanded bundles. Every bundle referenced by the profile is
+  copied/unpacked into its own subdirectory.
+- One ArgoCD Application resource for each bundle.