Merge pull request #5 from Axway-API-Management-Plus/develop

Release v0.1.0

.mvn/wrapper/maven-wrapper.properties

@@ -0,0 +1,18 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.6.0/apache-maven-3.6.0-bin.zip
+wrapperUrl=https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.1.1/maven-wrapper-3.1.1.jar

CHANGELOG.adoc

@@ -9,11 +9,12 @@ ifdef::env-github[]
 :warning-caption: :warning:
 endif::[]
 
-== Version 0.0.4
+== Version 0.1.0
 [cols="1,2,<10a", options="header"]
 |===
 |ID|Type|Description
 |0
 |Enhancement
-|Initial release.
+|After a complete redesign, this version provides a stable configuration format.
+
 |===

README.adoc

@@ -12,22 +12,25 @@ ifdef::env-github[]
 :tip-caption: :bulb:
 :warning-caption: :warning:
 endif::[]
-:project-ver: 0.0.4
+:project-ver: 0.1.0
 
 image:https://github.com/Axway-API-Management-Plus/yamles-utils/actions/workflows/maven-publish.yml/badge.svg[Workflow Status]
 image:https://img.shields.io/github/license/Axway-API-Management-Plus/yamles-utils?style=plastic[]
 image:https://img.shields.io/github/v/release/Axway-API-Management-Plus/yamles-utils?style=plastic[]
 
-== Introduction
+
+IMPORTANT: The project is community supported and not supported by Axway Support. If you encounter any issues, please raise an GitHub issue on this project.
 
 [sidebar]
 .tl;dr
 --
 Utilities to verify coding rules of YAML Entity Store based projects and to configure projects for target environments using various configuration sources.
 --
 
+== Introduction
+
 Since the 7.7 February'22 release of the Axway API Gateway, Policy Studio supports to edit policy projects in the YAML Entity Store (YAML-ES) format.
-This format enables fine grained storage of project entities in easy to read and easy to compare YAML files.
+This format enables fine grained storage of project entities, in easy to read and easy to compare YAML files.
 
 The YAML Entity Store has many advantages compared to the legacy XML based Entity Store:
 
@@ -37,8 +40,15 @@ The YAML Entity Store has many advantages compared to the legacy XML based Entit
 * Storing in a SCM (e.g, Git) changes can be easily checked.
 * Environmentalized values configured in a single YAML file.
 
-With the experience of the link:https://github.com/Axway-API-Management-Plus/apigw-maven-plugin[Maven Plugin for API Gateway] project, storing configuration in one single file doesn't always met the requirements.
-This project is created to provide the flexibility of the Maven Plugin (for XML based ES only) to the YAML-ES based projects also (and even more).
+Beside these advantages, there are still some challenges.
+From the experience of the link:https://github.com/Axway-API-Management-Plus/apigw-maven-plugin[Maven Plugin for API Gateway] project, storing configuration in one single file, may not always met the requirements.
+On staging, self-signed certificates may have to be replaced by certificates provided by the in-house CA.
+Credentials are stored in a separate database and have to be configured on deployment time.
+For K8s deployments, the API Gateway may has to be configured via an init container on startup of the pod. And other scenarios you may think of.
+
+This project provides the flexibility, to collect configuration parameters, credentials, and certificates from various sources, and use them to configure the YAML-ES of API Gateway for runtime.
+
+image:docs/asciidoc/images/concept.drw.png[YAML ES Utils Concepts]
 
 == Use Cases
 
@@ -78,49 +88,43 @@ After successful verification the project will be deployed to the target environ
 During the deployment a passphrase is added to protect the configurations.
 
 .CI/CD Pipeline
-image:docs/manual/images/pipeline.png[]
+image:docs/asciidoc/images/deployment-classic-topology.drw.png[]
 
 The picture above visualize the CI/CD pipeline.
 The _YAML-ES Utilities_ tool provided by this project is used for linting and for the configuration of the stage specific project.
 
 ==== Example
 
-The `examples` folder provides a demo of the previously described pipeline (except the deployment task).
-
-The `examples/apim` folder contains a YAML entity store of an API Manager.
-
-The `examples/cicd` folder contains a sample script for configuring a project for different stages.
-
-Execute the following command in a Bash shell to generate a YAML-ES archive for the test environment.
-
-[source,shell]
-----
-examples/cicd/bin/build-archive.sh -e test
-----
-
+The `examples` folder provides a demo of the previously described pipeline (except the deployment task) - see link:examples/README.adoc[examples] for details.
 
 == Features
 
-The YAMLES Utility provides the following features:
+The _YAML-ES Utilities_ provide the following features:
 
 * Linting.
 ** Lint API Gateway project against customized rules.
-** Rules can be sourced from different locations.
+** Rules can be sourced from multiple locations.
 * Merge configuration from multiple files into a result `values.yaml` file.
 * Lookup values from multiple data sources.
 *** YAML files
-*** Environment variables
+*** JSON files
+*** Environment variables (plain text or JSON document)
 *** System properties
 *** Keepass DB
 *** Hashicorp Vault
+*** AWS Secrets Manager
 * Merge certificates from multiple sources into the API Gateway project.
 ** Keystore: Add/overwrite certificates from PKCS#12 or JKS key stores
 ** Simple: Add/overwrite certificates by simple Base64 encoded values (can be combined with value lookup, see above).
 ** Remover: Remove certificates from the project
+** AWS Certificate Manager
 
 == More to Read
 
-* link:manual/user-guide.adoc[User Guide]
+* https://axway-api-management-plus.github.io/yamles-utils/[User Guide]
+* https://docs.axway.com/bundle/axway-open-docs/page/docs/api_mgmt_overview/index.html[Amplify API Management]
+* https://docs.axway.com/bundle/axway-open-docs/page/docs/apim_yamles/index.html[YAML Configuration]
+* https://docs.axway.com/bundle/axway-open-docs/page/docs/apim_yamles/yamles_structure/index.html[YAML Entity Store Structure]
 
 == Contributing
 Please read https://github.com/Axway-API-Management-Plus/Common/blob/master/Contributing.md[Contributing] for details on our code of conduct, and the process for submitting pull requests to us.

docs/asciidoc/_cert_providers.adoc

@@ -0,0 +1,184 @@
+= Certificate Providers
+ifdef::env-github[]
+:outfilesuffix: .adoc
+:!toc-title:
+:caution-caption: :fire:
+:important-caption: :exclamation:
+:note-caption: :paperclip:
+:tip-caption: :bulb:
+:warning-caption: :warning:
+endif::[]
+
+The purpose of a certificate provider is to retrieve certificates or private keys from external sources.
+Various providers exists to support various certificate sources.
+
+[NOTE]
+====
+To get a list of all available _Certificate Providers_ and their documentation, use the following describe command:
+
+[source, shell]
+----
+$ yamlesutils merge describe cert-providers --full
+----
+====
+
+Some lookup provider configurations require a path to a file.
+If the path represents a relative path, it is relative to the location of the certificate configuration file.
+
+== Simple
+
+[cols="2,6a"]
+|===
+|*Name*
+|`simple`
+
+|*Synopsis*
+|Provides certificates directly from configuration file.
+
+2+|*Configuration Parameters*
+|`cert`
+|PEM encoded certificate (single line).
+
+This parameter supports a _Mustache_ template, to lookup the certificates via lookup functions.
+|`key`
+|Optional PEM encoded private key (single line).
+
+This parameter supports a _Mustache_ template, to lookup the certificates via lookup functions.
+|===
+
+*Example*
+
+.Certificate Configuration
+[source, yaml]
+----
+certificates:
+  cassandra-ca:
+    provider: simple
+    config:
+      cert: "MIID...uB" #<1>
+
+  apim-server:
+    provider: simple
+      cert: "MIIDt...xdI=" #<2>
+      key: "MIIEv...CL+X"
+----
+<1> Public certificate
+<2> Server certificate including private key.
+
+In combination with a lookup function, the `simple` provider can also be used to retrieve certificates from a KeePass DB or AWS Secrets Manager.
+
+== Keystore
+
+[cols="2,6a"]
+|===
+|*Name*
+|`keystore`
+
+|*Synopsis*
+|Provides certificates from a keystore file (PKCS#12 or JKS).
+
+2+|*Configuration Parameters*
+|`path`
+|Path to the keystore file.
+If a relative path is specified, the path is relative to the location of the _Certificate Configuration_ file.
+|`pass`
+|Optional passphrase to access the keystore.
+
+This parameter supports a _Mustache_ template, to lookup the certificates via lookup functions.
+|`alias`
+|Optional alias of the certificate within the keystore.
+If not specified, the alias of the entity store certificate is used.
+|`type`
+|Type of the keystore.
+If not specified, the type is determined from the file name extension.
+
+* `JKS`: Java Key Store
+* `P12`: PKCS#12 file
+|===
+
+
+*Example*
+
+.Certificate Configuration
+[source, yaml]
+----
+certificates:
+  example-server:
+    provider: keystore
+    config:
+      path: keystore.p12 #<1>
+      pass: "{{ _kdb('/Test/Sever Certificate', 'password') }}" #<2>
+      alias: server #<3>
+----
+<1> Path to the keystore.
+As the extension is `.p12` the keystore is assumed to be in the PKCS#12 format.
+<2> The passphrase for the keystore is retrieved from a lookup function.
+<3> Alias of the certificate within the keystore.
+
+== AWS Certificate Manager
+
+[cols="2,6a"]
+|===
+|*Name*
+|`aws_cm`
+
+|*Synopsis*
+|Provides certificates from the AWS Certificate Manager.
+
+2+|*Configuration Parameters*
+|`arn`
+|ARN of the certificate stored in the AWS Certificate Manager.
+|`chain`
+|Set to `true` to add the full certificate chain to policy project.
+|===
+
+The https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials-chain.html[default credentials chain] used by the SDK are also used by this certificate provider.
+On systems directly running on EC2 instances, IAM roles can be used to authorize the request without specifying any credentials.
+For systems running on AWS EKS, a K8s service account can be bound to an IAM role to enable the access to the Secrets Manager without using additional credentials.
+But using environment variables (`AWS_SECRET_ACCESS_KEY`, `AWS_ACCESS_KEY_ID`), system properties (`aws.accessKeyId`, `aws.secretAccessKey`), or configuration file are also supported.
+
+*Example*
+
+In this example, the public certificate of CA, issued the Cassandra cluster certificate, is stored in the policy project under the alias `cassandra-ca`.
+For deployment, the certificate must be replaced by the public certificate stored in AWS Certificate Manager.
+The full certificate chain, has to be imported also. 
+
+.Certificate Configuration
+[source, yaml]
+----
+certificates:
+  cassandra-ca:
+    provider: aws_cm
+    config:
+      arn: "arn:aws:acm:us-west-1:000000000000:certificate/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
+      chain: true
+----
+
+== Remover
+
+[cols="2,6a"]
+|===
+|*Name*
+|`remover`
+
+|*Synopsis*
+|Removes a certificates from the policy project.
+
+2+|*Configuration Parameters*
+2+|not required
+|===
+
+The is not really a certificate provider.
+Instead it removes certificates from the policy project.
+The certificates are specified by their alias.
+
+*Example*
+
+.Certificate Configuration
+[source, yaml]
+----
+certificates:
+  acme: #<1>
+    provider: remover
+----
+<1> Alias of the certificate within the policy project.