PSCE-249 - Add entrypoint for trestle YAML rule transformation (#55)

* feat(action): prep for multiple actions Related: PSCE-243 Signed-off-by: Alex Flom <[email protected]> * docs: updates CONTRIBUTING.md with developer guide Signed-off-by: Jennifer Power <[email protected]> * feat(cli): adds rules tranformation entrypoint to infra module Moves cli.py and cli_base.py to entrypoint.py and entrypoint_base.py Adds rules_transform.py to `infra/entrypoints` Update CONTRIBUTING.md, pyproject.toml, and __main__.py with new name Signed-off-by: Jennifer Power <[email protected]> * chore: moves all entrypoints to a new directory Moves autosync entrypoint to entrypoints directory Removes infra directory Signed-off-by: Jennifer Power <[email protected]> * docs: updates documentation for new rules transform action and updates repo layout Signed-off-by: Jennifer Power <[email protected]> * feat: updates Dockerfile and actions.yaml and entrypoint script to support rules-transform Signed-off-by: Jennifer Power <[email protected]> * chore: change workdir in dependency stage to remove extra files Signed-off-by: Jennifer Power <[email protected]> * docs: adds steps for running as a container to README.md getting started Signed-off-by: Jennifer Power <[email protected]> * fix(actions): adds fixes to Dockerfile location and actions.yml for each each Signed-off-by: Jennifer Power <[email protected]> * docs: adds PR feedback and moves "add action" steps to actions README.md Signed-off-by: Jennifer Power <[email protected]> --------- Signed-off-by: Jennifer Power <[email protected]> Co-authored-by: Alex Flom <[email protected]>
complytime · Oct 16, 2023 · d60adda · d60adda
1 parent 8c4172f
commit d60adda
Show file tree

Hide file tree

Showing 21 changed files with 623 additions and 99 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -1,4 +1,54 @@
-## Contributing
+# Contributing to trestlebot
+
+Thank you for your interest in the trestlebot project.
+
+Before you start contributing, please take a moment to read through the guide below.
+
+> WARNING: This project is currently under initial development. APIs may be changed incompatibly from one commit to another.
+
+### How To Contribute
+
+Some initial contributions could be:
+
+- Improving the documentation
+- Adding test coverage
+- Try out issues that have the label `good first issue`
+- Opening an issue for bugs or feature requests
+
+## Opening a Pull Request
+
+When submitting a pull request, please follow these guidelines:
+
+1. Ensure that you have an issue submitted first and reference it in your pull request.
+2. Ensure that your code passes all CI tests.
+3. Please keep the pull request focused on a single issue or feature, if possible.
+
+## Developer Guide
+
+## Prerequisites
+
+- [Python](https://www.python.org/downloads/) - v3.8+
+- [Poetry](https://python-poetry.org/)
+- [Podman](https://podman.io/docs/installation)
+
+### How It Works
+
+For workflow diagrams, see the [diagrams](./docs/diagrams/) under the `docs` folder.
+
+#### Components
+
+1. CI Provider - Runs or builds and runs trestle-bot container
+2. Trestle Bot - Provides logic for managing workspace and containerized environment for use in workflows
+3. Compliance-Trestle - Upstream library that provided core logic for how OSCAL content is managed
+
+#### Code structure
+
+- `actions` - Provides specific logic for `trestle-bot` tasks that are packaged as Actions. See [README.md](./actions/README.md) for more information.
+- `entrypoints` - Provides top level logic for specific tasks. These tasks are not necessarily related in any so they are not organized into a hierarchical command structure, but they do inherit logic and flag from a base class.
+- `provider.py, github.py, and gitlab.py` - Git provider abstract class and concrete implementations for interacting with the API.
+- `tasks` - Pre-tasks can be configured before the main git logic is run. Any task that does workspace management should go here.
+- `tasks/authored` - The `authored` package contains logic for managing authoring tasks for single instances of a top-level OSCAL model. These encapsulate logic from the `compliance-trestle` library and allows loose coupling between `tasks` and `authored` types.
+- `transformers` - This contains data transformation logic; specifically for rules. 
 
 ### Format and Styling
 
@@ -14,7 +64,10 @@ make test
 
 ### Run with poetry
 ```
-poetry run trestle-bot
+poetry shell
+poetry install
+poetry run trestlebot-autosync
+poetry run trestlebot-rules-transform
 ```
 
 ### Local testing

diff --git a/Dockerfile b/Dockerfile
@@ -44,14 +44,17 @@ RUN  python3.9 -m pip install --no-cache-dir --upgrade pip \
     && pip install --no-cache-dir poetry=="$POETRY_VERSION"
 
 # Cache runtime deps
-WORKDIR $PYSETUP_PATH
-COPY ./ $PYSETUP_PATH
+WORKDIR "/build"
+COPY . "/build"
 
 # Install runtime deps
-RUN poetry install --without tests,dev --no-root
+RUN python -m venv $VENV_PATH && \
+  . $VENV_PATH/bin/activate && \
+  poetry install --without tests,dev --no-root
 
 # install the root project in non-editable mode
-RUN  poetry build -f wheel -n && \
+RUN . $VENV_PATH/bin/activate && \
+  poetry build -f wheel -n && \
   pip install --no-cache-dir --no-deps dist/*.whl && \
   rm -rf dist *.egg-info
 
@@ -64,9 +67,12 @@ RUN microdnf install -y git \
     && microdnf clean all \
     && rm -rf /var/lib/apt/lists/*
 
-COPY ./entrypoint.sh /
+# Add wrappers for entrypoints that provide support the actions
+COPY ./actions/autosync/auto-sync-entrypoint.sh /
+RUN chmod +x /auto-sync-entrypoint.sh
 
-RUN chmod +x /entrypoint.sh
+COPY ./actions/rules-transform/rules-transform-entrypoint.sh /
+RUN chmod +x /rules-transform-entrypoint.sh
 
 ENTRYPOINT ["python3.9", "-m" , "trestlebot"]
 CMD ["--help"]

diff --git a/README.md b/README.md
@@ -1,94 +1,46 @@
 # trestle-bot
 
-trestle-bot assists users in leveraging [Compliance-Trestle](https://github.com/IBM/compliance-trestle) in automated workflows for [OSCAL](https://github.com/usnistgov/OSCAL) formatted compliance content management.
+trestle-bot assists users in leveraging [Compliance-Trestle](https://github.com/IBM/compliance-trestle) in CI/CD workflows for [OSCAL](https://github.com/usnistgov/OSCAL) formatted compliance content management.
 
 > WARNING: This project is currently under initial development. APIs may be changed incompatibly from one commit to another.
 
-## Basic Configuration
+## Getting Started
 
+### GitHub Actions
 
-```yaml
+For detailed documentation on how to use each action see the README.md each each folder under [actions](./actions/).
 
-name: Example Workflow
-...
+The `autosync` action will sync trestle-generated Markdown files to OSCAL JSON files in a trestle workspace. All content under the provided markdown directory when the action is run. This action supports all top-level models [supported by compliance-trestle for authoring](https://ibm.github.io/compliance-trestle/tutorials/ssp_profile_catalog_authoring/ssp_profile_catalog_authoring/).
 
-    steps:
-      - uses: actions/checkout@v3
-      - name: Run trestlebot
-        id: trestlebot
-        uses: RedHatProductSecurity/trestle-bot@main
-        with:
-          markdown_path: "markdown/profiles"
-          oscal_model: "profile"
-```
+The `rules-transform` actions can be used when managing [OSCAL Component Definitions](https://pages.nist.gov/OSCAL-Reference/models/v1.1.1/component-definition/json-outline/) in a trestle workspace. The action will transform rules defined in the rules YAML view to an OSCAL Component Definition JSON file.
 
-## Inputs and Outputs
+### GitLab CI
 
-Checkout [`action.yml`](./action.yml) for a full list of supported inputs and outputs.
+> Coming Soon
 
-### Additional information on workflow inputs
+### Run as a Container
 
-- `markdown_path`: This is the location for Markdown generated by the `trestle author <model>-generate` commands
-- `ssp_index_path`: This is a text file that stores the component definition information by name in trestle with the ssp name. Example below
+Build and run the container locally:
 
-```json
- "ssp1": {
-            "profile": "profile1",
-            "component definitions": [
-                "comp1",
-                "comp2"
-            ]
-        },
+```bash
+podman build -f Dockerfile -t trestle-bot .
+podman run -v $(pwd):/data -w /data trestle-bot
 ```
 
-## Action Behavior
+Container images are available in `quay.io`:
 
-The purpose of this action is to sync JSON and Markdown data with `compliance-trestle` and commit changes back to the branch or submit a pull request (if desired). Below are the main use-cases/workflows available:
+```bash
+podman run -v $(pwd):/data -w /data quay.io/continuouscompliance/trestle-bot:<tag>
+```
 
-- The default behavior of this action is to run a trestle `assemble` and `regenerate` tasks with the given markdown directory and model and commit the changes back to the branch the workflow ran from ( `github.ref_name` ). The branch can be changed by setting the field `branch`. If no changes exist or the changes do not exist with the file pattern set, no changes will be made and the action will exit successfully.
+## Contributing
 
-```yaml
-  steps:
-    - uses: actions/checkout@v3
-    - name: Run trestlebot
-      id: trestlebot
-      uses: RedHatProductSecurity/trestle-bot@main
-      with:
-        markdown_path: "markdown/profiles"
-        oscal_model: "profile"
-        branch: "another-branch"
-```
+For information about contributing to trestle-bot see the [CONTRIBUTING.md](./CONTRIBUTING.md) file.
 
-- If the `target_branch` field is set, a pull request will be made using the `target_branch` as the base branch and `branch` as the head branch.
-
-```yaml
-  steps:
-    - uses: actions/checkout@v3
-    - name: Run trestlebot
-      id: trestlebot
-      uses: RedHatProductSecurity/trestle-bot@main
-      with:
-        markdown_path: "markdown/profiles"
-        oscal_model: "profile"
-        branch: "autoupdate-${{ github.run_id }}"
-        target_branch: "main"
-        github_token: ${{ secret.GITHUB_TOKEN }}
-```
+## License
 
-- When `check_only` is set, the trestle `assemble` and `regenerate` tasks are run and the repository is checked for changes. If changes exists, the action with exit with an error.
-
-```yaml
-    steps:
-      - uses: actions/checkout@v3
-      - name: Run trestlebot
-        id: trestlebot
-        uses: RedHatProductSecurity/trestle-bot@main
-        with:
-          markdown_path: "markdown/profiles"
-          oscal_model: "profile"
-          check_only: true
-```
+This project is licensed under the Apache 2.0 License - see the [LICENSE.md](LICENSE) file for details.
 
-> Note: Trestle `assemble` or `regenerate` tasks may be skipped if desired using `skip_assemble: true` or `skip_regenerate: true`, respectively. 
+## Troubleshooting
 
-See `TROUBLESHOOTING.md` for additional information.
+See [TROUBLESHOOTING.md](./TROUBLESHOOTING.md) for troubleshooting tips.
diff --git a/actions/README.md b/actions/README.md
@@ -0,0 +1,11 @@
+# Actions
+
+
+Tasks in trestle-bot that we want to expose as a GitHub Action are located here. 
+
+# Adding a new Action
+
+> Note to contributors: We are trying to limit the task that we expose as actions to workspace manage operations and checks only.
+
+First, create an entrypoint script for the new action in the `trestlebot/entrypoints` directory. Then add the action by creating a new directory in the `actions` directory with an `action.yml` that references your new entrypoint. See the [GitHub Actions documentation](https://docs.github.com/en/actions/creating-actions/creating-a-docker-container-action) for more information.
+
diff --git a/actions/autosync/README.md b/actions/autosync/README.md
@@ -0,0 +1,88 @@
+# trestlebot AutoSync Action
+
+## Basic Configuration
+
+
+```yaml
+
+name: Example Workflow
+...
+
+    steps:
+      - uses: actions/checkout@v3
+      - name: Run trestlebot
+        id: trestlebot
+        uses: RedHatProductSecurity/trestle-bot/actions/autosync@main
+        with:
+          markdown_path: "markdown/profiles"
+          oscal_model: "profile"
+```
+
+## Inputs and Outputs
+
+Checkout [`action.yml`](./action.yml) for a full list of supported inputs and outputs.
+
+### Additional information on workflow inputs
+
+- `markdown_path`: This is the location for Markdown generated by the `trestle author <model>-generate` commands
+- `ssp_index_path`: This is a text file that stores the component definition information by name in trestle with the ssp name. Example below
+
+```json
+ "ssp1": {
+            "profile": "profile1",
+            "component definitions": [
+                "comp1",
+                "comp2"
+            ]
+        },
+```
+
+## Action Behavior
+
+The purpose of this action is to sync JSON and Markdown data with `compliance-trestle` and commit changes back to the branch or submit a pull request (if desired). Below are the main use-cases/workflows available:
+
+- The default behavior of this action is to run a trestle `assemble` and `regenerate` tasks with the given markdown directory and model and commit the changes back to the branch the workflow ran from ( `github.ref_name` ). The branch can be changed by setting the field `branch`. If no changes exist or the changes do not exist with the file pattern set, no changes will be made and the action will exit successfully.
+
+```yaml
+  steps:
+    - uses: actions/checkout@v3
+    - name: Run trestlebot
+      id: trestlebot
+      uses: RedHatProductSecurity/trestle-bot/actions/autosync@main
+      with:
+        markdown_path: "markdown/profiles"
+        oscal_model: "profile"
+        branch: "another-branch"
+```
+
+- If the `target_branch` field is set, a pull request will be made using the `target_branch` as the base branch and `branch` as the head branch.
+
+```yaml
+  steps:
+    - uses: actions/checkout@v3
+    - name: Run trestlebot
+      id: trestlebot
+      uses: RedHatProductSecurity/trestle-bot/actions/autosync@main
+      with:
+        markdown_path: "markdown/profiles"
+        oscal_model: "profile"
+        branch: "autoupdate-${{ github.run_id }}"
+        target_branch: "main"
+        github_token: ${{ secret.GITHUB_TOKEN }}
+```
+
+- When `check_only` is set, the trestle `assemble` and `regenerate` tasks are run and the repository is checked for changes. If changes exists, the action with exit with an error.
+
+```yaml
+    steps:
+      - uses: actions/checkout@v3
+      - name: Run trestlebot
+        id: trestlebot
+        uses: RedHatProductSecurity/trestle-bot/actions/autosync@main
+        with:
+          markdown_path: "markdown/profiles"
+          oscal_model: "profile"
+          check_only: true
+```
+
+> Note: Trestle `assemble` or `regenerate` tasks may be skipped if desired using `skip_assemble: true` or `skip_regenerate: true`, respectively.