merge upstream (#28)

* Refac/github_actions (#1)

* feat(dev_requirements): Add pytest

* refac(workspace_settings): Specify python interpreter path

* refac(cicd): Combine actions in composition action and add unit tests.

* fix(cicd): Remove custom arg from pytest.

* fix(cicd): Add missing requirements for pytest

* feat(tasks): Add task to install all dependencies.

* feat(pytest_typehint): Add extension for pytest hints.

* feat(tests): Add initial test.

* ruff format

* fix(pytest): Set pythonpath

* fix(cicd): Pass python version to sub actions with input.

* fix(cicd): Add input for sub actions

* fix(cicd): Read python version from project file instead of env.

* fix(cicd): python version var assignment

* Feat/migrate git logic to python (#5)

* feat(action): Add configuration provider

* feat(vscode): Change auto save to onFocusChange since ruff does not work with afterDelay.

* feat(vscode): Add test task

* refac(tests): Remove __main__.py tests

* feat(vscode): Exclude __pycache__ folders in workspace

* refac(ActionConfig): Remove default value for registry_domain

* feat(git): Add git helper

* refac(action): Pass configuration over env. instead of cli args.

* refac(action): Add git handling and use new configuration provider.

* refac(action_integration_test): Provider github token in env. instead of input.

* fix(action): Remove git logic and rename env vars

* feat(ActionConfig): Log substring of secrets

* fix(ConfigProvider): Make flag registry_secrets as secret.

* refac(action): Change github_token back to input variable.

* fix(action): Add exception handling if target branch does not exist

* fix(config): Fix conversion from env_string to bool.

* refac(cli): Move cli resources to __main__

* feat(action): Add input to set relative working dir.

* feat(git): Set git root to new repo_root value.

* doc(README): Update documentation of the github action.

* Feat/refac_core (#10)

* refac(hcledit): Move bin in subdirectory

* refac(terraform_utils): Move in own directory.

* refac(terraform_utils): Move to own directory

* feat(vscode): Add some project settings.

* feat(pip): Add py-markdown-table as requirement.

* refac(terraform_utils): Add init file.

* feat(providers): Add Providers.

* feat(provider_handler): Add provider_handler and builder.

* refac(models): Update versioned resources models.

* feat(composition): Remove composition.

* feat(models): Add statistics model.

* feat(cli): Switch to provider implementation.

* refac(builder): Change secret input from file path to dict.

* refac(action): Switch to provider implementation.

* ruff auto-fix

* feat(vscode): Add task to auto-fix with ruff.

* doc(README): Update doc.

* fix(provider): Fix get markdown table

* feat(Statistics): Add function to get markdown table.

* feat(action): Add report to pr body.

* fix(upgrade_resources): Add catch to handle unsuccesfull upgrades

* ruff format

* fix(terraform_provider): remove exception handling.

* refac(exception): Remove base exception.

* fix(cicd): Change error text.

* fix(hcledit_cli): Check if binary exists.

* fix(hcledit): Make binary executable.

* fix(hcledit_cli): REmove exist check.

* Revert "fix(hcledit_cli): REmove exist check."

This reverts commit c4a29a2.

* fix(setup): Change package_data path.

* fix(Statistics): Fix markdown table.

* fix(Statistics): Change markdown input to list.

* feat(vscode): Disable justmycode in launch.json

* doc(README): Add supported platforms.

* fix(pr_body): Add missing newline after provider table.

* fix(pr_body): Remove second newline.

* fix(pr_body): Format tables.

* fix(pr_body): Add quotes.

* refac(markdown_table): Switch to new library for markdown tables.

* fix(pip): Fix pytablewriter version

* fix(terraform_provider): Set table_name

* feat(action): Implement handling of existing PRs

* fix(action): Save upgradable resources instead of all.

* fix(cicd): Disable checks on merge event.

* refac(action): Rename registry_secrets to terraform_registry_secrets

* Fix/action_file_references (#23)

* feat(action): Implement correct handling of repo root and action files path

* fix(tests): Change inputs in config tests.

* refac(rich): Change text in progress bar.

* fix(action): Change working directory

* dic(README): Update documentation and add pr image.

* doc(README): Change infrapatch Name to Upper-Case.

* Feat/add_tests (#27)

* feat(tests): Add tests for models

* ruff auto-fix

* test(terraform_resource): Add to_dict() test.

* doc(checkout): Add note and update action example regarding fetch-depth.

* feat(action): Add log message when updating pr.

* fix(provider_cache): Fix provider cache not working.

* fix(markdown_tables): Skip table generate for providers without changes.

* fix(pr_update): Fix handling of existing prs

* fix(action): Fix searching existing pr.

* fix(action): Filter pr in python instead of search over github api.

* feat(action): Update pr even when no upgrades where performed.

* fix(provider_handler): Switch reference of object.

* fix(provider_handler): Fix resource update.

* fix(provider_handler): Fix resource replace in cache.

* fix(provider_handler): REmove indent.

* fix(hcl_handler): Skip modules without version attribute.

* fix(action_integration_test): Fix repo name to source repo. (#37)

* Revert "fix(action_integration_test): Fix repo name to source repo. (#37)"

This reverts commit 67adea4.

* fix(action_integration_test): Switch update test to secrets.github_token, so that it can be replaced with a pat.

* feat(action_integration_test): Add condition to use token from secrets if available.

* fix(action_integration_test): Fix token variable assignment.

* Revert "fix(action_integration_test): Fix token variable assignment."

This reverts commit d488e16.

* Revert "feat(action_integration_test): Add condition to use token from secrets if available."

This reverts commit e99cc37.

* Revert "fix(action_integration_test): Switch update test to secrets.github_token, so that it can be replaced with a pat."

This reverts commit 0ea227c.

.devcontainer/devcontainer.json

@@ -32,7 +32,8 @@
 				"evendead.help-me-add",
 				"charliermarsh.ruff",
 				"streetsidesoftware.code-spell-checker",
-				"njqdev.vscode-python-typehint"
+				"njqdev.vscode-python-typehint",
+				"Cameron.vscode-pytest"
 			]
 		}
 	}

.github/workflows/action_integration_test.yml

@@ -1,62 +1,56 @@
-name: "GitHub Action integration test"
-
-permissions:
-  contents: write
-  pull-requests: write
-
-on:
-  pull_request:
-    branches:
-      - main
-  workflow_dispatch:
-
-jobs:
-  integration-test:
-    env:
-      report_json_file: InfraPatch_Statistics.json
-
-    name: "Run GitHub Action integration test"
-    runs-on: ubuntu-latest
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v3
-
-      - name: Run in report only mode
-        uses: ./
-        with:
-          report_only: true
-
-      - name: Run in update mode
-        id: update
-        uses: ./
-        with:
-          report_only: false
-          target_branch_name: "feat/infrapatch_test_${{ github.run_number }}"
-
-      - name: Check update result
-        shell: pwsh
-        run: |
-          $report = Get-Content $env:report_json_file -Raw | ConvertFrom-Json
-          if ( -not $report.total_resources -gt 0 ) {
-            throw "Failed to get resources"
-          }
-          if ( -not ( $report.resources_patched -gt 3 ) ) {
-              throw "No resources should be patched"
-          }
-          if ( $report.errors -gt 0 ) {
-              throw "Errors have been detected"
-          }
-
-      - name: Delete created branch$
-        if: always()
-        uses: dawidd6/action-delete-branch@v3
-        with:
-          github_token: ${{github.token}}
-          branches: ${{ steps.update.outputs.target_branch }}
-          soft_fail: true
-
-
-
-
-
-
+name: "GitHub Action integration test"
+
+on:
+  workflow_call:
+
+jobs:
+  integration-test:
+    env:
+      report_json_file: InfraPatch_Statistics.json
+
+    name: "Run GitHub Action integration test"
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Run in report only mode
+        uses: ./
+        with:
+          report_only: true
+
+      - name: Run in update mode
+        id: update
+        uses: ./
+        with:
+          report_only: false
+          target_branch_name: "feat/infrapatch_test_${{ github.run_number }}"
+
+      - name: Check update result
+        shell: pwsh
+        run: |
+          $report = Get-Content $env:report_json_file -Raw | ConvertFrom-Json
+          if ( -not $report.total_resources -gt 0 ) {
+            throw "Failed to get resources"
+          }
+          if ( -not ( $report.resources_patched -gt 3 ) ) {
+              throw "No resources should be patched"
+          }
+          if ( $report.errors -gt 0 ) {
+              throw "Errors have been detected"
+          }
+
+      - name: Delete created branch$
+        if: always()
+        uses: dawidd6/action-delete-branch@v3
+        with:
+          branches: ${{ steps.update.outputs.target_branch }}
+          soft_fail: true
+        env:
+          GITHUB_TOKEN: ${{github.token}}
+
+
+
+
+
+

.github/workflows/check_format_and_lint.yml

@@ -1,24 +1,27 @@
+name: "Check Format and Lint Code"
+
 on:
-  pull_request:
-    types:
-      - opened
-      - synchronize
-      - reopened
-      - closed
-    branches:
-      - main
+  workflow_call:
 
 jobs:
   check_code:
+    name: "Check Format and Lint Code"
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
         uses: actions/checkout@v4
+
+      - name: Get Python Version
+        id: get_python_verion
+        run: |
+          python_version=$(cat python_version.txt)
+          echo "Using Python version $python_version"
+          echo "::set-output name=python_version::$(echo $python_version)"
         
       - name: Install Python
         uses: actions/setup-python@v2
         with:
-            python-version: 3.11
+            python-version: ${{ steps.get_python_verion.outputs.python_version }}
 
       - name: Install Dependencies
         run: |
@@ -31,4 +34,9 @@ jobs:
 
       - name: Check code with ruff
         run: ruff check .
-
+
+
+
+
+
+

.github/workflows/cli-integration-test.yml → .github/workflows/cli_integration_test.yml

@@ -1,78 +1,79 @@
-name: CLI Integration test
-
-on:
-  pull_request:
-    types:
-      - opened
-      - synchronize
-      - reopened
-      - closed
-    branches:
-      - main
-
-jobs:
-  test:
-    # only run if not closed or closed with merge
-    if: ${{ github.event.pull_request.merged == true || github.event.pull_request.state != 'closed' }}
-    env:
-      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      report_json_file: InfraPatch_Statistics.json
-
-    strategy:
-      matrix:
-        os:
-          - macos-latest
-          - ubuntu-latest
-          # - windows-latest Windows does currently not work because of pygohcl
-    runs-on: ${{ matrix.os }}
-    steps:
-      - name: Checkout code
-        uses: actions/checkout@v4
-
-      - name: Install Python
-        uses: actions/setup-python@v4
-        with:
-          python-version: "3.11"
-
-      - name: Install InfraPatch CLI
-        run: |
-          python -m pip install .
-        shell: bash
-
-      - name: Run InfraPatch report
-        shell: bash
-        run: infrapatch --debug report --dump-json-statistics
-
-      - name: Check report result
-        shell: pwsh
-        run: |
-          $report = Get-Content $env:report_json_file -Raw | ConvertFrom-Json
-          if ( -not $report.total_resources -gt 0 ) {
-            throw "Failed to get resources"
-          }
-          if ( $report.resources_patched -ne 0 ) {
-              throw "No resources should be patched"
-          }
-          if ( $report.errors -gt 0 ) {
-              throw "Errors have been detected"
-          }
-
-      - name: Run InfraPatch update
-        shell: bash
-        run: infrapatch --debug update --dump-json-statistics --confirm
-
-      - name: Check update result
-        shell: pwsh
-        run: |
-          $report = Get-Content $env:report_json_file -Raw | ConvertFrom-Json
-          if ( -not $report.total_resources -gt 0 ) {
-            throw "Failed to get resources"
-          }
-          if ( -not ( $report.resources_patched -gt 3 ) ) {
-              throw "No resources should be patched"
-          }
-          if ( $report.errors -gt 0 ) {
-              throw "Errors have been detected"
-          }
-          
-          
+name: CLI Integration test
+
+on:
+  workflow_call:
+
+jobs:
+  cli_integration_test:
+    name: CLI Integration test
+    # only run if not closed or closed with merge
+    if: ${{ github.event.pull_request.merged == true || github.event.pull_request.state != 'closed' }}
+    env:
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      report_json_file: InfraPatch_Statistics.json
+
+    strategy:
+      matrix:
+        os:
+          - macos-latest
+          - ubuntu-latest
+          # - windows-latest Windows does currently not work because of pygohcl
+    runs-on: ${{ matrix.os }}
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Get Python Version
+        id: get_python_verion
+        run: |
+          python_version=$(cat python_version.txt)
+          echo "Using Python version $python_version"
+          echo "::set-output name=python_version::$(echo $python_version)"
+
+      - name: Install Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ steps.get_python_verion.outputs.python_version }}
+
+      - name: Install InfraPatch CLI
+        run: |
+          python -m pip install .
+        shell: bash
+
+      - name: Run InfraPatch report
+        shell: bash
+        run: infrapatch --debug report --dump-json-statistics
+
+      - name: Check report result
+        shell: pwsh
+        run: |
+          $report = Get-Content $env:report_json_file -Raw | ConvertFrom-Json
+          if ( -not $report.total_resources -gt 0 ) {
+            throw "Failed to get resources"
+          }
+          if ( $report.resources_patched -ne 0 ) {
+              throw "No resources should be patched"
+          }
+          if ( $report.errors -gt 0 ) {
+              throw "Errors have been detected"
+          }
+
+      - name: Run InfraPatch update
+        shell: bash
+        run: infrapatch --debug update --dump-json-statistics --confirm
+
+      - name: Check update result
+        shell: pwsh
+        run: |
+          $report = Get-Content $env:report_json_file -Raw | ConvertFrom-Json
+          if ( -not $report.total_resources -gt 0 ) {
+            throw "Failed to get resources"
+          }
+          if ( -not ( $report.resources_patched -gt 3 ) ) {
+              throw "At least 3 resources should be patched"
+          }
+          if ( $report.errors -gt 0 ) {
+              throw "Errors have been detected"
+          }
+          
+          

.github/workflows/composition.yml

@@ -0,0 +1,35 @@
+name: InfraPatch Checks
+
+permissions:
+  contents: write
+  pull-requests: write
+
+on:
+  pull_request:
+    types:
+      - opened
+      - synchronize
+      - reopened
+    branches:
+      - main
+
+env:
+  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+jobs:
+
+  check_code:
+    uses: ./.github/workflows/check_format_and_lint.yml
+
+  unit_tests:
+    needs: check_code
+    uses: ./.github/workflows/unit_tests.yml
+
+  cli_integration_test:
+    needs:  unit_tests
+    uses: ./.github/workflows/cli_integration_test.yml
+
+  github_action_integration_test:
+    needs:  unit_tests
+    uses: ./.github/workflows/action_integration_test.yml
+