Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unit test cases and one click deploy pipeline #18

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Unit test cases and one click deploy pipeline #18

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
2d2acae
unit test cases and one click deploy pipeline
Priyanka-Microsoft Dec 4, 2024
41054c9
changes in deployment file
Priyanka-Microsoft Dec 4, 2024
1a51871
added output in main.bicep
Priyanka-Microsoft Dec 4, 2024
a5e164c
scheduled the pipeline at 6 AM and 6 PM GMT
Priyanka-Microsoft Dec 4, 2024
b59e9a9
validated through deployment.json
Priyanka-Microsoft Dec 9, 2024
ee9be99
Validated deleting the main resource group
Priyanka-Microsoft Dec 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
190 changes: 190 additions & 0 deletions .github/workflows/deploy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
name: CI-Validate Deployment

on:
push:
branches:
- main
schedule:
- cron: '0 6,18 * * *' # Runs at 6:00 AM and 6:00 PM GMT

jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout@v3

- name: Setup Azure CLI
run: |
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
az --version # Verify installation

- name: Login to Azure
run: |
az login --service-principal -u ${{ secrets.AZURE_CLIENT_ID }} -p ${{ secrets.AZURE_CLIENT_SECRET }} --tenant ${{ secrets.AZURE_TENANT_ID }}

- name: Install Bicep CLI
run: az bicep install

- name: Generate Resource Group Name
id: generate_rg_name
run: |
echo "Generating a unique resource group name..."
TIMESTAMP=$(date +%Y%m%d%H%M%S)
COMMON_PART="pslautomationRes"
UNIQUE_RG_NAME="${COMMON_PART}${TIMESTAMP}"
echo "RESOURCE_GROUP_NAME=${UNIQUE_RG_NAME}" >> $GITHUB_ENV
echo "Generated Resource_GROUP_PREFIX: ${UNIQUE_RG_NAME}"

- name: Check and Create Resource Group
id: check_create_rg
run: |
set -e
echo "Checking if resource group exists..."
rg_exists=$(az group exists --name ${{ env.RESOURCE_GROUP_NAME }})
if [ "$rg_exists" = "false" ]; then
echo "Resource group does not exist. Creating..."
az group create --name ${{ env.RESOURCE_GROUP_NAME }} --location northcentralus || { echo "Error creating resource group"; exit 1; }
else
echo "Resource group already exists."
fi

- name: Generate Unique Solution Prefix
id: generate_solution_prefix
run: |
set -e
COMMON_PART="pslr"
TIMESTAMP=$(date +%s)
UPDATED_TIMESTAMP=$(echo $TIMESTAMP | tail -c 3)
UNIQUE_SOLUTION_PREFIX="${COMMON_PART}${UPDATED_TIMESTAMP}"
echo "SOLUTION_PREFIX=${UNIQUE_SOLUTION_PREFIX}" >> $GITHUB_ENV
echo "Generated SOLUTION_PREFIX: ${UNIQUE_SOLUTION_PREFIX}"

- name: Deploy Bicep Template
id: deploy
run: |
set -e
az deployment group create \
--resource-group ${{ env.RESOURCE_GROUP_NAME }} \
--template-file infrastructure/deployment.json \
--parameters \
HostingPlanName="${{ env.SOLUTION_PREFIX }}-plan" \
ApplicationInsightsName="appins-${{ env.SOLUTION_PREFIX }}" \
WebsiteName="webapp-${{ env.SOLUTION_PREFIX }}" \
CosmosDBName="db-cosmos-${{ env.SOLUTION_PREFIX }}" \
CosmosDBRegion="NorthCentralUS" \
AzureSearchService="search-${{ env.SOLUTION_PREFIX }}" \
AzureOpenAIResource="aoai-${{ env.SOLUTION_PREFIX }}" \
WorkspaceName="worksp-${{ env.SOLUTION_PREFIX }}"

- name: Delete Bicep Deployment
if: success()
run: |
set -e
echo "Checking if resource group exists..."
rg_exists=$(az group exists --name ${{ env.RESOURCE_GROUP_NAME }})
if [ "$rg_exists" = "true" ]; then
echo "Resource group exists. Cleaning..."
az group delete \
--name ${{ env.RESOURCE_GROUP_NAME }} \
--yes \
--no-wait
echo "Resource group deleted... ${{ env.RESOURCE_GROUP_NAME }}"
else
echo "Resource group does not exist."
fi

- name: Wait for Resource Deletion to Complete
run: |
# List of resources to check based on SOLUTION_PREFIX
resources_to_check=(
"aoai-${{ env.SOLUTION_PREFIX }}"
"appins-${{ env.SOLUTION_PREFIX }}"
"db-cosmos-${{ env.SOLUTION_PREFIX }}"
"${{ env.SOLUTION_PREFIX }}-plan"
"search-${{ env.SOLUTION_PREFIX }}"
"webapp-${{ env.SOLUTION_PREFIX }}"
"worksp-${{ env.SOLUTION_PREFIX }}"
)

# Get the list of resources in YAML format
resource_list=$(az resource list --subscription ${{ secrets.AZURE_SUBSCRIPTION_ID }} --output yaml)

# Maximum number of retries and retry intervals
max_retries=3
retry_intervals=(30 60 120)
retries=0

while true; do
resource_found=false
# Iterate through the resources to check
for resource in "${resources_to_check[@]}"; do
echo "Checking resource: $resource"
if echo "$resource_list" | grep -q "name: $resource"; then
echo "Resource '$resource' exists in the subscription."
resource_found=true
else
echo "Resource '$resource' does not exist in the subscription."
fi
done

# If any resource exists, retry
if [ "$resource_found" = true ]; then
retries=$((retries + 1))
if [ "$retries" -ge "$max_retries" ]; then
echo "Maximum retry attempts reached. Exiting."
break
else
echo "Waiting for ${retry_intervals[$retries-1]} seconds before retrying..."
sleep ${retry_intervals[$retries-1]}
fi
else
echo "No resources found. Exiting."
break
fi
done

- name: Purging the Resources
if: success()
run: |
set -e
# Purging resources based on solution prefix
echo "Purging resources..."

# List of resources to purge
resources_to_purge=(
"aoai-${{ env.SOLUTION_PREFIX }}"
"appins-${{ env.SOLUTION_PREFIX }}"
"db-cosmos-${{ env.SOLUTION_PREFIX }}"
"${{ env.SOLUTION_PREFIX }}-plan"
"search-${{ env.SOLUTION_PREFIX }}"
"webapp-${{ env.SOLUTION_PREFIX }}"
"worksp-${{ env.SOLUTION_PREFIX }}"
)

for resource in "${resources_to_purge[@]}"; do
echo "Purging resource: $resource"
if ! az resource delete --ids /subscriptions/${{ secrets.AZURE_SUBSCRIPTION_ID }}/providers/Microsoft.CognitiveServices/locations/uksouth/deletedAccounts/$resource --verbose; then
echo "Failed to purge resource: $resource"
else
echo "Purged the resource: $resource"
fi
done

echo "Resource purging completed successfully"

- name: Send Notification on Failure
if: failure()
run: |
RUN_URL="https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}"

EMAIL_BODY=$(cat <<EOF
{
"body": "<p>Dear Team,</p><p>We would like to inform you that the Research Assistant Automation process has encountered an issue and has failed to complete successfully.</p><p><strong>Build URL:</strong> ${RUN_URL}<br> ${OUTPUT}</p><p>Please investigate the matter at your earliest convenience.</p><p>Best regards,<br>Your Automation Team</p>"
}
EOF
)

curl -X POST "${{ secrets.LOGIC_APP_URL }}" \
-H "Content-Type: application/json" \
-d "$EMAIL_BODY" || echo "Failed to send notification"
111 changes: 111 additions & 0 deletions .github/workflows/tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
name: Tests

on:
push:
branches:
- main # Trigger on push to the main branch
pull_request:
branches:
- main # Trigger on pull requests to the main branch
types:
- opened
- ready_for_review
- reopened
- synchronize

jobs:
backend_tests:
name: Backend Tests
runs-on: ubuntu-latest # Use the latest Ubuntu runner

steps:
- uses: actions/checkout@v4 # Checkout the repository

# Set up Python environment for Backend
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11" # Set Python version

- name: Install Backend Dependencies
run: |
python -m pip install -r requirements.txt
python -m pip install coverage pytest-cov
python -m pip install azure-keyvault-secrets

- name: Run Backend Tests with Coverage
run: |
if python -m pytest --cov=. --cov-report=xml --cov-report=html --cov-report=term-missing --junitxml=coverage-junit.xml; then
echo "Tests completed, checking coverage."
# Only fail if coverage does not meet criteria
if [ -f coverage.xml ]; then
COVERAGE=$(python -c "import xml.etree.ElementTree as ET; tree = ET.parse('coverage.xml'); root = tree.getroot(); coverage = root.find('coverage').get('lines').split('%')[0]; print(float(coverage))")
if (( $(echo "$COVERAGE < 80" | bc -l) )); then
echo "Coverage is below 80%, failing the job."
exit 1
fi
fi
else
echo "No tests found, skipping coverage check."
fi

- uses: actions/upload-artifact@v4
with:
name: backend-coverage
path: |
coverage.xml # Correct path to backend coverage
coverage-junit.xml # Correct path to backend JUnit report
htmlcov/ # Correct path to backend HTML coverage report

frontend_tests:
name: Frontend Tests
runs-on: ubuntu-latest # Use the latest Ubuntu runner

steps:
- uses: actions/checkout@v4 # Checkout the repository

# Set up Node.js environment for Frontend
- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: '20' # Set the Node.js version

- name: Cache npm dependencies
uses: actions/cache@v3
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-

- name: Navigate to frontend directory
run: cd frontend

- name: Install Frontend Dependencies
run: |
cd frontend # Change to the frontend directory
npm install # Install dependencies from frontend/package.json

- name: Run Frontend Tests with Coverage
run: |
cd frontend # Change to the frontend directory
if npm run test -- --coverage; then
echo "Tests completed, checking coverage."
# Check coverage report and ensure it meets threshold
if [ -f frontend/coverage/lcov-report/index.html ]; then
COVERAGE=$(cat frontend/coverage/lcov-report/index.html | grep -oP 'total: \K[0-9]+(\.[0-9]+)?')
if (( $(echo "$COVERAGE < 80" | bc -l) )); then
echo "Coverage is below 80%, failing the job."
exit 1
fi
fi
else
echo "No tests found, skipping coverage check."
fi

- uses: actions/upload-artifact@v4
with:
name: frontend-coverage
path: |
frontend/coverage/ # Correct path to frontend coverage
frontend/coverage/lcov-report/ # Correct path to frontend lcov report
Loading