-
Notifications
You must be signed in to change notification settings - Fork 149
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
7 changed files
with
346 additions
and
4 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
104 changes: 104 additions & 0 deletions
104
docs/learning/solutions/cost-management/aws-list-unused-lambda.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| # AWS - Identify Unused Lambda Functions | ||
|
|
||
| ## Description | ||
|
|
||
| This automation job generates a listing of AWS Lambda functions and highlights any that may be eligible for deletion based on modification and execution dates provided as job inputs. It provides a detailed report of Lambda functions, including their last modified and last execution dates, and recommends whether to keep or delete each function. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - Turn on "[Runner as Node](/administration/runner/runner-management/node-dispatch.html#runner-as-a-node)" setting on your Runner. | ||
| - This requires version 5.8.0 or higher. Adjustments to Node tab may be required for earlier versions. | ||
| - AWS CLI installed on the runner node | ||
| - jq tool for JSON parsing installed on the runner node | ||
| - Proper AWS credentials configured on the runner node | ||
|
|
||
| ## AWS IAM Permissions | ||
|
|
||
| The AWS IAM role or user associated with this job requires the following permissions: | ||
|
|
||
| - `lambda:ListFunctions` | ||
| - `logs:DescribeLogGroups` | ||
| - `logs:DescribeLogStreams` | ||
|
|
||
| These permissions should be applied to all resources (`"Resource": "*"`). | ||
|
|
||
| ```json | ||
| { | ||
| "Version": "2012-10-17", | ||
| "Statement": [ | ||
| { | ||
| "Effect": "Allow", | ||
| "Action": [ | ||
| "lambda:ListFunctions", | ||
| "logs:DescribeLogGroups", | ||
| "logs:DescribeLogStreams" | ||
| ], | ||
| "Resource": "*" | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| ## Job Options | ||
|
|
||
| | Option Name | Description | Default Value | | ||
| |------------------|-----------------------------------------------------------|---------------| | ||
| | `Region` | AWS region to query for Lambda functions | N/A | | ||
| | `Execution Date` | List functions that have not been called since this date | N/A | | ||
| | `Modified Date` | List functions older than this date | N/A | | ||
|
|
||
|
|
||
| ## Job Workflow | ||
|
|
||
| 1. The job runs on a node with the tag "RUNNER" | ||
| 2. It uses the AWS CLI to list all Lambda functions in the specified region | ||
| 3. For each function, it retrieves: | ||
| - The last modified date | ||
| - The last execution date (from CloudWatch Logs) | ||
| 4. It compares these dates against the provided execution and modification thresholds | ||
| 5. The job generates a report for each function, including: | ||
| - Function name | ||
| - Last modified date | ||
| - Last execution date | ||
| - Recommendation to keep or delete the function | ||
|
|
||
| ## Output | ||
|
|
||
| The job produces a detailed report with the following information for each Lambda function: | ||
|
|
||
| - Function name | ||
| - Last modified date | ||
| - Last execution date | ||
| - Recommendation: "Delete" or "Keep" | ||
|
|
||
| The recommendation output is color-coded for easy reading: | ||
| - Red background: Functions recommended for deletion | ||
| - Green background: Functions recommended to keep | ||
|
|
||
| ## Script Details | ||
|
|
||
| The job uses a Bash script to perform the following tasks: | ||
|
|
||
| 1. Set up variables for the AWS region and date thresholds | ||
| 2. Convert input dates to Unix timestamps and ISO 8601 format | ||
| 3. List all Lambda functions in the specified region | ||
| 4. For each function: | ||
| - Retrieve the last modified date | ||
| - Check for associated CloudWatch Logs | ||
| - Retrieve the last execution date from logs (if available) | ||
| - Compare dates against thresholds | ||
| - Generate a recommendation | ||
|
|
||
| ## Notes | ||
|
|
||
| - The job does not actually delete any functions; it only provides recommendations | ||
| - Functions are recommended for deletion if both the last modified date and the last execution date are earlier than the provided thresholds | ||
| - If a function has no associated CloudWatch Logs, its last execution date will be shown as "No logs found" | ||
| - If a function has logs but no executions, its last execution date will be shown as "No execution found" | ||
| - The script is designed to work on both Linux and macOS systems | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| If you encounter issues running this job: | ||
| 1. Ensure that the AWS CLI and jq are properly installed on the runner node | ||
| 2. Verify that the AWS credentials on the runner node have the necessary permissions |
98 changes: 98 additions & 0 deletions
98
docs/learning/solutions/cost-management/aws-list-unused-securitygroups.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| # AWS - Identify Unused Security Groups | ||
|
|
||
| ## Description | ||
|
|
||
| This automation job generates a listing of AWS security groups that are not associated with any network interfaces and are therefore eligible for deletion. It checks various AWS services to ensure comprehensive coverage. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - Turn on "[Runner as Node](/administration/runner/runner-management/node-dispatch.html#runner-as-a-node)" setting on your Runner. | ||
| - This requires version 5.8.0 or higher. Adjustments to Node tab may be required for earlier versions. | ||
| - AWS CLI installed on the runner node. | ||
| - Proper AWS credentials configured on the runner node. | ||
|
|
||
| ## AWS IAM Permissions | ||
|
|
||
| The AWS IAM role or user associated with this job requires the following permissions: | ||
|
|
||
| - `ec2:DescribeSecurityGroups` | ||
| - `ec2:DescribeNetworkInterfaces` | ||
| - `elb:DescribeLoadBalancers` | ||
| - `elbv2:DescribeLoadBalancers` | ||
| - `rds:DescribeDBInstances` | ||
| - `elasticache:DescribeCacheClusters` | ||
| - `redshift:DescribeClusters` | ||
|
|
||
| These permissions should be applied to all resources in the specified region. | ||
|
|
||
| ```json | ||
| { | ||
| "Version": "2012-10-17", | ||
| "Statement": [ | ||
| { | ||
| "Effect": "Allow", | ||
| "Action": [ | ||
| "ec2:DescribeSecurityGroups", | ||
| "ec2:DescribeNetworkInterfaces", | ||
| "elb:DescribeLoadBalancers", | ||
| "elbv2:DescribeLoadBalancers", | ||
| "rds:DescribeDBInstances", | ||
| "elasticache:DescribeCacheClusters", | ||
| "redshift:DescribeClusters" | ||
| ], | ||
| "Resource": "*" | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| ## Job Options | ||
|
|
||
| | Option Name | Description | Default Value | | ||
| |-------------|-------------|---------------| | ||
| | `region` | AWS region to query for security groups | N/A | | ||
| | `always-show-results` | Show results even when checking AWS services results in Access Errors | false | | ||
|
|
||
| ## Job Workflow | ||
|
|
||
| 1. It uses the AWS CLI to list all security groups in the specified region. | ||
| 2. The script then checks for security groups associated with: | ||
| - Network interfaces | ||
| - Classic load balancers | ||
| - Application/Network load balancers | ||
| - RDS instances | ||
| - ElastiCache clusters | ||
| - Redshift clusters | ||
| 3. It compares the list of all security groups against those associated with the above services. | ||
| 4. The job generates a report of security groups that are not associated with any of these services and are eligible for deletion. | ||
|
|
||
| ## Output | ||
|
|
||
| The job produces a detailed report with the following information: | ||
|
|
||
| - List of all security groups in the region | ||
| - List of security groups associated with various AWS services | ||
| - Security groups that can be safely deleted (not associated with any service) | ||
| - Warnings for default security groups (which cannot be deleted) | ||
|
|
||
| ## Script Details | ||
|
|
||
| The job uses a Bash script to perform the following tasks: | ||
|
|
||
| 1. Fetch all security groups in the specified region | ||
| 2. Retrieve security groups associated with various AWS services | ||
| 3. Compare the lists to identify unused security groups | ||
| 4. Generate a report of security groups eligible for deletion | ||
|
|
||
| ## Notes | ||
|
|
||
| - The job does not actually delete any security groups; it only provides recommendations. | ||
| - Default security groups are excluded from the deletion recommendations. | ||
| - The script includes error handling and can optionally show the recommendation results even if some AWS API calls result in errors. | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| If you encounter issues running this job: | ||
| 1. Ensure that the AWS CLI is properly installed on the runner node | ||
| 2. Verify that the AWS credentials on the runner node have the necessary permissions | ||
| 3. Check the `always-show-results` option if you want to see partial results in case of API errors |
99 changes: 99 additions & 0 deletions
99
docs/learning/solutions/cost-management/aws-list-unused-vpcs.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| # AWS - Identify Unused VPCs | ||
|
|
||
| ## Overview | ||
|
|
||
| This job is designed to identify unused Virtual Private Clouds (VPCs) in a specified AWS region. It helps in cost management and resource optimization by highlighting VPCs that are not associated with any active AWS resources and may be candidates for deletion. | ||
|
|
||
| ## Functionality | ||
|
|
||
| The job performs the following tasks: | ||
|
|
||
| 1. Retrieves a list of all VPCs in the specified AWS region. | ||
| 2. Identifies the default VPC and excludes it from further processing. | ||
| 3. Checks each non-default VPC for associations with various AWS resources, including: | ||
| - EC2 instances | ||
| - RDS instances | ||
| - Classic Load Balancers (ELB) | ||
| - Application and Network Load Balancers (ALB/NLB) | ||
| - NAT Gateways | ||
| - VPN Connections | ||
| - Transit Gateway attachments | ||
| 4. Compiles a list of VPCs that are not associated with any of the above resources. | ||
| 5. Outputs a list of VPCs that can potentially be deleted. | ||
|
|
||
| ## Setup | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| - Turn on "[Runner as Node](/administration/runner/runner-management/node-dispatch.html#runner-as-a-node)" setting on your Runner. | ||
| - This requires version 5.8.0 or higher. Adjustments to Node tab may be required for earlier versions. | ||
| - AWS CLI installed on the Enterprise Runner node. | ||
| - Appropriate AWS IAM permissions (see below). | ||
|
|
||
| ### Job Configuration | ||
|
|
||
| 1. **Node Filter**: The job is configured to run on nodes tagged with "RUNNER". | ||
| 2. **Options**: | ||
| - `region`: AWS Region (required, uses aws-regions-job-options plugin) | ||
| - `always-show-results`: Show results even after an error (true/false) | ||
| 3. **Execution**: The job runs a bash script that utilizes AWS CLI commands. | ||
|
|
||
| ## AWS IAM Permissions | ||
|
|
||
| The IAM role or user executing this job needs the following permissions: | ||
|
|
||
| ```json | ||
| { | ||
| "Version": "2012-10-17", | ||
| "Statement": [ | ||
| { | ||
| "Effect": "Allow", | ||
| "Action": [ | ||
| "ec2:DescribeVpcs", | ||
| "ec2:DescribeInstances", | ||
| "ec2:DescribeNatGateways", | ||
| "ec2:DescribeVpnConnections", | ||
| "ec2:DescribeTransitGatewayVpcAttachments", | ||
| "rds:DescribeDBInstances", | ||
| "elasticloadbalancing:DescribeLoadBalancers" | ||
| ], | ||
| "Resource": "*" | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| These permissions allow the script to describe various AWS resources across different services to determine VPC usage. | ||
|
|
||
| ## Running the Job | ||
| 1. Select the job in the Cost Management / AWS folder. | ||
| 2. Choose the target AWS region from the dropdown. | ||
| 3. Set the always-show-results option as needed. | ||
| 4. Execute the job. | ||
|
|
||
| ## Output | ||
|
|
||
| The job will provide: | ||
|
|
||
| - A list of all VPCs in the region. | ||
| - Information about associated resources for each VPC. | ||
| - A list of VPCs that appear to be unused and can potentially be deleted. | ||
|
|
||
| ## Important Notes | ||
|
|
||
| - The job does not automatically delete any VPCs; it only identifies potential candidates for deletion. | ||
| - Always verify the results manually before deleting any VPC. | ||
| - The default VPC is automatically excluded from the list of deletable VPCs. | ||
| - If errors occur during execution, the job can be configured to show partial results. | ||
|
|
||
| ## Troubleshooting | ||
| - Turn on "[Runner as Node](/administration/runner/runner-management/node-dispatch.html#runner-as-a-node)" setting on your Runner. This requires version 5.8.0 or higher. Adjustments to Node tab may be required for earlier versions. | ||
| - Ensure the Enterprise Runner node has the AWS CLI properly configured. There are helper jobs in the _Getting Started_ folder of the project. | ||
| - Verify that the IAM role or user has the necessary permissions. (See above) | ||
| - Check execution logs for any execution errors. | ||
|
|
||
|
|
||
| ## Security Considerations | ||
| - Follow the principle of least privilege when assigning IAM permissions. | ||
| - Regularly review and update the permissions as needed. | ||
| - Ensure that sensitive information, like AWS credentials, are securely managed. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # Cost Management | ||
|
|
||
| ### Overview | ||
|
|
||
| Leveraging Runbook Automation for Cost Management tasks in cloud environments can significantly improve efficiency and reduce operational costs. Such tools allow teams to create, schedule, and manage complex workflows across multiple cloud platforms, ensuring consistent execution of cost-saving measures. By automating these tasks, organizations can minimize human error, save time, and maintain better control over their cloud resources. Regular, automated cleanup processes help prevent unnecessary charges from idle or forgotten resources, optimize infrastructure usage, and enforce cost governance policies. Additionally, automation tools often provide role-based access control and audit trails, enhancing security and compliance. This makes them invaluable for organizations looking to streamline their cloud cost management efforts, regardless of the specific cloud provider or infrastructure setup. | ||
|
|
||
| ### Use Cases | ||
|
|
||
| - **Identify and Remove Unused Resources**: Identifying and removing unused resources in cloud solutions saves money by eliminating unnecessary costs associated with idle or forgotten assets that continue to incur charges without providing any value to the organization. | ||
| - **Right-size Computing Resources**: Automated resizing of compute resources in cloud solutions optimizes costs by dynamically adjusting capacity to match actual workload demands, ensuring you're not overpaying for underutilized resources or suffering performance issues due to undersized instances. | ||
|
|
||
|
|
||
| ### Prebuilt Automation | ||
| PagerDuty provides a solution that helps users start automating diagnostics quickly. This Solution consists of **prebuilt Automation Jobs** that show how the use cases above can be implemented in your environment. (Note: Some of these solutions may exist in other Solution Packages.) | ||
|
|
||
|
|
||
| | Examples | | ||
| | --- | | ||
| | <img src="/assets/img/aws-logo.png" width="30" height="30"> [AWS - Identify Unused VPCs](/learning/solutions/cost-management/aws-list-unused-vpcs.md) | | ||
| | <img src="/assets/img/aws-logo.png" width="30" height="30"> [AWS - Identify Unused Lambda Functions](/learning/solutions/cost-management/aws-list-unused-lambda.md) | | ||
| | <img src="/assets/img/aws-logo.png" width="30" height="30"> [AWS - Identify Unused Security Groups](/learning/solutions/cost-management/aws-list-unused-securitygroups.md) | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters