-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add debugging steps for DiscoverEC2 User Task issues (#47513)
* Add debugging steps for DiscoverEC2 User Task issues This PR adds a new field when getting an User Task. Description contains a markdown document that should be displayed to the user which helps them fix the issue with the task. * move descriptions from api to lib
- Loading branch information
1 parent
f50547e
commit 22ce158
Showing
9 changed files
with
156 additions
and
5 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
/* | ||
* Teleport | ||
* Copyright (C) 2024 Gravitational, Inc. | ||
* | ||
* This program is free software: you can redistribute it and/or modify | ||
* it under the terms of the GNU Affero General Public License as published by | ||
* the Free Software Foundation, either version 3 of the License, or | ||
* (at your option) any later version. | ||
* | ||
* This program is distributed in the hope that it will be useful, | ||
* but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
* GNU Affero General Public License for more details. | ||
* | ||
* You should have received a copy of the GNU Affero General Public License | ||
* along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
*/ | ||
|
||
package usertasks | ||
|
||
import ( | ||
"embed" | ||
"fmt" | ||
) | ||
|
||
//go:embed descriptions/*.md | ||
var descriptionsFS embed.FS | ||
|
||
// DescriptionForDiscoverEC2Issue returns the description of the issue and fixing steps. | ||
// The returned string contains a markdown document. | ||
// If issue type is not recognized or doesn't have a specific description, them an empty string is returned. | ||
func DescriptionForDiscoverEC2Issue(issueType string) string { | ||
filename := fmt.Sprintf("descriptions/%s.md", issueType) | ||
bs, err := descriptionsFS.ReadFile(filename) | ||
if err != nil { | ||
return "" | ||
} | ||
return string(bs) | ||
} |
25 changes: 25 additions & 0 deletions
25
lib/usertasks/descriptions/ec2-ssm-agent-connection-lost.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
Auto enrolling EC2 instances requires the SSM Agent to be installed and running on them. | ||
Some instances appear to have lost connection to Amazon Systems Manager. | ||
|
||
You can see which instances lost connection using the [SSM Fleet Manager](https://console.aws.amazon.com/systems-manager/fleet-manager/managed-nodes). | ||
|
||
The most common issues for instances losing connection: | ||
|
||
**SSM Agent is not running** | ||
|
||
Ensure the SSM Agent is running in the instance and is not reporting any error. | ||
Please check the instructions [here](https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-status-and-restart.html). | ||
|
||
**SSM Agent can't reach the Amazon Systems Manager service** | ||
|
||
Ensure the instance's security groups allows outbound connections to Amazon Systems Manager endpoints. | ||
Allowing outbound on port 443 is enough for the agent to connect to AWS. | ||
|
||
**Instance is missing IAM policy** | ||
|
||
The SSM Agent requires the `AmazonSSMManagedInstanceCore` managed policy. | ||
Ensure the instance has an IAM Profile and that it includes the above policy. | ||
For more information please refer to [this page](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-getting-started-instance-profile.html). | ||
|
||
After following the steps above, you can mark the task as resolved. | ||
Teleport will try to auto-enroll these instances again. |
25 changes: 25 additions & 0 deletions
25
lib/usertasks/descriptions/ec2-ssm-agent-not-registered.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
Auto enrolling EC2 instances requires the SSM Agent to be installed and running on them. | ||
Some instances failed to connect to Amazon Systems Manager. | ||
|
||
You can see which instances were able to connect by opening the [SSM Fleet Manager](https://console.aws.amazon.com/systems-manager/fleet-manager/managed-nodes). | ||
|
||
The most common issues for instances not being visible are: | ||
|
||
**SSM Agent is not running** | ||
|
||
Ensure the SSM Agent is installed and running in the instance. | ||
Please check the instructions [here](https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-status-and-restart.html). | ||
|
||
**SSM Agent can't reach the Amazon Systems Manager service** | ||
|
||
Ensure the instance's security groups allows outbound connections to Amazon Systems Manager endpoints. | ||
Allowing outbound on port 443 is enough for the agent to connect to AWS. | ||
|
||
**Instance is missing IAM policy** | ||
|
||
The SSM Agent requires the `AmazonSSMManagedInstanceCore` managed policy. | ||
Ensure the instance has an IAM Profile and that it includes the above policy. | ||
For more information please refer to [this page](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-getting-started-instance-profile.html). | ||
|
||
After following the steps above, you can mark the task as resolved. | ||
Teleport will try to auto-enroll these instances again. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
Teleport failed to access the SSM Agent to auto enroll the instance. | ||
Some instances failed to communicate with the AWS Systems Manager service to execute the install script. | ||
|
||
Usually this happens when: | ||
|
||
**Missing policies** | ||
|
||
The IAM Role used by the integration might be missing some required permissions. | ||
Ensure the following actions are allowed in the IAM Role used by the integration: | ||
- `ec2:DescribeInstances` | ||
- `ssm:DescribeInstanceInformation` | ||
- `ssm:GetCommandInvocation` | ||
- `ssm:ListCommandInvocations` | ||
- `ssm:SendCommand` | ||
|
||
**SSM Document is invalid** | ||
|
||
Teleport uses an SSM Document to run an installation script. | ||
If the document is changed or removed, it might no longer work. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
Teleport was able to reach the SSM Agent inside the EC2 instance, however the install script returned an error. | ||
|
||
You can click below in the Invocation URL and get further details on why the script failed. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
Auto enrolling EC2 instances requires a compatible Operating System. | ||
|
||
Teleport only supports Linux instances when auto-enrolling them into the cluster. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
/* | ||
* Teleport | ||
* Copyright (C) 2024 Gravitational, Inc. | ||
* | ||
* This program is free software: you can redistribute it and/or modify | ||
* it under the terms of the GNU Affero General Public License as published by | ||
* the Free Software Foundation, either version 3 of the License, or | ||
* (at your option) any later version. | ||
* | ||
* This program is distributed in the hope that it will be useful, | ||
* but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
* GNU Affero General Public License for more details. | ||
* | ||
* You should have received a copy of the GNU Affero General Public License | ||
* along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
*/ | ||
|
||
package usertasks | ||
|
||
import ( | ||
"testing" | ||
|
||
"github.com/stretchr/testify/require" | ||
|
||
usertasksapi "github.com/gravitational/teleport/api/types/usertasks" | ||
) | ||
|
||
func TestAllDescriptions(t *testing.T) { | ||
for _, issueType := range usertasksapi.DiscoverEC2IssueTypes { | ||
require.NotEmpty(t, DescriptionForDiscoverEC2Issue(issueType), "issue type %q is missing descriptions/%s.md file", issueType, issueType) | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters