Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request to IMDS stucks forever when using MSIAuthentication #46

Open
jwxd opened this issue Feb 15, 2022 · 0 comments
Open

Request to IMDS stucks forever when using MSIAuthentication #46

jwxd opened this issue Feb 15, 2022 · 0 comments

Comments

@jwxd
Copy link

jwxd commented Feb 15, 2022

In rare occasion, init container stucks at Init stage indefinitely because MSIAuthentication library makes a http request to IMDS and the connection hangs. The MSIAuthentication library does not specified timeout in the actual request, it will stuck forever when that happen.
Although it's rare but it becomes a lot more likely to happen given the scale of our system.
The problem is at this line of code
The timeout argument provided by the library doesn't work either, because it's not used on the http request level
Related issue: Azure/msrestazure-for-python#148

https://github.com/Azure/msrestazure-for-python/blob/master/msrestazure/azure_active_directory.py
I tried to make a PR but this library has been deprecated and will not accept new feature.
and it is replaced by azure-identity.
https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/identity/azure-identity
We should migrate the current implementation to use azure-identity ASAP.

jwxd pushed a commit to jwxd/acs-keyvault-agent that referenced this issue Mar 1, 2022
This is a fork of open source project acs-keyvault-agent.
Addressing issue in Hexadite#46

Updating the code to migrate to use azure-identity
Azure-identity sets a default timeout to 2sec when making http request to IMDS.

The changes has been tested and verify working

Related work items: #38164308
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant