Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incompatible http streaming protocol with python requests #16

Open
Maxersh opened this issue Mar 22, 2023 · 1 comment
Open

Incompatible http streaming protocol with python requests #16

Maxersh opened this issue Mar 22, 2023 · 1 comment

Comments

@Maxersh
Copy link

Maxersh commented Mar 22, 2023

How to reproduce the issue:

import requests

token = "token"
cluster_id = "cluster_id"
url = f"https://mdb.api.cloud.yandex.net/managed-postgresql/v1/clusters/{cluster_id}:stream_logs"

with requests.get(url,
                  params={"serviceType":"POSTGRESQL"},
                  headers={"Authorization":f"Bearer {token}"},
                  stream=True
                  ) as response:
    for chunk in response.iter_content(chunk_size=None, decode_unicode=True):
        print(chunk)

The above code outputs the logs as they come in. However, an exception like the one below occurs when a timeout occurs.

Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 748, in _update_chunk_length
self.chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 443, in _error_catcher
yield
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 815, in read_chunked
self._update_chunk_length()
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 752, in _update_chunk_length
raise InvalidChunkLength(self, line)
urllib3.exceptions.InvalidChunkLength: InvalidChunkLength(got length b'', 0 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/requests/models.py", line 760, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 623, in stream
for line in self.read_chunked(amt, decode_content=decode_content):
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 803, in read_chunked
with self._error_catcher():
File "/usr/lib64/python3.10/contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 460, in _error_catcher
raise ProtocolError("Connection broken: %r" % e, e)
urllib3.exceptions.ProtocolError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/user/scripts/test.py", line 12, in <module>
for ind, chunk in enumerate(response.iter_content(chunk_size=None, decode_unicode=True)):
File "/usr/lib/python3.10/site-packages/requests/utils.py", line 549, in stream_decode_response_unicode
for chunk in iterator:
File "/usr/lib/python3.10/site-packages/requests/models.py", line 763, in generate
raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

Apparently this only happens if any logs were output. If there are no logs, one message about stream timeout will be output and the program will end properly.
I found a discussion of a similar error. Maintainer sad it's not a bug. This is a faulty implementation of the protocol on the server side.

@nikolaymatrosov
Copy link

It would help if you tried to use the gRPC version of the API. As Cloud API is gRPC first and the REST version is just a gateway, the gRRC endpoint should work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants