`ReverseProxyPlugin` doesn't return a response to the client #1433

eaftan · 2024-07-23T21:08:20Z

Check FAQs
Please check Frequently Asked Questions
before opening a bug report.

Describe the bug

The included example ReverseProxyPlugin doesn't seem to work. It doesn't return any response to the client.

To Reproduce
Steps to reproduce the behavior:

In one terminal, run the following command: proxy --enable-reverse-proxy --plugins proxy.plugin.ReverseProxyPlugin
In another terminal, run this command to send a request: curl -v localhost:8899/get
The curl command never gets a response; it reports "Empty reply from server"

Here are the outputs I see on my machine when I do this.

Proxy

% proxy --enable-reverse-proxy --plugins proxy.plugin.ReverseProxyPlugin
2024-07-23 14:02:31,090 - pid:2514 [I] plugins.load:89 - Loaded plugin proxy.http.proxy.HttpProxyPlugin
2024-07-23 14:02:31,091 - pid:2514 [I] plugins.load:89 - Loaded plugin proxy.http.server.HttpWebServerPlugin
2024-07-23 14:02:31,094 - pid:2514 [I] plugins.load:89 - Loaded plugin proxy.http.server.reverse.ReverseProxy
2024-07-23 14:02:31,094 - pid:2514 [I] plugins.load:89 - Loaded plugin proxy.plugin.ReverseProxyPlugin
2024-07-23 14:02:43,471 - pid:2519 [I] reverse.on_access_log:168 - 127.0.0.1:65505 - GET /get - curl/8.6.0 -> None - 10198.24ms

Curl

% curl -v localhost:8899/get
* Host localhost:8899 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:8899...
* connect to ::1 port 8899 from ::1 port 65504 failed: Connection refused
*   Trying 127.0.0.1:8899...
* Connected to localhost (127.0.0.1) port 8899
> GET /get HTTP/1.1
> Host: localhost:8899
> User-Agent: curl/8.6.0
> Accept: */*
> 
* Empty reply from server
* Closing connection
curl: (52) Empty reply from server

Expected behavior
I expect the output that httpbin.org/get returns:

{
  "args": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "httpbin.org", 
    "User-Agent": "curl/8.6.0", 
    "X-Amzn-Trace-Id": "Root=1-66a01aff-7ebaa35e304be47c40746c5b"
  }, 
  "origin": "24.7.24.238", 
  "url": "http://httpbin.org/get"
}

Version information
I see this behavior on two different machines.

OS: MacOS 14.5 and Ubuntu Linux 20.04.1
Browser: N/A
Device: M1 Macbook Pro, an Azure VM for Ubuntu
proxy.py Version: 2.4.5.dev9+ga7077cf

Additional context
Add any other context about the problem here.

Screenshots
If applicable, add screenshots to help explain your problem.

The text was updated successfully, but these errors were encountered:

abhinavsingh · 2024-07-28T04:54:34Z

@eaftan Thank you very much for reporting this. In last release, I pushed a bug while adding support for dynamic routes. I went to trace back the changes and looks like I even modified tests to make it work thinking change was just a side effect of changing httpbin to httpbingo 🤦‍♂️

abhinavsingh · 2024-07-28T05:16:40Z

Looks into the tests a bit further, looks like there is an issue with httpbingo (just like httpbin) when using from GitHub workflows. I'll investigate into tests later, but for now reverse proxy with static routes must work fine. I'll make a new release early coming week.

eaftan · 2024-07-30T13:48:51Z

Thank you! 🙏

creaexo · 2024-09-18T14:53:48Z

Hello, @abhinavsingh!
I have the same problem as described above.

To Reproduce

Steps to reproduce the behavior:

Install in a new virtual environment proxy.py==2.4.7
In one terminal, run the following command: .venv3/bin/proxy --enable-reverse-proxy --plugins proxy.plugin.ReverseProxyPlugin
In another terminal, run this command to send a request: curl -v localhost:8899/get
The curl command never gets a response; it reports "Empty reply from server"

Proxy (I've shortened the error tracing a bit)

.venv3/bin/proxy --enable-reverse-proxy --plugins proxy.plugin.ReverseProxyPlugin
2024-09-18 19:36:35,182 - pid:115447 [I] plugins.load:89 - Loaded plugin proxy.http.proxy.HttpProxyPlugin
2024-09-18 19:36:35,182 - pid:115447 [I] plugins.load:89 - Loaded plugin proxy.http.server.HttpWebServerPlugin
2024-09-18 19:36:35,182 - pid:115447 [I] plugins.load:89 - Loaded plugin proxy.http.server.reverse.ReverseProxy
2024-09-18 19:36:35,182 - pid:115447 [I] plugins.load:89 - Loaded plugin proxy.plugin.ReverseProxyPlugin
2024-09-18 19:36:41,471 - pid:115454 [W] handler.handle_readables:240 - Exception when receiving from client 
"....................................................LONG ERROR TRACE................................................................."
    self.upstream.wrap(
  File "/home/path_to_my_project/proxer/.venv3/lib/python3.11/site-packages/proxy/core/connection/server.py", line 55, in wrap
    ctx = ssl.create_default_context(ssl.Purpose.SERVER_AUTH, cafile=ca_file)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 770, in create_default_context
    context.load_verify_locations(cafile, capath, cadata)
FileNotFoundError: [Errno 2] No such file or directory
2024-09-18 19:36:41,472 - pid:115454 [I] reverse.on_access_log:169 - 127.0.0.1:38500 - GET /get - curl/7.81.0 -> None - 371.14ms

Curl

curl -v localhost:8899/get
*   Trying 127.0.0.1:8899...
* Connected to localhost (127.0.0.1) port 8899 (#0)
> GET /get HTTP/1.1
> Host: localhost:8899
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Empty reply from server
* Closing connection 0
curl: (52) Empty reply from server

Version information

OS: Ubuntu 22.04.4 LTS
Browser: N/A
proxy.py Version: 2.4.5, 2.4.7

abhinavsingh · 2024-09-18T15:31:40Z

@creaexo Looks like Ubuntu doesn't have necessary CA certificates installed. Try:

pip install certifi

If above alone does not work, also try:

sudo apt-get install ca-certificates

Please let me know.

creaexo · 2024-09-19T07:09:22Z

Thank you for your reply! After the above recommendations, a successful response began to come alternating with an unsuccessful one. Why do you think not every answer is successful?

Proxy

.venv3/bin/proxy --enable-reverse-proxy --plugins proxy.plugin.ReverseProxyPlugin
2024-09-19 11:52:31,136 - pid:120607 [I] plugins.load:89 - Loaded plugin proxy.http.proxy.HttpProxyPlugin
2024-09-19 11:52:31,136 - pid:120607 [I] plugins.load:89 - Loaded plugin proxy.http.server.HttpWebServerPlugin
2024-09-19 11:52:31,136 - pid:120607 [I] plugins.load:89 - Loaded plugin proxy.http.server.reverse.ReverseProxy
2024-09-19 11:52:31,136 - pid:120607 [I] plugins.load:89 - Loaded plugin proxy.plugin.ReverseProxyPlugin
2024-09-19 11:52:34,166 - pid:120614 [I] reverse.on_access_log:169 - 127.0.0.1:38808 - GET /get - curl/7.81.0 -> None - 74.46ms
2024-09-19 11:52:38,290 - pid:120614 [I] tcp_upstream.read_from_descriptors:87 - Upstream SSLWantReadError, will retry
2024-09-19 11:52:38,550 - pid:120614 [I] reverse.on_access_log:169 - 127.0.0.1:38820 - GET /get - curl/7.81.0 -> None - 408.62ms

Two curl requests are executed, one after the other

curl -v localhost:8899/get

*   Trying 127.0.0.1:8899...
* Connected to localhost (127.0.0.1) port 8899 (#0)
> GET /get HTTP/1.1
> Host: localhost:8899
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Empty reply from server
* Closing connection 0
curl: (52) Empty reply from server


curl -v localhost:8899/get

*   Trying 127.0.0.1:8899...
* Connected to localhost (127.0.0.1) port 8899 (#0)
> GET /get HTTP/1.1
> Host: localhost:8899
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< access-control-allow-credentials: true
< access-control-allow-origin: *
< content-type: application/json; charset=utf-8
< date: Thu, 19 Sep 2024 06:52:38 GMT
< content-length: 555
< server: Fly/a5e9a2096 (2024-09-17)
< via: 1.1 fly.io
< fly-request-id: 01J84GVS02PZ98XB43E9335WJ5-waw
< 
{
  "args": {},
  "headers": {
    "Accept": [
      "*/*"
    ],
    "Host": [
      "localhost:8899"
    ],
    "User-Agent": [
      "curl/7.81.0"
    ],
    "Via": [
      "1.1 fly.io"
    ],
    "X-Forwarded-For": [
      "my_ip, my_ip"
    ],
    "X-Forwarded-Port": [
      "443"
    ],
    "X-Forwarded-Proto": [
      "https"
    ],
    "X-Forwarded-Ssl": [
      "on"
    ],
    "X-Request-Start": [
      "t=1726728758274304"
    ]
  },
  "method": "GET",
  "origin": "my_ip",
  "url": "https://localhost:8899/get"
}
* Connection #0 to host localhost left intact

creaexo · 2024-09-19T07:31:47Z

@abhinavsingh, also I have problem with standard ReverseProxyPlugin.handle_route, which must redirect http://localhost:8899/get/1 to http://httpbingo.org/get?id=1.

Proxy

.venv3/bin/proxy --enable-reverse-proxy --plugins proxy.plugin.ReverseProxyPlugin
2024-09-19 12:20:01,701 - pid:122607 [I] plugins.load:89 - Loaded plugin proxy.http.proxy.HttpProxyPlugin
2024-09-19 12:20:01,702 - pid:122607 [I] plugins.load:89 - Loaded plugin proxy.http.server.HttpWebServerPlugin
2024-09-19 12:20:01,702 - pid:122607 [I] plugins.load:89 - Loaded plugin proxy.http.server.reverse.ReverseProxy
2024-09-19 12:20:01,702 - pid:122607 [I] plugins.load:89 - Loaded plugin proxy.plugin.ReverseProxyPlugin
2024-09-19 12:20:17,308 - pid:122611 [I] reverse.on_access_log:169 - 127.0.0.1:33674 - GET /get?id=1 - curl/7.81.0 -> http://httpbingo.org/get?id=1 - 79.78ms
2024-09-19 12:20:25,383 - pid:122618 [I] reverse.on_access_log:169 - 127.0.0.1:58530 - GET /get?id=1 - curl/7.81.0 -> http://httpbingo.org/get?id=1 - 81.17ms
2024-09-19 12:20:27,641 - pid:122619 [I] reverse.on_access_log:169 - 127.0.0.1:58542 - GET /get?id=1 - curl/7.81.0 -> http://httpbingo.org/get?id=1 - 74.19ms

Thre curl requests

curl -v http://127.0.0.1:8899/get/1                                     
*   Trying 127.0.0.1:8899...
* Connected to 127.0.0.1 (127.0.0.1) port 8899 (#0)
> GET /get/1 HTTP/1.1
> Host: 127.0.0.1:8899
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Empty reply from server
* Closing connection 0
curl: (52) Empty reply from server

curl -v http://127.0.0.1:8899/get/1
*   Trying 127.0.0.1:8899...
* Connected to 127.0.0.1 (127.0.0.1) port 8899 (#0)
> GET /get/1 HTTP/1.1
> Host: 127.0.0.1:8899
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Empty reply from server
* Closing connection 0
curl: (52) Empty reply from server

curl -v http://127.0.0.1:8899/get/1
*   Trying 127.0.0.1:8899...
* Connected to 127.0.0.1 (127.0.0.1) port 8899 (#0)
> GET /get/1 HTTP/1.1
> Host: 127.0.0.1:8899
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Empty reply from server
* Closing connection 0
curl: (52) Empty reply from server

abhinavsingh · 2024-09-22T12:06:42Z

Unsure what the issue might be at your end, but I am unable to reproduce the empty reply from server on my Mac laptop. I tried about 10-15 hits and got response for all the requests.

I see in your logs you encountered Upstream SSLWantReadError, will retry, are you seeing it repeatedly every now and then?

creaexo · 2024-09-24T05:56:38Z

Yes, I regularly see the message Upstream SSLWantReadError, will retry, when using ReverseProxyPlugin

creaexo · 2024-09-24T08:00:11Z

After an unsuccessful attempt to implement a redirect scenario with http://127.0.0.1:8899/get/1, on https://httpbingo.org/get?id=1 in the browser, I decided to add prints to the synchronous methods proxy/plugin/reverse_proxy.ReverseProxy.
When I try run http://localhost:8899/get in browser and take expected result, returns this methods names:

write_to_descriptors
read_from_descriptors
handle_upstream_data
write_to_descriptors
read_from_descriptors
handle_upstream_data
write_to_descriptors
read_from_descriptors
write_to_descriptors
read_from_descriptors
on_client_connection_close
on_access_log

When I try run http://localhost:8899/get/1 in browser and always take unexpected result (empty response), returns this methods names:

routes
handle_request
on_client_connection_close
on_access_log

Also I haven't any exceptions.

Maybe this will be a clue...

abhinavsingh · 2024-10-15T04:36:31Z

Please use --rewrite-host-header flag if you are using a http upstream. See latest commit #1492

eaftan added the Bug Bug report in proxy server label Jul 23, 2024

eaftan assigned abhinavsingh Jul 23, 2024

abhinavsingh mentioned this issue Jul 28, 2024

Static route reverse proxy always needs_upstream #1434

Merged

abhinavsingh mentioned this issue Jul 28, 2024

Update project test dependencies to ensure green workflow #1371

Merged

abhinavsingh closed this as completed Jul 28, 2024

abhinavsingh reopened this Oct 15, 2024

abhinavsingh closed this as completed Oct 15, 2024

abhinavsingh mentioned this issue Oct 15, 2024

Reverse proxy returns "Empty reply from server" #285

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`ReverseProxyPlugin` doesn't return a response to the client #1433

`ReverseProxyPlugin` doesn't return a response to the client #1433

eaftan commented Jul 23, 2024

abhinavsingh commented Jul 28, 2024

abhinavsingh commented Jul 28, 2024

eaftan commented Jul 30, 2024

creaexo commented Sep 18, 2024

abhinavsingh commented Sep 18, 2024

creaexo commented Sep 19, 2024

creaexo commented Sep 19, 2024

abhinavsingh commented Sep 22, 2024

creaexo commented Sep 24, 2024 •

edited

Loading

creaexo commented Sep 24, 2024

abhinavsingh commented Oct 15, 2024

ReverseProxyPlugin doesn't return a response to the client #1433

ReverseProxyPlugin doesn't return a response to the client #1433

Comments

eaftan commented Jul 23, 2024

abhinavsingh commented Jul 28, 2024

abhinavsingh commented Jul 28, 2024

eaftan commented Jul 30, 2024

creaexo commented Sep 18, 2024

To Reproduce

abhinavsingh commented Sep 18, 2024

creaexo commented Sep 19, 2024

creaexo commented Sep 19, 2024

abhinavsingh commented Sep 22, 2024

creaexo commented Sep 24, 2024 • edited Loading

creaexo commented Sep 24, 2024

abhinavsingh commented Oct 15, 2024

`ReverseProxyPlugin` doesn't return a response to the client #1433

`ReverseProxyPlugin` doesn't return a response to the client #1433

creaexo commented Sep 24, 2024 •

edited

Loading