Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

does JavaScript rendering works? It seems like not working #79

Open
pbrns opened this issue Dec 30, 2024 · 7 comments
Open

does JavaScript rendering works? It seems like not working #79

pbrns opened this issue Dec 30, 2024 · 7 comments

Comments

@pbrns
Copy link

pbrns commented Dec 30, 2024

I am trying to make a test in a specific site, that it seems that the page when loading, it returns a javascript instead of html.
Because the library mentions that has JavaScript rendering I thought that it will return me the html. But the html never loaded. It returns me just the js code.

@SSujitX
Copy link

SSujitX commented Dec 30, 2024

I am trying to make a test in a specific site, that it seems that the page when loading, it returns a javascript instead of html. Because the library mentions that has JavaScript rendering I thought that it will return me the html. But the html never loaded. It returns me just the js code.

pip install -U hrequests[all]

import hrequests
session = hrequests.Session()
resp = session.get(url)
page = resp.render()
print(page.html)

@pbrns
Copy link
Author

pbrns commented Dec 30, 2024

I am trying to make a test in a specific site, that it seems that the page when loading, it returns a javascript instead of html. Because the library mentions that has JavaScript rendering I thought that it will return me the html. But the html never loaded. It returns me just the js code.

pip install -U hrequests[all]

import hrequests
session = hrequests.Session()
resp = session.get(url)
page = resp.render()
print(page.html)

I already do the install with hrequests[all] but it doesn't work.
It seems that stuck in the render and don't get the html.
Can I post the url that I try here for testing?
(it seems that the specific page it returns javascript to load dynamic html)

@SSujitX
Copy link

SSujitX commented Dec 30, 2024

I already do the install with hrequests[all] but it doesn't work. It seems that stuck in the render and don't get the html. Can I post the url that I try here for testing? (it seems that the specific page it returns javascript to load dynamic HTML)

sure

@pbrns
Copy link
Author

pbrns commented Dec 30, 2024

I am trying to test it with this site

https://www.public.gr

@SSujitX
Copy link

SSujitX commented Dec 30, 2024

I am trying to test it with this site

https://www.public.gr

image

@pbrns
Copy link
Author

pbrns commented Dec 30, 2024

it gets some html, but not the inside content.
Example: https://www.public.gr/product/gaming/playstation/games/ps5/stellar-blade--ps5/1919895

If you go to this link you will see a known game.
If you took the html for this you will get only a js not the actual html that you see in browser

The same happens in the first main page. It get some basic elements but not the entire html that loaded dynamically

@pbrns
Copy link
Author

pbrns commented Dec 30, 2024

also I don't know if this should post it here. I already use the hrequests in some case that I want with a proxy.
It seems that
When I do this:

with hrequests.Session() as session:
    session.proxy = proxies['https'] 
    session.verify = False
    resp = session.get("https://ipinfo.io/json")
    print(resp.text)

it works.

When i try this:

with hrequests.Session() as session:
    session.proxy = proxies['https'] 
    session.verify = False
    resp = session.get("https://ipinfo.io/json")
    page = resp.render()
    print(page.text)

I am getting an error:

File "/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 528, in wrap_api_call
raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None
playwright._impl._errors.Error: Page.goto: NS_ERROR_PROXY_CONNECTION_REFUSED
Call log:

and the script don't finish.

I don't know if is relative to that, but the above problem appeared only when render called.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants