You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I must reinstall my server and now im getting in 80% of the runs the following error.
this is a fresh docker installation, have someone the same Problems?
i crawl "Kleinanzeigen" and "immobilienscout24"
[2024/02/24 15:05:57|chrome_wrapper.py |INFO ]: Initializing Chrome WebDriver for crawler...
[2024/02/24 15:05:58|patcher.py |INFO ]: patching driver executable /root/.local/share/undetected_chromedriver/undetected_chromedriver
[2024/02/24 15:05:59|__init__.py |INFO ]: setting properties for headless
Traceback (most recent call last):
File "/usr/src/app/flathunt.py", line 99, in <module>
main()
File "/usr/src/app/flathunt.py", line 95, in main
launch_flat_hunt(config, heartbeat)
File "/usr/src/app/flathunt.py", line 35, in launch_flat_hunt
hunter.hunt_flats()
File "/usr/src/app/flathunter/hunter.py", line 56, in hunt_flats
for expose in processor_chain.process(self.crawl_for_exposes(max_pages)):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/flathunter/hunter.py", line 35, in crawl_for_exposes
return chain(*[try_crawl(searcher, url, max_pages)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/flathunter/hunter.py", line 35, in <listcomp>
return chain(*[try_crawl(searcher, url, max_pages)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/flathunter/hunter.py", line 27, in try_crawl
return searcher.crawl(url, max_pages)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/flathunter/abstract_crawler.py", line 151, in crawl
return self.get_results(url, max_pages)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/flathunter/abstract_crawler.py", line 139, in get_results
soup = self.get_page(search_url)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/flathunter/crawler/kleinanzeigen.py", line 56, in get_page
return self.get_soup_from_url(search_url, driver=self.get_driver())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/backoff/_sync.py", line 105, in retry
ret = target(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/flathunter/abstract_crawler.py", line 70, in get_soup_from_url
driver.get(url)
File "/usr/local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 629, in get_wrapped
return orig_get(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 665, in get
return super().get(url)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 356, in get
self.execute(Command.GET, {"url": url})
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
(Session info: chrome=122.0.6261.57)
Stacktrace:
#0 0x561cd54b1793 <unknown>
#1 0x561cd51a5017 <unknown>
#2 0x561cd518dca2 <unknown>
#3 0x561cd518d51f <unknown>
#4 0x561cd518c3e9 <unknown>
#5 0x561cd518c274 <unknown>
#6 0x561cd518acf4 <unknown>
#7 0x561cd518b3ff <unknown>
#8 0x561cd519b665 <unknown>
#9 0x561cd51b0d8c <unknown>
#10 0x561cd51b636b <unknown>
#11 0x561cd518ba8e <unknown>
#12 0x561cd51b0986 <unknown>
#13 0x561cd5231131 <unknown>
#14 0x561cd5212173 <unknown>
#15 0x561cd51e32d3 <unknown>
#16 0x561cd51e3c9e <unknown>
#17 0x561cd54758cb <unknown>
#18 0x561cd5479745 <unknown>
#19 0x561cd54622e1 <unknown>
#20 0x561cd547a2d2 <unknown>
#21 0x561cd544617f <unknown>
#22 0x561cd549fdc8 <unknown>
#23 0x561cd549ffc3 <unknown>
#24 0x561cd54b0944 <unknown>
#25 0x7fc668fce134 <unknown>
[2024/02/24 15:06:01|__init__.py |INFO ]: ensuring close
The text was updated successfully, but these errors were encountered:
I must reinstall my server and now im getting in 80% of the runs the following error.
this is a fresh docker installation, have someone the same Problems?
i crawl "Kleinanzeigen" and "immobilienscout24"
The text was updated successfully, but these errors were encountered: