You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add more params to the web/spider worker to control the spider.
limit (Integer)
max_depth (Integer)
delay (Integer)
open_timeout (Integer)
read_timeout (Integer)
ssl_timeout (Integer)
continue_timeout (Integer)
keep_alive_timeout (Integer)
proxy (URI) (eventually we will want to define a global HTTP proxy option for all web/* workers).
referer (String)
user_agent (String) (eventually we will want to define a global User-Agent option for all web/* workers).
robots (Boolean)
The web/spider worker should also have a agent_kwargs method which builds a Symbol Hash based on the params, which is then passed to Ronin::Web::Spider.site(base_uri,**agent_kwargs).
The text was updated successfully, but these errors were encountered:
Add more
param
s to theweb/spider
worker to control the spider.limit
(Integer)max_depth
(Integer)delay
(Integer)open_timeout
(Integer)read_timeout
(Integer)ssl_timeout
(Integer)continue_timeout
(Integer)keep_alive_timeout
(Integer)proxy
(URI) (eventually we will want to define a global HTTP proxy option for allweb/*
workers).referer
(String)user_agent
(String) (eventually we will want to define a globalUser-Agent
option for allweb/*
workers).robots
(Boolean)The
web/spider
worker should also have aagent_kwargs
method which builds a Symbol Hash based on the params, which is then passed toRonin::Web::Spider.site(base_uri,**agent_kwargs)
.The text was updated successfully, but these errors were encountered: