Skip to content

抓取网站源码并分析链接URL,用于生成sitemap,可开多个线程,速度非常快

License

Notifications You must be signed in to change notification settings

hainatravel/sitemap-url-spider

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 

Repository files navigation

sitemap-url-spider

抓取网站源码并分析链接URL,用于生成sitemap,可开多个线程,速度非常快

安装说明

pip install BeautifulSoup requests threading queue

运行说明

python s.py --domain https://www.xxxxx.com --thread 18

参数说明

--domain 必填 网站域名 如 https://wwww.xxxxx.com

--thread 可选 线程数 默认18 数字越大越快

About

抓取网站源码并分析链接URL,用于生成sitemap,可开多个线程,速度非常快

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages