Add support for sitemap.xml or robots.txt in 'urls' option

[humidair1999]

Since most sites requiring snapshots utilize a sitemap.xml or robots.txt listing all crawlable URLs already, it'd be nice to be able to utilize these files as the source for urls in the urls option, rather than a raw array of strings.

[gary-george]

id love to see this support also ,
im in abit of a pickle myself as my site will have lots of dynamic urls and it isnt really ideal to have to manually input urls each time.

[liquidvisual]

Yes, absolutely agree. Will this ever happen?