simple single-site crawler, crawls one site, notes all links from that site and all assets from that site. Keeps all retrieved data in memory, so no persistence. Didactic exercise primarily to confirm how simple a crawler could be.
crawl.pl www.example.com