use regex for traversals

[rrmerugu]

Is your feature request related to a problem? Please describe.
Currently only selectors are used for traversing. In case of traversing the urls that has the string "/blog/", there is no way to achieve this.

Describe the solution you'd like
Will send the regex to the LinkExtractor for more precise targeting of the urls to traverse and crawl.

Describe alternatives you've considered
NA

Additional context
Add any other context or screenshots about the feature request here.