Adding support for issue #182

[knirbhay]

Adding support for

1.Custom Headers and Cookies with Initial request
2.Shared cookies middleware to share cookies between crawl nodes

Linked Issue #182

[knirbhay]

This code was not getting executed because kafka scraper_schema.json is only forcing it to be string. So I added attribute cookies to scraper_schema. Not sure if it make sense.

[knirbhay]

Custom feed will look like this

python kafka_monitor.py feed {
"url": "http://dmoztools.net",
"appid": "testapp",
"crawlid": "ABC123",
"spiderid": "myspiderid",
"headers": {
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,ima/webp,/;q=0.8",
"Accept-Encoding": "gzip, deflate",
"X-Requested-With": "dmoztools.net",
"User-Agent": "My Custom User Agent"
},
"cookies": {
"device_id": "1",
"app_token": "guid"
}
}

[coveralls]

Coverage decreased (-0.4%) to 69.99% when pulling 1cd7940 on knirbhay:dev into 2c2075a on istresearch:dev.

[knirbhay]

Due to shared cookie middle ware the coverage has decreased by 0.4%. @madisonb do you think this can be managed? I improved few percentage in distributed_scheduler.