bug(scope): Nuclei only scan the first endpoint (https://domain.com/) (HTTPS) and do not scan the next endpoints (http://domain.com/) (HTTP)

[haryisharry]

Current Behavior

Run Full Scan and in the Vulnerability Scan, Nuclei only scan the first endpoint (https://domain.com/) (HTTPS) and do not scan the next endpoints (http://domain.com/) (HTTP) , in the target we have a bug to test. I search in the source code and found the file "input_endpoints_vulnerability_scan.txt" is called one time only before it be converted to urls_unfurled.txt. This may cause this issue.

Expected Behavior

Nuclei scan ALL found endpoints.

Steps To Reproduce

1. Install Branch: release/2.1.0 or any version
2. Run "Full Scan" on target with multi vulnerabilities on many endpoints, especially : "https" endpoint and "http" endpoints
3. No vulnerabilities endpoint on "http endpoints" found.

Environment

- reNgine: 2.1.0 PR
- OS: Ubuntu 22.04
- Python: 3.11
- Docker Engine: 
- Docker Compose: 2.33
- Browser: Chrome

Anything else?

By the way, the current log of scan only show 1 command per steps of the scan,for eg.

sort -u /home/rengine/scan_results/domain.net/scans/8ac36e99-8125-11ef-9a4a-9b45a032b0f2/urls_unfurled.txt -o /home/rengine/scan_results/domain.net/scans/8ac36e99-8125-11ef-9a4a-9b45a032b0f2/urls_unfurled.txt

It will better for debugging if we have a way to show full command had been run on the scan AND keep all the tempt files, for eg. "urls_unfurled.txt". Maybe we should add "Debug_Mode" in the configure file

Acknowledgements

• This issue is not a duplicate of an existing bug report.
• I have chosen an appropriate title.
• All requested information has been provided properly.

[psyray]

Hi, could you test the 2.1.1 branch and make feedback about this issue.

It should be resolved with #238