From 0dbfa99ed787f4aecae21603bc36704e99a8a747 Mon Sep 17 00:00:00 2001 From: Ravindu D Date: Tue, 15 Feb 2022 06:14:47 -0800 Subject: [PATCH 01/16] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 5527a3b..e2212c0 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -Scan8 +Scan8 (Open for GSoC 22) ======== Scan8 is a distributed scanning system for detecting trojans, viruses, malware, and other malicious threats embedded in files. The system will allow one to submit a list of URLs or files and get the scan results in return. From 15d6a12efcc9e0770cdb4d41f20e08c2024c2bad Mon Sep 17 00:00:00 2001 From: Prerak Mathur Date: Thu, 17 Feb 2022 06:55:01 +0530 Subject: [PATCH 02/16] Pull Request Tempelate added --- .../pull_request_template.md | 39 +++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 .github/PULL_REQUEST_TEMPLATE/pull_request_template.md diff --git a/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md b/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md new file mode 100644 index 0000000..4f7ec03 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md @@ -0,0 +1,39 @@ +# Description + +Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. + +Fixes # (issue) + + +## Type of change + +Please delete options that are not relevant. + +- [ ] Bug fix (non-breaking change which fixes an issue) +- [ ] New feature (non-breaking change which adds functionality) +- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) +- [ ] This change requires a documentation update + +# How Has This Been Tested? + +Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration + +- [ ] Test A +- [ ] Test B + +**Test Configuration**: +* Firmware version: +* Hardware: +* Toolchain: +* SDK: + +# Checklist: + +- [ ] My code follows the style guidelines of this project +- [ ] I have performed a self-review of my own code +- [ ] I have commented my code, particularly in hard-to-understand areas +- [ ] I have made corresponding changes to the documentation +- [ ] My changes generate no new warnings +- [ ] I have added tests that prove my fix is effective or that my feature works +- [ ] New and existing unit tests pass locally with my changes +- [ ] Any dependent changes have been merged and published in downstream modules \ No newline at end of file From 11d1d8fac6e20b0f5429eb231bfdf9792b5e35cb Mon Sep 17 00:00:00 2001 From: Prerak Mathur Date: Thu, 17 Feb 2022 06:59:26 +0530 Subject: [PATCH 03/16] Issue tempelate added --- .github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md | 41 ++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md diff --git a/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md new file mode 100644 index 0000000..145df5f --- /dev/null +++ b/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md @@ -0,0 +1,41 @@ +# Prerequisites + +Please answer the following questions for yourself before submitting an issue. **YOU MAY DELETE THE PREREQUISITES SECTION.** + +- [ ] I am running the latest version +- [ ] I checked the documentation and found no answer +- [ ] I checked to make sure that this issue has not already been filed +- [ ] I'm reporting the issue to the correct repository (for multi-repository projects) + +# Expected Behavior + +Please describe the behavior you are expecting + +# Current Behavior + +What is the current behavior? + +# Failure Information (for bugs) + +Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template. + +## Steps to Reproduce + +Please provide detailed steps for reproducing the issue. + +1. step 1 +2. step 2 +3. you get it... + +## Context + +Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions. + +* Firmware Version: +* Operating System: +* SDK version: +* Toolchain version: + +## Failure Logs + +Please include any relevant log snippets or files here. \ No newline at end of file From 6cb0509884b959e2206b9d390f0a287b84726a3a Mon Sep 17 00:00:00 2001 From: Prerak Mathur <76054330+PrerakMathur20@users.noreply.github.com> Date: Thu, 17 Feb 2022 07:02:46 +0530 Subject: [PATCH 04/16] Pull request template added --- .../{pull_request_template.md => PULL_REQUEST_TEMPLATE.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename .github/PULL_REQUEST_TEMPLATE/{pull_request_template.md => PULL_REQUEST_TEMPLATE.md} (98%) diff --git a/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md b/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md similarity index 98% rename from .github/PULL_REQUEST_TEMPLATE/pull_request_template.md rename to .github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md index 4f7ec03..7e33dc6 100644 --- a/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md +++ b/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md @@ -36,4 +36,4 @@ Please describe the tests that you ran to verify your changes. Provide instructi - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes -- [ ] Any dependent changes have been merged and published in downstream modules \ No newline at end of file +- [ ] Any dependent changes have been merged and published in downstream modules From 8783689a4d5008e73dde7e2ad2e5d85f49352a38 Mon Sep 17 00:00:00 2001 From: Prerak Mathur <76054330+PrerakMathur20@users.noreply.github.com> Date: Fri, 25 Feb 2022 03:15:41 +0530 Subject: [PATCH 05/16] Update PULL_REQUEST_TEMPLATE.md --- .../PULL_REQUEST_TEMPLATE.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md index 7e33dc6..74a695a 100644 --- a/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md @@ -14,18 +14,17 @@ Please delete options that are not relevant. - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update -# How Has This Been Tested? +# Testing -Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration +The application comes with a test suite to help users ensure correct installation and help developers verify any updates. -- [ ] Test A -- [ ] Test B +- [ ] `Results` and `Uploads` directories are empty. +- [ ] MongoDB collections are empty. +- [ ] **Scan8** application is up and running. +- [ ] Run the test suite using `python3 app.py -v`. +- [ ] Run a single scan using the Scan8 dashboard and wait till completion. +- [ ] Run the test suite again using `python3 app.py -v`. -**Test Configuration**: -* Firmware version: -* Hardware: -* Toolchain: -* SDK: # Checklist: From 12090ff2686e9630bade13ea3ba897e4060acda4 Mon Sep 17 00:00:00 2001 From: Prerak Mathur Date: Fri, 25 Feb 2022 03:24:05 +0530 Subject: [PATCH 06/16] Update ISSUE_TEMPELATE.md --- .github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md index 145df5f..3840b28 100644 --- a/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md +++ b/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md @@ -27,7 +27,8 @@ Please provide detailed steps for reproducing the issue. 2. step 2 3. you get it... -## Context +## Context +**[THIS SECTION MAY BE DELETED IF IRREVELANT TO THE ISSUE]** Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions. From f7244a91ca50865f5ba669a68984a92c181e3d42 Mon Sep 17 00:00:00 2001 From: Prerak Mathur Date: Wed, 2 Mar 2022 05:14:34 +0530 Subject: [PATCH 07/16] Added unit test cases in PULL_REQUEST_TEMPLATE.md --- .../PULL_REQUEST_TEMPLATE.md | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md index 74a695a..9648cf1 100644 --- a/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md @@ -16,16 +16,15 @@ Please delete options that are not relevant. # Testing -The application comes with a test suite to help users ensure correct installation and help developers verify any updates. - -- [ ] `Results` and `Uploads` directories are empty. -- [ ] MongoDB collections are empty. -- [ ] **Scan8** application is up and running. -- [ ] Run the test suite using `python3 app.py -v`. -- [ ] Run a single scan using the Scan8 dashboard and wait till completion. -- [ ] Run the test suite again using `python3 app.py -v`. +The application comes with a test suite to help users ensure correct installation and help developers verify any updates. +The following tests should return **ok** status upon running the unit-test command `python3 app.py -v` from the `/Testing` directory. +- [ ] `testResultsJSON` +- [ ] `testResults` +- [ ] `testUploads` +- [ ] `testResultsDirectoryPresent` +- [ ] `testUploadsDirectoryPresent` # Checklist: - [ ] My code follows the style guidelines of this project From cf5701ccf13b6823ec42a7c803f416002b00111d Mon Sep 17 00:00:00 2001 From: Aju100 Date: Wed, 2 Mar 2022 07:25:33 +0545 Subject: [PATCH 08/16] fix: update env file --- .env | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.env b/.env index a4786c8..f5db948 100644 --- a/.env +++ b/.env @@ -1,5 +1,5 @@ UPLOAD_DIRECTORY="../common/Uploads" -MONGODB_HOST="mongo-service" +MONGODB_HOST="127.0.0.1" MONGODB_PORT=27017 UTILITIES_PATH="../Utilities" REDIS_URL="redis://redis-service:6379" From df44f6e9f50cd1ca0a11be3fcc882bf21864be5b Mon Sep 17 00:00:00 2001 From: Maanas Talwar <54113320+maanas-talwar@users.noreply.github.com> Date: Sat, 22 Oct 2022 20:02:08 +0530 Subject: [PATCH 09/16] Updated README --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index e2212c0..5cdf64f 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -Scan8 (Open for GSoC 22) +Scan8 (Open for GSoC and Hacktoberfest) ======== Scan8 is a distributed scanning system for detecting trojans, viruses, malware, and other malicious threats embedded in files. The system will allow one to submit a list of URLs or files and get the scan results in return. From 7fc8fc26637db804bb35cc9a1290c9eb5598aba9 Mon Sep 17 00:00:00 2001 From: Maanas Talwar <54113320+maanas-talwar@users.noreply.github.com> Date: Sat, 22 Oct 2022 20:12:49 +0530 Subject: [PATCH 10/16] Added coordinator readme --- Coordinator/README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Coordinator/README.md b/Coordinator/README.md index eb140fd..23057f6 100644 --- a/Coordinator/README.md +++ b/Coordinator/README.md @@ -1,2 +1,3 @@ -### Master Node +### Coordinator Node +The Coordinator Node listens to updates for new scans, subsequently creating and adding scan jobs to the Redis Queue. From d258b479ab14441bc18b2fe43fd12123dbbce2cb Mon Sep 17 00:00:00 2001 From: Maanas Talwar <54113320+maanas-talwar@users.noreply.github.com> Date: Sat, 22 Oct 2022 20:16:06 +0530 Subject: [PATCH 11/16] Added dashboard readme --- Dashboard/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/Dashboard/README.md b/Dashboard/README.md index fdb0443..a6a84f4 100644 --- a/Dashboard/README.md +++ b/Dashboard/README.md @@ -1,2 +1,3 @@ ### Dashboard +The Dashboard provides a responsive web interface for uploading files for new scans and tracking the status of all the submitted scans. From f092564fba501d3300645d2addab107be3e4182f Mon Sep 17 00:00:00 2001 From: Maanas Talwar <54113320+maanas-talwar@users.noreply.github.com> Date: Sat, 22 Oct 2022 20:18:47 +0530 Subject: [PATCH 12/16] Added worker node readme --- Worker/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/Worker/README.md b/Worker/README.md index b52a5e7..47a0446 100644 --- a/Worker/README.md +++ b/Worker/README.md @@ -1,2 +1,3 @@ ### Worker Node +The Worker Node listens to updates for new jobs in Redis Queue and executes them. From 646abb1d66baeddafce009e4b0815971212be0e8 Mon Sep 17 00:00:00 2001 From: Madushan98 Date: Sun, 19 Mar 2023 07:16:30 +0530 Subject: [PATCH 13/16] Added delete function for scanned records --- Dashboard/app.py | 20 ++++++++++++++++++-- Dashboard/templates/index.html | 3 +++ 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/Dashboard/app.py b/Dashboard/app.py index 3519e97..a283d4c 100644 --- a/Dashboard/app.py +++ b/Dashboard/app.py @@ -3,7 +3,7 @@ from werkzeug.utils import secure_filename import os import uuid -from pymongo import MongoClient +from pymongo import MongoClient, errors import json from hurry.filesize import size, si from dotenv import load_dotenv @@ -31,7 +31,7 @@ def index(): queued = queuedScans.find() running = runningScans.find() completed = completedScans.find() - return render_template('index.html', prequeued=prequeued, queued=queued, running=running, completed=completed, newScanUrl=url_for('newScan')) + return render_template('index.html', prequeued=prequeued, queued=queued, running=running, completed=completed, newScanUrl=url_for('newScan'),delete_completed=url_for('deleteCompleted', file='')) def new_scan(): @@ -79,6 +79,19 @@ def generate(): return Response(generate(), mimetype='text/event-stream') +def delete_completed(file): + try: + file = completedScans.find_one({'_id': file}) + print(file) + if file: + # Get the file's ID and delete it + file_id = file['_id'] + completedScans.delete_one({'_id': file_id}) + return redirect(url_for('dashboard')) + except errors.OperationFailure as e: + return {"message": f"Error: {e}"}, 400 + + app.add_url_rule("/", endpoint="dashboard", view_func=index, methods=['GET']) app.add_url_rule("/newScan", endpoint="newScan", view_func=new_scan, methods=['GET']) @@ -86,5 +99,8 @@ def generate(): view_func=progress, methods=['GET']) app.add_url_rule("/upload", endpoint="upload", view_func=upload_files, methods=['GET', 'POST']) +app.add_url_rule("/deleteCompleted/", endpoint="deleteCompleted", + view_func=delete_completed, methods=['GET', 'POST']) + if __name__ == "__main__": app.run(host="0.0.0.0",debug=True) diff --git a/Dashboard/templates/index.html b/Dashboard/templates/index.html index e53bf03..efcc750 100644 --- a/Dashboard/templates/index.html +++ b/Dashboard/templates/index.html @@ -100,6 +100,9 @@
Number of files: {{ item['files']['total'] }}
+
+ +
{% endfor %} From 47108a93c008b87a4dc046b34d8e123c92f54cb3 Mon Sep 17 00:00:00 2001 From: Madushan98 Date: Sun, 19 Mar 2023 07:24:49 +0530 Subject: [PATCH 14/16] issue template updated --- .github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md | 29 ++---------------------- 1 file changed, 2 insertions(+), 27 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md index 3840b28..6ad7bd8 100644 --- a/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md +++ b/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md @@ -8,35 +8,10 @@ Please answer the following questions for yourself before submitting an issue. * - [ ] I'm reporting the issue to the correct repository (for multi-repository projects) # Expected Behavior +The delete button has been added to the "Completed Scans" section of the Scan8 tool, and is easily accessible to users. It is labeled with a delete tag match the existing Scan8 user interface. The delete button works as expected, and removes the completed scan files without causing any unintended consequences. -Please describe the behavior you are expecting # Current Behavior -What is the current behavior? +Currently, there is no way to delete completed scan files in Scan8, which is problematic for users who want to remove unnecessary scan files from their system. -# Failure Information (for bugs) - -Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template. - -## Steps to Reproduce - -Please provide detailed steps for reproducing the issue. - -1. step 1 -2. step 2 -3. you get it... - -## Context -**[THIS SECTION MAY BE DELETED IF IRREVELANT TO THE ISSUE]** - -Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions. - -* Firmware Version: -* Operating System: -* SDK version: -* Toolchain version: - -## Failure Logs - -Please include any relevant log snippets or files here. \ No newline at end of file From 7b1df52d4ad3d1f754868d6fc4e57edeef322a71 Mon Sep 17 00:00:00 2001 From: Madushan98 Date: Sun, 19 Mar 2023 07:25:17 +0530 Subject: [PATCH 15/16] pull request templete updated --- .github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md index 9648cf1..d80a111 100644 --- a/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md @@ -1,18 +1,20 @@ # Description -Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. +Currently, there is no way to delete completed scan files in Scan8, which is problematic for users who want to remove unnecessary scan files from their system. However, I have implemented a delete button that can be used to delete completed scans. When pressed, the button will remove the completed scan files immediately. + + Fixes # (issue) +The delete button has been added to the "Completed Scans" section of the Scan8 tool, and is easily accessible to users. It is labeled with a trash can icon and has been styled to match the existing Scan8 user interface. The delete button works as expected, and removes the completed scan files without causing any unintended consequences. ## Type of change Please delete options that are not relevant. -- [ ] Bug fix (non-breaking change which fixes an issue) + - [ ] New feature (non-breaking change which adds functionality) -- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) -- [ ] This change requires a documentation update + # Testing From 04feb609a490b51b4a19f35dd16b844cb3e7575f Mon Sep 17 00:00:00 2001 From: Madushan98 Date: Tue, 21 Mar 2023 13:00:42 +0530 Subject: [PATCH 16/16] refactor code --- Dashboard/app.py | 2 -- 1 file changed, 2 deletions(-) diff --git a/Dashboard/app.py b/Dashboard/app.py index a283d4c..28243ee 100644 --- a/Dashboard/app.py +++ b/Dashboard/app.py @@ -82,9 +82,7 @@ def generate(): def delete_completed(file): try: file = completedScans.find_one({'_id': file}) - print(file) if file: - # Get the file's ID and delete it file_id = file['_id'] completedScans.delete_one({'_id': file_id}) return redirect(url_for('dashboard'))