Skip to content

GSoC 2018 Work Product Submission

Jump to bottom
Viswak Hanumanth edited this page Aug 7, 2018 · 3 revisions

SNARE

SNARE unittesting

It involves writing unittests for all SNARE's functionality.

Code:

  1. test_converter.py: https://github.com/mushorg/snare/commit/841af31c1f6439a563be9d224cec25ae4b51854f
  2. test_add_meta_tag.py: https://github.com/mushorg/snare/commit/16859bd3932550abe012f573f6dfa3c1b87b6569
  3. test_get_dorks.py: https://github.com/mushorg/snare/commit/94ee15de6d413e086d635ac924ec6fafc978d852
  4. test_submit_data.py: https://github.com/mushorg/snare/commit/98085e37568c295072efd041b840a906f7872cd3
  5. test_handle_html_content.py: https://github.com/mushorg/snare/commit/3487d50747ad707cddf8b4df046b31e9ddd89ac2
  6. test_handle_error: https://github.com/mushorg/snare/commit/b7642764b1af10d306f4bb5c0cb22963038da613
  7. test_handle_request : https://github.com/mushorg/snare/commit/3b8bb1a991ff267e7a11f8630fcf40207e8fda02
  8. test_create_data: https://github.com/mushorg/snare/commit/3162c5d44c9e6bd5763a0981c020514d25995958
  9. test_parse_tanner_response: https://github.com/mushorg/snare/commit/65771a65b6ab4d38f4e73a193f8036ab1f96e045

SNARE Reconstruction

Previously, all the SNARE's code were written in a single and used a deprecated low level aiohttp server. I have improved its architecture by breaking the code into different files also maintaining its integrity while in parallel rewriting the code in latest high level aiohttp (3.3.0) server.

Code:

  1. Reconstruction of snare and cloner: https://github.com/mushorg/snare/commit/9875203b759c965367a55f8d01319b20b075efee
  2. Rewriting unittests after reconstruction: https://github.com/mushorg/snare/commit/83b36c68a4e197f9165189b2ee50e388f75aedef

HTTP server fingerprinting on SNARE:

It involves performing HTTP fingerprinting on SNARE server, and evaluate how SNARE performs. I used (httprint)[http://www.net-square.com/httprint.html] for automated http fingerprinting. Initially we faced an issue (#103), which got fixed after we migrated to aioredis.

Evaluation result:

httprint -h 127.0.0.1:8080 -s signatures.txt -P0
httprint v0.301 (beta) - web server fingerprinting tool
(c) 2003-2005 net-square solutions pvt. ltd. - see readme.txt
http://net-square.com/httprint/
[email protected]

Finger Printing on http://127.0.0.1:8080/
Finger Printing Completed on http://127.0.0.1:8080/
--------------------------------------------------
Host: 127.0.0.1
Derived Signature:
nignx/1.3.8
811C9DC56ED3C295811C9DC5811C9DC5811C9DC594DF1BD04276E4BB811C9DC5
0D7645B5811C9DC5811C9DC5811C9DC5811C9DC5811C9DC5811C9DC5811C9DC5
E2CE6923E2CE6923E2CE6923811C9DC56ED3C295811C9DC56ED3C295811C9DC5
6ED3C2956ED3C295811C9DC56ED3C2956ED3C2956ED3C2956ED3C2956ED3C295
6ED3C2956ED3C295811C9DC56ED3C2956ED3C295

Banner Reported: nignx/1.3.8
Banner Deduced: Microsoft-IIS/5.0 ASP.NET, Microsoft-IIS/5.1
Score: 69
Confidence: 41.57

TANNER

TANNER Session-analyzer

Improve analyzer

Previous analyzing process used a basic approach to make simple suggestions about the peer status and other info. I have applied a heuristic approach to improve the analyzing technique.

This technique involves:

  • A binary-tree structured detection process, where every node is a detection test.
  • Assigning confidence-factor (cf), depending on the detection tests.
  • One binary tree for each of the possible_owner: attacker and crawler/tool.
  • possible_owner 'user' is assigned cf value based on the cf values of the rest of the owners.

The following is the binary tree diagram for attacker detection:

(Note: cf is confidence factor of attacker)

The following is the binary tree diagram for crawler/tool detection:

(Note: cf-t is confidence factor of tool; cf-c is confidence factor of crawler)

Equation for user cf calculation:

(Note: cf-u is the confidence factor of user)

  • Cf-u = 1 - MAX(Cf-a, Cf-c, Cf-t)

Code:

  1. Implementation of the new analyzing technique: https://github.com/mushorg/tanner/commit/2f97e815a0821f7ed559895fb9f1024ff1dd7174
  2. Improving available data for crawler-detection: https://github.com/mushorg/tanner/commit/cb3ef016bdde89597c81ca950d1cd4ca3545bb74

Implement 'location' stats

It involves adding 'location' to the tanner-analyzer stats. This is done by using a python-lib geoip2 which uses an open-source database to locate a session using its ip-address.

Code: https://github.com/mushorg/tanner/commit/a73d8dac1b81ada23d5f15e4389f8216e636fa5c

Migrate to aioredis:

Previously tanner used asycnio_redis. As this library created exceptions, we decided to migrate TANNER to aioredis.

Code: https://github.com/mushorg/tanner/commit/3e96b6d15723e423f92957de6fc46e16af644136

TANNER API-server authorisation:

Previously TANNER API was completely open, so, if someone knew the address of the tanner server, he or she could get the list of all snare's uuids and info about all sessions. I have implemented authorisation for this API. It involves usage of JWT token with a desired signature as the API-key. This signature can be modified by each TANNER owner.

Code:

  1. API-auth implementation: https://github.com/mushorg/tanner/commit/92666d71ebe41bae5589e966d88e44cf1fb7b0a6
  2. Doc update and improvement: https://github.com/mushorg/tanner/commit/43f2eb32fca570d8933dd38942959c5749bcf412

TANNER WEB-UI improvements:

Previously tanner-web had very limited functionality with index-page looking very much blank. I have implemented features and stats in the index page as well as other pages:

Features:

  1. Mushorg logo in the index-page.
  2. Snares button in the index-page.
  3. Home and Doc button on top right corner of every page.

Stats:

  1. Latest session ​info in index-page.
  2. Number of snares info in index-page.
  3. Changed ‘attack_types’ to ‘attacks’ info. This will now show the count of every attack types in the info.
  4. new identity ‘admin’ for the info ‘Possible Owners’ of session-stats
  5. Make location a clickable in session stats, which will redirect to filter of location in sessions.

Code: https://github.com/mushorg/tanner/commit/4b2077557bddb2e9313119b34686b7bb8c713188

Other

  1. Fix issue #253: https://github.com/mushorg/tanner/commit/fa893ed9f5910cdf1999474ef63708b50ceec1af
  2. Fix failing test: https://github.com/mushorg/snare/commit/7634cb8b746bcbd5b32c1d56b19e5ea4f1d151f0
  3. Fix detection of attack requests from Snare: https://github.com/mushorg/snare/commit/14bfbe7d479ee1da74e125020bbcb3f1b7c173d2
  4. Tanner-develop fixes:
  1. Update docs after SNARE's new release: https://github.com/mushorg/snare/commit/9015a00bd574d45d3bb63c8f9a4e73d9c2fd03bc
  2. Fix 'status_404' with request containing prefixed protocol (issue #154): https://github.com/mushorg/snare/commit/30c4dbcbd549746e1cad98bf39b39d4fe8e90b7f
Clone this wiki locally