Skip to content

Latest commit

 

History

History
195 lines (170 loc) · 21.3 KB

GAP Help.md

File metadata and controls

195 lines (170 loc) · 21.3 KB
Raw

GAP by @xnl_h4ck3r


This is an evolution of the original getAllParams extension for Burp. Not only does it find more potential parameters for you to investigate, but it also finds potential links to try these parameters on, and produces a target specific wordlist to use for fuzzing. This code is far from perfect, but any constructive criticism is very much welcome! I hope this tool helps you.

Acknowledgments:

Respect and thanks go to @HolyBugx for help with ideas, testing and patience!
A shout out to Gerben Javado and his amazing tool Link Finder who's regular expression (regex) provided the starting point for the Link mode in GAP.

How to Install

  1. Visit Jython Offical Site, and download the latest stand alone JAR file, e.g. jython-standalone-2.7.3.jar.
  2. Open Burp, go to Extensions -> Extension Settings -> Python Environment, set the Location of Jython standalone JAR file and Folder for loading modules to the directory where the Jython JAR file was saved.
  3. On a command line, go to the directory where the jar file is and run java -jar jython-standalone-2.7.3.jar -m ensurepip.
  4. Download the GAP.py and requirements.txt from this project and place in the same directory.
  5. Install Jython modules by running java -jar jython-standalone-2.7.3.jar -m pip install -r requirements.txt.
  6. Go to the Extensions -> Installed and click Add under Burp Extensions.
  7. Set Extension type to Python and select the GAP.py file
  8. Click Next and you're good to go 🤘

How to Run

You can run GAP from a single request/response, or multiple, from any context in Burp. For example, you can run for a single request in Repeater, a group of requests in Proxy History, request in the Site Map contents, etc. However, the most common option will probably be from the Site Map tree view. IMPORTANT: Make sure you have scope set before running from this context.

On the Target -> Site map tab of Burp you can select a specific host, a selection of hosts (holding down Ctrl or Shift), or all hosts (using Ctrl-A), or even select a specific sub folder or specific endpoints. Once the required endpoints are selected, right click and select Extensions -> GAP to run the tool. Go to the GAP tab and see the results. What gets returned will depend on the options selected, and these will all be described below. For very large projects (and depending on what options were selected), it can sometimes take GAP a little while to run. If for some reason it hasn't completed and you want to cancel the current run to change options for example, you can do this by pressing the CANCEL GAP button. If you try running GAP again while it is still running, it will CANCEL the current run before starting the new one.

GAP Mode

There are 3 different modes for GAP, Parameters,Links and Words. They can either be run separately, or together, depending on what you select. What each mode does will be explained below, but if you don't need all enabled then unselecting them can use less memory and get results back quicker.

Parameters Mode

When the GAP Mode of Parameters is selected then GAP will try to find as many potential parameters based the following options:

  • Include URL path words? - The words in the response URL path are included as potential parameters if the URL is in scope.
  • Report "sus" parameters? - If a "sus" parameter is identified, a Burp custom Issue will be raised (unavailable in Burp Community Edition). There will be no markers in the Request/Response of the Issue showing where the named parameter can be found because including this functionality seriously increases the time GAP can take to run, so this is not a feature at the moment. For Burp Community Edition, the details of the parameter will be written to the extension output.
  • Inc. Tentative? - If a "sus" parameter is identified, the Report "sus" parameters option is checked, and the confidence is Tentative, this option determines whether it is raised or not.

Request Parameters

These are mainly parameters that Burp itself identifies from HTTP requests and are part of the Burp Extender API IParameter interface
  • Query string params - PARAM_URL; a parameter within the URL query string
  • Message body params - PARAM_BODY; a parameter within the message body
  • Param attribute in multi-part message body - PARAM_MULTIPART_ATTR; the value of a parameter attribute within a multi-part message body (such as the name of an uploaded file)
  • JSON params - PARAM_JSON; an item of data within a JSON structure
  • Cookie names - PARAM_COOKIE; an HTTP cookie name
  • Items of data in XML structure - PARAM_XML
  • Value of tag attribute in XML structure - PARAM_XML_ATTR
Additionally, GAP will also report any parameters in the Request where the Burp API doesn't always successfully detect them:
  • JSON format
  • GraphQL request (COMING SOON)
  • XML request (COMING SOON)

Response Parameters

These are potential parameters that can be found in the HTTP responses. These are identified by GAP itself rather than through the Burp Extender API.

  • JSON params - if the response has a MIME type of JSON then the Key names will be retrieved
  • Value of tag attributes in XML structure - if the response has a MIME type of XML then the XML attributes are retrieved
  • Name and Id attributes of HTML input fields - if the response has a MIME type of HTML then the value of the NAME and ID attributes of any INPUT tags are retrieved
  • Javascript variables and constants - javascript variables set with var, let or const are retrieved. Also, if there is a variable set with one of those keywords that is set to a nested object, the keys for that object are also returned as parameters. In addition to this, any key value is nested objects for dataLayer.push are also returned. NOTE: Improvements are needed to retrieve more variables as there are many ways that these can be declared and difficult to retrieve all from regex.
  • Params from links found - THIS OPTION IS ONLY ENABLED IF LINKS MODE IS ALSO USED. Any URL query string parameters in potential Links found will be retrieved, only if they are clearly in scope, or there is just a path and no way of determining if it is in scope.

Links Mode

When the GAP Mode of Links is selected then GAP will try to find possible links based on the following. Also, only requests of a certain Content-Type are checked for potential links. This is determined by the constant CONTENTTYPE_EXCLUSIONS in the code (these are types such as images, video, audio, fonts, etc.)

  • Prefix with selected target(s) - If checked, the root of each target selected in the Site Map will be prefixed to any links found that do not have a domain, e.g. /api/user/1
  • Prefix with link(s) - If checked, the value(s) in the text field will be prefixed to any links found that do not have a domain, e.g. /api/user/1. Multiple domains can be provided, separated by a semicolon, e.g. http://example.com;https://sub.example.com
  • Also include un-prefixed links - If the Prefix with selected target(s) or Prefix with link(s) option is checked then this option can be checked to include the original un-prefixed link in addition to the prefixed link.
  • Include site map endpoints? - This will include endpoints from the Burp Site map (what was selected) in the potential Link list, if they are in scope.
  • Include relative links? - If checked, links found that start with `./` or `../` will be included in the results.
  • Link exclusions - If the option is selected it will be applied when run. The text field contains a comma separated list of values. If any of these values exists in a potential link found, then it will be excluded from the final list. There is a initial default list determined by the DEFAULT_EXCLUSIONS constant, but you can change this and save your settings. If the option is not selected, all links will be returned.

Words Mode

When the GAP Mode of Words is selected then GAP will produce a target specific wordlist from the responses searched.

  • Create lowercase words? - Any word found that contains an uppercase letter will also be added as an all lowercase word.
  • Create singular/plural word? - If checked, then for each word found, a suitable singular or plural version will also be added to the output.
  • Include HTML comments? - If checked, all words within HTML comments will be considered.
  • Include IMG ALT attribute? -If checked, all words with the ALT attribute of IMG tags will be considered.
  • Include words with digits? - If un-checked, then any words with numeric digits will be excluded from output.
  • Include URL path words? - Any path words in selected links will be added as words.
  • Include potential params - This option is only shown if the Parameters Mode is enabled. If selected, all potential params will also be added to the word list.
  • Maximum length of words - The maximum length of words that will be output (this excludes plurals of minimum length words). This can be a minimum of 3.
  • Stop words - The term stop words comes from Natural Language Processing where they are common words that will be excluded from content. If a word exists in this list before running, then it will be excluded from output.

In addition to the options above, words will be taken from all responses with certain conditions:

  • Only responses with content types are searched. The defaults are text/html,application/xml,application/json,text/plain,application/xhtml+xml,application/ld+json,text/xml
  • Words from <meta> tag content where:
    • Property is og:title, og:description, title, og:site_name or fb:admins
    • Name is description, keywords, twitter:title, twitter:descriptionapplication-name, author, subject, copyright, abstract, topic, summary, owner, directory, category, og:title, og:type, og:site_name, og:description, csrf-param, apple-mobile-web-app-title, twitter:label1, twitter:data1, twitter:label2, twitter:data2 or twitter:title
  • Words from <link> tag title where:
    • Rel is alternate, index, start, prev, next or search
  • Words from the rest of the inner HTML of the page, excluding tags <style>, <script> and <link>

GAP Output

Below is an explanation of the output given when GAP has completed running. When running has been completed, you can click the right mouse button in the parameter, words or links pane to get a Copy link. Clicking this will copy whatever is in the pane to your clipboard.

Potential Parameters

  • Potential parameters found - This text are will show all unique potential parameters, one per line.
  • Show origin - If this feature is ticked, the potential parameter will be followed by the HTTP request endpoint (in square brackets) that the parameter was found in. A parameter could have been found in more than one request, so this view can show duplicate links, one per origin endpoint.
  • Show "sus" - If this feature is ticked, only potential parameters that are "sus" are shown followed by the associated vulnerability type(s) (in square brackets). NOTE: If you right click and select Copy you will have the parameter names WITHOUT the vuln types copied to your clipboard.
  • Show query string with value - This checkbox can be used to switch between the list of parameters and a concatenated query string with all parameters with a value given in the following text box.
  • Param Value - This defaults to XNLV and is a value that is used to create the concatenated query string, with each parameter given this value followed by a unique number of the parameter. This query string can be used to manually append to a URL and check for reflections.

Potential Links

  • Potential links found - This text area will show potential links found. Without any of the other options described below selected, all unique endpoints found are displayed, one per line.
  • Show origin endpoint - If this feature is ticked, the potential link will be followed by the HTTP request endpoint (in square brackets) that the link was found in. A link could have been found in more than one request, so this view can show duplicate links, one per origin endpoint.
  • In scope only - If this feature is ticked, and the potential links contain a host, then this link will be checked against the Burp Target Scope. If it is not in scope then the link will be removed from the output. NOTE: If it is not possible to determine the scope (e.g. it may just be a path without a host) then it will be included as in scope to avoid omitting anything potentially useful.
  • Link filter - any value entered in the Filter input field followed by ENTER or pressing Apply filter will determine which links will be displayed. This can depend on the values of the following two options:
  • Negative match - If selected, any link containing the Filter text will NOT be displayed. If unselected, then only links containing the filter will be displayed.
  • Case sensitive - If selected, the value is the Filter input field will be case sensitive when determining which Links to display.
The filter is something that is applied after GAP has run. It allows you to look for specific things when there are many results. For example, enter .js to only show the links to javascript files. As soon as you clear the filter, the original results are redisplayed.

An additional feature of GAP is to automatically include links of valid .js.map (javascript source map) files. These are identified by responses that contain the //# sourceMappingURL line, or have a HTTP header of SourceMap or X-SourceMap.

To find links, a complex regex is used to look for different formats and contexts for potential links and files. This regex was initially based on the one used in Link Finder by Gerben Javado, but has been evolved to try and identify more with minimal false positives.

If you have the Show origin endpoint options unchecked and the In scope only option checked, then when you right click the pane, you will get another menu option of Request all prefixed URLs and send to Site Map. Clicking this will request all URLs shown in the current pane (so can be filtered with Link filter too) and make a GET request for each URL. The URLs will be requested in 2 separate threads, with 10 milliseconds between each request, and they will be added to the Site Map.
NOTE: Sometimes java errors can be written to the extensions Error tab if a hoist is unreachable. There doesn't seem to be a way to catch these, but they can be safely ignored.
When requests are being made, you can right click on the pane and select the Cancel all requests being made menu item to stop all requests previously scheduled.

Words

  • Words found - This text are will show all unique words, one per line.
  • Show origin - If this feature is ticked, the words will be followed by the HTTP request endpoint (in square brackets) that the word was found in. A word could have been found in more than one request, so this view can show duplicate links, one per origin endpoint. If the word was generated by GAP (e.g. a plural or singular version) then it will be followed by [GAP] instead of an origin endpoint.

Other options

  • Show contextual help - If selected, hovering over any features of GAP will give contextual help for that feature.
  • Auto save output to directory - If this option is checked then when GAP completes a run, a file will be created with the potential parameters, with potential links, and target specific wordlist. These files will be created in the specified directory. If the directory is invalid then the users home directory will be used.
  • Choose... - the button can be used to select the required directory to store output files.
If the Auto save output to directory option is checked, then files are written as follows:

  • Create a sub folder for each root in the target site that was selected in Site Map
  • Create a file in the main folder with the name of the Burp project and timestamp, e.g. if the project is called target, the files might be target_20230416_133700_links.txt, target_20230416_133700_parans.txt and target_20230416_133700_words.txt where 20230416 is the current date in YYYYMMDD format and 133700 is the current time in HHMMSS format. These files will contain what was found for ALL roots selected.
  • Within each sub folder, the files will only contain findings for that particular root. If there are no findings for a mode, then a file will not be created for that mode.
NOTE: The project name is taken from the Burp title. If you use the Sharpener Burp Extension then changing the Title will affect the naming of the files.

GAP Settings

When GAP is first started, it will start with default settings. Any changes made to the configuration settings of GAP can be saved for future use by clicking the Save options button. If for any reason you want to revert to the default configuration options, you can click the Restore defaults button.

Troubleshooting and Feedback

It is hard to design GAP to display all controls for all screen resolutions and font sizes. I have tried to deal with the most common setups, but if you find you cannot see all the controls, you can hold down the Ctrl button and click the GAP logo header image to remove it to make more space.

If you have any problems with GAP, you can report an issue on Github. Before you report an issue, please look at the Extender -> Extensions tab in Burp, click on the GAP extension in the list and include details of any output displayed on the Errors tab with your issue. If you know of a parameter or link that you believe GAP should/shouldn't have identified then please provide as much info as possible, e.g. the options you had selected, the relevant endpoint, etc.

Important Notes

If you don't need one of the modes, then un-check it as results will be quicker.

If you run GAP for one or more targets from the Site Map view, don't have them expanded when you run GAP... unfortunately this can make it a lot slower. It will be more efficient if you run for one or two target in the Site Map view at a time, as huge projects can have consume a lot of resources.

If you want to run GAP on one of more specific requests, do not select them from the Site Map tree view. It will be a lot quicker to run it from the Site Map Contents view if possible, or from proxy history.

It is hard to design GAP to display all controls for all screen resolutions and font sizes. I have tried to deal with the most common setups, but if you find you cannot see all the controls, you can hold down the Ctrl button and click the GAP logo header image to remove it to make more space.

Thank you for trying out GAP!
Good luck and good hunting! If you really love the tool (or any others), or they helped you find an awesome bounty, consider BUYING ME A COFFEE! ☕ (I could use the caffeine!)

@xnl-h4ck3r 🤘