diff --git a/doc/language.md b/doc/language.md index bbaa38d..97090bd 100644 --- a/doc/language.md +++ b/doc/language.md @@ -139,132 +139,27 @@ If you choose the body section, the meaning of the tags is different, infact: Note that the content lenght of the body section is automatically updated or removed if the content of the body is edited. -## Decode operation - -A list of decode operations can be added to each Operation (passive or active). These operations are used to decode encoded content taken from inside an intercepted message. A decode operation can have its own list of decode operations; these are called recursive decode ops. and they take as input the previously decoded content. - -The HTTP parameter containing the content to be decoded has to be specified with the `decode param` If the section of the message is 'body,' the `decode param` takes a regex (for more information, see the section below). - -Next, a list of encodings (or the `type`) has to be specified. You can specify an encodings list with the tag `encodings` the order of these encodings will be followed while decoding, and when the message is re-encoded, the order will be reversed. With the tag `type` the content is decoded following a fixed set of rules (e.g., jwts or XML). If you specify the `type` tag, the `encodings` tag will be ignored. If you use the `type` tag when using Edit Operations, you may be allowed to use more tags (e.g., with jwts or XML) to edit easily; otherwise, the decoded content will be used as plain text. - -In decode operations is possible to use check operations to check the decoded content, depending on the specified type, the allowed check operations differ. - -- JWT type -> See 'Checks on JSON content' section -- No type (encodings used) -> See check section - -Needed tags: - -- `type` and/or `encodings` -- `decode param` -- `from` select from where decode the content (HTTP message section or previous decode output) - -optional tags: - -- `decode operations` -- `checks` -- `edits` -- `force regex` set it to true to enable using regex inside decode param in a recursive decode operation. This is useful when you are decoding the body a JWE when it contains a raw JWT not inside a json element. - -#### Body section - -An important note about the body (from) section is that the input of `decode param` has to be a regex, and whatever is matched with that regex is decoded. -An useful regex to match a parameter's value could be `(?<=SAMLResponse=)[^$\n& ]*` which searches the SAMLResponse= string in the body, and matches everything that is not $ or \n or & or whitespace - -#### Decoding JWTs - -When decoding a jwt (by using tag type=jwt) it is possible to check the signature of that jwt by providing a public key. To do this, use the `jwt check sig` tag with value the PEM-encoded public key to be used to check. - -Note: The supported algorithms for signing are: - -- RS256 -- RS512 +## Edit Operation -#### Decoding JWE +The edit operation can be used to modify the content of a message, a jwt, or other sources. -Note that when decrypting a JWE, a JWS is expected in the payload. Other payloads are not supproted. +### Using Edit Operation in a Operation -To decrypt a JWE to access the JWT in its payload use these tags +By using an Edit Operation inside an Operation, you are able to edit the intercepted message content. There are multiple ways of editing it, in the following section they will be described. You can use the following tags: -- `jwe decrypt` with the private key in PEM string format -- `jwe encrypt` with the public key in PEM string format +- `from` to select the section of the message you need +- `edit` to edit the value of the given parameter. (only for url and head sections) use `value` to specify the new value. +- `edit regex` to edit with a regex the section of the message you selected. use `value` to specify the new value +- `add` to add some content to the given section. in case of url and head, you need to specify the name of the parameter in this tag, and the value with `value`. If the parameter is not found, a new parameter is added, if the parameter is already present, the new value will be appended to the old one. For the body section, the content will be always appended to the end of the body, so you can leave this tag value empty and put the content to append in the `value` tag. +- `remove` used to specify the name of a parameter to remove in url and head. Not available on body. +- `value` used to specify the new value for the edit operations +- `use` used in place of `value` to use the given variable value as new value. You should give a variable name to this tag. -Note: You can decrypt without specifying encrypt, this will prevent the JWE from being edited (as no encryption key is passed) -Note: You can't encrypt, without decrypt +>Note: the url section is the entire request url such as "" -Note: Supported algorithms are: +>Note: if you use edit regex in the head, the regex is executed over all the headers -- RSA_OAEP -- RSA_OAEP_256 -- ECDH_ES_A128KW -- ECDH_ES_A256KW - -## Tag table - -In this table you can find a description of all the tags available for this Operation based on the input Module. - -| input module (container) | Available tags | Required | value type | allowed values | -| --------------------------- | ----------------- | -------- | ---------------------- | -------------------------------------- | -| standard Operation | from | yes | str | head, body, url | -| | decode param | yes | str | \* | -| decode Operation (type=jwt) | from | yes | str | jwt header, jwt payload, jwt signature | -| | decode param | yes | str(JSON path) | \* | -| \* | type | | str | xml, jwt | -| | encodings | | list[str] | base64, url, .. | -| | decode operations | | list[decode Operation] | \* | -| | checks | | list[check Operation] | \* | -| | edits | | list[edit Operation] | \* | -| | jwt check sig | | str(PEM) | PEM-encoded public key | -| | jwe decrypt | | str(PEM) | PEM-encoded private key | -| | jwe encrypt | | str(PEM) | PEM-encoded public key | - -### Recursive Decode operations - -When using the `from` tag in a recursive decode (one that is inside another), you can use - depending on the previous decode type - other sections, such as "jwt header" "jwt payload" .. - -Source: standard Operation (HTTP intercepted message) - -- `from`: (url, head, body) - -Source: decode Operation JWT - -- `from`: (jwt header, jwt payload, jwt signature) - -in recurdsive decode op, the decode param accepts different inputs, e.g. if previous decoded content is a jwt, decoded content will accept a JSON path - -Syntax example of a recursive decode operation: - -```json -"decode operations": [ - { - ..., - "decode operations": [ - { - "from": "somwhere in the previous decode", - "encodings": "asdasd", - } - ] - } -] -``` - -Example of decoding a jwt from the url of a message in the asd parameter, and then decode the jwt found inside of the jwt. - -```json -"decode operations": [ - { - "from": "url", - "type": "jwt", - "decode param": "asd", - "decode operations": [ - { - "from": "jwt header", - "type": "jwt", - "decode param": "$.something" - } - ] - } -] -``` +>Note: Save from message is not possible in edit, is should be done in message operations ### Using Edit Operation in Decode Operations @@ -426,10 +321,154 @@ It is possible to use the tag `use` instead of the tag `value` to use the text s #### SAML signature -There's the possibility to remove the signature from a saml request or response and resign it with a test private key, just specify `self-sign`: true in the message operation. +There's the possibility to remove the signature from a saml request or response and resign it with a test private key, just specify `self-sign`: true in the message operation.## Decode operation Another possibility is just to remove the signature, using `remove signature` set to true in the message operation. Note that these keys are avaiable and applied only on decoded parameters, also if `decode param` is defined. +## Decode operation + +A list of decode operations can be added to each Operation (passive or active). These operations are used to decode encoded content taken from inside an intercepted message. A decode operation can have its own list of decode operations; these are called recursive decode ops. and they take as input the previously decoded content. + +The HTTP parameter containing the content to be decoded has to be specified with the `decode param` If the section of the message is 'body,' the `decode param` takes a regex (for more information, see the section below). + +Next, a list of encodings (or the `type`) has to be specified. You can specify an encodings list with the tag `encodings` the order of these encodings will be followed while decoding, and when the message is re-encoded, the order will be reversed. With the tag `type` the content is decoded following a fixed set of rules (e.g., jwts or XML). If you specify the `type` tag, the `encodings` tag will be ignored. If you use the `type` tag when using Edit Operations, you may be allowed to use more tags (e.g., with jwts or XML) to edit easily; otherwise, the decoded content will be used as plain text. + +In decode operations is possible to use check operations to check the decoded content, depending on the specified type, the allowed check operations differ. + +- JWT type -> See 'Checks on JSON content' section +- No type (encodings used) -> See check section + +Needed tags: + +- `type` and/or `encodings` +- `decode param` or `decode regex` +- `from` select from where decode the content (HTTP message section or previous decode output) + +optional tags: + +- `decode operations` +- `checks` +- `edits` + +#### Body section + +An important note about the body (from) section is that the input of `decode param` has to be a regex, and whatever is matched with that regex is decoded. +An useful regex to match a parameter's value could be `(?<=SAMLResponse=)[^$\n& ]*` which searches the SAMLResponse= string in the body, and matches everything that is not $ or \n or & or whitespace + +#### Decoding JWTs + +When decoding a jwt (by using tag type=jwt) it is possible to check the signature of that jwt by providing a public key. To do this, use the `jwt check sig` tag with value the PEM-encoded public key to be used to check. + +Note: The supported algorithms for signing are: + +- RS256 +- RS512 + +#### Decoding JWE + +To decrypt a JWE to access the JWT in its payload use these tags + +- `jwe decrypt` with the private key in PEM string format +- `jwe encrypt` with the public key in PEM string format + +Note: You can decrypt without specifying encrypt, this will prevent the JWE from being edited (as no encryption key is passed) +> Note: You can't encrypt, without decrypt + +> Note: When decrypting a JWE, if you then access the jwt header, or jwt payload, you are accessing the ones of the JWE. If the JWE contains a JWT in his payload (nested JWT) then you will have to first decrypt the JWE, and then do another decode operation to decode the nested JWT, by selecting the payload of the JWE with a regex. + +Note: Supported algorithms are: + +- RSA_OAEP +- RSA_OAEP_256 +- ECDH_ES_A128KW +- ECDH_ES_A256KW + +## Tag table + +In this table you can find a description of all the tags available for this Operation based on the input Module. + +| input module (container) | Available tags | Required | value type | allowed values | +| --------------------------- | ----------------- | -------- | ---------------------- | -------------------------------------- | +| standard Operation | from | yes | str | head, body, url | +| | decode param or decode regex | yes | str or str(regex) | \* | +| decode Operation (type=jwt) | from | yes | str | jwt header, jwt payload, jwt signature | +| | decode param | yes | str(JSON path) | \* | +| | decode param or decode regex | yes | str(JSON path) or str(regex)| \* | +| \* | type | | str | xml, jwt | +| | encodings | | list[str] | base64, url, .. | +| | decode operations | | list[decode Operation] | \* | +| | checks | | list[check Operation] | \* | +| | edits | | list[edit Operation] | \* | +| | jwt check sig | | str(PEM) | PEM-encoded public key | +| | jwe decrypt | | str(PEM) | PEM-encoded private key | +| | jwe encrypt | | str(PEM) | PEM-encoded public key | + +### Recursive Decode operations + +When using the `from` tag in a recursive decode (one that is inside another), you can use - depending on the previous decode type - other sections, such as "jwt header" "jwt payload" .. + +Source: standard Operation (HTTP intercepted message) + +- `from`: (url, head, body) + +Source: decode Operation JWT + +- `from`: (jwt header, jwt payload, jwt signature) + +in recurdsive decode op, the decode param accepts different inputs, e.g. if previous decoded content is a jwt, decoded content will accept a JSON path + +Syntax example of a recursive decode operation: + +```json +"decode operations": [ + { + ..., + "decode operations": [ + { + "from": "somwhere in the previous decode", + "encodings": "asdasd", + } + ] + } +] +``` + +Example of decoding a jwt from the url of a message in the asd parameter, and then decode the jwt found inside of the jwt. + +```json +"decode operations": [ + { + "from": "url", + "type": "jwt", + "decode param": "asd", + "decode operations": [ + { + "from": "jwt header", + "type": "jwt", + "decode param": "$.something" + } + ] + } +] +``` + +Example of decoding a value in the url with a regex + +```json +{ + "from": "url", + "decode regex": "(?<=request=)([^&]+)", + "type": "jwt", + "edits": [ + { + "jwt from": "payload", + "jwt edit": "iss", + "value": "https://www.example.com/" + } + ] +} +``` + ## Session Operation Session operations are used to edit the session tracks given the actual test progress. @@ -522,6 +561,7 @@ The Checks tag inside an operation has a list of Check elements, which can be de - `is subset of` used to check that a matched JSON array is a subset of the given array. Is subset means that all the values in the matched array are present in the given array. - `matches regex` when using `check param` or `check` in json, you can use this tag to execute a regex to the matched value of the parameter or the value of the json key specified with the jsonpath - `not matches regex` as `match regex` but the result is true when the regex doesn't match + - `json schema compliant` to pass a json schema to be validated against what you match with `check`. Can only be used in jwt checks. Make sure to match the json schema with what you are selecting with the json PATH with `check`. > Note that you can use `check regex` OR `check` OR `check param`. @@ -968,3 +1008,8 @@ Examples:
- Removed OAuth metadata tag in test - Added signing of decoded jwt with private key inside Edit Operation - Added check of signature of jwt inside the decode operation +- Added edit operation also in Operations: message editing +- Added encode option to edit operation +- Removed "remove match word" from edit operation, just use edit regex with empty substitution +- Added decode regex in Decode Operations +- Added json schema validation in checks for jwts \ No newline at end of file diff --git a/tool/pom.xml b/tool/pom.xml index 6c17f4d..e75f03e 100644 --- a/tool/pom.xml +++ b/tool/pom.xml @@ -60,6 +60,21 @@ json-smart 2.5.0 + + org.apache.httpcomponents + httpclient + 4.5.13 + + + org.apache.httpcomponents + httpcore + 4.4.14 + + + com.networknt + json-schema-validator + 1.0.87 + diff --git a/tool/src/main/java/migt/BurpExtender.java b/tool/src/main/java/migt/BurpExtender.java index 945c997..818d1c8 100644 --- a/tool/src/main/java/migt/BurpExtender.java +++ b/tool/src/main/java/migt/BurpExtender.java @@ -11,8 +11,6 @@ /** * Main class executed by Burp - * - * @author Matteo Bitussi */ public class BurpExtender implements IBurpExtender, ITab, IProxyListener { @@ -58,7 +56,6 @@ public void registerExtenderCallbacks(IBurpExtenderCallbacks callbacks) { errorStream = new PrintStream(stdErr); mainPane = new GUI(); - mainPane.helpers = callbacks.getHelpers(); mainPane.callbacks = callbacks; mainPane.messageViewer = callbacks.createMessageEditor(mainPane.controller, false); mainPane.splitPane.setRightComponent(mainPane.messageViewer.getComponent()); @@ -165,7 +162,6 @@ private void processMatchedMsg(MessageType msg_type, HTTPReqRes message) { messageInfo.setHighlight("red"); - mainPane.act_active_op.helpers = helpers; mainPane.act_active_op.setAPI(new Operation_API(message, msg_type.msg_to_process_is_request)); mainPane.act_active_op.execute(); diff --git a/tool/src/main/java/migt/Check.java b/tool/src/main/java/migt/Check.java index ead0881..90b077d 100644 --- a/tool/src/main/java/migt/Check.java +++ b/tool/src/main/java/migt/Check.java @@ -1,6 +1,13 @@ package migt; +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; import com.jayway.jsonpath.JsonPath; +import com.networknt.schema.JsonSchema; +import com.networknt.schema.JsonSchemaFactory; +import com.networknt.schema.SpecVersion; +import com.networknt.schema.ValidationMessage; import org.json.JSONArray; import org.json.JSONException; import org.json.JSONObject; @@ -10,6 +17,7 @@ import java.util.ArrayList; import java.util.Iterator; import java.util.List; +import java.util.Set; import java.util.regex.Matcher; import java.util.regex.Pattern; @@ -17,8 +25,6 @@ /** * Check Object class. This object is used in Operations to check that a parameter or some text is in as specified. - * - * @author Matteo Bitussi */ public class Check extends Module { String what; // what to search @@ -138,6 +144,10 @@ public Check(JSONObject json_check) throws ParsingException { value_list.add(act_enc); } break; + case "json schema compliant": + this.op = JSON_SCHEMA_COMPLIANT; + this.op_val = json_check.getString("json schema compliant"); + break; case "matches regex": this.op = MATCHES_REGEX; this.op_val = json_check.getString("matches regex"); @@ -149,6 +159,8 @@ public Check(JSONObject json_check) throws ParsingException { case "url decode": url_decode = json_check.getBoolean("url decode"); break; + default: + throw new ParsingException("Invalid key:\"" + key + "\" used in Check Operation"); } } catch (JSONException e) { throw new ParsingException("error in parsing check: " + e); @@ -250,14 +262,13 @@ private boolean execute_http(HTTPReqRes message, if (msg_str.length() == 0) { applicable = true; - if (this.op != null && op == IS_NOT_PRESENT) { - return true; - } - return false; + return this.op != null && op == IS_NOT_PRESENT; } + msg_str = url_decode(msg_str); + // if a regex is present, execute it - if (!regex.equals("")) { + if (!regex.isEmpty()) { return execute_regex(msg_str); } @@ -302,6 +313,22 @@ private boolean execute_http(HTTPReqRes message, return true; } + private String url_decode(String string) { + if (url_decode) { + if (string.contains("+")) { + System.err.println("Warning! During a check on the value\"" + string + "\" a '+' symbol has been" + + "converted to a space, as it has been interpreted as url-encoded character. If you want to avoid" + + "this behaviour use 'url decode' tag set to false inside the check to disable url-decoding "); + } + try { + string = URLDecoder.decode(string, StandardCharsets.UTF_8); + } catch (IllegalArgumentException e) { + throw new RuntimeException("Failed URL-decode in check: " + e); + } + } + return string; + } + /** * Execute the json version of the check * @@ -455,6 +482,21 @@ private boolean execute_json() throws ParsingException { Matcher m = p.matcher(found); return !m.find(); } + case JSON_SCHEMA_COMPLIANT: { + JsonSchema schema = null; + JsonNode node = null; + try { + // parse the schema + schema = getJsonSchemaFromStringContent(op_val); + ObjectMapper mapper = new ObjectMapper(); + node = mapper.readTree(found); + } catch (JsonProcessingException e) { + throw new RuntimeException(e); + } + + Set errors = schema.validate(node); + return errors.isEmpty(); + } } return false; @@ -556,25 +598,10 @@ public void execute(List vars) throws ParsingException { if (use_variable) { // Substitute to the op_val variable (that contains the name), the value of the variable op_val = Tools.getVariableByName(op_val, vars).value; - } - // URL-decode matched content - // when a string contains a "+" character then, it is replaced with a space. - if (url_decode) { - /* - Pattern p = Pattern.compile("%[0-9a-fA-F]{2}"); - Matcher m = p.matcher(op_val); - if (m.find()) { - // if the content contains url-encoded characters then, url-decode the content - op_val = URLDecoder.decode(op_val, StandardCharsets.UTF_8); - } - */ - if (op_val.contains("+")) { - System.err.println("Warning! During a check on the value\"" + op_val + "\" a '+' symbol has been" + - "converted to a space, as it has been interpreted as url-encoded character. If you want to avoid" + - "this behaviour use 'url decode' tag set to false inside the check to disable url-decoding " ); - } - op_val = URLDecoder.decode(op_val, StandardCharsets.UTF_8); + // URL-decode variable value + // when a string contains a "+" character then, it is replaced with a space. + op_val = url_decode(op_val); } if (imported_api instanceof Operation_API) { @@ -612,6 +639,11 @@ public String toString() { return "check: " + what + (op == null ? "" : " " + op + ": " + op_val); } + protected JsonSchema getJsonSchemaFromStringContent(String schemaContent) { + JsonSchemaFactory factory = JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V4); + return factory.getSchema(schemaContent); + } + /** * enum containing all the possible check operations */ @@ -626,7 +658,8 @@ public enum CheckOps { IS_NOT_IN, IS_SUBSET_OF, MATCHES_REGEX, - NOT_MATCHES_REGEX; + NOT_MATCHES_REGEX, + JSON_SCHEMA_COMPLIANT; /** * Function that given a String, returns the corresponding CheckOps enum's value diff --git a/tool/src/main/java/migt/DecodeOperation.java b/tool/src/main/java/migt/DecodeOperation.java index 9b2b663..2772ddc 100644 --- a/tool/src/main/java/migt/DecodeOperation.java +++ b/tool/src/main/java/migt/DecodeOperation.java @@ -1,6 +1,5 @@ package migt; -import burp.IExtensionHelpers; import com.jayway.jsonpath.JsonPath; import org.json.JSONArray; import org.json.JSONObject; @@ -25,6 +24,7 @@ public class DecodeOperation extends Module { public String decoded_content; // the decoded content public String decode_target; // aka decode_param how to decode the raw content + public boolean is_regex; public DecodeOperationFrom from; // where the raw content is. Depending on the containing module, can be other things public List encodings; // the list of encoding to decode and rencode public DecodeOpType type; // the type of the decoded param (used only to edit its content) @@ -32,7 +32,6 @@ public class DecodeOperation extends Module { public List decodeOperations; // a list of decode operations to execute them recursevly public List editOperations; // a list of edit operations public boolean check_jwt = false; - boolean force_regex = false; // true if you want to decode by using a regex instead of a path in "decode param" JWT jwt; String what; @@ -59,6 +58,10 @@ public DecodeOperation(JSONObject decode_op_json) throws ParsingException { case "decode param": decode_target = decode_op_json.getString("decode param"); break; + case "decode regex": + decode_target = decode_op_json.getString("decode regex"); + is_regex = true; + break; case "encodings": JSONArray encodings = decode_op_json.getJSONArray("encodings"); Iterator it = encodings.iterator(); @@ -101,55 +104,12 @@ public DecodeOperation(JSONObject decode_op_json) throws ParsingException { jwt.decrypt = true; jwt.private_key_pem_enc = decode_op_json.getString("jwe encrypt"); jwt.public_key_pem_enc = decode_op_json.getString("jwe decrypt"); - case "force regex": - force_regex = decode_op_json.getBoolean("force regex"); - break; default: - throw new ParsingException("Unsupported key \"" + key + "\" in decode operation"); + throw new ParsingException("Invalid key:\"" + key + "\" used in decode operation"); } } } - /** - * Decodes a parameter from a message, given the message section and the list of encodings to be applied during - * decoding - * - * @param helpers IExtensionHelpers helpers object from Burp - * @param ms The message section that contains the parameter to be decoded - * @param encodings The list of encodings to be applied to decode the parameter - * @param messageInfo The message to be decoded - * @param isRequest True if the message containing the parameter is a request - * @param decode_param The name of the parameter to be decoded - * @return The decoded parameter as a string - * @throws ParsingException If problems are encountered during decoding - */ - public static String decodeParam(IExtensionHelpers helpers, - DecodeOperationFrom ms, - List encodings, - HTTPReqRes messageInfo, - Boolean isRequest, - String decode_param) throws ParsingException { - String decoded_param = ""; - switch (ms) { - case HEAD: - decoded_param = decode( - encodings, messageInfo.getHeadParam(isRequest, decode_param), helpers); - break; - case BODY: - decoded_param = decode( - encodings, messageInfo.getBodyRegex(isRequest, decode_param), helpers); - break; - case URL: - decoded_param = decode( - encodings, messageInfo.getUrlParam(decode_param), helpers); - break; - } - - decoded_param = Tools.removeNewline(decoded_param); - - return decoded_param; - } - /** * Decode the given string, with the given ordered encodings * Example taken from @@ -160,13 +120,12 @@ public static String decodeParam(IExtensionHelpers helpers, * @return the decoded string * @throws ParsingException if the decoding fails */ - public static String decode(List encodings, String encoded, IExtensionHelpers helpers) throws ParsingException { - // TODO: remove dependency from helpers + public static String decode(List encodings, String encoded) throws ParsingException { String actual = encoded; byte[] actual_b = null; boolean isActualString = true; - if (encoded.length() == 0) { + if (encoded.isEmpty()) { return ""; } @@ -174,18 +133,17 @@ public static String decode(List encodings, String encoded, IExtension switch (e) { case BASE64: if (isActualString) { - actual_b = helpers.base64Decode(actual); + actual_b = Base64.getDecoder().decode(actual); isActualString = false; } else { - actual_b = helpers.base64Decode(actual_b); + actual_b = Base64.getDecoder().decode(actual_b); } break; case URL: - if (isActualString) { - actual = helpers.urlDecode(actual); + actual = java.net.URLDecoder.decode(actual, StandardCharsets.UTF_8); } else { - actual = helpers.urlDecode(new String(actual_b)); + actual = java.net.URLDecoder.decode(new String(actual_b), StandardCharsets.UTF_8); isActualString = true; } break; @@ -250,7 +208,7 @@ public static String decode(List encodings, String encoded, IExtension * @param decoded the string to be encoded * @return the encoded string */ - public static String encode(List encodings, String decoded, IExtensionHelpers helpers) { + public static String encode(List encodings, String decoded) { String actual = decoded; byte[] actual_b = null; boolean isActualString = true; @@ -259,9 +217,9 @@ public static String encode(List encodings, String decoded, IExtension case BASE64: if (isActualString) { - actual = helpers.base64Encode(actual); + actual = Base64.getEncoder().encodeToString(actual.getBytes()); } else { - actual = helpers.base64Encode(actual_b); + Base64.getEncoder().encodeToString(actual_b); isActualString = true; } break; @@ -363,6 +321,61 @@ public static byte[] compress(byte[] data, boolean gzip) throws IOException { return output; } + /** + * Decodes a parameter from a message, given the message section and the list of encodings to be applied during + * decoding + * + * @param ms The message section that contains the parameter to be decoded + * @param encodings The list of encodings to be applied to decode the parameter + * @param messageInfo The message to be decoded + * @param isRequest True if the message containing the parameter is a request + * @param decode_param The name of the parameter to be decoded + * @return The decoded parameter as a string + * @throws ParsingException If problems are encountered during decoding + */ + public String decodeParam(DecodeOperationFrom ms, + List encodings, + HTTPReqRes messageInfo, + Boolean isRequest, + String decode_param) throws ParsingException { + String decoded_param = ""; + if (is_regex) { + switch (ms) { + case HEAD: + decoded_param = decode( + encodings, messageInfo.getHeadRegex(isRequest, decode_param)); + break; + case BODY: + decoded_param = decode( + encodings, messageInfo.getBodyRegex(isRequest, decode_param)); + break; + case URL: + decoded_param = decode( + encodings, messageInfo.getUrlRegex(decode_param)); + break; + } + } else { + switch (ms) { + case HEAD: + decoded_param = decode( + encodings, messageInfo.getHeadParam(isRequest, decode_param)); + break; + case BODY: + decoded_param = decode( + encodings, messageInfo.getBodyRegex(isRequest, decode_param)); + break; + case URL: + decoded_param = decode( + encodings, messageInfo.getUrlParam(decode_param)); + break; + } + } + + decoded_param = Tools.removeNewline(decoded_param); + + return decoded_param; + } + public void init() { decoded_content = ""; decode_target = ""; @@ -373,6 +386,7 @@ public void init() { type = DecodeOpType.NONE; editOperations = new ArrayList<>(); jwt = new JWT(); + is_regex = false; } @Override @@ -401,12 +415,10 @@ public void setAPI(DecodeOperation_API dop_api) { * Loads an Operation API * * @param api - * @param helpers * @throws ParsingException */ - public void loader(Operation_API api, IExtensionHelpers helpers) { + public void loader(Operation_API api) { // load api, extract needed things - this.helpers = helpers; this.imported_api = api; } @@ -415,10 +427,8 @@ public void loader(Operation_API api, IExtensionHelpers helpers) { * * @param api */ - public void loader(DecodeOperation_API api, IExtensionHelpers helpers) { + public void loader(DecodeOperation_API api) { this.imported_api = api; - this.helpers = helpers; - } /** @@ -430,20 +440,44 @@ public void loader(DecodeOperation_API api, IExtensionHelpers helpers) { @Override public API exporter() throws ParsingException { Collections.reverse(encodings); // Set the right order for encoding - String encoded = encode(encodings, decoded_content, helpers); + String encoded = encode(encodings, decoded_content); if (imported_api instanceof Operation_API) { - Tools.editMessageParam( - helpers, - decode_target, - from, - ((Operation_API) imported_api).message, - ((Operation_API) imported_api).is_request, - encoded, - true); + switch (from) { + case HEAD: + if (is_regex) { + ((Operation_API) imported_api).message.editHeadRegex( + ((Operation_API) imported_api).is_request, decode_target, encoded); + } else { + ((Operation_API) imported_api).message.editHeadParam( + ((Operation_API) imported_api).is_request, decode_target, encoded + ); // TODO test + } + break; + case BODY: + ((Operation_API) imported_api).message.editBodyRegex( + ((Operation_API) imported_api).is_request, decode_target, encoded + ); // TODO test + break; + case URL: + if (is_regex) { + ((Operation_API) imported_api).message.editUrlRegex( + decode_target, encoded + ); + } else { + ((Operation_API) imported_api).message.editUrlParam( + decode_target, encoded + ); //TODO test + } + break; + case JWT_HEADER: + case JWT_PAYLOAD: + case JWT_SIGNATURE: + throw new ParsingException("invalid from section in decode operation should be a message section"); + } // the previous function should already have updated the message inside api - return ((Operation_API) imported_api); + return imported_api; } else if (imported_api instanceof DecodeOperation_API) { return imported_api; } @@ -458,7 +492,6 @@ public API exporter() throws ParsingException { public void execute(List vars) throws ParsingException { if (imported_api instanceof Operation_API) { decoded_content = decodeParam( - helpers, from, encodings, ((Operation_API) imported_api).message, @@ -474,15 +507,16 @@ public void execute(List vars) throws ParsingException { String j = ((DecodeOperation_API) imported_api).getDecodedContent(from); String found = ""; - if (!force_regex) { - // https://github.com/json-path/JsonPath - try { - found = JsonPath.read(j, decode_target); // select what to decode - } catch (com.jayway.jsonpath.PathNotFoundException e) { + if (is_regex) { + Pattern p = Pattern.compile(decode_target); + Matcher m = p.matcher(j); + + if (m.find()) { + decoded_content = decode(encodings, m.group()); + } else { applicable = false; - result = false; - return; } + break; } else { Pattern pattern = Pattern.compile(decode_target); Matcher matcher = pattern.matcher(j); @@ -491,8 +525,9 @@ public void execute(List vars) throws ParsingException { break; } } - decoded_content = decode(encodings, found, helpers); + decoded_content = decode(encodings, found); break; + default: throw new UnsupportedOperationException( "the from you selected in the recursive decode operation is not yet supported"); @@ -519,7 +554,7 @@ public void execute(List vars) throws ParsingException { // executes recursive decode operations if (decodeOperations.size() != 0) { - executeDecodeOps(this, helpers, vars); + executeDecodeOps(this, vars); } // execute checks diff --git a/tool/src/main/java/migt/EditOperation.java b/tool/src/main/java/migt/EditOperation.java index 12767c9..0a43434 100644 --- a/tool/src/main/java/migt/EditOperation.java +++ b/tool/src/main/java/migt/EditOperation.java @@ -1,12 +1,15 @@ package migt; import com.jayway.jsonpath.PathNotFoundException; +import org.json.JSONArray; import org.json.JSONObject; import org.w3c.dom.Document; import org.xml.sax.SAXException; import samlraider.application.SamlTabController; import samlraider.helpers.XMLHelpers; +import java.util.ArrayList; +import java.util.Iterator; import java.util.List; import java.util.regex.Matcher; import java.util.regex.Pattern; @@ -40,6 +43,12 @@ public class EditOperation extends Module { // TXT TxtAction txt_action; String txt_action_name; + // Encode + List encodings; + + // Http message + MessageOperation.MessageOperationActions action; + HTTPReqRes.MessageSection msg_from; public EditOperation(JSONObject eop_json) throws ParsingException { init(); @@ -56,6 +65,7 @@ public EditOperation(JSONObject eop_json) throws ParsingException { break; case "value": // value of xml or other edits + // TODO add in doc, used also for messages value = eop_json.getString("value"); break; case "add tag": @@ -147,6 +157,56 @@ public EditOperation(JSONObject eop_json) throws ParsingException { txt_action = TxtAction.SAVE; txt_action_name = eop_json.getString("txt save"); break; + case "encodings": + JSONArray encodings = eop_json.getJSONArray("encodings"); + Iterator it = encodings.iterator(); + + while (it.hasNext()) { + String act_enc = (String) it.next(); + this.encodings.add( + DecodeOperation.Encoding.fromString(act_enc)); + } + break; + case "encode": + action = MessageOperation.MessageOperationActions.ENCODE; + what = eop_json.getString("encode"); + break; + case "from": + msg_from = HTTPReqRes.MessageSection.fromString(eop_json.getString("from")); + break; + case "edit": + action = MessageOperation.MessageOperationActions.EDIT; + what = eop_json.getString("edit"); + break; + case "edit regex": + action = MessageOperation.MessageOperationActions.EDIT_REGEX; + what = eop_json.getString("edit regex"); + break; + case "add": + action = MessageOperation.MessageOperationActions.ADD; + what = eop_json.getString("add"); + break; + case "remove": + action = MessageOperation.MessageOperationActions.REMOVE_PARAMETER; + what = eop_json.getString("remove"); + break; + // todo add action of message operation actions + default: + throw new ParsingException("Invalid key \"" + key + "\" in Edit Operation"); + } + } + + validate(); + } + + /** + * Validate this object's content. Used to check if the parsed tags are valid. + */ + @Override + public void validate() throws ParsingException { + if (action == MessageOperation.MessageOperationActions.ENCODE) { + if (encodings.isEmpty()) { + throw new ParsingException("Using encode in Edit Operation, but not providing encodings"); } } } @@ -165,203 +225,332 @@ public void init() { sign = false; txt_action_name = ""; what = ""; + encodings = new ArrayList<>(); } - public void loader(DecodeOperation_API dop_api) { - // TODO - if (dop_api == null) { - throw new RuntimeException("loaded api is null"); - } - this.imported_api = dop_api; + public void setAPI(Operation_API api) { + imported_api = api; } - public DecodeOperation_API exporter() { - return (DecodeOperation_API) imported_api; + public API exporter() { + return imported_api; } - public void execute(List vars) throws ParsingException { - if (imported_api instanceof DecodeOperation_API) { - // the edit operation is being executed inside a Decode Operation - // If a variable value has to be used, read the value of the variable at execution time - if (!use.equals("")) { - Var v = getVariableByName(use, vars); - if (!v.isMessage) { - value = v.value; - } else { - throw new ParsingException("Error while using variable, expected text var, got message var"); + public void execute_decodeOperation_API(List vars) throws ParsingException { + // the edit operation is being executed inside a Decode Operation + DecodeOperation_API tmp_imported_api = (DecodeOperation_API) imported_api; + + switch (((DecodeOperation_API) imported_api).type) { + case XML: + //SAML Remove signatures + if (self_sign | remove_signature) { + Document document = null; + try { + XMLHelpers xmlHelpers = new XMLHelpers(); + document = xmlHelpers.getXMLDocumentOfSAMLMessage(((DecodeOperation_API) imported_api).xml); + saml_original_cert = xmlHelpers.getCertificate(document.getDocumentElement()); + if (saml_original_cert == null) { + System.out.println("SAML Certificate not found in decoded parameter"); + applicable = false; + } + edited_xml = SamlTabController.removeSignature_edit(((DecodeOperation_API) imported_api).xml); + } catch (SAXException e) { + e.printStackTrace(); + } } - } - DecodeOperation_API tmp_imported_api = (DecodeOperation_API) imported_api; - - switch (((DecodeOperation_API) imported_api).type) { - case XML: - //SAML Remove signatures - if (self_sign | remove_signature) { - Document document = null; - try { - XMLHelpers xmlHelpers = new XMLHelpers(); - document = xmlHelpers.getXMLDocumentOfSAMLMessage(((DecodeOperation_API) imported_api).xml); - saml_original_cert = xmlHelpers.getCertificate(document.getDocumentElement()); - if (saml_original_cert == null) { - System.out.println("SAML Certificate not found in decoded parameter"); - applicable = false; - } - edited_xml = SamlTabController.removeSignature_edit(((DecodeOperation_API) imported_api).xml); - } catch (SAXException e) { - e.printStackTrace(); - } + switch (xml_action) { + case ADD_TAG: + edited_xml = XML.addTag(edited_xml, + xml_tag, + xml_action_name, + value, + xml_occurrency); + break; + case ADD_ATTR: + edited_xml = XML.addTagAttribute(edited_xml, + xml_tag, + xml_action_name, + value, + xml_occurrency); + break; + case EDIT_TAG: + edited_xml = XML.editTagValue(edited_xml, + xml_action_name, + value, + xml_occurrency); + break; + case EDIT_ATTR: + edited_xml = XML.editTagAttributes(edited_xml, + xml_tag, + xml_action_name, + value, + xml_occurrency); + break; + case REMOVE_TAG: + edited_xml = XML.removeTag(edited_xml, + xml_action_name, + xml_occurrency); + break; + case REMOVE_ATTR: + edited_xml = XML.removeTagAttribute(edited_xml, + xml_tag, + xml_action_name, + xml_occurrency); + break; + case SAVE_TAG: { + String to_save = XML.getTagValaue(edited_xml, + xml_action_name, + xml_occurrency); + Var v = new Var(); + v.name = save_as; + v.isMessage = false; + v.value = to_save; + vars.add(v); + break; } + case SAVE_ATTR: + String to_save = XML.getTagAttributeValue(edited_xml, + xml_tag, xml_action_name, + xml_occurrency); + Var v = new Var(); + v.name = save_as; + v.isMessage = false; + v.value = to_save; + vars.add(v); + break; + } - switch (xml_action) { - case ADD_TAG: - edited_xml = XML.addTag(edited_xml, - xml_tag, - xml_action_name, - value, - xml_occurrency); - break; - case ADD_ATTR: - edited_xml = XML.addTagAttribute(edited_xml, - xml_tag, - xml_action_name, - value, - xml_occurrency); - break; - case EDIT_TAG: - edited_xml = XML.editTagValue(edited_xml, - xml_action_name, - value, - xml_occurrency); - break; - case EDIT_ATTR: - edited_xml = XML.editTagAttributes(edited_xml, - xml_tag, - xml_action_name, - value, - xml_occurrency); - break; - case REMOVE_TAG: - edited_xml = XML.removeTag(edited_xml, - xml_action_name, - xml_occurrency); - break; - case REMOVE_ATTR: - edited_xml = XML.removeTagAttribute(edited_xml, - xml_tag, - xml_action_name, - xml_occurrency); - break; - case SAVE_TAG: { - String to_save = XML.getTagValaue(edited_xml, - xml_action_name, - xml_occurrency); - Var v = new Var(); - v.name = save_as; - v.isMessage = false; - v.value = to_save; - vars.add(v); - break; + if (self_sign && !edited_xml.equals("")) { + // SAML re-sign + edited_xml = SamlTabController.resignAssertion_edit(edited_xml, saml_original_cert); + } + + tmp_imported_api.xml = edited_xml; + applicable = true; + break; + + case JWT: + if (jwt_section != null) { // if only sign, there will be no jwt section + try { + switch (jwt_section) { + case HEADER: + tmp_imported_api.jwt.header = Tools.editJson( + jwt_action, tmp_imported_api.jwt.header, what, vars, save_as, value); + break; + case PAYLOAD: + // TODO: pass newvalue + tmp_imported_api.jwt.payload = Tools.editJson( + jwt_action, tmp_imported_api.jwt.payload, what, vars, save_as, value); + break; + case SIGNATURE: + tmp_imported_api.jwt.signature = Tools.editJson( + jwt_action, tmp_imported_api.jwt.signature, what, vars, save_as, value); + break; } - case SAVE_ATTR: - String to_save = XML.getTagAttributeValue(edited_xml, - xml_tag, xml_action_name, - xml_occurrency); - Var v = new Var(); - v.name = save_as; - v.isMessage = false; - v.value = to_save; - vars.add(v); - break; + } catch (PathNotFoundException e) { + this.applicable = false; + this.result = false; + return; } + applicable = true; + } else if (sign) { + applicable = true; + tmp_imported_api.jwt.sign = true; + tmp_imported_api.jwt.private_key_pem = jwt_private_key_pem; + } else { + throw new ParsingException("missing jwt section in Edit operation"); + } - if (self_sign && !edited_xml.equals("")) { - // SAML re-sign - edited_xml = SamlTabController.resignAssertion_edit(edited_xml, saml_original_cert); - } + break; - tmp_imported_api.xml = edited_xml; - applicable = true; - break; + case NONE: + Pattern p = Pattern.compile(Pattern.quote(txt_action_name)); + Matcher m = p.matcher(tmp_imported_api.txt); + + if (txt_action == null) { + throw new ParsingException("txt action not specified"); + } - case JWT: - if (jwt_section != null) { // if only sign, there will be no jwt section - try { - switch (jwt_section) { - case HEADER: - tmp_imported_api.jwt.header = Tools.editJson( - jwt_action, tmp_imported_api.jwt.header, what, vars, save_as, value); - break; - case PAYLOAD: - // TODO: pass newvalue - tmp_imported_api.jwt.payload = Tools.editJson( - jwt_action, tmp_imported_api.jwt.payload, what, vars, save_as, value); - break; - case SIGNATURE: - tmp_imported_api.jwt.signature = Tools.editJson( - jwt_action, tmp_imported_api.jwt.signature, what, vars, save_as, value); - break; - } - } catch (PathNotFoundException e) { - this.applicable = false; - this.result = false; - return; + switch (txt_action) { + case REMOVE: + tmp_imported_api.txt = m.replaceAll(""); + + break; + case EDIT: + tmp_imported_api.txt = m.replaceAll(value); + + break; + case ADD: + while (m.find()) { + int index = m.end(); + String before = tmp_imported_api.txt.substring(0, index); + String after = tmp_imported_api.txt.substring(index); + tmp_imported_api.txt = before + value + after; + break; + } + break; + case SAVE: + String val = ""; + while (m.find()) { + val = m.group(); + break; } - applicable = true; - } else if (sign) { - applicable = true; - tmp_imported_api.jwt.sign = true; - tmp_imported_api.jwt.private_key_pem = jwt_private_key_pem; - } else { - throw new ParsingException("missing jwt section in Edit operation"); - } - break; + Var v = new Var(); + v.name = save_as; + v.isMessage = false; + v.value = val; + vars.add(v); + break; + } + applicable = true; + break; + } + imported_api = tmp_imported_api; + } - case NONE: - Pattern p = Pattern.compile(Pattern.quote(txt_action_name)); - Matcher m = p.matcher(tmp_imported_api.txt); + public void execute_Operation_API() throws ParsingException { + HTTPReqRes message = ((Operation_API) imported_api).message; + boolean is_request = ((Operation_API) imported_api).is_request; // if the message to edit is the request - if (txt_action == null) { - throw new ParsingException("txt action not specified"); - } + switch (msg_from) { + case URL: { + if (!is_request) { + throw new RuntimeException("trying to access the URL of a response message"); + } - switch (txt_action) { - case REMOVE: - tmp_imported_api.txt = m.replaceAll(""); + switch (action) { + case REMOVE_PARAMETER: + message.removeUrlParam(what); + break; + case REMOVE_MATCH_WORD: + // TODO: remove, can be done with edit regex + break; + case EDIT: + message.editUrlParam(what, value); + break; + case EDIT_REGEX: + message.editUrlRegex(what, value); + break; + case ADD: + message.addUrlParam(what, value); + break; + case ENCODE: + String old_value = message.getUrlParam(what); + String new_value = DecodeOperation.encode( + encodings, + old_value + ); + message.editUrlParam(what, new_value); + break; + } + break; + } - break; - case EDIT: - tmp_imported_api.txt = m.replaceAll(value); + case HEAD: { + switch (action) { + case REMOVE_PARAMETER: + message.removeHeadParameter(is_request, what); + break; + case REMOVE_MATCH_WORD: + // TODO: remove, can be done with edit regex + break; + case EDIT: + message.editHeadParam(is_request, what, value); + break; + case EDIT_REGEX: + // For each header applies regex + message.editHeadRegex(is_request, what, value); + break; + case ADD: + message.addHeadParameter(is_request, what, value); + break; + case ENCODE: + String old_value = message.getHeadParam(is_request, what); + String new_value = DecodeOperation.encode( + encodings, + old_value + ); + message.editHeadParam(is_request, what, new_value); + break; + } + break; + } - break; - case ADD: - while (m.find()) { - int index = m.end(); - String before = tmp_imported_api.txt.substring(0, index); - String after = tmp_imported_api.txt.substring(index); - tmp_imported_api.txt = before + value + after; - break; - } - break; - case SAVE: - String val = ""; - while (m.find()) { - val = m.group(); - break; - } + case BODY: { + switch (action) { + // TODO add also edits based on Content-Type? + case REMOVE_PARAMETER: + // nothing + break; + case REMOVE_MATCH_WORD: + // nothing + break; + case EDIT: + // nothing + break; + case EDIT_REGEX: + // edit matched value + message.editBodyRegex(is_request, what, value); + break; + case ADD: + // append value + message.addBody(is_request, value); + break; + case ENCODE: + // encode matched value + String old_value = message.getBodyRegex(is_request, what); + String new_value = DecodeOperation.encode( + encodings, + old_value + ); + message.editBodyRegex(is_request, what, new_value); + break; + } + break; + } - Var v = new Var(); - v.name = save_as; - v.isMessage = false; - v.value = val; - vars.add(v); - break; - } - applicable = true; - break; + case RAW: { + switch (action) { + //TODO + case REMOVE_PARAMETER: + break; + case REMOVE_MATCH_WORD: + break; + case EDIT: + break; + case EDIT_REGEX: + // TODO + break; + case ADD: + break; + case ENCODE: + //TODO + break; + } + break; + } + } + applicable = true; // check if there is a better place for this + ((Operation_API) imported_api).message = message; + } + + public void execute(List vars) throws ParsingException { + // If a variable value has to be used, read the value of the variable at execution time + if (!use.equals("")) { + Var v = getVariableByName(use, vars); + if (!v.isMessage) { + value = v.value; + } else { + throw new ParsingException("Error while using variable, expected text var, got message var"); } - imported_api = tmp_imported_api; + } + + if (imported_api instanceof DecodeOperation_API) { + execute_decodeOperation_API(vars); + } else if (imported_api instanceof Operation_API) { + execute_Operation_API(); } } diff --git a/tool/src/main/java/migt/ExecuteActiveListener.java b/tool/src/main/java/migt/ExecuteActiveListener.java index 3487a4c..a339a12 100644 --- a/tool/src/main/java/migt/ExecuteActiveListener.java +++ b/tool/src/main/java/migt/ExecuteActiveListener.java @@ -2,8 +2,6 @@ /** * Listener class for ExecuteActive class - * - * @author Matteo Bitussi */ public interface ExecuteActiveListener { diff --git a/tool/src/main/java/migt/ExecuteActives.java b/tool/src/main/java/migt/ExecuteActives.java index 6ef2ff7..5b74a61 100644 --- a/tool/src/main/java/migt/ExecuteActives.java +++ b/tool/src/main/java/migt/ExecuteActives.java @@ -8,8 +8,6 @@ /** * Class which executes actives tests, has to be run as a thread - * - * @author Matteo Bitussi */ public class ExecuteActives implements Runnable { final Object waiting; // the lock on which the thread will wait diff --git a/tool/src/main/java/migt/ExecutePassives.java b/tool/src/main/java/migt/ExecutePassives.java index 411832b..bf06cb6 100644 --- a/tool/src/main/java/migt/ExecutePassives.java +++ b/tool/src/main/java/migt/ExecutePassives.java @@ -1,7 +1,5 @@ package migt; -import burp.IExtensionHelpers; - import java.util.ArrayList; import java.util.HashMap; import java.util.List; @@ -9,13 +7,10 @@ /** * Class used to execute passive tests, it implements Runnable, it should be executed as a Thread. To communicate with * the tread you can use the ExecutePassivesListener listener class. - * - * @author Matteo Bitussi */ public class ExecutePassives implements Runnable { final Object lock = new Object(); public List passives; - IExtensionHelpers helpers; ExecutePassiveListener listener; List messageTypes; boolean finished; @@ -24,17 +19,14 @@ public class ExecutePassives implements Runnable { /** * Used to instantiate an ExecutePassives object * - * @param helpers IExtensionHelpers instance of Burp * @param passiveTests The list of passive tests to execute * @param listener the listener for this ExecutePassives Object, used to communicate with the thread * @param msg_types the list of message types needed by the tests */ - public ExecutePassives(IExtensionHelpers helpers, - List passiveTests, + public ExecutePassives(List passiveTests, ExecutePassiveListener listener, List msg_types) { this.passives = passiveTests; - this.helpers = helpers; this.listener = listener; this.messageTypes = msg_types; this.finished = false; @@ -103,11 +95,7 @@ public void run() { boolean res = false; try { - res = Tools.executePassiveTest( - actual_test, - executedSession.messages, - helpers, - messageTypes); + res = actual_test.execute(executedSession.messages, messageTypes); } catch (ParsingException e) { actual_test.applicable = false; } diff --git a/tool/src/main/java/migt/ExecuteTrack.java b/tool/src/main/java/migt/ExecuteTrack.java index 4094273..908c35c 100644 --- a/tool/src/main/java/migt/ExecuteTrack.java +++ b/tool/src/main/java/migt/ExecuteTrack.java @@ -15,8 +15,6 @@ /** * Class that executes a Session Track (series of user actions). It launches a browser with Selenium to automate the * actions - * - * @author Matteo Bitussi */ public class ExecuteTrack implements Runnable { private static String snapshot = ""; diff --git a/tool/src/main/java/migt/ExecuteTrackListener.java b/tool/src/main/java/migt/ExecuteTrackListener.java index df85888..f341947 100644 --- a/tool/src/main/java/migt/ExecuteTrackListener.java +++ b/tool/src/main/java/migt/ExecuteTrackListener.java @@ -2,8 +2,6 @@ /** * Listener for the ExectuteTrack Object - * - * @author Matteo Bitussi */ public interface ExecuteTrackListener { diff --git a/tool/src/main/java/migt/GUI.java b/tool/src/main/java/migt/GUI.java index ef01a0d..1327dd0 100644 --- a/tool/src/main/java/migt/GUI.java +++ b/tool/src/main/java/migt/GUI.java @@ -1,6 +1,9 @@ package migt; -import burp.*; +import burp.IBurpExtenderCallbacks; +import burp.IHttpService; +import burp.IMessageEditor; +import burp.IMessageEditorController; import com.google.gson.Gson; import org.json.JSONArray; import org.json.JSONException; @@ -43,9 +46,6 @@ public void write(int b) { /** * This class contains the GUI for the plugin, also a lot of functionality methods - * - * @author Matteo Bitussi - * @author Wendy Barreto */ public class GUI extends JSplitPane { private static DefaultTableModel resultTableModel; @@ -97,7 +97,6 @@ public class GUI extends JSplitPane { JTabbedPane bot_tabbed; Map bot_tabs_index; HTTPReqRes viewedMessage; - IExtensionHelpers helpers; IBurpExtenderCallbacks callbacks; List sessions_names; Map session_port; @@ -245,7 +244,7 @@ private void readMsgDefFile() { tmp += myReader.nextLine(); } myReader.close(); - messageTypes = Tools.readMsgTypeFromJson(tmp); + messageTypes = Tools.readMsgTypesFromJson(tmp); } catch (ParsingException e) { lblOutput.setText("Invalid message type in message type definition file"); e.printStackTrace(); @@ -256,7 +255,7 @@ private void readMsgDefFile() { FileWriter w = new FileWriter(MSG_DEF_PATH); w.write(Tools.getDefaultJSONMsgType()); w.close(); - messageTypes = Tools.readMsgTypeFromJson(Tools.getDefaultJSONMsgType()); + messageTypes = Tools.readMsgTypesFromJson(Tools.getDefaultJSONMsgType()); } } catch (ParsingException e) { e.printStackTrace(); @@ -524,6 +523,7 @@ public void onExecuteStart() { public void onExecuteDone() { if (passives.size() == 0) { update_gui_test_results(); + testSuite.log_test_suite(LOG_FOLDER); lblOutput.setText("Done. Executed Passive Tests: " + (passives.isEmpty() ? 0 : passives.size()) @@ -684,7 +684,7 @@ public ArrayList onTrackExecuteDone() { } }; - ExecutePassives expa = new ExecutePassives(helpers, + ExecutePassives expa = new ExecutePassives( passives, listener, messageTypes); diff --git a/tool/src/main/java/migt/HTTPReqRes.java b/tool/src/main/java/migt/HTTPReqRes.java index 7388cbc..d450231 100644 --- a/tool/src/main/java/migt/HTTPReqRes.java +++ b/tool/src/main/java/migt/HTTPReqRes.java @@ -4,7 +4,14 @@ import burp.IHttpRequestResponse; import burp.IHttpRequestResponsePersisted; import burp.IHttpService; - +import org.apache.http.NameValuePair; +import org.apache.http.client.utils.URLEncodedUtils; +import org.apache.http.message.BasicNameValuePair; + +import java.net.MalformedURLException; +import java.net.URI; +import java.net.URISyntaxException; +import java.net.URL; import java.nio.charset.StandardCharsets; import java.util.ArrayList; import java.util.Arrays; @@ -14,8 +21,6 @@ /** * Class which is intended to substitute the IHTTPRequestResponse one, because of serialization support - * - * @author Matteo Bitussi */ public class HTTPReqRes implements Cloneable { static public int instances; @@ -82,7 +87,7 @@ public HTTPReqRes(IHttpRequestResponsePersisted message, IExtensionHelpers helpe * @param helpers an istance of the IExtensionHelpers * @param isRequest true if the message is a request, false otherwise */ - public HTTPReqRes(IHttpRequestResponse message, IExtensionHelpers helpers, Boolean isRequest, int index) { + public HTTPReqRes(IHttpRequestResponse message, IExtensionHelpers helpers, boolean isRequest, int index) { if (!isRequest) { this.isResponse = true; this.setResponse(message.getResponse()); @@ -154,11 +159,24 @@ public void setUrlHeader(String url_header) { this.headers_req.set(0, url_header); } + /** + * returns true if the message has the body + * + * @param isRequest select the request or the response message + * @return true if it has body, false otherwise + */ + public boolean hasBody(boolean isRequest) { + if (isRequest && this.body_offset_req == 0) { + return false; + } + return isRequest || this.body_offset_resp != 0; + } + public byte[] getBody(boolean isRequest) { - if (isRequest && (this.body_offset_req == 0 | this.request == null | this.request.length == 0)) { + if (isRequest && (!this.hasBody(isRequest) | this.request == null | this.request.length == 0)) { throw new RuntimeException("called getBody, but class is not properly initialized"); } - if (!isRequest && (this.body_offset_resp == 0 | this.response == null | this.response.length == 0)) { + if (!isRequest && (!this.hasBody(isRequest) | this.response == null | this.response.length == 0)) { throw new RuntimeException("called getBody, but class is not properly initialized"); } @@ -178,21 +196,6 @@ public List getHeaders(boolean isRequest) { return isRequest ? this.headers_req : this.headers_resp; } - /** - * Used to build the message based on the changes made - */ - private byte[] build_message(IExtensionHelpers helpers, boolean isRequest) { - // TODO: this could be written avoiding helpers class - // TODO: url is not updated - if (isRequest) { - this.request = helpers.buildHttpMessage(headers_req, getBody(true)); - return this.request; - } else { - this.response = helpers.buildHttpMessage(headers_resp, getBody(false)); - return this.response; - } - } - /** * Builds the message taking the headers and the body, without using the burp's helpers. * @@ -231,29 +234,18 @@ public byte[] build_message(boolean isRequest) { * @param isRequest to specify the request or the response * @return the message */ - public byte[] getMessage(boolean isRequest, IExtensionHelpers helpers) { + public byte[] getMessage(boolean isRequest) { if (isRequest && this.request == null) { throw new RuntimeException("Called getMessage on a message that is not initialized"); } else if (!isRequest && this.response == null) { throw new RuntimeException("Called getMessage on a message that is not initialized"); } - build_message(helpers, isRequest); - - return isRequest ? request : response; - } + build_message(isRequest); - /** - * Get the message without updating it with the changes - * - * @param isRequest - * @return - */ - public byte[] getMessage(boolean isRequest) { - // TODO: this is probably a source of bugs, called without noticing that it doesnt get the updated message - if (isRequest && this.request == null) { - throw new RuntimeException("Called getMessage on a message that is not initialized"); - } else if (!isRequest && this.response == null) { - throw new RuntimeException("Called getMessage on a message that is not initialized"); + if (isRequest) { + request = build_message(isRequest); + } else { + response = build_message(isRequest); } return isRequest ? request : response; @@ -326,7 +318,12 @@ public String getRequest_url() { } public void setRequest_url(String request_url) { - this.request_url = request_url; + if (this.request_url == null) { + this.request_url = request_url; + } else { + this.request_url = request_url; + updateHeadersWHurl(); + } } public String getHost() { @@ -369,16 +366,217 @@ public String getUrlParam(String param) { throw new RuntimeException("Trying to access the url of a response message"); } - Pattern pattern = Pattern.compile("(?<=" + Pattern.quote(param) + "=)[^$\\n&\\s]*"); - Matcher matcher = pattern.matcher(this.request_url); + List params = new ArrayList<>(); + + try { + params = URLEncodedUtils.parse( + new URI(request_url), StandardCharsets.UTF_8 + ); + } catch (URISyntaxException e) { + throw new RuntimeException(e); + } + + for (NameValuePair p : params) { + if (p.getName().equals(param)) { + return p.getValue(); + } + } + + return ""; + } + + /** + * Execute a regex over the complete url and return the value matched + * + * @param regex the regex to execute + * @return the matched value + */ + public String getUrlRegex(String regex) { + if (!isRequest || request_url == null) { + throw new RuntimeException("Trying to access the url of a response message"); + } + String res = ""; - while (matcher.find()) { - res = matcher.group(); - break; + Pattern p = Pattern.compile(regex); + Matcher m = p.matcher(request_url); + if (m.find()) { + res = m.group(); } return res; } + /** + * Edit this message's URL with a regex, everything matched will be replaced by new_value + * + * @param regex the regex to execute + * @param new_value the value to substitute to matched content + */ + public void editUrlRegex(String regex, String new_value) { + String old_url = getUrl(); + Pattern p = Pattern.compile(regex); + Matcher m = p.matcher(old_url); + String new_url = m.replaceAll(new_value); + setRequest_url(new_url); + } + + /** + * Edits the given parameter value with the new given value + * + * @param param the parameter name + * @param value the new value of the parameter + */ + public void editUrlParam(String param, String value) throws ParsingException { + if (!isRequest || request_url == null) { + throw new RuntimeException("Trying to access the url of a response message"); + } + + List params = new ArrayList<>(); + + try { + params = URLEncodedUtils.parse( + new URI(request_url), StandardCharsets.UTF_8 + ); + } catch (URISyntaxException e) { + throw new RuntimeException(e); + } + + int indx = -1; + int c = 0; + for (NameValuePair p : params) { + if (p.getName().equals(param)) { + indx = c; + } + c++; + } + + if (indx == -1) { + throw new ParsingException("Could not find parameter " + param + " in url"); + } + + params.set(indx, new BasicNameValuePair(param, value)); + + String new_query = URLEncodedUtils.format(params, "utf-8"); + + URL url = null; + try { + url = new URL(request_url); + } catch (MalformedURLException e) { + throw new RuntimeException(e); + } + + request_url = request_url.replaceAll( + "\\Q" + java.util.regex.Matcher.quoteReplacement(url.getQuery()) + "\\E", + new_query); + + updateHeadersWHurl(); + } + + /** + * Removes the given param from the request url query parameters + * + * @param name param name + */ + public void removeUrlParam(String name) throws ParsingException { + if (!isRequest || request_url == null) { + throw new RuntimeException("Trying to access the url of a response message"); + } + + List params = new ArrayList<>(); + + try { + params = URLEncodedUtils.parse( + new URI(request_url), StandardCharsets.UTF_8 + ); + } catch (URISyntaxException e) { + throw new RuntimeException(e); + } + + int indx = -1; + int c = 0; + for (NameValuePair p : params) { + if (p.getName().equals(name)) { + indx = c; + } + c++; + } + + if (indx == -1) { + throw new ParsingException("Could not find parameter " + name + " in url"); + } + + params.remove(indx); + + String new_query = URLEncodedUtils.format(params, "utf-8"); + + URL url = null; + try { + url = new URL(request_url); + } catch (MalformedURLException e) { + throw new RuntimeException(e); + } + + request_url = request_url.replaceAll( + "\\Q" + java.util.regex.Matcher.quoteReplacement(url.getQuery()) + "\\E", + new_query); + + updateHeadersWHurl(); + } + + /** + * Adds an url query parameter to the request url. If parameter already present, concatenate new value to old. + * + * @param name the name of the new parameter + * @param value the value of the new parameter + */ + public void addUrlParam(String name, String value) { + if (!isRequest || request_url == null) { + throw new RuntimeException("Trying to access the url of a response message"); + } + + List params = new ArrayList<>(); + + try { + params = URLEncodedUtils.parse( + new URI(request_url), StandardCharsets.UTF_8 + ); + } catch (URISyntaxException e) { + throw new RuntimeException(e); + } + + int c = 0; + boolean found = false; + for (NameValuePair p : params) { + if (p.getName().equals(name)) { + found = true; + break; + } + c += 1; + } + + if (found) { + String old_value = params.get(c).getValue(); + old_value += value; + params.set(c, new BasicNameValuePair(name, old_value)); + } else { + params.add(new BasicNameValuePair(name, value)); + } + + String new_query = URLEncodedUtils.format(params, "utf-8"); + + URL url = null; + try { + url = new URL(request_url); + } catch (MalformedURLException e) { + throw new RuntimeException(e); + } + + request_url = request_url.replaceAll( + "\\Q" + java.util.regex.Matcher.quoteReplacement(url.getQuery()) + "\\E", + new_query); + + updateHeadersWHurl(); + } + /** * Get the given parameter value from the head * @@ -386,7 +584,7 @@ public String getUrlParam(String param) { * @param param the parameter name to be searched * @return the value of the parameter */ - public String getHeadParam(Boolean isRequest, String param) { + public String getHeadParam(boolean isRequest, String param) { List headers = isRequest ? this.headers_req : this.headers_resp; for (String s : headers) { @@ -398,8 +596,35 @@ public String getHeadParam(Boolean isRequest, String param) { return ""; } + /** + * Execute a regex over the headers and return the first value matched + * + * @param regex the regex to execute + * @return the matched value + */ + public String getHeadRegex(boolean isRequest, String regex) { + List headers = isRequest ? this.headers_req : this.headers_resp; + + String res = ""; + Pattern p = Pattern.compile(regex); + for (String s : headers) { + Matcher m = p.matcher(s); + if (m.find()) { + res = m.group(); + } + } + return res; + } - public void editHeadParam(Boolean isRequest, String param, String new_value) { + + /** + * Edits the Header of the given message + * + * @param isRequest select the request or teh response + * @param param the name of the header + * @param new_value the new value + */ + public void editHeadParam(boolean isRequest, String param, String new_value) { List headers = isRequest ? this.headers_req : this.headers_resp; int indx = -1; @@ -418,10 +643,62 @@ public void editHeadParam(Boolean isRequest, String param, String new_value) { } } + /** + * Edit the header of the message with a regex + * + * @param isRequest select the request or response message + * @param regex the regex to execute + * @param new_value the new value to substitute + */ + public void editHeadRegex(boolean isRequest, String regex, String new_value) { + if (!isResponse) { + throw new RuntimeException("tried to edit headers of response not yet received"); + } + + getHeaders(isRequest).replaceAll(header -> { + Pattern p = Pattern.compile(regex); + Matcher m = p.matcher(header); + header = m.replaceAll(new_value); + return header; + }); + } + + /** + * Adds a Header to the given message + * + * @param isRequest if the message to edit is the request or the response + * @param name the name of the new header + * @param value the value of the new header + */ public void addHeadParameter(boolean isRequest, String name, String value) { - (isRequest ? this.headers_req : this.headers_resp).add(name + ": " + value); + List headers = isRequest ? this.headers_req : this.headers_resp; + + int c = 0; + boolean found = false; + + for (String h : headers) { + if (h.startsWith(name + ":")) { + found = true; + break; + } + c += 1; + } + + if (found) { + String old_header = headers.get(c); + old_header += value; + headers_req.set(c, old_header); + } else { + headers.add(name + ": " + value); + } } + /** + * Removes the header from the given message + * + * @param isRequest select the request or the response + * @param name the name of the header + */ public void removeHeadParameter(boolean isRequest, String name) { List headers = isRequest ? this.headers_req : this.headers_resp; @@ -444,11 +721,11 @@ public void removeHeadParameter(boolean isRequest, String name) { * everything matched will be returned as a value * * @param isRequest if the message is a request - * @param param the parameter to be searched as a regex, everything matched by this will be returned as a value + * @param regex the parameter to be searched as a regex, everything matched by this will be returned as a value * @return the value of the parameter */ - public String getBodyRegex(Boolean isRequest, String param) { - Pattern pattern = Pattern.compile(param); + public String getBodyRegex(boolean isRequest, String regex) { + Pattern pattern = Pattern.compile(regex); Matcher matcher = pattern.matcher(new String(getBody(isRequest), StandardCharsets.UTF_8)); String res = ""; @@ -459,6 +736,86 @@ public String getBodyRegex(Boolean isRequest, String param) { return res; } + /** + * Edit the body of the message. Replaces the matched content of the regex with the new value. + * + * @param isRequest select the request or response message + * @param regex the regex to execute + * @param new_value the new value to substitute + */ + public void editBodyRegex(boolean isRequest, String regex, String new_value) { + Pattern pattern = Pattern.compile(regex); + Matcher matcher = pattern.matcher(new String(getBody(isRequest), StandardCharsets.UTF_8)); + + String new_body = matcher.replaceFirst(new_value); + setBody(isRequest, new_body); + } + + /** + * Append to the body of the message the given value. If the message doesn't have a body it creates it. + * + * @param isRequest select the request or response message + * @param new_value the value to append + */ + public void addBody(boolean isRequest, String new_value) { + if (hasBody(isRequest)) { + String body = new String(getBody(isRequest)); + body += new_value; + setBody(isRequest, body); + } else { + if (isRequest) { + body_offset_req = 1; // this is for recognizing body in hasBody method + body_req = new_value.getBytes(StandardCharsets.UTF_8); + } else { + body_offset_resp = 1; // this is for recognizing body in hasBody method + body_resp = new_value.getBytes(StandardCharsets.UTF_8); + } + } + } + + /** + * Updates the headers in this request message with the acctual url value + */ + public void updateHeadersWHurl() throws RuntimeException { + if (!isRequest || request_url == null) { + throw new RuntimeException("Trying to access the url of a response message"); + } + + URL url = null; + try { + url = new URL(request_url); + } catch (MalformedURLException e) { + throw new RuntimeException(e); + } + + if (headers_req.isEmpty()) { + throw new RuntimeException("Message headers not properly initialized"); + } + + String[] header_0 = headers_req.get(0).split(" "); + + String new_header_0 = header_0[0] + " " + url.getPath(); + if (url.getQuery() != null) { + new_header_0 += "?" + url.getQuery(); + } + + //if (url.getRef() != null) { + // new_header_0 += "#" + url.getRef(); + //} + + new_header_0 += " " + url.getProtocol().toUpperCase(); + new_header_0 += "/" + header_0[2].split("/")[1]; + + String new_header_1 = "Host: " + url.getHost(); + + if (!headers_req.get(1).contains("Host")) { + throw new RuntimeException("could not find Host header in header"); + } + + headers_req.set(0, new_header_0); + headers_req.set(1, new_header_1); + } + /** * Function to check if the given message matches a message_type * @@ -502,6 +859,17 @@ public boolean matches_msg_type(MessageType msg_type) { return matchedMessage; } + /** + * Returns a string representation of all the headers of the message + * + * @param isRequest select the request or the response + * @return + */ + public String getHeadersString(boolean isRequest) { + List headers_string = getHeaders(isRequest); + return String.join("\r\n", headers_string); + } + /** * An enum representing the possible message sections */ diff --git a/tool/src/main/java/migt/JWT.java b/tool/src/main/java/migt/JWT.java index b3c8a15..f72739a 100644 --- a/tool/src/main/java/migt/JWT.java +++ b/tool/src/main/java/migt/JWT.java @@ -15,7 +15,6 @@ /** * Class to manage JWT tokens * Uses https://connect2id.com/products/nimbus-jose-jwt - * {@code @Author} Matteo Bitussi */ public class JWT { public String header; diff --git a/tool/src/main/java/migt/Marker.java b/tool/src/main/java/migt/Marker.java index 2309f5a..752bb1f 100644 --- a/tool/src/main/java/migt/Marker.java +++ b/tool/src/main/java/migt/Marker.java @@ -4,8 +4,6 @@ /** * Class used to mark User Actions to be managed by session actions - * - * @author Matteo Bitussi */ public class Marker { String name; diff --git a/tool/src/main/java/migt/MessageOperation.java b/tool/src/main/java/migt/MessageOperation.java index f91ca8a..e10fe1c 100644 --- a/tool/src/main/java/migt/MessageOperation.java +++ b/tool/src/main/java/migt/MessageOperation.java @@ -1,6 +1,5 @@ package migt; -import burp.IExtensionHelpers; import org.json.JSONObject; import java.io.File; @@ -12,12 +11,10 @@ import java.util.regex.Matcher; import java.util.regex.Pattern; -import static migt.Tools.getAdding; +import static migt.Tools.getVariableByName; /** * The class storing a MessageOperation object - * - * @author Matteo Bitussi */ public class MessageOperation extends Module { HTTPReqRes.MessageSection from; @@ -101,12 +98,28 @@ public MessageOperation(JSONObject message_op_json) throws ParsingException { output_path = message_op_json.getString("output_path"); break; default: - System.err.println(key); - throw new ParsingException("Message operation not valid"); + throw new ParsingException("Message operation key \" " + key + "\" not valid"); } } } + /** + * Returns the adding of a message operation, decides if the value to be inserted/edited should be a variable or + * a typed value and return it + * + * @param m the message operation which has to be examined + * @return the adding to be used in add/edit + * @throws ParsingException if the variable name is not valid or the variable has not been initiated + */ + public static String getAdding(MessageOperation m, List vars) throws ParsingException { + if (!m.use.isEmpty()) { + return getVariableByName(m.use, vars).value; + } else { + + return m.to; + } + } + public void init() { this.what = ""; this.to = ""; @@ -135,8 +148,7 @@ public Operation_API exporter() { * @return the updated Operation with the result * @throws ParsingException if parsing of names is not successfull */ - public Operation execute(Operation op, - IExtensionHelpers helpers) throws ParsingException { + public Operation execute(Operation op) throws ParsingException { for (MessageOperation mop : op.getMessageOperations()) { Pattern pattern; Matcher matcher; @@ -182,12 +194,12 @@ public Operation execute(Operation op, matcher = pattern.matcher(url_header); String new_url = matcher.replaceFirst(""); op.api.message.setUrlHeader(new_url); - op.processed_message = op.api.message.getMessage(op.api.is_request, helpers); + op.processed_message = op.api.message.getMessage(op.api.is_request); break; case HEAD: op.api.message.removeHeadParameter(op.api.is_request, mop.what); - op.processed_message = op.api.message.getMessage(op.api.is_request, helpers); + op.processed_message = op.api.message.getMessage(op.api.is_request); break; case BODY: @@ -196,7 +208,7 @@ public Operation execute(Operation op, matcher = pattern.matcher(body); op.api.message.setBody(op.api.is_request, matcher.replaceAll("")); //Automatically update content-lenght - op.processed_message = op.api.message.getMessage(op.api.is_request, helpers); + op.processed_message = op.api.message.getMessage(op.api.is_request); break; } break; @@ -209,7 +221,7 @@ public Operation execute(Operation op, switch (mop.from) { case HEAD: { op.api.message.addHeadParameter(op.api.is_request, mop.what, getAdding(mop, op.api.vars)); - op.processed_message = op.api.message.getMessage(op.api.is_request, helpers); + op.processed_message = op.api.message.getMessage(op.api.is_request); break; } case BODY: { @@ -217,7 +229,7 @@ public Operation execute(Operation op, tmp = tmp + getAdding(mop, op.api.vars); op.api.message.setBody(op.api.is_request, tmp); //Automatically update content-lenght - op.processed_message = op.api.message.getMessage(op.api.is_request, helpers); + op.processed_message = op.api.message.getMessage(op.api.is_request); break; } case URL: @@ -238,14 +250,13 @@ public Operation execute(Operation op, found = true; } op.api.message.setUrlHeader(newHeader_0); - op.processed_message = op.api.message.getMessage(op.api.is_request, helpers); + op.processed_message = op.api.message.getMessage(op.api.is_request); break; } break; case EDIT: op.processed_message = Tools.editMessageParam( - helpers, mop.what, mop.from, op.api.message, @@ -256,7 +267,6 @@ public Operation execute(Operation op, case EDIT_REGEX: op.processed_message = Tools.editMessage( - helpers, mop.what, mop, op.api.message, @@ -277,7 +287,7 @@ public Operation execute(Operation op, } op.api.message.setHeaders(op.api.is_request, new_headers); - op.processed_message = op.api.message.getMessage(op.api.is_request, helpers); + op.processed_message = op.api.message.getMessage(op.api.is_request); break; } case BODY: { @@ -286,7 +296,7 @@ public Operation execute(Operation op, op.api.message.setBody(op.api.is_request, matcher.replaceAll("")); //Automatically update content-lenght - op.processed_message = op.api.message.getMessage(op.api.is_request, helpers); + op.processed_message = op.api.message.getMessage(op.api.is_request); break; } case URL: @@ -301,7 +311,7 @@ public Operation execute(Operation op, String newHeader_0 = matcher.replaceFirst(""); op.api.message.setUrlHeader(newHeader_0); - op.processed_message = op.api.message.getMessage(op.api.is_request, helpers); + op.processed_message = op.api.message.getMessage(op.api.is_request); break; } break; @@ -390,10 +400,6 @@ public Operation execute(Operation op, } else { op.api.message.setResponse(op.processed_message); } - if (op.processed_message_service != null) { - // TODO: check if ok to remove - //op.api.message.setHttpService(op.processed_message_service); - } } } catch (StackOverflowError e) { e.printStackTrace(); @@ -412,7 +418,8 @@ public enum MessageOperationActions { EDIT_REGEX, ADD, SAVE, - SAVE_MATCH; + SAVE_MATCH, + ENCODE; /** * From a string get the corresponding enum value @@ -438,8 +445,10 @@ public static MessageOperationActions fromString(String input) throws ParsingExc return SAVE; case "save match": return SAVE_MATCH; + case "encode": + return ENCODE; default: - throw new ParsingException("invalid check operation"); + throw new ParsingException("invalid Message operation action \"" + input + "\""); } } else { throw new NullPointerException(); diff --git a/tool/src/main/java/migt/MessageType.java b/tool/src/main/java/migt/MessageType.java index b07d331..4c9b5d8 100644 --- a/tool/src/main/java/migt/MessageType.java +++ b/tool/src/main/java/migt/MessageType.java @@ -5,8 +5,6 @@ /** * Class storing a MessageType - * - * @author Matteo Bitussi */ public class MessageType implements Cloneable { String name; diff --git a/tool/src/main/java/migt/Module.java b/tool/src/main/java/migt/Module.java index a4ccc74..4cb1785 100644 --- a/tool/src/main/java/migt/Module.java +++ b/tool/src/main/java/migt/Module.java @@ -1,6 +1,5 @@ package migt; -import burp.IExtensionHelpers; import org.json.JSONObject; /** @@ -11,7 +10,6 @@ public class Module { // These variables should be present in each module boolean result = true; boolean applicable = false; - IExtensionHelpers helpers; API api; // the api of this module API imported_api; // the api imported from a previous module @@ -28,15 +26,9 @@ public Module(JSONObject json_module) { // Parse } - public Module(IExtensionHelpers helpers) { - this.helpers = helpers; - } - /** * This function should be called to check that after an initialization of a module all the necessary parameters - * are set correctly. - * - * @return + * are set correctly. And the JSON has been parsed correctly with all the required tags present. */ public void validate() throws ParsingException { @@ -63,7 +55,10 @@ public void setAPI(API api) { * @param api the imported API */ public void loader(API api) { - + if (api == null) { + throw new RuntimeException("loaded api is null"); + } + this.imported_api = api; } /** diff --git a/tool/src/main/java/migt/Operation.java b/tool/src/main/java/migt/Operation.java index fcacf1d..33c7221 100644 --- a/tool/src/main/java/migt/Operation.java +++ b/tool/src/main/java/migt/Operation.java @@ -1,6 +1,5 @@ package migt; -import burp.IHttpService; import burp.IInterceptedProxyMessage; import org.json.JSONArray; import org.json.JSONObject; @@ -13,8 +12,6 @@ /** * Class storing an Operation in a Test - * - * @author Matteo Bitussi */ public class Operation extends Module { public List messageOperations; @@ -29,11 +26,11 @@ public class Operation extends Module { public boolean isSessionOp = false; public List matchedMessages; public byte[] processed_message; - public IHttpService processed_message_service; // null if it is not changed public List log_messages; public List session_operations; // Decode operations public List decodeOperations; + public List editOperations; // Session operation // API Operation_API api; @@ -140,6 +137,11 @@ public Operation(JSONObject operation_json, decodeOperations.add(decode_op); } } + + // Edit operations + if (operation_json.has("edits")) { + editOperations = Tools.parseEditsFromJSON(operation_json.getJSONArray("edits")); + } } private void init() { @@ -151,6 +153,7 @@ private void init() { this.session_operations = new ArrayList<>(); this.log_messages = new ArrayList<>(); this.decodeOperations = new ArrayList<>(); + editOperations = new ArrayList<>(); this.from_session = ""; this.to_session = ""; this.save_name = ""; @@ -159,7 +162,6 @@ private void init() { this.replace_request_name = ""; this.messageType = ""; this.session = ""; - this.processed_message_service = null; this.processed_message = null; } @@ -368,6 +370,7 @@ public Operation_API getAPI() { /** * Sets the api of this Operation with the given api. Note that the variables are added, not substituted + * * @param api the new api to be set */ public void setAPI(Operation_API api) { @@ -419,7 +422,6 @@ public void execute() { try { applicable = true; processed_message = getVariableByName(replace_request_name, api.vars).message; - processed_message_service = getVariableByName(replace_request_name, api.vars).service_info; //return op; } catch (ParsingException e) { e.printStackTrace(); @@ -432,7 +434,6 @@ public void execute() { try { applicable = true; processed_message = getVariableByName(replace_response_name, api.vars).message; - processed_message_service = getVariableByName(replace_response_name, api.vars).service_info; //return op; } catch (ParsingException e) { e.printStackTrace(); @@ -445,10 +446,13 @@ public void execute() { // execute the message operations and the decode ops try { applicable = true; - executeMessageOperations(this, helpers); // TOOD: change to edits + executeMessageOperations(this); + if (!applicable | !result) + return; + executeEditOps(this, api.vars); if (!applicable | !result) return; - executeDecodeOps(this, helpers, api.vars); + executeDecodeOps(this, api.vars); if (!applicable | !result) return; executeChecks(this, api.vars); @@ -466,7 +470,6 @@ public void execute() { v.name = save_name; v.isMessage = true; v.message = api.is_request ? api.message.getRequest() : api.message.getResponse(); - v.service_info = api.message.getHttpService(helpers); api.vars.add(v); } } diff --git a/tool/src/main/java/migt/ParsingException.java b/tool/src/main/java/migt/ParsingException.java index 9afda57..4083c35 100644 --- a/tool/src/main/java/migt/ParsingException.java +++ b/tool/src/main/java/migt/ParsingException.java @@ -2,8 +2,6 @@ /** * Exception raised when the parsing of the language fails - * - * @author Matteo Bitussi */ public class ParsingException extends Exception { /** diff --git a/tool/src/main/java/migt/Session.java b/tool/src/main/java/migt/Session.java index 1f173e7..0b5ee2f 100644 --- a/tool/src/main/java/migt/Session.java +++ b/tool/src/main/java/migt/Session.java @@ -10,8 +10,6 @@ /** * Class to manage Sessions - * - * @author Matteo Bitussi */ public class Session { // Session actions diff --git a/tool/src/main/java/migt/SessionOperation.java b/tool/src/main/java/migt/SessionOperation.java index ead878d..28f582c 100644 --- a/tool/src/main/java/migt/SessionOperation.java +++ b/tool/src/main/java/migt/SessionOperation.java @@ -11,8 +11,6 @@ /** * Class containing a session Operation - * - * @author Matteo Bitussi */ public class SessionOperation { public String from_session; @@ -88,7 +86,7 @@ public static List parseFromJson(JSONObject act_operation) thr break; default: - throw new ParsingException("Unexpected value: " + key); + throw new ParsingException("Unexpected value: " + key + " in session operation"); } } lsop.add(sop); diff --git a/tool/src/main/java/migt/SessionTrackAction.java b/tool/src/main/java/migt/SessionTrackAction.java index 92b6a09..aa201c7 100644 --- a/tool/src/main/java/migt/SessionTrackAction.java +++ b/tool/src/main/java/migt/SessionTrackAction.java @@ -6,8 +6,6 @@ /** * This class represents an user action in a session - * - * @author Matteo Bitussi */ public class SessionTrackAction { public SessionOperation.SessAction action; diff --git a/tool/src/main/java/migt/Test.java b/tool/src/main/java/migt/Test.java index 074ef13..d24ba33 100644 --- a/tool/src/main/java/migt/Test.java +++ b/tool/src/main/java/migt/Test.java @@ -17,8 +17,6 @@ /** * Class to store a test - * - * @author Matteo Bitussi */ public class Test { public ResultType result; @@ -90,7 +88,7 @@ public Test(JSONObject test_json, affected_entity = test_json.getString("affected_entity"); break; default: - throw new ParsingException("Invalid key \"" + key + "\""); + throw new ParsingException("Invalid key \"" + key + "\" in test"); } } @@ -388,6 +386,52 @@ public void logTest(String log_folder) { } } + /** + * Function that execute the given passive test. + * + * @param messageList a list of HTTPReqRes messages + * @param msg_types the message types used by the test + * @return true if a test is passed, false otherwise + */ + public boolean execute(List messageList, + List msg_types) throws ParsingException { + int i, j; + boolean res = true; + + for (i = 0; i < messageList.size(); i++) { + j = 0; + while (j < operations.size() && res) { + Operation currentOP = operations.get(j); + MessageType msg_type = MessageType.getFromList(msg_types, currentOP.getMessageType()); + + if (currentOP.api == null) { + currentOP.api = new Operation_API(vars); + } else { + currentOP.api.vars = vars; + } + + if (messageList.get(i).matches_msg_type(msg_type)) { + currentOP.setAPI(new Operation_API(messageList.get(i), msg_type.msg_to_process_is_request)); + currentOP.execute(); + res = currentOP.getResult(); + } + + vars = currentOP.api.vars; + j++; + } + } + + for (Operation op : operations) { + if (!op.applicable) { + res = false; + applicable = false; + break; + } + } + + return res; + } + /** * The result type of (also the oracle) of an Active test */ diff --git a/tool/src/main/java/migt/TestSuite.java b/tool/src/main/java/migt/TestSuite.java index 5740d2e..a41b585 100644 --- a/tool/src/main/java/migt/TestSuite.java +++ b/tool/src/main/java/migt/TestSuite.java @@ -1,12 +1,15 @@ package migt; +import java.io.BufferedWriter; +import java.io.File; +import java.io.FileWriter; +import java.io.IOException; +import java.text.SimpleDateFormat; import java.util.ArrayList; import java.util.List; /** * Class to store a TestSuite - * - * @author Matteo Bitussi */ public class TestSuite { String name; @@ -35,6 +38,41 @@ public TestSuite(String name, String description, List tests) { this.tests = tests; } + public void log_test_suite(String log_folder_path) { + String timestamp = new SimpleDateFormat("yyyy_MM_dd_HH_mm").format(new java.util.Date()); + String test_log_folder = log_folder_path + "/" + timestamp + "/suite_" + this.name + "/"; + + File directory = new File(test_log_folder); + if (!directory.exists()) { + if (!directory.mkdirs()) { + System.err.println("cannot create log directory at " + test_log_folder); + } + } + + String log_content = ""; + log_content += "| test name | description | type | result | applicable |\n"; + log_content += "|-----------|-------------|------|--------|------------|\n"; + + for (Test t : tests) { + log_content += "|" + t.name + + "|" + t.description + + "|" + (t.isActive ? "active" : "passive") + + "|" + t.success + + "|" + t.applicable + "|\n"; + } + + File log_message = new File(test_log_folder + "results.md"); + try { + FileWriter fw = new FileWriter(log_message.getAbsoluteFile()); + BufferedWriter bw = new BufferedWriter(fw); + bw.write(log_content); + bw.close(); + } catch (IOException e) { + e.printStackTrace(); + } + } + + public List getTests() { return tests; } diff --git a/tool/src/main/java/migt/Tools.java b/tool/src/main/java/migt/Tools.java index 5d66fc9..580cc35 100644 --- a/tool/src/main/java/migt/Tools.java +++ b/tool/src/main/java/migt/Tools.java @@ -1,6 +1,5 @@ package migt; -import burp.IExtensionHelpers; import com.google.gson.JsonElement; import com.google.gson.JsonParser; import com.jayway.jsonpath.Configuration; @@ -9,7 +8,6 @@ import org.json.JSONArray; import org.json.JSONObject; - import java.nio.charset.StandardCharsets; import java.util.ArrayList; import java.util.HashMap; @@ -20,77 +18,8 @@ /** * Class with methods to process messages and execute tests - * - * @author Matteo Bitussi */ public class Tools { - /** - * Function that execute the given passive test. - * - * @param test a Test element, it has to be a passive test - * @param messageList a list of HTTPReqRes messages - * @param helpers an istance of IExtensionHelpers - * @param msg_types the message types used by the test - * @return true if a test is passed, false otherwise - */ - public static boolean executePassiveTest(Test test, - List messageList, - IExtensionHelpers helpers, - List msg_types) throws ParsingException { - int i, j; - boolean res = true; - - for (i = 0; i < messageList.size(); i++) { - j = 0; - while (j < test.operations.size() && res) { - Operation currentOP = test.operations.get(j); - MessageType msg_type = MessageType.getFromList(msg_types, currentOP.getMessageType()); - - if (currentOP.api == null) { - currentOP.api = new Operation_API(test.vars); - } else { - currentOP.api.vars = test.vars; - } - - if (messageList.get(i).matches_msg_type(msg_type)) { - currentOP.helpers = helpers; - - currentOP.setAPI(new Operation_API(messageList.get(i), msg_type.msg_to_process_is_request)); - currentOP.execute(); - res = currentOP.getResult(); - } - - test.vars = currentOP.api.vars; - j++; - } - } - - for (Operation op : test.operations) { - if (!op.applicable) { - res = false; - test.applicable = false; - break; - } - } - - return res; - } - - /** - * Function that given a list of headers, concatenates them in a single string - * - * @param headers the list of headers - * @return the string - */ - public static String getAllHeaders(List headers) { - StringBuilder out = new StringBuilder(); - for (Object o : headers) { - out.append(o.toString()); - out.append("\n"); - } - return out.toString(); - } - /** * This function execute a list of checks over a message, returning true if all the checks are successful * @@ -131,17 +60,15 @@ public static Operation executeChecks(Operation op, List vars) throws Parsi /** * Executes the decode operations in an operation. Uses APIs. Sets the result to the operation * - * @param op the operation to execute the decode operations from - * @param helpers the Burp helpers + * @param op the operation to execute the decode operations from * @return The operation (edited) * @throws ParsingException if something goes wrong */ public static Operation executeDecodeOps(Operation op, - IExtensionHelpers helpers, List vars) throws ParsingException { Operation_API api = op.getAPI(); for (DecodeOperation dop : op.getDecodeOperations()) { - dop.loader(api, helpers); + dop.loader(api); dop.execute(vars); if (!op.setResult(dop)) break; @@ -154,17 +81,15 @@ public static Operation executeDecodeOps(Operation op, /** * Executes the decode operations in a decode operation. This is the recursive step. * - * @param op the decode operation executing its child decode operations - * @param helpers the burp helpers + * @param op the decode operation executing its child decode operations * @return The operation (edited) * @throws ParsingException if something goes wrong */ public static DecodeOperation executeDecodeOps(DecodeOperation op, - IExtensionHelpers helpers, List vars) throws ParsingException { DecodeOperation_API api = op.getAPI(); for (DecodeOperation dop : op.decodeOperations) { - dop.loader(api, helpers); + dop.loader(api); dop.execute(vars); if (!op.setResult(dop)) break; @@ -195,10 +120,30 @@ public static DecodeOperation executeEditOps(DecodeOperation op, return op; } - public static Operation executeMessageOperations(Operation op, IExtensionHelpers helpers) throws ParsingException { + /** + * Executes the edit operations inside a standard Operation + * + * @param op the Operation to run the edit operations from + * @return the Operation (edited) + * @throws ParsingException if something goes wrong + */ + public static Operation executeEditOps(Operation op, List vars) throws ParsingException { + Operation_API api = op.getAPI(); + for (EditOperation eop : op.editOperations) { + eop.loader(api); + eop.execute(vars); + if (!op.setResult(eop)) + break; + op.setAPI((Operation_API) eop.exporter()); + } + + return op; + } + + public static Operation executeMessageOperations(Operation op) throws ParsingException { for (MessageOperation mop : op.messageOperations) { mop.loader(op.api); - mop.execute(op, helpers); + mop.execute(op); op.setAPI(mop.exporter()); if (op.setResult(op)) break; @@ -251,7 +196,7 @@ public static List parseEditsFromJSON(JSONArray edits_array) thro * @return a List of messagetype objects * @throws ParsingException if the input is malformed */ - public static List readMsgTypeFromJson(String input) throws ParsingException { + public static List readMsgTypesFromJson(String input) throws ParsingException { List msg_types = new ArrayList<>(); JSONObject obj = new JSONObject(input); @@ -283,23 +228,6 @@ public static List readMsgTypeFromJson(String input) throws Parsing return msg_types; } - /** - * Returns the adding of a message operation, decides if the value to be inserted/edited should be a variable or - * a typed value and return it - * - * @param m the message operation which has to be examined - * @return the adding to be used in add/edit - * @throws ParsingException if the variable name is not valid or the variable has not been initiated - */ - public static String getAdding(MessageOperation m, List vars) throws ParsingException { - if (!m.use.isEmpty()) { - return getVariableByName(m.use, vars).value; - } else { - - return m.to; - } - } - /** * Returns the default string that contains the default message types that fill a msg_def.json file * @@ -422,7 +350,7 @@ public static String buildStringWithVars(List vars, String s) throws Parsin req_var.put(act_match, getVariableByName(act_match, vars).value); } - if (req_var.size() == 0) { + if (req_var.isEmpty()) { return s; } @@ -604,7 +532,7 @@ public static String generate_CSRF_POC(HTTPReqRes message) { public static HashMap> batchPassivesFromSession(List testList) throws ParsingException { HashMap> batch = new HashMap<>(); for (Test t : testList) { - if (t.sessions.size() == 0) { + if (t.sessions.isEmpty()) { throw new ParsingException("Undefined session in test " + t.name); } @@ -640,7 +568,6 @@ public static List debatchPassive(HashMap> batch) { /** * Edit a message treating it as a string using a regex * - * @param helpers an instance of Burp's IExtensionHelper * @param regex the regex used to match the things to change * @param mop the message operation containing information about the section to match the regex * @param messageInfo the message as IHttpRequestResponse object @@ -649,13 +576,12 @@ public static List debatchPassive(HashMap> batch) { * @return the edited message as byte array * @throws ParsingException if problems are encountered in editing the message */ - public static byte[] editMessage(IExtensionHelpers helpers, - String regex, + public static byte[] editMessage(String regex, MessageOperation mop, HTTPReqRes messageInfo, boolean isRequest, String new_value) throws ParsingException { - // TODO: remove dependency from Helpers + //TODO: remove in future versions Pattern pattern = null; Matcher matcher = null; switch (mop.from) { @@ -669,7 +595,7 @@ public static byte[] editMessage(IExtensionHelpers helpers, new_head.add(matcher.replaceAll(new_value)); } messageInfo.setHeaders(isRequest, new_head); - return messageInfo.getMessage(isRequest, helpers); + return messageInfo.getMessage(isRequest); case BODY: pattern = Pattern.compile(regex); @@ -677,7 +603,7 @@ public static byte[] editMessage(IExtensionHelpers helpers, matcher = pattern.matcher(new String(messageInfo.getBody(isRequest))); messageInfo.setBody(isRequest, matcher.replaceAll(new_value)); //Automatically update content-lenght - return messageInfo.getMessage(isRequest, helpers); + return messageInfo.getMessage(isRequest); case URL: if (!isRequest) { @@ -689,7 +615,7 @@ public static byte[] editMessage(IExtensionHelpers helpers, String replaced = matcher.replaceAll(new_value); messageInfo.setUrlHeader(replaced); - return messageInfo.getMessage(isRequest, helpers); + return messageInfo.getMessage(isRequest); } return null; } @@ -697,7 +623,6 @@ public static byte[] editMessage(IExtensionHelpers helpers, /** * Edit a message parameter * - * @param helpers an instance of Burp's IExtensionHelper * @param param_name the name of the parameter to edit * @param message_section the message section to edit * @param messageInfo the message as IHttpRequestResponse object @@ -708,19 +633,19 @@ public static byte[] editMessage(IExtensionHelpers helpers, * @return the edited message as byte array * @throws ParsingException if problems are encountered in editing the message */ - public static byte[] editMessageParam(IExtensionHelpers helpers, - String param_name, + public static byte[] editMessageParam(String param_name, HTTPReqRes.MessageSection message_section, HTTPReqRes messageInfo, boolean isRequest, String new_value, boolean isBodyRegex) throws ParsingException { + //TODO: remove in future versions Pattern pattern = null; Matcher matcher = null; switch (message_section) { case HEAD: messageInfo.editHeadParam(isRequest, param_name, new_value); - byte[] message = messageInfo.getMessage(isRequest, helpers); + byte[] message = messageInfo.getMessage(isRequest); messageInfo.setHost(new_value); // this should be set when the message is converted to the burp class return message; @@ -735,7 +660,7 @@ public static byte[] editMessageParam(IExtensionHelpers helpers, String new_body = matcher.replaceFirst(new_value); messageInfo.setBody(isRequest, new_body); //Automatically update content-lenght - return messageInfo.getMessage(isRequest, helpers); + return messageInfo.getMessage(isRequest); case URL: if (!isRequest) { @@ -748,48 +673,11 @@ public static byte[] editMessageParam(IExtensionHelpers helpers, messageInfo.setUrlHeader(matcher.replaceAll(param_name + "=" + new_value)); // problema - return messageInfo.getMessage(isRequest, helpers); + return messageInfo.getMessage(isRequest); } return null; } - public static byte[] editMessageParam(IExtensionHelpers helpers, - String param_name, - DecodeOperation.DecodeOperationFrom decodeOperationFrom, - HTTPReqRes messageInfo, - boolean isRequest, - String new_value, - boolean isBodyRegex) throws ParsingException { - - HTTPReqRes.MessageSection ms = null; - - switch (decodeOperationFrom) { - case HEAD: - ms = HTTPReqRes.MessageSection.HEAD; - break; - case BODY: - ms = HTTPReqRes.MessageSection.BODY; - break; - case URL: - ms = HTTPReqRes.MessageSection.URL; - break; - case JWT_HEADER: - case JWT_PAYLOAD: - case JWT_SIGNATURE: - throw new ParsingException("invalid from section in decode operation should be a message section"); - } - - return editMessageParam( - helpers, - param_name, - ms, - messageInfo, - isRequest, - new_value, - isBodyRegex - ); - } - /** * Finds the parent div of an http element * @@ -847,6 +735,7 @@ public static String editJson(EditOperation.Jwt_action action, List vars, String save_as, String newValue) throws PathNotFoundException { + //TODO: remove in future versions Object document = Configuration.defaultConfiguration().jsonProvider().parse(content); JsonPath jsonPath = JsonPath.compile(j_path); @@ -873,6 +762,7 @@ public static String editJson(EditOperation.Jwt_action action, /** * Checks that the json resulting by parsing the two strings is equals. Equals in this case means that the same keys * and values are present, but the order is ignored + * * @param s1 the string 1 * @param s2 the string 2 * @return true or false depending if the json parsing of the two strings is the same or not diff --git a/tool/src/main/java/migt/Var.java b/tool/src/main/java/migt/Var.java index 0815b64..b09ef77 100644 --- a/tool/src/main/java/migt/Var.java +++ b/tool/src/main/java/migt/Var.java @@ -1,18 +1,13 @@ package migt; -import burp.IHttpService; - /** * The class storing the variables used in the test and sessions - * - * @author Matteo Bitussi */ public class Var { public String name; public String value; public byte[] message; public boolean isMessage; // tells if a variable contains a message - public IHttpService service_info; /** * Istantiate a Var object diff --git a/tool/src/main/java/migt/XML.java b/tool/src/main/java/migt/XML.java index 53905cf..464d7bc 100644 --- a/tool/src/main/java/migt/XML.java +++ b/tool/src/main/java/migt/XML.java @@ -20,8 +20,6 @@ /** * Class used to parse and edit xml strings - * - * @author Matteo Bitussi */ public class XML { diff --git a/tool/src/test/java/Checks_Test.java b/tool/src/test/java/Checks_Test.java index 958604c..cc82f58 100644 --- a/tool/src/test/java/Checks_Test.java +++ b/tool/src/test/java/Checks_Test.java @@ -334,4 +334,50 @@ void test_check_json_not_matches_regex_wrong() throws ParsingException { c.execute(new ArrayList()); assertFalse(c.getResult()); } + + @Test + void test_check_json_schema_validation_ok() throws ParsingException { + String check_str = "{\n" + + " \"in\": \"header\",\n" + + " \"check\": \"$.pageInfo.imaninteger\",\n" + + " \"json schema compliant\": \"{\\\"type\\\":\\\"integer\\\"}\" " + + "}"; + + Check c = initCheck_json(check_str); + + c.execute(new ArrayList<>()); + assertTrue(c.getResult()); + } + + @Test + void test_check_json_schema_validation_wrong() throws ParsingException { + String check_str = "{\n" + + " \"in\": \"header\",\n" + + " \"check\": \"$.pageInfo.imaninteger\",\n" + + " \"json schema compliant\": \"{\\\"type\\\":\\\"string\\\"}\" " + + "}"; + + Check c = initCheck_json(check_str); + + c.execute(new ArrayList<>()); + assertFalse(c.getResult()); + } + + @Test + void test_check_json_schema_validation_wrong_schema() throws ParsingException { + String check_str = "{\n" + + " \"in\": \"header\",\n" + + " \"check\": \"$.pageInfo.imaninteger\",\n" + + " \"json schema compliant\": \"wrongschema\" " + + "}"; + + Check c = initCheck_json(check_str); + try { + c.execute(new ArrayList<>()); + } catch (RuntimeException e) { + assertEquals(1,1); + return; + } + assertEquals(1,0); + } } diff --git a/tool/src/test/java/EditOperation_test.java b/tool/src/test/java/EditOperation_test.java new file mode 100644 index 0000000..4be558a --- /dev/null +++ b/tool/src/test/java/EditOperation_test.java @@ -0,0 +1,158 @@ +import migt.EditOperation; +import migt.HTTPReqRes; +import migt.Operation_API; +import migt.ParsingException; +import org.json.JSONObject; +import org.junit.jupiter.api.Test; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertTrue; + +public class EditOperation_test { + HTTPReqRes message = HTTPReqRes_Test.initMessage_ok(); + HTTPReqRes message_w_body; + + @Test + public void test_encode_url_param() throws ParsingException { + String input = "{\"from\": \"url\", \"encode\": \"format\"," + "\"encodings\": [\"base64\"]}"; + EditOperation eop = new EditOperation(new JSONObject(input)); + Operation_API api = new Operation_API(message, true); + + eop.setAPI(api); + eop.execute(null); + assertTrue(eop.getResult()); + Operation_API res = (Operation_API) eop.exporter(); + assertEquals("anNvbg==", res.message.getUrlParam("format")); + } + + @Test + public void test_encode_head_param() throws ParsingException { + String input = "{\"from\": \"head\", \"encode\": \"Host\"," + "\"encodings\": [\"base64\"]}"; + EditOperation eop = new EditOperation(new JSONObject(input)); + Operation_API api = new Operation_API(message, true); + + eop.setAPI(api); + eop.execute(null); + assertTrue(eop.getResult()); + Operation_API res = (Operation_API) eop.exporter(); + assertEquals("cGxheS5nb29nbGUuY29t", res.message.getHeadParam(true, "Host")); + } + + @Test + public void test_encode_body_param() throws ParsingException { + String input = "{\"from\": \"body\", \"encode\": \".*\"," + "\"encodings\": [\"base64\"]}"; + EditOperation eop = new EditOperation(new JSONObject(input)); + Operation_API api = new Operation_API(message, true); + + eop.setAPI(api); + eop.execute(null); + assertTrue(eop.getResult()); + Operation_API res = (Operation_API) eop.exporter(); + assertEquals("Ym9keWNvbnRlbnQ=", new String(res.message.getBody(true))); + } + + @Test + public void test_edit_url_regex() throws ParsingException { + String input = "{\"from\": \"url\", \"edit regex\": \"format=json\"," + "\"value\": \"test=testone\"}"; + EditOperation eop = new EditOperation(new JSONObject(input)); + Operation_API api = new Operation_API(message, true); + + eop.setAPI(api); + eop.execute(null); + assertTrue(eop.getResult()); + Operation_API res = (Operation_API) eop.exporter(); + assertEquals("testone", res.message.getUrlParam("test")); + } + + @Test + public void test_edit_head_regex() throws ParsingException { + String input = "{\"from\": \"head\", \"edit regex\": \"Host:\"," + "\"value\": \"Hosted:\"}"; + EditOperation eop = new EditOperation(new JSONObject(input)); + Operation_API api = new Operation_API(message, true); + + eop.setAPI(api); + eop.execute(null); + assertTrue(eop.getResult()); + Operation_API res = (Operation_API) eop.exporter(); + assertEquals("play.google.com", res.message.getHeadParam(true, "Hosted")); + } + + @Test + public void test_edit_body_regex() throws ParsingException { + String input = "{\"from\": \"body\", \"edit regex\": \"ent\"," + "\"value\": \"123\"}"; + EditOperation eop = new EditOperation(new JSONObject(input)); + Operation_API api = new Operation_API(message, true); + + eop.setAPI(api); + eop.execute(null); + assertTrue(eop.getResult()); + Operation_API res = (Operation_API) eop.exporter(); + assertEquals("bodycont123", new String(res.message.getBody(true))); + } + + @Test + public void test_add_url_param() throws ParsingException { + String input = "{\"from\": \"url\", \"add\": \"codechallenge\"," + "\"value\": \"12345\"}"; + EditOperation eop = new EditOperation(new JSONObject(input)); + Operation_API api = new Operation_API(message, true); + + eop.setAPI(api); + eop.execute(null); + assertTrue(eop.getResult()); + Operation_API res = (Operation_API) eop.exporter(); + assertEquals("12345", res.message.getUrlParam("codechallenge")); + } + + @Test + public void test_add_url_param_already_present() throws ParsingException { + String input = "{\"from\": \"url\", \"add\": \"authuser\"," + "\"value\": \"1\"}"; + EditOperation eop = new EditOperation(new JSONObject(input)); + Operation_API api = new Operation_API(message, true); + + eop.setAPI(api); + eop.execute(null); + assertTrue(eop.getResult()); + Operation_API res = (Operation_API) eop.exporter(); + assertEquals("01", res.message.getUrlParam("authuser")); + } + + @Test + public void test_add_head_param() throws ParsingException { + String input = "{\"from\": \"head\", \"add\": \"Magicheader\"," + "\"value\": \"123123\"}"; + EditOperation eop = new EditOperation(new JSONObject(input)); + Operation_API api = new Operation_API(message, true); + + eop.setAPI(api); + eop.execute(null); + assertTrue(eop.getResult()); + Operation_API res = (Operation_API) eop.exporter(); + assertEquals("123123", res.message.getHeadParam(true,"Magicheader")); + } + + @Test + public void test_add_head_param_already_present() throws ParsingException { + String input = "{\"from\": \"head\", \"add\": \"Accept\"," + "\"value\": \"1\"}"; + EditOperation eop = new EditOperation(new JSONObject(input)); + Operation_API api = new Operation_API(message, true); + + eop.setAPI(api); + eop.execute(null); + assertTrue(eop.getResult()); + Operation_API res = (Operation_API) eop.exporter(); + assertEquals("*/*1", res.message.getHeadParam(true,"Accept")); + } + + @Test + public void test_add_body() throws ParsingException { + String input = "{\"from\": \"body\", \"add\": \"anything\"," + "\"value\": \"&appended\"}"; + EditOperation eop = new EditOperation(new JSONObject(input)); + Operation_API api = new Operation_API(message, true); + + eop.setAPI(api); + eop.execute(null); + assertTrue(eop.getResult()); + Operation_API res = (Operation_API) eop.exporter(); + assertEquals("bodycontent&appended", + new String(res.message.getBody(true))); + } +} diff --git a/tool/src/test/java/HTTPReqRes_Test.java b/tool/src/test/java/HTTPReqRes_Test.java index 1dada6d..58c564b 100644 --- a/tool/src/test/java/HTTPReqRes_Test.java +++ b/tool/src/test/java/HTTPReqRes_Test.java @@ -1,7 +1,9 @@ import migt.HTTPReqRes; +import migt.ParsingException; import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; +import java.net.MalformedURLException; import java.nio.charset.StandardCharsets; import java.util.ArrayList; import java.util.Arrays; @@ -12,8 +14,8 @@ public class HTTPReqRes_Test { - public HTTPReqRes initMessage_ok() { - String raw = "POST /log?format=json&hasfast=true&authuser=0 HTTP/2\r\n" + + public static HTTPReqRes initMessage_ok() { + String raw = "POST /log?format=json&hasfast=true&authuser=0 HTTPS/2\r\n" + "Host: play.google.com\r\n" + "Cookie: CONSENT=PENDING+392; SOCS=CAISHAgCEhJnd3NfMjAyMzAyMjgtMF9SQzIaAml0IAEaBgiA2pSgBg; AEC=AUEFqZdSS4hmP6dNNRrldXefJFuHK2ldiLrZLJG24hUqaFA2L0jJxZwSBA; NID=511=SPj3DZBbWBMVstxl414okznEMUOaUHRzxZehEHxoaTi0Fr_X9RQ6UmFDBvI6wWn1Iivh7lzi_q7Ktri2q8hHc9nVY3XNgQP-IQ4AHNz7lCKra72IjxzhBvEBQFdXy7lEaIVC3wK5TfPIXLX3TWhKwrZAVEg77UkqV2oHYohcSXg\r\n" + "Content-Length: 11\r\n" + @@ -55,6 +57,46 @@ public HTTPReqRes initMessage_ok() { return message; } + @Test + public HTTPReqRes init_message_no_body() { + String raw = "POST /log?format=json&hasfast=true&authuser=0 HTTPS/2\r\n" + + "Host: play.google.com\r\n" + + "Cookie: CONSENT=PENDING+392; SOCS=CAISHAgCEhJnd3NfMjAyMzAyMjgtMF9SQzIaAml0IAEaBgiA2pSgBg; AEC=AUEFqZdSS4hmP6dNNRrldXefJFuHK2ldiLrZLJG24hUqaFA2L0jJxZwSBA; NID=511=SPj3DZBbWBMVstxl414okznEMUOaUHRzxZehEHxoaTi0Fr_X9RQ6UmFDBvI6wWn1Iivh7lzi_q7Ktri2q8hHc9nVY3XNgQP-IQ4AHNz7lCKra72IjxzhBvEBQFdXy7lEaIVC3wK5TfPIXLX3TWhKwrZAVEg77UkqV2oHYohcSXg\r\n" + + "Content-Length: 11\r\n" + + "Sec-Ch-Ua: \"Chromium\";v=\"111\", \"Not(A:Brand\";v=\"8\"\r\n" + + "Content-Type: application/x-www-form-urlencoded;charset=UTF-8\r\n" + + "X-Goog-Authuser: 0\r\n" + + "Sec-Ch-Ua-Mobile: ?0\r\n" + + "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.5563.65 Safari/537.36\r\n" + + "Sec-Ch-Ua-Platform: \"Linux\"\r\n" + + "Accept: */*\r\n" + + "Origin: https://www.google.com\r\n" + + "X-Client-Data: CNiKywE=\r\n" + + "Sec-Fetch-Site: same-site\r\n" + + "Sec-Fetch-Mode: cors\r\n" + + "Sec-Fetch-Dest: empty\r\n" + + "Referer: https://www.google.com/\r\n" + + "Accept-Encoding: gzip, deflate\r\n" + + "Accept-Language: en-US,en;q=0.9\r\n" + + "\r\n"; + + List headers = new ArrayList<>(); + + Collections.addAll(headers, raw.split("\r\n")); + + byte[] raw_b = raw.getBytes(StandardCharsets.UTF_8); + + HTTPReqRes message = new HTTPReqRes(raw_b, null); + + message.body_offset_req = 0; + message.setHeaders(true, headers); + message.isRequest = true; + message.isResponse = false; + message.setRequest_url("https://play.google.com/log?format=json&hasfast=true&authuser=0"); + + return message; + } + @Test @DisplayName("") public void test_build() { @@ -184,7 +226,7 @@ public void test_getUrlHeader() { HTTPReqRes message = initMessage_ok(); String header_0 = message.getUrlHeader(); - assertEquals("POST /log?format=json&hasfast=true&authuser=0 HTTP/2", header_0); + assertEquals("POST /log?format=json&hasfast=true&authuser=0 HTTPS/2", header_0); } @Test @@ -201,6 +243,55 @@ public void test_getUrlParam() { @Test @DisplayName("") + public void test_editUrlParam() throws ParsingException { + HTTPReqRes message = initMessage_ok(); + message.editUrlParam("format", "new"); + assertEquals("https://play.google.com/log?format=new&hasfast=true&authuser=0", message.getUrl()); + message.setRequest_url("https://play.google.com:8080/log?format=new&hasfast=true&authuser=0#123123123"); + message.editUrlParam("format", "newnew"); + assertEquals("https://play.google.com:8080/log?format=newnew&hasfast=true&authuser=0#123123123", message.getUrl()); + assertEquals("POST /log?format=newnew&hasfast=true&authuser=0 HTTPS/2", message.getUrlHeader()); + } + + @Test + public void test_url_update_header_0() { + HTTPReqRes message = initMessage_ok(); + message.setRequest_url("https://play.google.com:8080/log?format=newnew&hasfast=true&authuser=0#123123123"); + assertEquals("https://play.google.com:8080/log?format=newnew&hasfast=true&authuser=0#123123123", message.getUrl()); + assertEquals("POST /log?format=newnew&hasfast=true&authuser=0 HTTPS/2", message.getUrlHeader()); + } + + @Test + public void test_editUrlHeaders() throws ParsingException { + HTTPReqRes message = initMessage_ok(); + message.updateHeadersWHurl(); + assertEquals("POST /log?format=json&hasfast=true&authuser=0 HTTPS/2", message.getHeaders(true).get(0)); + message.editUrlParam("format", "new"); + assertEquals("POST /log?format=new&hasfast=true&authuser=0 HTTPS/2", message.getHeaders(true).get(0)); + message.removeUrlParam("hasfast"); + assertEquals("POST /log?format=new&authuser=0 HTTPS/2", message.getHeaders(true).get(0)); + message.addUrlParam("prova", "provona"); + assertEquals("POST /log?format=new&authuser=0&prova=provona HTTPS/2", message.getHeaders(true).get(0)); + } + + @Test + @DisplayName("") + public void test_removeUrlParam() throws ParsingException { + HTTPReqRes message = initMessage_ok(); + message.removeUrlParam("format"); + assertEquals("https://play.google.com/log?hasfast=true&authuser=0", message.getUrl()); + } + + @Test + @DisplayName("") + public void test_addUrlParam() throws ParsingException { + HTTPReqRes message = initMessage_ok(); + message.addUrlParam("test", "test"); + assertEquals("https://play.google.com/log?format=json&hasfast=true&authuser=0&test=test", message.getUrl()); + } + + @Test + @DisplayName("") public void test_getHeadParam() { HTTPReqRes message = initMessage_ok(); String value = message.getHeadParam(true, "Origin"); @@ -235,4 +326,29 @@ public void test_removeHeadParameter() { String value = message.getHeadParam(true, "Origin"); assertEquals("", value); } + + @Test + public void test_add_body() { + // add body to a message that does not have it + HTTPReqRes message = init_message_no_body(); + assertFalse(message.hasBody(true)); + + message.addBody(true, "testbodycontent"); + + assertTrue(message.hasBody(true)); + assertEquals("testbodycontent", new String(message.getBody(true))); + + message.addBody(true, "1"); + assertEquals("testbodycontent1", new String(message.getBody(true))); + } + + @Test + public void test_edit_body_regex() { + HTTPReqRes message = initMessage_ok(); + + message.editBodyRegex(true, "conte", "1234"); + assertEquals("body1234nt", new String(message.getBody(true))); + } + + }