Skip to content

Commit

Permalink
javy-fuzz: Compare values instead of strings. (#706)
Browse files Browse the repository at this point in the history
This commit introduces value comparison for the json-differential
target.

The motivation behind this change is to overcome the complexity of float
value formatting.

JavaScript has very complex rules for Number formatting, for example,
numbers that have magnitude of 10^21, independent of the sign, will be
formatted via scientific notation. I was able to achieve certain format
parity by writing a custom serde serializer formatter and relying on
crates like `ryu` and `lexical`. Even though this worked to an extent,
there are some unwritten rules to my knowledge that make it harder to
achieve full formatting parity. For example, in some cases the
scientific notation would include the exponent with a sign e.g., `e+` or
`e-` depending on how big the number is. In the case of `lexical` or
`ryu` even though the numerical value is the same, no signs are used,
which makes it harder to perform differential testing.

Having to post-process the resulting string, will most likely incur in
a performance penalty that I'm not sure is worth, especially given that
no issues have been reported regarding the numerical value during my
testing.

Due to all this reasons, this commit switches toward value comparison
rahter than format/string comparison by parsing the stringified JSON
into `serde_json::Value` and asserting equality on top of it.
  • Loading branch information
saulecabrera authored Jul 24, 2024
1 parent 51a6da5 commit dc232b9
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions fuzz/fuzz_targets/json_differential.rs
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,8 @@ fn exec(data: &ArbitraryValue) -> Result<()> {
panic!("{}\n{}", from_js_error(cx.clone(), e), **data,);
}

output = globals.get("OUTPUT")?;
let result: String = globals.get("OUTPUT")?;
output = serde_json::from_str(&result).ok();

Ok::<(), Error>(())
})?;
Expand All @@ -63,7 +64,8 @@ fn exec(data: &ArbitraryValue) -> Result<()> {
panic!("{}\n{}", from_js_error(cx.clone(), e), **data);
}

ref_output = globals.get("OUTPUT")?;
let result: String = globals.get("OUTPUT")?;
ref_output = serde_json::from_str(&result).ok();

Ok::<(), Error>(())
})?;
Expand Down

0 comments on commit dc232b9

Please sign in to comment.