netx: investigate refraction-networking/utls (1/n)

[bassosimone]

https://github.com/refraction-networking/utls contains code to parrot popular TLS handshakes and it may be a good starting point to have pluggable TLS. This work is potentially important to avoid being flagged as MITM because our ClientHello signature does not match the User-Agent that we are using. It's also important to keep in mind that the User-Agent is being deprecated by the Chrome team, in favour of more specific headers, so we should also keep this in mind.

[bassosimone]

I've started looking into how to use this library when trying to understand the 400 Bad Request returned by WhatsApp when we claim we're Chrome latest and we use Golang's TLS stack. More work to be done in next sprint.

[bassosimone]

I am going to write down the results of this investigation in Sprint 33.

[bassosimone]

I have spent some time working to understand how to integrate refraction-networking/utls inside our http.Client. I firstly created a wrapper with connection caching just for it in the research/netx3 branch. Then, there was an open issue about doing something similar at, except where both http/1.1 and h2 can cohexist, so I adapted that code and submitted it here: refraction-networking/utls#74 (and it's also available at https://github.com/bassosimone/utlstransport). The connection-cache-with-ErrNoCachedConn solution seems great for us (also, http3 certainly has this functionality and I need to check whether http2 has it as well, maybe it has?). I think this way of organising an HTTP transport is ~better than the current one, especially in regards to how to pass around the proxy. I have also created a follow-up issue ooni/probe#1424.

[bassosimone]

I think it's reasonable to close this issue now!

[hellais]

The goal of this is to make our HTTPS client look more similar to Firefox, therefore ensuring that what we are measuring is as close as possible to a real browser, hence the data quality tag.