Change testclient to work with encryption.

[Sp3ctr3]

The current version of testclient does not work with encryption. If we are working with urllib2 we'll need to overload a load of function to get the public key. I think we need to use the twisted HTTPClient and overload it's functions.

[boxtown]

Couldn't we just implement a HTTP request to grab the key on the proxy side? Like if the proxy see's a GET with a header or query string that says 'get-key=true' or something? Then we could just call that first before trying to open another website using the proxy? How is the client side going to retrieve the key anyways? I imagine if the client side doesn't already have that implemented this is how we could do it

[Sp3ctr3]

That sounds great actually! The client side was going to retrieve the key the same way, but this sounds better. I'll try to change the implementation accordingly.

[Sp3ctr3]

When calling urllib2.urlopen twice, both start a factory, complete the transaction and then end it. This means that if we were to implement encryption with the recently mentioned idea we'll need some sort of logging system so as to track encryption keys across clients. Right now what happens is that each connection gets their own encryption keys.

[boxtown]

Well since this is for a test, you could just generate a public key once and keep as a global in the script or something. Then you can just send that key each time. No need to generate new keys per connection for a test script.

[admwx7]

I was thinking the flow would go as follows:
Client: sends encrypted (via public key) request to server with a 'get-key=true' (or whatever we decide on) and 'temp-key=a quick temporary key'
Server: receives clients request, decrypts client request using secret, creates a one-time id:encKey pair and stores it then responds to the client with the encKey encrypted using the clients temp key. The id would have to be generated based on something specific to each client, so maybe something based on the IP and MAC address.
Client: gets server response, decrypts using the temp key, stores the servers encKey, all following traffic would be encrypted using the encKey.

Now, that being said after writing it all out, it might make more sense to generate our keys client side, then on initial request the client sends the key it wants to communicate using encrypted via the servers public key, the server would store it using some key it generates (I would guess based on the IP/MAC address so it can grab it later easily with no extra work on the client-side). When the plugin starts up it could generate the key before the first request is even sent.

I can see upsides and downsides to both options.

[admwx7]

A major benefit we could do to the client-side generated keys would be sending the encryption algorithm used to the server, so we could support custom encryption modules a lot easier.

[boxtown]

I definitely think the client should be generating it's own public-private key pair. Then you can encrypt as follows:

Client: Send request with 'get-key=true' and 'pub-key='.
Server: Generate and store shared secret, client-id. Send back server public key, shared secret, and client-id encrypted by client public key

Now the client has the shared secret and the server's public key and vice versa. There's no need for a temporary key

[admwx7]

Agreed, client doesn't need to know it's own ID if we generate it based on client information that's natively stored in a request, such as the IP and MAC address since we don't provide them information based on this, just attempt to decrypt what was sent so spoofing them doesn't provide anything to the user. The downside is if it changes, such as using a proxy to make requests to our service for example.

If we send 'pub-key' as a header we don't need the 'get-key' actually, we can key off of that. This would work until the client key times out, which could be set to 30 minutes or something. This should work pretty well.

[admwx7]

Asymmetric key gens are pretty costly so adding this to the client-side doesn't seem like our best option unfortunately...The only solution I can come up with that keeps this level of security is as follows:

Client: Sends a connection request to the server (unencrypted, no information aside from 'get-key' in header).
Client: Starts generating a shared key for the session (symmetric). Starts processing actual request, adds the session key to the header 'session-key'.
Server: Responds to 'get-key' with the servers public key.
Client: Receives servers public key then sends the request along with session-key encrypted using the server's public key.
Server: Receives shared key, stores it with some identifiable name for the session and processes the request, all following data uses the session-key.

This setup gives the client full control of what encryption it wants to use, we can add a header to notify the server what algorithm is being used. It also ensures the client has control to change it's own key whenever it wants and allows the server to generate a new pub/priv key every day or hour if it wants and the client will always get updated with the new key when it makes the request. Downside is that there's going to be a bit of overhead associated with the initial requests.

Nathan and I are working on a pitch deck for Thursday and I know Yashin has been trying to get this encryption implemented so lets see if we can hammer this out here soon.

[Sp3ctr3]

Hmmm.in either approach we will need to have a key storage system on the server.

[admwx7]

Using a session system will require that no matter our solution. It's either that or the client sends it's shared key to the server as part of the header encrypted with the servers public key on every request and only the responses are encrypted using the session key pulled straight from the header. Then the client could send a different key for it's data on every request if it wanted to and the server would never store anything aside from temporarily while it waited for the responses from other servers.