Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Uploading small files is slow #15

Open
jffry opened this issue Mar 1, 2016 · 0 comments
Open

Uploading small files is slow #15

jffry opened this issue Mar 1, 2016 · 0 comments

Comments

@jffry
Copy link
Contributor

jffry commented Mar 1, 2016

Uploading tiny files can be very slow. Here we upload a 1081-byte JSON file, with conversion to a Google Doc explicitly disabled:

(time
  (let [auth (google-apps-clj.credentials/default-credential ["https://www.googleapis.com/auth/drive"])
        folder-id "FOLDERIDFOLDERIDFOLDERIDFOLD"
        raw-req (google-apps-clj.google-drive/upload-file folder-id "test.json" "desc" "text/plain" "/path/to/test.json" false)
        req (assoc raw-req :fields ["id"])]
    (google-apps-clj.google-drive/execute-query! auth req)))

"Elapsed time: 2312.169 msecs"

It seems that by default, the Java client library uses resumable media uploads, which will upload the file using multiple requests: https://developers.google.com/api-client-library/java/google-api-java-client/media-upload#direct

For tiny files, we don't need this. In ee4931b I added an option to support direct media uploads. For this small file I ran some tests both with and without direct upload enabled, to see the response times in ms:

without with
943.888 1027.120
2420.539 1615.611
1102.272 624.711
800.267 625.270
2845.130 668.712
869.747 689.082
753.111 691.276
4449.890 744.944
2416.990 834.301
1096.242 908.461
---------- ---------- -------
1769.808 842.949 mean
1226.861 301.284 stdev

While this is a big improvement, a mean of 842ms to upload a 1081-byte file is still a bit too large and leaves room to improve.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant