Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

general gene regulatory network link not working with wget #31

Open
RubenVanEsch opened this issue May 7, 2024 · 4 comments
Open

general gene regulatory network link not working with wget #31

RubenVanEsch opened this issue May 7, 2024 · 4 comments

Comments

@RubenVanEsch
Copy link

Hi LINGER team,
paper looked really cool! So wanted to try and run on some of my own data and tried to download the general GRN using the wget code provided:

wget --load-cookies /tmp/cookies.txt "https://drive.usercontent.google.com/download?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://drive.usercontent.google.com/download?id=1lAlzjU5BYbpbr4RHMlAGDOh9KWdCMQpS' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=1lAlzjU5BYbpbr4RHMlAGDOh9KWdCMQpS" -O data_bulk.tar.gz && rm -rf /tmp/cookies.txt

however, I get the following error message:
<!DOCTYPE html><html><head><title>Google Drive - Quota exceeded</title><meta http-equiv="content-type" content="text/html; charset=utf-8"/><style nonce="xMKJdxGwxulflmiIEboRhQ">.goog-link-button{position:relative;color:#15c;text-decoration:underline;cursor:pointer}.goog-link-button-disabled{color:#ccc;text-decoration:none;cursor:default}body{color:#222;font:normal 13px/1.4 arial,sans-serif;margin:0}.grecaptcha-badge{visibility:hidden}.uc-main{padding-top:50px;text-align:center}#uc-dl-icon{display:inline-block;margin-top:16px;padding-right:1em;vertical-align:top}#uc-text{display:inline-block;max-width:68ex;text-align:left}.uc-error-caption,.uc-warning-caption{color:#222;font-size:16px}#uc-download-link{text-decoration:none}.uc-name-size a{color:#15c;text-decoration:none}.uc-name-size a:visited{color:#61c;text-decoration:none}.uc-name-size a:active{color:#d14836;text-decoration:none}.uc-footer{color:#777;font-size:11px;padding-bottom:5ex;padding-top:5ex;text-align:center}.uc-footer a{color:#15c}.uc-footer a:visited{color:#61c}.uc-footer a:active{color:#d14836}.uc-footer-divider{color:#ccc;width:100%}.goog-inline-block{position:relative;display:-moz-inline-box;display:inline-block}* html .goog-inline-block{display:inline}*:first-child+html .goog-inline-block{display:inline}sentinel{}</style><link rel="icon" href="//ssl.gstatic.com/docs/doclist/images/drive_2022q3_32dp.png"/></head><body><div class="uc-main"><div id="uc-text"><p class="uc-error-caption">Sorry, you can&#39;t view or download this file at this time.</p><p class="uc-error-subcaption">Too many users have viewed or downloaded this file recently. Please try accessing the file again later. If the file you are trying to access is particularly large or is shared with many people, it may take up to 24 hours to be able to view or download the file. If you still can't access a file after 24 hours, contact your domain administrator.</p></div></div><div class="uc-footer"><hr class="uc-footer-divider"></div></body></html>

going to the drive link on my laptop and downloading also does not seem to work, so both options seem to be blocked

@withermatt
Copy link

withermatt commented May 7, 2024

Hi, I am having the same issue downloading the general grn.
wget --load-cookies /tmp/cookies.txt "https://drive.usercontent.google.com/download?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt
--keep-session-cookies --no-check-certificate 'https://drive.usercontent.google.com/download?id=1lAlzjU5BYbpbr4RHMlAGDOh9KWdCMQpS' -O- | sed -rn 's/.confirm=([0-9A-Za-z_]+)./\1\n/p')&id=1lAlzjU5BYbpbr4RHMlAG
DOh9KWdCMQpS" -O data_bulk.tar.gz && rm -rf /tmp/cookies.txt

My output is:
--2024-05-07 13:19:54-- https://drive.usercontent.google.com/download?export=download&confirm=&id=1lAlzjU5BYbpbr4RHMlAGDOh9KWdCMQpS
Resolving drive.usercontent.google.com (drive.usercontent.google.com)... 142.251.211.225, 2607:f8b0:400a:804::2001
Connecting to drive.usercontent.google.com (drive.usercontent.google.com)|142.251.211.225|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2004 (2.0K) [text/html]

When I unzip,
tar -xzf data_bulk.tar.gz

My output is:
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

Downloading from firefox initiates (file size 19.4 GB) but ends up failing after about 75% completion.
UPDATE: I retried to download from google drive in firefox and it was successful. Must have timed out or something my first try.

@RubenVanEsch
Copy link
Author

@withermatt for the wget, you get the same error, but the extension of your file is wrong because the download failed. If you change the extension to .txt and then open it with vim or something you will see the same message

@amssyqy
Copy link
Collaborator

amssyqy commented May 10, 2024

Thanks for using LINGER~ There are 2 more options to download the data.

  1. wget --no-check-certificate -O data_bulk.tar.gz "https://drive.usercontent.google.com/download?id=1lAlzjU5BYbpbr4RHMlAGDOh9KWdCMQpS&export=download&authuser=0&confirm=t&uuid=1dd434ea-bada-46dd-80bc-3c80945066cd&at=APZUnTW25jguC0k7Lljqk6Bo71nF%3A1705074384034"
  2. Open the link in Chrome: https://drive.google.com/file/d/1lAlzjU5BYbpbr4RHMlAGDOh9KWdCMQpS/view?usp=sharing
    We are checking the reason why the original command does not work. Maybe one reason is 'Too many users have viewed or downloaded this file recently.' We are trying to solve this problem.
    Thank you~

@RubenVanEsch
Copy link
Author

RubenVanEsch commented May 21, 2024

That new wget link works great!
Though it might have also just been the time-out being lifted by trying it a week later

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants