I want to connect to mirror website of huggingface instead of huggingface itself.

[l1ghtyagam1Jin]

Due to the block, I am not able to connect to huggingface in China.
I tried to set HF_ENDPOINT to ''https://hf-mirror.com' but this doesn't work. It still tries to connect to huggingface.com.

[ZipingL]

You probably need to use a VPN. I'm not sure what the mirror site is, but it seems to be from China. How would you be able to access Hugging Face from a site hosted in China as a mirror @l1ghtyagam1Jin?

It seems to be an endpoint mirror. Did you check if the site is down? Have you tried the other methods provided on their page? You mentioned setting the environment variable, but it appears they offer three methods. Did you try them all? Perhaps try again later. Did you install the cli did you try that? There's a lot of missing information right now. Did you follow all the steps from the mirror site, it provides a set of instructions, and did you try watching their tutorial?

The domain name of this website is hf-mirror.com , which is used to mirror the domain name huggingface.co . As a public welfare project, it is committed to helping domestic AI developers download models and data sets quickly and stably. Please see the lower left corner of the page for donation support. Thank you for your support!

For more detailed usage, please see "This Tutorial" .

Method 1: Download from the web

Search on this site and Files and Versiondownload the file from the model homepage.

Method 2: huggingface-cli

huggingface-cli It is a command line tool officially provided by Hugging Face, which comes with a complete download function.

1. Install dependencies

pip install -U huggingface_hub

2. Set environment variables

Linux

 export HF_ENDPOINT=https://hf-mirror.com

Windows Powershell

 $env:HF_ENDPOINT = "https://hf-mirror.com"

It is recommended to write the above line ~/.bashrc.

3.1 Download the model

 huggingface-cli download --resume-download gpt2 --local-dir gpt2

3.2 Download the dataset

huggingface-cli download --repo-type dataset --resume-download wikitext --local-dir wikitext

You can add --local-dir-use-symlinks False the parameter to disable file soft links, so that the download path is WYSIWYG. Please refer to the tutorial mentioned above for detailed explanation.

Method 3: Using hfd

HFD is a huggingface-specific download tool developed by our website. It is based on mature toolsaria2and can achieve stable and high-speed downloading without interruption.

1. Download hfd

wget https://hf-mirror.com/hfd/hfd.shchmod a+x hfd.sh

2. Set environment variables
Linux

export HF_ENDPOINT=https://hf-mirror.com

Windows Powershell

  $env:HF_ENDPOINT = "https://hf-mirror.com"

3.1 Download the model

./hfd.sh gpt2

3.2 Download the dataset

  ./hfd.sh wikitext --dataset

Method 4: Using environment variables (non-intrusive)

Non-intrusive, can solve most cases. The huggingface toolchain will obtain HF_ENDPOINT environment variables to determine the URL used to download files, so you can use it to solve it by setting variables.

  HF_ENDPOINT=https://hf-mirror.com python your_script.py

However, some data sets have built-in download scripts, so you need to manually change the address in the script to achieve it.

Frequently asked questions

Q: Some items require login, how to download?

A : Some Gated Repos require login and permission. To ensure account security, this site does not support login. You must first log in to the Hugging Face official website and apply for permission. After obtaining the Access Token on the official website , return to the mirror site and download using the command line.
Methods for downloading Gated Repo with some tools:

huggingface-cli: add --tokenparameters

      huggingface-cli download --token hf_*** --resume-download meta-llama/Llama-2-7b-hf --local-dir Llama-2-7b-hf

hfd: Add --hf_username``--hf_tokenparameters

  hfd meta-llama/Llama-2-7b --hf_username YOUR_HF_USERNAME --hf_token hf_***

For details from_pretrainedon wgethow curlto set up the authentication token, see the tutorial mentioned in the first paragraph above.