Skip to content

v0.17.0: Inference, CLI and Space API
Compare
Choose a tag to compare
@Wauplin Wauplin released this 08 Sep 14:41
· 543 commits to main since this release
v0.17.0
7d82cd9

InferenceClient

All tasks are now supported! 💥

Thanks to a massive community effort, all inference tasks are now supported in InferenceClient. Newly added tasks are:

Documentation, including examples, for each of these tasks can be found in this table.

All those methods also support async mode using AsyncInferenceClient.

Get InferenceAPI status

Sometimes knowing which models are available or not on the Inference API service can be useful. This release introduces two new helpers:

  1. list_deployed_models aims to help users discover which models are currently deployed, listed by task.
  2. get_model_status aims to get the status of a specific model. That's useful if you already know which model you want to use.

Those two helpers are only available for the Inference API, not Inference Endpoints (or any other provider).

>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()

# Discover zero-shot-classification models currently deployed 
>>> models = client.list_deployed_models()
>>> models["zero-shot-classification"]
['Narsil/deberta-large-mnli-zero-cls', 'facebook/bart-large-mnli', ...]

# Get status for a specific model
>>> client.get_model_status("bigcode/starcoder")
ModelStatus(loaded=True, state='Loaded', compute_type='gpu', framework='text-generation-inference')

Few fixes

  • Send Accept: image/png as header for image tasks by @Wauplin in #1567
  • FIX text_to_image and image_to_image parameters by @Wauplin in #1582
  • Distinguish _bytes_to_dict and _bytes_to_list + fix issues by @Wauplin in #1641
  • Return whole response from feature extraction endpoint instead of assuming its shape by @skulltech in #1648

Download and upload files... from the CLI 🔥 🔥 🔥

This is a long-awaited feature finally implemented! huggingface-cli now offers two new commands to easily transfer file from/to the Hub. The goal is to use them as a replacement for git clone, git pull and git push. Despite being less feature-complete than git (no .git/ folder, no notion of local commits), it offers the flexibility required when working with large repositories.

Download

# Download a single file
>>> huggingface-cli download gpt2 config.json
/home/wauplin/.cache/huggingface/hub/models--gpt2/snapshots/11c5a3d5811f50298f278a704980280950aedb10/config.json

# Download files to a local directory
>>> huggingface-cli download gpt2 config.json --local-dir=./models/gpt2
./models/gpt2/config.json

# Download a subset of a repo
>>> huggingface-cli download bigcode/the-stack --repo-type=dataset --revision=v1.2 --include="data/python/*" --exclu
de="*.json" --exclude="*.zip"
Fetching 206 files:   100%|████████████████████████████████████████████| 206/206 [02:31<2:31, ?it/s]
/home/wauplin/.cache/huggingface/hub/datasets--bigcode--the-stack/snapshots/9ca8fa6acdbc8ce920a0cb58adcdafc495818ae7

Upload

# Upload single file
huggingface-cli upload my-cool-model model.safetensors

# Upload entire directory
huggingface-cli upload my-cool-model ./models

# Sync local Space with Hub (upload new files except from logs/, delete removed files)
huggingface-cli upload Wauplin/space-example --repo-type=space --exclude="/logs/*" --delete="*" --commit-message="Sync local Space with Hub"

Docs

For more examples, check out the documentation:

🚀 Space API

Some new features have been added to the Space API to:

  • request persistent storage for a Space
  • set a description to a Space's secrets
  • set variables on a Space
  • configure your Space (hardware, storage, secrets,...) in a single call when you create or duplicate it
>>> from huggingface_hub import HfApi
>>> api = HfApi()
>>> api.create_repo(
...     repo_id=repo_id,
...     repo_type="space",
...     space_sdk="gradio",
...     space_hardware="t4-medium",
...     space_sleep_time="3600",
...     space_storage="large",
...     space_secrets=[{"key"="HF_TOKEN", "value"="hf_api_***"}, ...],
...     space_variables=[{"key"="MODEL_REPO_ID", "value"="user/repo"}, ...],
... )

A special thank to @martinbrose who largely contributed on those new features.

📚 Documentation

A new section has been added to the upload guide with some tips about how to upload large models and datasets to the Hub and what are the limits when doing so.

🗺️ The documentation organization has been updated to support multiple languages. The community effort has started to translate the docs to non-English speakers. More to come in the coming weeks!

Breaking change

The behavior of InferenceClient.feature_extraction has been updated to fix a bug happening with certain models. The shape of the returned array for transformers models has changed from (sequence_length, hidden_size) to (1, sequence_length, hidden_size) which is the breaking change.

  • Return whole response from feature extraction endpoint instead of assuming its shape by @skulltech in #1648

QOL improvements

HfApi helpers:

Two new helpers have been added to check if a file or a repo exists on the Hub:

>>> from huggingface_hub import file_exists
>>> file_exists("bigcode/starcoder", "config.json")
True
>>> file_exists("bigcode/starcoder", "not-a-file")
False

>>> from huggingface_hub import repo_exists
>>> repo_exists("bigcode/starcoder")
True
>>> repo_exists("bigcode/not-a-repo")
False

Also, hf_hub_download and snapshot_download are now part of HfApi (keeping the same syntax and behavior).

  • Add download alias for hf_hub_download to HfApi by @Wauplin in #1580

Download improvements:

  1. When a user tries to download a model but the disk is full, a warning is triggered.
  2. When a user tries to download a model but a HTTP error happen, we still check locally if the file exists.
  • Check local files if (RepoNotFound, GatedRepo, HTTPError) while downloading files by @jiamings in #1561
  • Implemented check_disk_space function by @martinbrose in #1590

Small fixes and maintenance

⚙️ Doc fixes

⚙️ Other fixes

⚙️ Internal

  • Prepare for 0.17 by @Wauplin in #1540
  • update mypy version + fix issues + remove deprecatedlist helper by @Wauplin in #1628
  • mypy traceck by @Wauplin (direct commit on main)
  • pin pydantic version by @Wauplin (direct commit on main)
  • Fix ci tests by @Wauplin in #1630
  • Fix test in contrib CI by @Wauplin (direct commit on main)
  • skip gated repo test on contrib by @Wauplin (direct commit on main)
  • skip failing test by @Wauplin (direct commit on main)
  • Fix fsspec tests in ci by @Wauplin in #1635
  • FIX windows CI by @Wauplin (direct commit on main)
  • FIX style issues by pinning black version by @Wauplin (direct commit on main)
  • forgot test case by @Wauplin (direct commit on main)
  • shorter is better by @Wauplin (direct commit on main)

🤗 Significant community contributions

The following contributors have made significant changes to the library over the last release:

  • @dulayjm
    • Add object detection to inference client (#1548)
  • @martinbrose
    • Added support for secret description (#1594)
    • Check if repo or file exists (#1591)
    • Implemented check_disk_space function (#1590)
    • Added support for space variables (#1592)
    • Add settings for creating and duplicating spaces (#1625)
    • Implemented CLI download functionality (#1617)
    • Implemented CLI upload functionality (#1618)
    • Add text classification to inference client (#1606)
    • Add token classification to inference client (#1607)
    • Add translation to inference client (#1608)
    • Add question answering to inference client (#1609)
    • Add table question answering to inference client (#1612)
    • Add fill mask to inference client (#1613)
    • Add visual question answering to inference client (#1621)
    • Add document question answering to InferenceClient (#1620)
    • Add tabular classification to inference client (#1614)
    • Add tabular regression to inference client (#1615)
    • Add list_deployed_models to inference client (#1622)
  • @sifisKoen
    • Add get_model_status function (#1558) (#1559)
    • Add documentation for modelcard Metadata. Resolves (#1448) (#1631)

Contributors

julien-c, severo, and 11 other contributors
Assets 2
Loading