Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document the new prefix.dev robotology channel mirror and upload new robotology conda packages to both anaconda.org and prefix.dev #1716

Merged
merged 3 commits into from
Sep 19, 2024

Conversation

traversaro
Copy link
Member

This permit us to keep latest packages on robotology channel on anaconda.org so that -c robotology continue to work for latest packages even on vanilla conda, but we keep the full history of packages in prefix.dev robotology.

To kickstart the upload, I used the following two scripts (that I do not upload in the repo as they will not be necessary in the future):

download_all.sh:

conda-mirror --upstream-channel robotology --target-directory robotology_mirror --platform noarch
conda-mirror --upstream-channel robotology --target-directory robotology_mirror --platform linux-64
conda-mirror --upstream-channel robotology --target-directory robotology_mirror --platform osx-64
conda-mirror --upstream-channel robotology --target-directory robotology_mirror --platform win-64

upload_all.sh:

# The script requires PREFIX_DEV_TOKEN to be defined to upload files to prefix.dev
if [ -z "${PREFIX_DEV_TOKEN}" ]; then
  echo "Error: The environment variable PREFIX_DEV_TOKEN is not defined, please define it with your prefix.dev API key"
  exit 1
fi

pixi auth login https://prefix.dev --token $PREFIX_DEV_TOKEN

# Function to check if a file exists on the server
file_exists_on_server() {
  local file_path="$1"
  # Extract the filename
  local filename=$(basename "$file_path")
  # Extract the platform directory from the file path
  local platform=$(echo "$file_path" | grep -oE 'noarch|linux-64|osx-64|win-64')
  # Construct the URL where the file would be hosted
  local url="https://repo.prefix.dev/robotology/$platform/$filename"
  # Perform a HEAD request to check if the file exists
  local status_code=$(curl -s -o /dev/null -w "%{http_code}" -I "$url")
  if [ "$status_code" -ne 404 ]; then
    return 0  # File exists
  else
    return 1  # File does not exist
  fi
}

# Collect all files into an array
files=(
  ./robotology_mirror/noarch/*.tar.bz2
  ./robotology_mirror/noarch/*.conda
  ./robotology_mirror/linux-64/*.tar.bz2
  ./robotology_mirror/linux-64/*.conda
  ./robotology_mirror/osx-64/*.tar.bz2
  ./robotology_mirror/osx-64/*.conda
  ./robotology_mirror/win-64/*.tar.bz2
  ./robotology_mirror/win-64/*.conda
)

# Loop over the array and upload each file if not already present
for file in "${files[@]}"; do
  # Check if the file exists locally before proceeding
  if [ -f "$file" ]; then
    if file_exists_on_server "$file"; then
      echo "Skipping upload of '$file'; it already exists on the server."
    else
      echo "Uploading '$file'..."
      pixi upload https://prefix.dev/api/v1/upload/robotology "$file"
    fi
  else
    echo "File '$file' does not exist locally. Skipping."
  fi
done

pixi auth logout https://prefix.dev

@traversaro traversaro merged commit 2821bd0 into master Sep 19, 2024
29 checks passed
@traversaro traversaro deleted the documentprefixdevrobotology branch September 19, 2024 11:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant