Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Ubuntu Focal gcc-10 setup and update related usage instructions. #371

Merged
merged 1 commit into from
Apr 29, 2024

Conversation

kirkrodrigues
Copy link
Member

@kirkrodrigues kirkrodrigues commented Apr 29, 2024

Description

In #131, we made GCC v10+ a requirement. Since the default version of GCC on Ubuntu Focal is v9, #131 also added set up steps to configure v10 as the compiler on Focal. However, these steps were flawed:

  • They were configuring gcc and g++ as the names for the alternatives whereas Ubuntu usually sets cc and c++ as the names.
  • Adding the alternatives was done as a configuration step rather than as part of the installation.
  • The priority of the added alternatives was set to 10/20 rather than something low that wouldn't affect any alternatives the user already had installed and prioritized.
  • They were not configuring slave links for the man pages (something the Ubuntu GCC package maintainers usually do).
    • To investigate the alternatives created by the package maintainers, you can use update-alternatives --query <program> where <program> is one of cc, c++, or cpp.

This PR addresses these limitations:

  • We create low-priority alternatives for cc, c++, and cpp (the same alternatives created by the Ubuntu GCC package maintainers, except for GCC v10) as part of the dependency install scripts.
    • We no longer create alternatives for gcc and g++.
  • The configuration setup steps only switch to the alternative.

Validation performed

  • Validated that workflow for building and testing the Ubuntu core deps container image succeeded.
  • Built and ran the container image locally.
  • Validated that cc --version, c++ --version and /lib/cpp --version all print a v10 version string.
    • We use /lib/cpp based on the alternative that the Ubuntu gcc package usually creates (see ls /etc/alternatives).
  • Validated that building clp showed that it was using v10.

@junhaoliao
Copy link
Member

In a Ubuntu Focal distribution where package gcc is not installed, /usr/bin/x86_64-linux-gnu-gcc does not exist, which can cause task "package" to fail when it pip installs mariadb.

Building wheels for collected packages: mariadb
  Building wheel for mariadb (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for mariadb (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-38
      creating build/lib.linux-x86_64-cpython-38/mariadb
      copying mariadb/__init__.py -> build/lib.linux-x86_64-cpython-38/mariadb
      copying mariadb/__init__.py -> build/lib.linux-x86_64-cpython-38/mariadb
      creating build/lib.linux-x86_64-cpython-38/mariadb/constants
      copying mariadb/constants/__init__.py -> build/lib.linux-x86_64-cpython-38/mariadb/constants
      copying mariadb/constants/CLIENT.py -> build/lib.linux-x86_64-cpython-38/mariadb/constants
      copying mariadb/constants/INDICATOR.py -> build/lib.linux-x86_64-cpython-38/mariadb/constants
      copying mariadb/constants/CURSOR.py -> build/lib.linux-x86_64-cpython-38/mariadb/constants
      copying mariadb/constants/FIELD_TYPE.py -> build/lib.linux-x86_64-cpython-38/mariadb/constants
      running build_ext
      building 'mariadb._mariadb' extension
      creating build/temp.linux-x86_64-cpython-38
      creating build/temp.linux-x86_64-cpython-38/mariadb
      x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DPY_MARIADB_MAJOR_VERSION=1 -DPY_MARIADB_MINOR_VERSION=0 -DPY_MARIADB_PATCH_VERSION=11 -I/usr/include/mariadb -I/usr/include/mariadb/mysql -I./include -I/home/junhao/workspace/clp/build/package-venv/include -I/usr/include/python3.8 -c mariadb/mariadb.c -o build/temp.linux-x86_64-cpython-38/mariadb/mariadb.o -DDEFAULT_PLUGINS_SUBDIR=\"/usr/lib/x86_64-linux-gnu/mariadb19/plugin\"
      error: command 'x86_64-linux-gnu-gcc' failed: No such file or directory
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for mariadb
Failed to build mariadb
ERROR: Could not build wheels for mariadb, which is required to install pyproject.toml-based projects

After we install gcc-10, only /usr/bin/x86_64-linux-gnu-gcc-10 exists. Doing sudo ln -s /usr/bin/x86_64-linux-gnu-gcc-10 /usr/bin/x86_64-linux-gnu-gcc can get the pip install to succeed without seeing any compilation error; however, it might not be the best approach, given gcc-10 is not the version used in the Ubuntu Focal toolchain (i.e. Python 3.8 + GCC 9 + ...). Maybe in some future PR, we can add sudo apt install gcc into the non-core dep setup scripts.

@junhaoliao junhaoliao self-requested a review April 29, 2024 06:07
@kirkrodrigues
Copy link
Member Author

After we install gcc-10, only /usr/bin/x86_64-linux-gnu-gcc-10 exists. Doing sudo ln -s /usr/bin/x86_64-linux-gnu-gcc-10 /usr/bin/x86_64-linux-gnu-gcc can get the pip install to succeed without seeing any compilation error; however, it might not be the best approach, given gcc-10 is not the version used in the Ubuntu Focal toolchain (i.e. Python 3.8 + GCC 9 + ...). Maybe in some future PR, we can add sudo apt install gcc into the non-core dep setup scripts.

Yep, I agree. In the upcoming PR to automate the package build, we'll have a container + scripts which install the normal gcc toolchain so that Python packages with C extensions can be installed without issue.

Copy link
Member

@junhaoliao junhaoliao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on a clean WSL Ubuntu 20.04.

How about

Refactor Ubuntu Focal gcc-10 setup and update related usage instructions.

@kirkrodrigues kirkrodrigues changed the title Clean-up usage instructions and setup for gcc-10 on Ubuntu Focal. Refactor Ubuntu Focal gcc-10 setup and update related usage instructions. Apr 29, 2024
@kirkrodrigues kirkrodrigues merged commit 90214ed into y-scope:main Apr 29, 2024
9 checks passed
@kirkrodrigues kirkrodrigues deleted the update-alternatives branch April 29, 2024 06:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants