Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Packaging] BREAKING CHANGE: Remove unnecessary packages in docker image #27567

Merged
merged 7 commits into from
Oct 25, 2023
Merged

[Packaging] BREAKING CHANGE: Remove unnecessary packages in docker image #27567

Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
b886850
Init
bebound Oct 11, 2023
5516ed5
Minor fix
bebound Oct 11, 2023
9c8e701
Merge branch 'dev' into remove-unused
bebound Oct 18, 2023
c5c78d1
Add jp jq
bebound Oct 18, 2023
acc0cd5
Minor fix
bebound Oct 18, 2023
57469a7
Add openssh-keygen
bebound Oct 19, 2023
ed48481
Minor fix
bebound Oct 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 14 additions & 34 deletions alpine.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,49 +25,29 @@ LABEL maintainer="Microsoft" \
org.label-schema.vcs-url="https://github.com/Azure/azure-cli.git" \
org.label-schema.docker.cmd="docker run -v \${HOME}/.azure:/root/.azure -it mcr.microsoft.com/azure-cli:$CLI_VERSION"

# bash gcc make openssl-dev libffi-dev musl-dev - dependencies required for CLI
# openssh - included for ssh-keygen
# ca-certificates

# curl - required for installing jp
# jq - we include jq as a useful tool
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jq is a very useful tool and we frequently see users using jq to parse Azure CLI's output. Removing tools like this will cause a breaking change.

# pip wheel - required for CLI packaging
# jmespath-terminal - we include jpterm as a useful tool
# libintl and icu-libs - required by azure devops artifact (az extension add --name azure-devops)

# We don't use openssl (3.0) for now. We only install it so that users can use it.
RUN apk add --no-cache bash openssh ca-certificates jq curl openssl perl git zip \
&& apk add --no-cache --virtual .build-deps gcc make openssl-dev libffi-dev musl-dev linux-headers \
Copy link
Contributor Author

@bebound bebound Oct 23, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We used to remove .build-deps until #8291.
PS: az extension add -n azure-batch-cli-extensions works without those dependencies. I guess some dependencies finally provide the wheel packages over time.

Choosing Alpine as a base image is not a good idea, as some packages do not provide binary packages and the user must build them from source, such as the psutil package.
It's common to use cibuildwheel to build wheel. However, Alpine Linux uses musl, while manylinux wheels are based on glibc, so some packages are not available on Alpine. (Cibuildwheel supports build musllinux wheel, but not all package provide musl wheel)

Ref: pypa/pip#9735

&& apk add --no-cache libintl icu-libs libc6-compat \
&& apk add --no-cache bash-completion \
&& update-ca-certificates
# ca-certificates bash bash-completion jq jp openssh-keygen - for convenience
# libintl icu-libs - required by azure-devops https://github.com/Azure/azure-cli/pull/9683
# libc6-compat - required by az storage blob sync https://github.com/Azure/azure-cli/issues/10381
# gcc python3-dev musl-dev linux-headers libffi-dev - temporarily required by psutil
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

python3-dev is not in the original alpine.dockerfile. Is it really needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When install psutil in alpine, it prints

psutil could not be installed from sources because gcc is not installed. Try running:
sudo apk add gcc python3-dev

Weird, it can be built without python3-dev.

# curl - temporarily required by jp

ARG JP_VERSION="0.2.1"

RUN arch=$(arch | sed s/aarch64/arm64/ | sed s/x86_64/amd64/) && curl -L https://github.com/jmespath/jp/releases/download/${JP_VERSION}/jp-linux-$arch -o /usr/local/bin/jp \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jp is also demanded by some users. See the discussion at #25590.

Copy link
Contributor Author

@bebound bebound Oct 18, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added jp and jq again.

Should we also make ssh-keygen available by default? #1343

PS: Our docker image is more like a Linux image than a CLI image. The entrypoint is bash rather than az

&& chmod +x /usr/local/bin/jp

WORKDIR azure-cli
COPY . /azure-cli
RUN apk add --no-cache ca-certificates bash bash-completion libintl icu-libs libc6-compat jq openssh-keygen \
&& apk add --no-cache --virtual .build-deps gcc python3-dev musl-dev linux-headers libffi-dev curl \
&& update-ca-certificates && ./scripts/install_full.sh && python ./scripts/trim_sdk.py \
&& cat /azure-cli/az.completion > ~/.bashrc \
&& dos2unix /root/.bashrc /usr/local/bin/az \
&& arch=$(arch | sed s/aarch64/arm64/ | sed s/x86_64/amd64/) && curl -L https://github.com/jmespath/jp/releases/download/${JP_VERSION}/jp-linux-$arch -o /usr/local/bin/jp \
&& chmod +x /usr/local/bin/jp \
&& apk del .build-deps

# 1. Build packages and store in tmp dir
# 2. Install the cli and the other command modules that weren't included
RUN ./scripts/install_full.sh && python ./scripts/trim_sdk.py \
&& cat /azure-cli/az.completion > ~/.bashrc \
&& runDeps="$( \
scanelf --needed --nobanner --recursive /usr/local \
| awk '{ gsub(/,/, "\nso:", $2); print "so:" $2 }' \
| sort -u \
| xargs -r apk info --installed \
| sort -u \
)" \
&& apk add --virtual .rundeps $runDeps
RUN rm -rf /azure-cli

WORKDIR /

# Remove CLI source code from the final image and normalize line endings.
RUN rm -rf ./azure-cli && \
dos2unix /root/.bashrc /usr/local/bin/az

ENV AZ_INSTALLER=DOCKER
CMD bash