Dockerfile optimization proposal #660

benbrummer · 2024-11-27T12:43:07Z

docker image seems to be quite big, so I try to identify ways to shrink it

Ideas

apt-get --no-install-recommends
Review apt packages:
- snappdf requirements: https://github.com/beganovich/snappdf?tab=readme-ov-file#headless-chrome-doesnt-launch-on-unix
Review php modules
- https://invoiceninja.github.io/en/self-host-installation/
https://docs.docker.com/build/building/multi-stage/
php 8.3

apt packages

required:

curl => google-chrome-stable
gnupg2 => google-chrome-stable
gosu => init.sh
default-mysql-client => initial user creation

obsolete (?):

git
unzip
wget
zip
...

php modules

[PHP Modules]
bcmath
Core
ctype
curl
date
dom
exif
fileinfo
filter
gd
gmp
hash
iconv
imagick
json
libxml
mbstring
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
pdo_sqlite
Phar
posix
random
readline
redis
Reflection
saxonc
session
SimpleXML
soap
sodium
SPL
sqlite3
standard
tokenizer
xml
xmlreader
xmlwriter
Zend OPcache
zip
zlib
[Zend Modules]
Zend OPcache

Misc

dockerfiles/debian/Dockerfile

Line 149 in a6d8f66

RUN sed -i "s/user = www-data/user = www-data/g" /usr/local/etc/php-fpm.d/www.conf \

seems to do nothing
Exact versions for php modlues e.g saxon vs no version for other modules?

The text was updated successfully, but these errors were encountered:

turbo124 · 2024-11-28T00:30:56Z

@benbrummer we can certainly start with the obsolete ones like zip/unzip

turbo124 · 2024-11-29T02:58:35Z

@benbrummer

related #663 I had to roll back the architecture if/fi blocks as they were preventing required dependencies from installing.

benbrummer · 2024-11-29T08:28:30Z

Apart of the default-mysql-client, did you face any other issues?

Most of them will be installed as dependencies of google-chrome-stable, so we could move them to arm64 section. The other ones are not clear to me.

    fonts-liberation \ google-chrome-stable
    fonts-noto-cjk \ ?
    fonts-noto-cjk-extra \ ?
    fonts-wqy-microhei \ ?
    fonts-wqy-zenhei \ ?
    libasound2 \ google-chrome-stable
    libatk-bridge2.0-0 \ google-chrome-stable
    libatk1.0-0 \ google-chrome-stable
    libatspi2.0-0 \ google-chrome-stable
    libcups2 \ google-chrome-stable
    libdbus-1-3 \ google-chrome-stable
    libdrm2 \ google-chrome-stable
    libgbm1 \ google-chrome-stable
    libgtk-3-0 \ google-chrome-stable
    libnspr4 \ google-chrome-stable
    libnss3 \ google-chrome-stable
    libonig-dev \ ?
    libpng-dev \ ? => google-chrome-stable (libpng16-16)
    libwayland-client0 \ google-chrome-stable
    libxcomposite1 \ google-chrome-stable
    libxdamage1 \ google-chrome-stable
    libxfixes3 \ google-chrome-stable
    libxkbcommon0 \ google-chrome-stable
    libxml2-dev \ ?
    libxrandr2 \ google-chrome-stable
    xdg-utils \ google-chrome-stable
    xfonts-wqy \ ?

turbo124 · 2024-11-29T09:47:25Z

The fonts are definitely needed to provide unicode support for PDF generation is regions like China.

The others, if they are dependencies of chrome won't hurt to have in place I would have thought.

benbrummer · 2024-12-02T11:02:24Z

@turbo124

Are these apt packages needed?
I assume the *-dev can be ommited and xfonts-wgy is propably required for pdf?

    libonig-dev \
    libpng-dev \
    libxml2-dev \
    xfonts-wqy \

turbo124 · 2024-12-02T11:06:50Z

I'm not sure if any of these are subdependencies of anything

benbrummer · 2024-12-02T11:49:14Z

If I compare the php list https://invoiceninja.github.io/en/self-host-installation/#server-requirements with the modules, which are installed in the image, the intl module is missing.

Additional modules in the image installed with install-php-extensions

```
exif 
```
```
opcache => Needed for caching/jit
```
```
pcntl
```
```
pdo_mysql
```
```
redis => Needed for redis
```
```
saxon-12.5.0
```

@turbo124 can you tell me what are the required ones. E.g pdo_mysql vs mysqli?

Update:
After disabling some of the modules i get

turbo124 · 2024-12-02T19:46:48Z

mysqli is fine for use,.
exif would be to extract metadata from certain filetypes,
gd is also for image processing
pcntl I think is required for Laravel
saxon definitely needed for xslt transformations
redis - for cache access

benbrummer · 2024-12-02T21:34:24Z

mysqli is fine for use,. exif would be to extract metadata from certain filetypes, gd is also for image processing pcntl I think is required for Laravel saxon definitely needed for xslt transformations redis - for cache access

I reduced it for a test to just

RUN ( curl -sSLf https://github.com/mlocati/docker-php-extension-installer/releases/latest/download/install-php-extensions -o - || echo 'return 1' ) | sh -s \
    bcmath \
    # exif \
    gd \
    # gmp \
    # imagick \
    # intl \
    # mysqli \
    # opcache \
    # pcntl \
    pdo_mysql \
    # redis \
    # saxon-${saxon} \
    # soap \
    zip \
    @composer

Everything seems to work. Even without opcache and redis it is quick and responsive. pdo_mysql is required, otherwise the database connection fails, mysqli seems to be unnecessary. All Laravel requirements are installed https://laravel.com/docs/11.x/deployment#server-requirements.

Predis => https://laracasts.com/discuss/channels/laravel/what-is-difference-between-phpredis-and-predis-in-laravel => phpredis is not needed

turbo124 · 2024-12-03T01:41:50Z

predis is more than sufficient here, i agree.

benbrummer · 2024-12-03T10:05:00Z

@turbo124 I'm not able to find a usage for these modules. Do you have an example, where these modules are used in invoiceninja. Can I somehow produce an error somewhere?

Modules, which seem to be unused

exif
gmp
imagick
intl
mysqli
pcntl
saxon-12.5.0
soap

Required/used modules

RUN ( curl -sSLf https://github.com/mlocati/docker-php-extension-installer/releases/latest/download/install-php-extensions -o - || echo 'return 1' ) | sh -s \
    bcmath \
    gd \
    opcache \
    pdo_mysql \
    zip \
    @composer

turbo124 · 2024-12-03T10:50:24Z

Soap needed for vat number verification
Saxon for xslt
Imagick for image compression
Intl also required the exact reason currently escapes me, but I believe around some mb4 support

Pcntl, I believe is needed by Laravel
GMP or bcmath needed
Some form of php MySQL extension required.

benbrummer · 2024-12-03T12:17:30Z

I'm not able to find any evidenz for the packages.

benbrummer · 2024-12-03T12:58:35Z

composer suggest --all

awobaz/compoships suggests:
 - awobaz/blade-active: Blade directives for the Laravel 'Active' package
 - awobaz/eloquent-auto-append: Automatically append accessors to model serialization
 - awobaz/eloquent-mutators: Reusable mutators (getters/setters) for Laravel 5's Eloquent
 - awobaz/syntactic: Syntactic sugar for named and indexed parameters call.

aws/aws-crt-php suggests:
 - ext-awscrt: Make sure you install awscrt native extension to use any of the functionality.

aws/aws-sdk-php suggests:
 - aws/aws-php-sns-message-validator: To validate incoming SNS notifications
 - doctrine/cache: To use the DoctrineCacheAdapter
 - ext-sockets: To use client-side monitoring

barryvdh/reflection-docblock suggests:
 - dflydev/markdown: ~1.0
 - erusev/parsedown: ~1.0

dompdf/dompdf suggests:
 - ext-gmagick: Improves image processing performance

endroid/qr-code suggests:
 - khanamiryan/qrcode-detector-decoder: Enables you to use the image validator
 - roave/security-advisories: Makes sure package versions with known security issues are not installed

fakerphp/faker suggests:
 - doctrine/orm: Required to use Faker\ORM\Doctrine

filp/whoops suggests:
 - whoops/soap: Formats errors as SOAP responses

google/apiclient suggests:
 - cache/filesystem-adapter: For caching certs and tokens (using Google\Client::setCache)

guzzlehttp/psr7 suggests:
 - laminas/laminas-httphandlerrunner: Emit PSR-7 responses

intervention/image suggests:
 - intervention/imagecache: Caching extension for the Intervention Image library

jms/serializer suggests:
 - doctrine/collections: Required if you like to use doctrine collection types as ArrayCollection.
 - symfony/cache: Required if you like to use cache functionality.

larastan/larastan suggests:
 - orchestra/testbench: Using Larastan for analysing a package needs Testbench

laravel/framework suggests:
 - ably/ably-php: Required to use the Ably broadcast driver (^1.0).
 - ext-apcu: Required to use the APC cache driver.
 - ext-ftp: Required to use the Flysystem FTP driver.
 - ext-memcached: Required to use the memcache cache driver.
 - ext-redis: Required to use the Redis cache and queue drivers (^4.0|^5.0|^6.0).
 - league/flysystem-ftp: Required to use the Flysystem FTP driver (^3.25.1).
 - league/flysystem-path-prefixing: Required to use the scoped driver (^3.25.1).
 - league/flysystem-read-only: Required to use read-only disks (^3.25.1)
 - league/flysystem-sftp-v3: Required to use the Flysystem SFTP driver (^3.25.1).
 - pda/pheanstalk: Required to use the beanstalk queue driver (^5.0).
 - resend/resend-php: Required to enable support for the Resend mail transport (^0.10.0).
 - symfony/cache: Required to PSR-6 cache bridge (^7.0).

laravel/scout suggests:
 - algolia/algoliasearch-client-php: Required to use the Algolia engine (^3.2).
 - meilisearch/meilisearch-php: Required to use the Meilisearch engine (^1.0).
 - typesense/typesense-php: Required to use the Typesense engine (^4.9).

league/fractal suggests:
 - pagerfanta/pagerfanta: Pagerfanta Paginator
 - zendframework/zend-paginator: Zend Framework Paginator

maximebf/debugbar suggests:
 - kriswallsmith/assetic: The best way to manage assets

mindee/mindee suggests:
 - ghostscript/ghostscript: Required for PDF rasterization features

mollie/mollie-api-php suggests:
 - mollie/oauth2-mollie-php: Use OAuth to authenticate with the Mollie API. This is needed for some endpoints. Visit https://docs.mollie.com/ for more information.

monolog/monolog suggests:
 - doctrine/couchdb: Allow sending log messages to a CouchDB server
 - ext-amqp: Allow sending log messages to an AMQP server (1.0+ required)
 - ext-mongodb: Allow sending log messages to a MongoDB server (via driver)
 - ext-sockets: Allow sending log messages to a Syslog server (via UDP driver)
 - mongodb/mongodb: Allow sending log messages to a MongoDB server (via library)
 - php-amqplib/php-amqplib: Allow sending log messages to an AMQP server using php-amqplib
 - rollbar/rollbar: Allow sending log messages to Rollbar
 - ruflin/elastica: Allow sending log messages to an Elastic Search server

nelexa/zip suggests:
 - ext-bz2: Needed to support BZIP2 compression

open-telemetry/context suggests:
 - ext-ffi: To allow context switching in Fibers

paragonie/random_compat suggests:
 - ext-libsodium: Provides a modern crypto API that can be used to generate random bytes.

php-http/client-common suggests:
 - php-http/cache-plugin: PSR-6 Cache plugin
 - php-http/logger-plugin: PSR-3 Logger plugin
 - php-http/stopwatch-plugin: Symfony Stopwatch plugin

php-http/message suggests:
 - laminas/laminas-diactoros: Used with Diactoros Factories
 - slim/slim: Used with Slim Framework PSR-7 implementation

phpmyadmin/sql-parser suggests:
 - phpmyadmin/motranslator: Translate messages to your favorite locale

phpoffice/phpspreadsheet suggests:
 - mitoteam/jpgraph: Option for rendering charts, or including charts with PDF or HTML Writers
 - tecnickcom/tcpdf: Option for rendering PDF with PDF Writer

phpseclib/phpseclib suggests:
 - ext-libsodium: SSH2/SFTP can make use of some algorithms provided by the libsodium-php extension.
 - ext-mcrypt: Install the Mcrypt extension in order to speed up a few other cryptographic operations.

phpunit/php-code-coverage suggests:
 - ext-pcov: PHP extension that provides line coverage
 - ext-xdebug: PHP extension that provides line coverage as well as branch and path coverage

predis/predis suggests:
 - ext-relay: Faster connection with in-memory caching (>=0.6.2)

psy/psysh suggests:
 - ext-pdo-sqlite: The doc command requires SQLite to work.

ramsey/uuid suggests:
 - paragonie/random-lib: Provides RandomLib for use with the RandomLibAdapter
 - ramsey/uuid-doctrine: Allows the use of Ramsey\Uuid\Uuid as Doctrine field type.

rmccue/requests suggests:
 - art4/requests-psr18-adapter: For using Requests as a PSR-18 HTTP Client

socialiteproviders/apple suggests:
 - ahilmurugesan/socialite-apple-helper: Automatic Apple client key generation and management.

spatie/error-solutions suggests:
 - openai-php/client: Require get solutions from OpenAI
 - simple-cache-implementation: To cache solutions from OpenAI

spatie/ignition suggests:
 - openai-php/client: Require get solutions from OpenAI
 - simple-cache-implementation: To cache solutions from OpenAI

spatie/laravel-ignition suggests:
 - openai-php/client: Require get solutions from OpenAI

spaze/phpstan-stripe suggests:
 - phpstan/extension-installer: Allows automatic requirement of extension.neon

sprain/swiss-qr-bill suggests:
 - fpdf/fpdf: Needed to create pdfs with FpdfOutput
 - tecnickcom/tcpdf: Needed to create pdfs with TcPdfOutput

Findings:

imagick:
There is one interesting suggestion for dompdf => gmagick, but from the docs, imagick is supported, too

 dompdf/dompdf suggests:
 - ext-gmagick: Improves image processing performance

soup:
only mentioned here, but not the ext-soap

filp/whoops suggests:
 - whoops/soap: Formats errors as SOAP responses

no match for exif, gmp, imagick, intl, mysqli, pcntl, saxon

I think the dependency confusion, discrepancy of docs and composer feels like an issue, which should be handled in https://github.com/invoiceninja/invoiceninja.

turbo124 · 2024-12-03T16:13:24Z

whilst there may be no composer related dependencies, the app directly engages the ones I have mentioned above.

benbrummer · 2024-12-03T18:06:37Z

whilst there may be no composer related dependencies, the app directly engages the ones I have mentioned above.

So than they should be listed here? https://github.com/invoiceninja/invoiceninja/blob/7b02668d4bd1de6462717a98a88e6ebea706eaa5/composer.json#L33

turbo124 · 2024-12-03T18:16:12Z

Not necessarily. The app can certainly work without some of these. We do not enforce all possible extensions.

The app is installed on a wide range of services including shared hosting which can present challenges for meeting some dependencies.

In environments where we can control the underlying extensions, then it makes sense to use them.

benbrummer · 2024-12-03T22:12:55Z

Understand :-)

Is there a documentation for all the possible extensions and their purpuose?

https://invoiceninja.github.io/en/self-host-installation/#server-requirements should list pdo_mysql, database connection does not work (But it works without mysqli). Maybe a list of mandatory modules (or a link to the require section in composer.json) can be added, and a list of optional modules.

pcntl:
seems to be not required by laravel: https://laravel.com/docs/11.x/deployment#server-requirements

turbo124 · 2024-12-03T22:19:45Z

"suggest": {
        "ext-pcntl": "Required for process control (CLI only)",
        "ext-gd": "Required for image processing",
        "ext-mbstring": "Required for multibyte string handling"
    }

We can certainly add the list of non-required extensions into a suggest block.

In regards to pdo_mysql, I believe this is automatically included (at least when using apt) when running apt-get install php-mysql

pnctl is part of PHP core, not really an extension, my guess here is upstream they check for this flag and replace with a binary that has pnctl enabled or disabled.

given pnctl's role in process control, and laravel does use SIG* commands for process controls, I would assume this is very much needed.

benbrummer · 2024-12-04T09:07:19Z

@turbo124 I tried to to my best and combined all Information (composer.json, laravel docs, invoiceninja docs, the base dockerimages 8.2-8.4-fpm)

"require": {
    // php 8.4 works with some notices like PHP Deprecated:  App\Models\CompanyGateway::driver(): Implicitly marking parameter $client as nullable is deprecated, the explicit nullable type must be used instead in /var/www/html/app/Models/CompanyGateway.php on line 206
    "php": ">=8.2",
    // https://github.com/invoiceninja/dockerfiles/issues/660#issuecomment-2511321698
    "ext-gd": "*, insall needed",
    "ext-curl": "*",    
    "ext-zip": "*, install needed", 
    "ext-openssl": "*",
    "ext-mbstring": "*",
    "ext-dom": "*",
    "ext-xml": "*",
    "ext-bcmath": "*, install needed",
    "ext-iconv": "*",
    // Additional mandatory module
    "pdo_mysql": "*, Connection.php... could not find driver..., install needed"
    }

"suggest": {
    "ext-simplexml": "*, ?, was listed in 'require'",
    // All of these are not preinstalled in php:8.2/8.3-fpm
    "ext-exif": "*, ",
    "ext-gmp": "*, ",
    "ext-imagick": "*, suggest for DomPDF suggested",
    "ext-intl": "*, Was missing in previous invoiceninja-debian images, twig/intl requires symfony/intl, but none of them require/suggest ext_intl",
    "ext-mysqli": "*, ?",
    "ext-opcache": "*, caching",
    "ext-pcntl": "*, Required for process control (CLI only), php needs to be compiled for it. Not listed in requirements for Laravel https://laravel.com/docs/11.x/deployment#server-requirements ",
    "ext-saxonc": ">=12.5.0, xslt processing during",
    "ext-soap": "*, VAT validation"        
    }

Additional modules listed in documentation https://invoiceninja.github.io/en/self-host-installation/#server-requirements, which are not already listed above

"suggest?": {
    "ctype": "*, ?",
    "tokenizer": "*, ?",
    "gmp": "*, ?, install needed",
}

php-common will be installed anyway, when any php-* package is installed, and it is no php modul itself, so I think it is not needed to list in the docs.

turbo124 · 2024-12-04T20:36:52Z

removed gmp requirement as we enforce bcmath.
removed mysqli ? as mysql, mysqlnd, mysqli are viable options?
removed opcache as it requires additional configuration outside the scope of the application

My Suggestion

"suggest": {
    "ext-exif": "*, Advanced image meta data collection",
    "ext-imagick": "*, Provides image compression, if available",
    "ext-intl": "*, Required for full intl support in twig",
    "ext-pcntl": "*, Improves parallel performance and provides signal handling for CLI commands",
    "ext-saxonc": ">=12.5.0, xslt processing and validation of xml documents",
    "ext-soap": "*, VAT validation"        
    }

benbrummer · 2024-12-05T10:56:17Z

@turbo124 I integrated yourt suggestion 806a340#diff-b98298742c5536fa90710aea60f865cdf0149b61c06288c4a0219ce4796ec807R3

Apart of a multistage implementation everything is implemented from this issue in #664. Multistage can be handled, if needed in a different issue.

intl seems to be unnecessary

benbrummer · 2024-12-08T07:10:39Z

Multistage is currently out of scope for this issue

benbrummer changed the title ~~Dockerfile optimizationproposal~~ Dockerfile optimization proposal Nov 27, 2024

benbrummer mentioned this issue Nov 29, 2024

dockerfile cleanup, apt reduction, php module configuration, allow to run custom CMD #664

Merged

benbrummer closed this as completed Dec 8, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dockerfile optimization proposal #660

Dockerfile optimization proposal #660

benbrummer commented Nov 27, 2024 •

edited

Loading

turbo124 commented Nov 28, 2024

turbo124 commented Nov 29, 2024

benbrummer commented Nov 29, 2024 •

edited

Loading

turbo124 commented Nov 29, 2024

benbrummer commented Dec 2, 2024

turbo124 commented Dec 2, 2024

benbrummer commented Dec 2, 2024 •

edited

Loading

turbo124 commented Dec 2, 2024

benbrummer commented Dec 2, 2024 •

edited

Loading

turbo124 commented Dec 3, 2024

benbrummer commented Dec 3, 2024 •

edited

Loading

turbo124 commented Dec 3, 2024

benbrummer commented Dec 3, 2024 •

edited

Loading

benbrummer commented Dec 3, 2024 •

edited

Loading

turbo124 commented Dec 3, 2024

benbrummer commented Dec 3, 2024

turbo124 commented Dec 3, 2024

benbrummer commented Dec 3, 2024

turbo124 commented Dec 3, 2024

benbrummer commented Dec 4, 2024 •

edited

Loading

turbo124 commented Dec 4, 2024

benbrummer commented Dec 5, 2024 •

edited

Loading

benbrummer commented Dec 8, 2024

Dockerfile optimization proposal #660

Dockerfile optimization proposal #660

Comments

benbrummer commented Nov 27, 2024 • edited Loading

Ideas

apt packages

php modules

Misc

turbo124 commented Nov 28, 2024

turbo124 commented Nov 29, 2024

benbrummer commented Nov 29, 2024 • edited Loading

turbo124 commented Nov 29, 2024

benbrummer commented Dec 2, 2024

turbo124 commented Dec 2, 2024

benbrummer commented Dec 2, 2024 • edited Loading

turbo124 commented Dec 2, 2024

benbrummer commented Dec 2, 2024 • edited Loading

turbo124 commented Dec 3, 2024

benbrummer commented Dec 3, 2024 • edited Loading

Modules, which seem to be unused

Required/used modules

turbo124 commented Dec 3, 2024

benbrummer commented Dec 3, 2024 • edited Loading

benbrummer commented Dec 3, 2024 • edited Loading

Findings:

turbo124 commented Dec 3, 2024

benbrummer commented Dec 3, 2024

turbo124 commented Dec 3, 2024

benbrummer commented Dec 3, 2024

turbo124 commented Dec 3, 2024

benbrummer commented Dec 4, 2024 • edited Loading

turbo124 commented Dec 4, 2024

benbrummer commented Dec 5, 2024 • edited Loading

benbrummer commented Dec 8, 2024

benbrummer commented Nov 27, 2024 •

edited

Loading

benbrummer commented Nov 29, 2024 •

edited

Loading

benbrummer commented Dec 2, 2024 •

edited

Loading

benbrummer commented Dec 2, 2024 •

edited

Loading

benbrummer commented Dec 3, 2024 •

edited

Loading

benbrummer commented Dec 3, 2024 •

edited

Loading

benbrummer commented Dec 3, 2024 •

edited

Loading

benbrummer commented Dec 4, 2024 •

edited

Loading

benbrummer commented Dec 5, 2024 •

edited

Loading