Override scripts for custom ckan images fail: sudo not installed

[themowski]

Overview

The ckan/setup/start_ckan.sh.override file uses sudo to impersonate the ckan user. However, sudo is not available in the Alpine-based container images. As a result, if you follow the instructions in step 5 of the top-level README.md and install the override script to create a custom ckan image, the script fails when a container is started with that custom image (see the "Demonstration" section).

I did not try to build the development image, but looking at the corresponding override file, this error also affects that one.

As a note, it appears that the start_ckan.sh.override script is a copy of ckan-2.10/base/setup/start_ckan.sh in the ckan/ckan-docker-base repo. (Note, however, that the ckan/ckan-docker-base version of this script does not use sudo, so presumably this override file got out of sync with upstream.) Similarly, the start_ckan_development.sh.override file seems to be a copy of ckan-2.10/dev/setup/start_ckan_development.sh in the ckan/ckan-docker-base repo.

It would probably be worth adding comments to the override files or to the README stating this, so that people who encounter issues with these in the future can understand how they relates to the base images.

Demonstration

I cloned the current master (commit 6bbc482) and made this change to ckan/Dockerfile:

$ git diff
diff --git a/ckan/Dockerfile b/ckan/Dockerfile
index 75001e9..236f3b0 100644
--- a/ckan/Dockerfile
+++ b/ckan/Dockerfile
@@ -10,3 +10,5 @@ RUN echo ${TZ} > /etc/timezone
 RUN if ! [ /usr/share/zoneinfo/${TZ} -ef /etc/localtime ]; then \
         cp /usr/share/zoneinfo/${TZ} /etc/localtime ;\
     fi ;
+
+COPY setup/start_ckan.sh.override ${APP_DIR}/start_ckan.sh

Then, I ran docker compose build and docker compose up. The ckan container eventually died. Here are its logs, minus some extremely lengthy traceback that I believe occurs because the invocation of sudo python3 prerun.py fails due to sudo:

$ docker logs ckan
beaker.session.secret =
Setting beaker.session.secret in ini file
/srv/app/start_ckan.sh: line 19: sudo: not found
Set up ckan.datapusher.api_token in the CKAN config file
2023-09-05 17:36:00,458 INFO  [ckan.cli] Using configuration file /srv/app/ckan.ini
2023-09-05 17:36:00,459 INFO  [ckan.config.environment] Loading static files from public
2023-09-05 17:36:00,680 INFO  [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
2023-09-05 17:36:01,115 WARNI [ckanext.reclineview.plugin] The Recline-based views are deprecated andwill be removed in future versions
2023-09-05 17:36:01,137 INFO  [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
2023-09-05 17:36:02,293 WARNI [ckan.config.middleware.flask_app] Extensions are excluded from CSRF protection! We allow extensions to run without CSRF protection but it will be forced future releases. Read the documentation for more information on how to add CSRF protection to your extension.
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/usr/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.UndefinedTable: relation "user" does not exist
LINE 2: FROM "user"
             ^
(... omitting lengthy traceback related to this ...)

/srv/app/start_ckan.sh: Ignoring /docker-entrypoint.d/* (not an sh or py file)

/srv/app/start_ckan.sh: line 54: sudo: not found

Suggested Fixes

For the start_ckan.sh.override file, remove the instances of sudo -u ckan -EH from the file. It's worth pointing out that this results in the application running as root, which might not be ideal.

It's also worth noting that because the script does not run with the -e flag, the first sudo call in the current script does not cause the container to die immediately. I verified that changing the shebang line to #!/bin/sh -e causes the script to fail as soon as that first sudo is hit, but I don't know if that's really desirable; are some failures OK or expected? (This is really an upstream ckan/ckan-docker-base question; if there is interest, I can log an issue there as well, if needed.)

For start_ckan_development.sh.override, the approach is less clear, since I haven't actually tested it. Removing sudo -u ckan -EH probably needs to happen. However, the ckan/ckan-docker-base version of the script has an extra su command on the last line that appears to be used to run CKAN as the ckan user.

Maybe using su ckan -c ... is the best path forward for both override scripts? I don't know enough about how CKAN runs / expects to run to say for sure, but it is worth noting that this difference exists.

[kowh-ai]

Thanks @themowski - yes when you say "(Note, however, that the ckan/ckan-docker-base version of this script does not use sudo, so presumably this override file got out of sync with upstream.)" thats exactly what has happened. I'll sort out a way to make this more robust. Thanks for your help. Oh and using the -e option is probably better. No reason for not using it

[iameskild]

@kowh-ai would a possible fix for this be to simply install sudo? Perhaps including apk add sudo at the top of the start_ckan.sh script? If that's the case, I would be happy to open a PR for this.

[kowh-ai]

sudo was removed about the same time last year commit: a7f14ddadd5d850dd7a8bd65666a87c2332e9cd0

[iameskild]

Thanks for the update @kowh-ai! I think the start_ckan.sh.override (in ckan-docker) will need to be updated to match.

ckan-docker/ckan/setup/start_ckan.sh.override

Line 19 in b70bd39

sudo -u ckan -EH python3 prerun.py

It wasn't clear to me what the process for deploying ckan using docker was, so we used the ckan/ckan-docker repo as a template to start from. If there's a better approach, we'd be glad to hear it.

Otherwise, I'm happy to open a PR in the ckan-docker repo to update both start_ckan.sh.override and start_ckan_development.sh.override to match their respective scripts in ckan-docker-base.

Thanks again for all your work on CKAN :)

EDIT: I see you have a PR opened already for this: #150