undefined: ibmmq.MQCNO after upgrading to golang 1.21.1

[Sparkz0629]

Hi.

We are busy with a golang upgrade from 1.15 to 1.21.1 across our microservices

One of the services uses the mq-golang-jms20 module.

When attempting to build the code in the new env, we get the following:

# github.com/ibm-messaging/mq-golang-jms20/jms20subset
/go/pkg/mod/github.com/ibm-messaging/[email protected]/jms20subset/MQOptions.go:5:32: undefined: ibmmq.MQCNO
/go/pkg/mod/github.com/ibm-messaging/[email protected]/jms20subset/MQOptions.go:8:25: undefined: ibmmq.MQCNO

this is our build stage:

build:
  stage: build
  only:
    refs:
    - master
  image: ${BUILD_IMAGE}
  script:
    - mkdir -p /opt/mqm
    - chmod a+rx /opt/mqm
    - export genmqpkg_incnls=1
    - export genmqpkg_incsdk=1
    - export genmqpkg_inctls=1
    - cp 9.2.4.0-IBM-MQC-Redist-LinuxX64.tar.gz /opt/mqm/
    - tar -zxf /opt/mqm/*.tar.gz
    - rm -f /opt/mqm/*.tar.gz
    - export LD_LIBRARY_PATH=.:/opt/mqm/lib64:/opt/mqm/java/lib64:$LIBPATH
    - export PATH=.:/opt/mqm/bin:$PATH
    - bin/genmqpkg.sh -b /opt/mqm
    - img build -t ${DOCKER_PUSH_URL}/mq-transfer-server:2.0 --build-arg GO_IMAGE=${GO_121_IMAGE}  .
    - echo "build done"
    - echo ${ENCODED_PASSWORD} | base64 -d | img login --username ${USER} --password-stdin ${DOCKER_PUSH_URL}
    - img push ${DOCKER_PUSH_URL}/mq-transfer-server:2.0
    - img logout ${DOCKER_PUSH_URL}

where GO_121_IMAGE is an image built from the below with our certificates added.

FROM golang:1.21.1-bookworm
ENV GOOS=linux 
ENV GOARCH=amd64 
ENV CGO_ENABLED=0 

BUILD_IMAGE is FROM debian:12.1-slim with our certificates added.

Looking for any advised or assistance on the above... out of 15 microservices, this is the only one that we are stuck on the upgrade with. (Also the only one to use to mq libraries)

[matrober-uk]

Hi @Sparkz0629 - thanks for getting in touch.

I've updated the "openshift-app-sample" directory (which uses docker/podman to build the image in a similar way that you're using above) to use Golang 1.21.1 using this commit and it ran successfully on my Ubuntu VM.

So I suggest looking to see what the differences are between your build and the openshift-app-sample Dockerfile.

Some particular areas to investigate are:

• Update your MQ client library to the latest 9.4.0.0-IBM-MQC-Redist-LinuxX64.tar.gz (it's backwards compatible with other queue manager versions)
• Check your go.mod (+go.sum) files to ensure they point at the correct 1.11.0 version of mq-golang-jms20 and the related v5.6.0 version of mq-golang respectively
• there is a similar error in this recent Stackoverflow question where in that case the issue looks to be a mismatch between different platform types, so that's worth checking into as well

Output from my successful execution of the openshift-app-sample/Dockerfile

matrober@matrober21:~/go/src/github.com/ibm-messaging/mq-golang-jms20/openshift-app-sample$ sudo docker build -t golang-app -f Dockerfile .
[+] Building 51.8s (16/16) FINISHED                                                                                            docker:default
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 2B                                                                                                          0.0s
 => [internal] load build definition from Dockerfile                                                                                     0.1s
 => => transferring dockerfile: 1.36kB                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/golang:1.21.1                                                                         0.5s
 => [internal] load build context                                                                                                        0.0s
 => => transferring context: 366B                                                                                                        0.0s
 => [builder 1/7] FROM docker.io/library/golang:1.21.1@sha256:19600fdcae402165dcdab18cb9649540bde6be7274dedb5d205b2f84029fe909           0.0s
 => CACHED [builder 2/7] RUN mkdir -p /opt/mqm    && chmod a+rx /opt/mqm                                                                 0.0s
 => [builder 3/7] RUN cd /opt/mqm    && curl -LO "https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redis  13.2s
 => [builder 4/7] RUN mkdir -p /go/src/openshift-app-sample                                                                              0.5s
 => [builder 5/7] WORKDIR /go/src/openshift-app-sample                                                                                   0.2s 
 => [builder 6/7] COPY src/ .                                                                                                            0.1s 
 => [builder 7/7] RUN go build -o openshift-app-sample                                                                                  31.1s 
 => CACHED [stage-1 2/5] RUN mkdir -p /go/src/openshift-app-sample    && mkdir -p /IBM/MQ/data/errors    && mkdir -p /.mqm    && chmod   0.0s 
 => CACHED [stage-1 3/5] WORKDIR /go/src/openshift-app-sample                                                                            0.0s 
 => [stage-1 4/5] COPY --chown=0:0 --from=builder /go/src/openshift-app-sample/openshift-app-sample /go/src/openshift-app-sample         0.1s
 => [stage-1 5/5] COPY --chown=0:0 --from=builder /opt/mqm /opt/mqm                                                                      0.6s
 => exporting to image                                                                                                                   4.2s
 => => exporting layers                                                                                                                  4.2s
 => => writing image sha256:1d01a8d00a9cb4a499e5fd648137df25c00177f3b1096472fbd51aac13fe7657                                             0.0s
 => => naming to docker.io/library/golang-app

# Run the built docker file - fails to connect to the queue manager because I didn't set up the target host details (expected)
matrober@matrober21:~/go/src/github.com/ibm-messaging/mq-golang-jms20/openshift-app-sample$ sudo docker run 1d01a8d00a9cb4a499e5fd648137df25c00177f3b1096472fbd51aac13fe7657
Beginning world!!!
2024/07/06 13:36:47 {errorCode=2277, reason=MQRC_CD_ERROR, linkedErr=MQCONNX: MQCC = MQCC_FAILED [2] MQRC = MQRC_CD_ERROR [2277]}

[Sparkz0629]

Thanks @matrober-uk for the quick response.
Ive updated to the latest Redist file, and also conformed that the correct versions of mq-golang-jms20 and the related mq-golang or where we want to see them.

Still combing through to see if there are other differences.

[Sparkz0629]

• there is a similar error in this recent Stackoverflow question where in that case the issue looks to be a mismatch between different platform types, so that's worth checking into as well

This is running on a gitlab runner based on gitlab/gitlab-runner:alpine-v14.9.0.
To the best of my knowledge, this should mean the IBM-MQC-Redist-LinuxX64.tar.gz should be fine.

Also, something to note, the genmqpkg.sh runs fine:

$ bin/genmqpkg.sh -b /opt/mqm
Generate MQ Runtime Package
---------------------------
This program will help determine a minimal set of runtime files that are
required for a queue manager installation or to be distributed with a
client application. The program will ask a series of questions and then
prompt for a filesystem location for the runtime files.
Note that IBM can only provide support assistance for an unmodified set
of runtime files.
The MQ runtime package will be created in
/opt/mqm
Generation complete !
MQ runtime package copied to '/opt/mqm'

The failure occurs during the build stage:

$ go build -o ${NAME} ./cmd/mq-transfer-server
go: downloading github.com/minio/minio-go/v7 v7.0.73
go: downloading github.com/gorilla/mux v1.8.1
go: downloading github.com/prometheus/client_golang v1.19.1
go: downloading github.com/segmentio/kafka-go v0.4.47
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading github.com/google/uuid v1.6.0
go: downloading github.com/ibm-messaging/mq-golang-jms20 v1.11.0
go: downloading github.com/ibm-messaging/mq-golang/v5 v5.6.0
go: downloading github.com/prometheus/common v0.48.0
go: downloading github.com/klauspost/compress v1.17.9
go: downloading github.com/go-ini/ini v1.67.0
go: downloading github.com/prometheus/client_model v0.5.0
go: downloading github.com/goccy/go-json v0.10.3
go: downloading google.golang.org/protobuf v1.33.0
go: downloading github.com/cespare/xxhash/v2 v2.2.0
go: downloading golang.org/x/net v0.26.0
go: downloading github.com/beorn7/perks v1.0.1
go: downloading golang.org/x/crypto v0.24.0
go: downloading github.com/prometheus/procfs v0.12.0
go: downloading github.com/minio/md5-simd v1.1.2
go: downloading github.com/rs/xid v1.5.0
go: downloading github.com/klauspost/cpuid/v2 v2.2.8
go: downloading github.com/pierrec/lz4/v4 v4.1.15
go: downloading golang.org/x/sys v0.21.0
go: downloading golang.org/x/text v0.16.0
# github.com/ibm-messaging/mq-golang-jms20/jms20subset
/go/pkg/mod/github.com/ibm-messaging/[email protected]/jms20subset/MQOptions.go:5:32: undefined: ibmmq.MQCNO
/go/pkg/mod/github.com/ibm-messaging/[email protected]/jms20subset/MQOptions.go:8:25: undefined: ibmmq.MQCNO

[ibmmqmet]

You have set ENV CGO_ENABLED=0 which disables the C compilation phase. If I add that to Matt's Dockerfile, then it fails in the same way. Either remove that line, or set the value to 1 which might be clearer as to intention.

[Sparkz0629]

You have set ENV CGO_ENABLED=0 which disables the C compilation phase. If I add that to Matt's Dockerfile, then it fails in the same way. Either remove that line, or set the value to 1 which might be clearer as to intention.

Thanks for this.

So i updated this and the build succeeds, the issue we have then is that the code doesnt run because of the below, which was the actual reason we added that flag:

/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /usr/bin/kaas-mq-transfer-server)
/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/bin/kaas-mq-transfer-server)

[ibmmqmet]

That is telling you that the container you are running on does not have an appropriate level of glibc installed.

The MQ 9.4 client library is built on RHEL8, so whatever glibc is in there will be the minimum for any other distro-based container. And that excludes anything using the musl libc typically used in alpine containers, and which is completely incompatible.

[Sparkz0629]

So that happens even when running with the golang:1.21.1 runner image

Our cluster has RHEL 8.9 boxes as the underlying servers.
The gitlab runner is alpine based
and the runner images are debian based.

We had this issue in the past, and the only way we got around it was by setting the CGO_ENABLED to 0 on the build stage.

What is weird, is that using the exact same process for version golang 1.15 and 9.2.4.0 of the MQ Distro, all works 100% (Without having to set CGO_ENABLED). That runner image was debian:11.3-slim. As soon as we attempt to run on a later image like debian:12.1-slim we get the above.

[Sparkz0629]

I decided to go back to the drawing board and attempt this step by step using the sample app initially supplied by Matt.

The code build works now.

Our image build and push step runs the following dockerfile:

FROM golang:1.21.1

COPY kaas-mq-transfer-server /usr/bin/kaas-mq-transfer-server

RUN mkdir -p /opt/mqm \
    && chmod a+rx /opt/mqm

ENV genmqpkg_incnls=1 \
    genmqpkg_incsdk=1 \
    genmqpkg_inctls=1 

COPY 9.4.0.0-IBM-MQC-Redist-LinuxX64.tar.gz /opt/mqm/

RUN cd /opt/mqm \
    && tar -zxf ./*.tar.gz \
    && rm -f ./*.tar.gz \
    && bin/genmqpkg.sh -b /opt/mqm

ENTRYPOINT [ "/usr/bin/kaas-mq-transfer-server" ]

I can also run this manually by spinning up the local image and running the commands one by one.
As soon as it tries to run in the pipeline, i get the following on the genmqpkg.sh step:

Building kaas-mq-transfer-server:v0.0.19
Setting up the rootfs... this may take a bit.
time="2024-07-11T09:56:36Z" level=warning msg="using host network as the default"
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 751B done
#2 DONE 0.0s
#1 [internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.0s
#3 [internal] load metadata for golang:1.21.1
#3 DONE 0.2s
#4 [1/5] FROM golang:1.21.1@sha256:19600fdcae...
#4 resolve golang:1.21.1@sha256:19600fdcae402165dcdab18cb9649540bde6be7274dedb5d205b2f84029fe909 done
#4 sha256:92b30a24413a45c9744b79bafa4c7717eafff586a9332210abbd384f77785d8c 155B / 155B 0.0s done
#4 sha256:2159148dcc0[81](kaas-mq-transfer-server/-/jobs/2309432#L81)245165b2aa99fc5a94ca9818bece66839d8eb11c9335ae9e688 7.22kB / 7.22kB done
#4 sha256:19600fdcae402165dcdab18cb9649540bde6be7274dedb5d205b2f84029fe909 2.36kB / 2.36kB done
#4 sha256:b17c35044f4062d83c815434615997eed97697daae8745c6dd39dc3673b87efb 1.58kB / 1.58kB done
#4 sha256:167b8a53ca4504bc6aa31[82](kaas-mq-transfer-server/-/jobs/2309432#L82)e336fa96f4ef76875d158c1933d3e2fa19c57e0c3 13.47MB / 49.56MB 0.1s
#4 sha256:b0ed6cc9b50977796e8eb9b270ad9c62922003c0090aa3e5ec26a165cfcb9c02 9.79MB / 66.99MB 0.1s
#4 sha256:debce5f9f3a9709885f7f2ad3cf41f036a3b57b406b27ba3a8[83](kaas-mq-transfer-server/-/jobs/2309432#L83)928315787042 9.06MB / 64.11MB 0.1s
#4 sha256:b47a222d28fa95680198398973d0a29b82a968f03e7ef361cc8ded562e4d[84](kaas-mq-transfer-server/-/jobs/2309432#L84)a3 11.79MB / 24.03MB 0.1s
#4 sha256:91b457aaf04f424db4f223ea7aad4b196d4a62da58d6f45938233e0f54bd162c 12.82MB / 92.30MB 0.1s
#4 sha256:167b8a53ca4504bc6aa3182e336fa96f4ef76875d158c1933d3e2fa19c57e0c3 34.24MB / 49.56MB 0.3s
#4 sha256:b0ed6cc9b50977796e8eb9b270ad9c62922003c0090aa3e5ec26a165cfcb9c02 22.07MB / 66.99MB 0.3s
#4 sha256:debce5f9f3a97098[85](kaas-mq-transfer-server/-/jobs/2309432#L85)f7f2ad3cf41f036a3b57b406b27ba3a883928315787042 21.36MB / 64.11MB 0.3s
#4 sha256:b47a222d28fa95680198398973d0a29b82a968f03e7ef361cc8ded562e4d84a3 24.03MB / 24.03MB 0.2s done
#4 sha256:91b457aaf04f424db4f223ea7aad4b196d4a62da58d6f45938233e0f54bd162c 33.19MB / 92.30MB 0.3s
#4 sha256:167b8a53ca4504bc6aa3182e336fa96f4ef76[87](kaas-mq-transfer-server/-/jobs/2309432#L87)5d158c1933d3e2fa19c57e0c3 49.56MB / 49.56MB 0.4s done
#4 sha256:b0ed6cc9b50977796e8eb9b270ad9c62922003c0090aa3e5ec26a165cfcb9c02 34.05MB / 66.99MB 0.4s
#4 sha256:debce5f9f3a9709[88](kaas-mq-transfer-server/-/jobs/2309432#L88)5f7f2ad3cf41f036a3b57b406b27ba3a883928315787042 33.33MB / 64.11MB 0.4s
#4 sha256:91b457aaf04f424db4f223ea7aad4b196d4a62da58d6f45938233e0f54bd162c 54.64MB / 92.30MB 0.4s
#4 sha256:b0ed6cc9b50977796e8eb9b270ad9c62922003c0090aa3e5ec26a165cfcb9c02 46.57MB / 66.99MB 0.6s
#4 sha256:debce5f9f3a9709885f7f2ad3cf41f036a3b57b406b27ba3a883928315787042 45.84MB / 64.11MB 0.6s
#4 sha256:91b457aaf04f424db4f223ea7aad4b196d4a62da58d6f45938233e0f54bd162c 73.04MB / 92.30MB 0.6s
#4 ...
#5 [internal] load build context
#5 transferring context: 102.54MB 0.8s done
#5 DONE 0.8s
#4 [1/5] FROM golang:1.21.1@sha256:19600fdcae...
#4 sha256:b0ed6cc9b50977796e8eb9b270ad9c62922003c0090aa3e5ec26a165cfcb9c02 58.31MB / 66.99MB 0.7s
#4 sha256:debce5f9f3a9709885f7f2ad3cf41f036a3b57b406b27ba3a883928315787042 57.07MB / 64.11MB 0.7s
#4 sha256:91b457aaf04f424db4f223ea7aad4b196d4a62da58d6f45938233e0f54bd162c [89](kaas-mq-transfer-server/-/jobs/2309432#L89).21MB / 92.30MB 0.7s
#4 sha256:b0ed6cc9b50977796e8eb9b270ad9c62922003c0090aa3e5ec26a165cfcb9c02 66.99MB / 66.99MB 1.0s done
#4 sha256:debce5f9f3a9709885f7f2ad3cf41f036a3b57b406b27ba3a883928315787042 64.11MB / 64.11MB 1.0s done
#4 sha256:91b457aaf04f424db4f223ea7aad4b196d4a62da58d6f45938233e0f54bd162c 92.30MB / 92.30MB 0.9s done
#4 unpacking golang:1.21.1@sha256:19600fdcae402165dcdab18cb9649540bde6be7274dedb5d205b2f84029fe[90](kaas-mq-transfer-server/-/jobs/2309432#L90)9
#4 unpacking golang:1.21.1@sha256:19600fdcae402165dcdab18cb9649540bde6be7274dedb5d205b2f84029fe909 25.7s done
#4 DONE 26.7s
#6 [2/5] COPY kaas-mq-transfer-server /usr/bin/kaas-mq-transfer-server
#6 DONE 5.6s
#7 [3/5] RUN mkdir -p /opt/mqm     && chmod a+rx /opt/mqm
#7 DONE 6.4s
#8 [4/5] COPY 9.4.0.0-IBM-MQC-Redist-LinuxX64.tar.gz /opt/mqm/
#8 DONE 6.1s
#9 [5/5] RUN cd /opt/mqm     && tar -zxf ./*.tar.gz     && rm -f ./*.tar.gz...
#9 9.253 
#9 9.253 Generate MQ Runtime Package
#9 9.253 ---------------------------
#9 9.253 This program will help determine a minimal set of runtime files that are
#9 9.253 required for a queue manager installation or to be distributed with a
#9 9.253 client application. The program will ask a series of questions and then
#9 9.253 prompt for a filesystem location for the runtime files.
#9 9.253 
#9 9.253 Note that IBM can only provide support assistance for an unmodified set
#9 9.253 of runtime files.
#9 9.253 
#9 9.2[95](kaas-mq-transfer-server/-/jobs/2309432#L95) 
#9 9.295 The MQ runtime package will be created in
#9 9.295 
#9 9.295 /opt/mqm
#9 9.295 
#9 9.295 The package MANIFEST does not exist; no files can be removed from the MQ runtime package.
#9 ERROR: executor failed running [/bin/sh -c cd /opt/mqm     && tar -zxf ./*.tar.gz     && rm -f ./*.tar.gz     && bin/genmqpkg.sh -b /opt/mqm]: runc did not terminate successfully
------
 > [5/5] RUN cd /opt/mqm     && tar -zxf ./*.tar.gz     && rm -f ./*.tar.gz     && bin/genmqpkg.sh -b /opt/mqm:
------

Ive edited the dockerfile and confirmed that the tar is getting extracted as expected and the file is present.

So atleast making progress as the initial build works, but now the image build and push doesnt.