cat issue with untagged text

[gngrossi]

09:01:47 RC=(0) [SYSA] bash-5.2$ which cat
/hewitt/zopentools/zopen_repo/usr/local/bin/cat

09:01:50 RC=(0) [SYSA] bash-5.2$ ls -T /SYSE/etc/sshd.pid

• untagged T=off /SYSE/etc/sshd.pid

09:01:53 RC=(0) [SYSA] bash-5.2$ cat /SYSE/etc/sshd.pid
50397225

09:01:59 RC=(0) [SYSA] bash-5.2$ ls -T /SYSF/etc/sshd.pid

• untagged T=off /SYSF/etc/sshd.pid

09:02:09 RC=(0) [SYSA] bash-5.2$ cat /SYSF/etc/sshd.pid
▒▒▒▒▒▒09:02:18 RC=(0) [SYSA] bash-5.2$

--

09:03:46 RC=(0) [SYSA] bash-5.2$ file /SYSE/etc/sshd.pid
/SYSE/etc/sshd.pid: text

09:03:59 RC=(0) [SYSA] bash-5.2$ file /SYSF/etc/sshd.pid
/SYSF/etc/sshd.pid: text

[IgorTodorovskiIBM]

@HarithaIBM I think this is related to the issue you were investigating (zoslib macros were not being passed in the builds).

[HarithaIBM]

@HarithaIBM I think this is related to the issue you were investigating (zoslib macros were not being passed in the builds).
Oh Ok! Let me check.

[HarithaIBM]

I ran zopen upgrade coreutils -r and tried cat on untagged header file env.h in /usr/include. It worked fine.

[HarithaIBM]

[gngrossi]

Never used upgrade with -r before. Hung up after the Download complete message.

bash-5.2$ zopen upgrade coreutils -r

• Querying repo for latest package information.
  Processing package: coreutils
• Reinstalling version '9.4.20231116_212509' of coreutils...
• Replacing coreutils version '9.4.20231116_212509' with '9.4.20231116_212509'
  % Total % Received % Xferd Average Speed Time Time Time Current
  Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
  100 2234 100 2234 0 0 10051 0 --:--:-- --:--:-- --:--:-- 10051
• Downloading coreutils-9.4.20231116_212509.zos.pax.Z file from remote to zopen package cache...
  -->--
• Processing coreutils-9.4.20231116_212509.zos.pax.Z...
  After this operation, 0.07 MB of additional disk space will be used.
• Download complete.

--

Had to Ctrl-c, then:
zopen remove coreutils
zopen install coreutils

--

bash-5.2$ which cat
/hewitt/zopentools/zopen_repo/usr/local/bin/cat

bash-5.2$ ls -T /SYSF/etc/sshd.pid

• untagged T=off /SYSF/etc/sshd.pid

bash-5.2$ cat /SYSF/etc/sshd.pid
▒▒▒▒▒▒07:56:57 RC=(0) [SYSA] bash-5.2$

bash-5.2$ /bin/cat /SYSF/etc/sshd.pid
131117

[HarithaIBM]

Ok, so looks like issue is with heuristic. We will fix it. But, if you know the file is in EBCDIC then you can set export __UNTAGGED_READ_MODE=ASCII and then run cat.

[gngrossi]

When OpenSSH starts, it creates the file containing the PID. It's untagged and in EBCDIC.
My question is why cat would work on one file and not the other. In the future, are there requirements for z/OS utilities like OpenSSH and others to tag the files they create? thanks

09:01:50 RC=(0) [SYSA] bash-5.2$ ls -T /SYSE/etc/sshd.pid

untagged T=off /SYSE/etc/sshd.pid
09:01:53 RC=(0) [SYSA] bash-5.2$ cat /SYSE/etc/sshd.pid
50397225

09:01:59 RC=(0) [SYSA] bash-5.2$ ls -T /SYSF/etc/sshd.pid

untagged T=off /SYSF/etc/sshd.pid
09:02:09 RC=(0) [SYSA] bash-5.2$ cat /SYSF/etc/sshd.pid
▒▒▒▒▒▒09:02:18 RC=(0) [SYSA] bash-5.2$

[IgorTodorovskiIBM]

Coreutils is linked with zoslib which applies heuristics to guess the encoding of untagged files (default behaviour).

You can change the default to avoid the heuristic by setting:

export __UNTAGGED_READ_MODE=ASCII

When there is not enough content in the file, the heuristic fails. We'll need to examine this in more detail - I think it's a bug in the heuristic of zoslib.

[gngrossi]

Looks good...closing
thanks

[gngrossi]

Sorry, I forgot to ask...do you plan to include export __UNTAGGED_READ_MODE=ASCII in zoslib?
thanks

[gngrossi]

Any update?
thanks

Using cat on an untagged 1047 text file on the local LPAR is ok.

Using cat on an untagged 1047 text file on a different LPAR is not ok.

[IgorTodorovskiIBM]

I'll look into this, this will be handled via zoslib changes in ibmruntimes/zoslib#58. I'll revisit it this week.

[gngrossi]

I was thinking of adding this to /etc/rc

file /etc/sshd.pid | grep text | sed 's/:.*text$//' | xargs chtag -tc IBM-1047
file /etc/syslog.pid | grep text | sed 's/:.*text$//' | xargs chtag -tc IBM-1047

thoughts?

[gngrossi]

perhaps a bit more complete...could also handle an empty untagged file

file /etc/sshd.pid | grep text | sed 's/:.*text$//' | xargs ls -T | grep untagged | awk '{print $4}' | xargs chtag -tc IBM-1047
file /etc/syslog.pid | grep text | sed 's/:.*text$//' | xargs ls -T | grep untagged | awk '{print $4}' | xargs chtag -tc IBM-1047

[IgorTodorovskiIBM]

perhaps a bit more complete...could also handle an empty untagged file

file /etc/sshd.pid | grep text | sed 's/:.*text$//' | xargs ls -T | grep untagged | awk '{print $4}' | xargs chtag -tc IBM-1047 file /etc/syslog.pid | grep text | sed 's/:.*text$//' | xargs ls -T | grep untagged | awk '{print $4}' | xargs chtag -tc IBM-1047

That would work, also future proof in case it gets tagged as something else in new OS releases.