"Buffer overflow detected"

[chengzhuzhang]

I'm using a tool called e3sm_to_cmip to cmorize e3sm model output. It is not clear what triggers this
"buffer overflow detected" error. It could be the updated cmor tables. I saw discussion on a new release of cmor could resolve this problem, but I'm wondering if there is a workaround to bypass this problem before the new release? A similar issue was reported in #543
I'm tagging @tomvothecoder in case here are questions on the e3sm_to_cmip side.

buffer overflow detected ***: /p/user_pub/e3sm_unified/envs/base/envs/e3sm_unified_1.7.0_nompi/bin/python3.9 terminated
======= Backtrace: =========
/lib64/libc.so.6(__fortify_fail+0x37)[0x7f825de757a7]
/lib64/libc.so.6(+0x116922)[0x7f825de73922]
/p/user_pub/e3sm_unified/envs/base/envs/e3sm_unified_1.7.0_nompi/lib/python3.9/site-packages/cmor/_cmor.cpython-39-x86_64-linux-gnu.so(cmor_CV_ValidateAttribute+0x36b)[0x7f7dccd8d7bb]
/p/user_pub/e3sm_unified/envs/base/envs/e3sm_unified_1.7.0_nompi/lib/python3.9/site-packages/cmor/_cmor.cpython-39-x86_64-linux-gnu.so(cmor_CV_checkGblAttributes+0x73)[0x7f7dccd8dc03]
/p/user_pub/e3sm_unified/envs/base/envs/e3sm_unified_1.7.0_nompi/lib/python3.9/site-packages/cmor/_cmor.cpython-39-x86_64-linux-gnu.so(cmor_setGblAttr+0x60d)[0x7f7dccd86a6d]
/p/user_pub/e3sm_unified/envs/base/envs/e3sm_unified_1.7.0_nompi/lib/python3.9/site-packages/cmor/_cmor.cpython-39-x86_64-linux-gnu.so(cmor_write+0x1d8)[0x7f7dccd88b68]
/p/user_pub/e3sm_unified/envs/base/envs/e3sm_unified_1.7.0_nompi/lib/python3.9/site-packages/cmor/_cmor.cpython-39-x86_64-linux-gnu.so(+0xe850)[0x7f7dccd73850]

[chengzhuzhang]

Update: it seems this problem can be resolved by using an old version of cmor tables...But this workaround won't be useful for creating cmor files for more recently registered files.

[durack1]

@chengzhuzhang thanks for raising this issue. Are you trying to CMORize E3SM 2.0 data? If yes, we have a standing issue regarding variable sizing (see PCMDI/cmip6-cmor-tables#377) which is yet to be solved - it requires a tweak to the registered content, which I had planned to get done today (with new tables generated soon thereafter)

[durack1]

@chengzhuzhang I assume that your issue will be fixed when you update your table files to the latest PCMDI/cmip6-cmor-tables@bf115fc.

I'll close this issue, please reopen if the problem continues

[chengzhuzhang]

Hey @durack1 I updated my local cmor tables. But tripped on the the same error. Any suggestion on debugging.
Tagging @TonyB9000 since I found he has updated the cmip6-cmor-tables in his staging area. Again, reverting back to use an old version of cmor tables works for E3SMv1 data.

[durack1]

@mauzey1 any ideas here?

[TonyB9000]

@chengzhuzhang I need to update the staging "cmip6-cmor-table" when they change. This is a hassle, because the way datasm is written, it wants to address the tables, independently of "e3sm_to_cmip" invocation. Not expert enough to know why just yet.

The latest "git clone https://github.com/PCMDI/cmip6-cmor-tables" is in
/p/user_pub/e3sm/staging/resource/cmor/cmip6-cmor-tables/