Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

some_dataset.listfiles() does not show a warning AFTER calling summary(some_dataset) #199

Open
jypeter opened this issue Aug 4, 2021 · 0 comments

Comments

@jypeter
Copy link

jypeter commented Aug 4, 2021

This is possibly linked to #193

When there is an ambiguity on the realization (I'm using an example from CMIP6 with CliMAF: basis and tips to work on ensembles):

  • I get a warning when using listfiles(), which is what I expect, even when using listfiles several times
  • the warning disappears if I call summary and then use listfiles again! I should keep on getting the warning
  • I get the warning back only if I add the force=True option

See detailed session below

[jypmce@ciclad-ng ~]$ module load climaf/2.0.0-python3.6
Loading climaf/2.0.0-python3.6
  Loading requirement: nco/4.5.2 ncl/6.3.0 cdo/1.9 netcdf4/4.3.3.1-gfortran

[jypmce@ciclad-ng PMIP4]$ python
Python 3.6.11 | packaged by conda-forge | (default, Aug  5 2020, 20:09:42)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from climaf.api import *
python => 3.6.11 | packaged by conda-forge | (default, Aug  5 2020, 20:09:42)
[GCC 7.5.0]
---
Required softwares to run CliMAF => you are using the following versions/installations:
ncl 6.6.2 => /modfs/modtools/miniconda3/envs/analyse_3.6_test/bin/ncl
cdo 1.9.6 => /opt/nco/1.9/bin/cdo
nco (ncks) 4.5.2 => /opt/nco-4.5.2/bin/ncks
ncdump such => /prodigfs/ipslfs/dods/jservon/miniconda/envs/cesmep_env/bin/ncdump
Check stamping requirements
nco (ncatted) found -> /opt/nco-4.5.2/bin/ncatted
convert found -> /usr/bin/convert
pdftk found -> /usr/bin/pdftk
exiv2 found -> /ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_V2.0.0/bin/exiv2
---
CliMAF version = 2.0.0
CliMAF install => /ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_V2.0.0
Cache directory set to : /data/jypmce/climafcache (use $CLIMAF_CACHE if set)
Cache directory for remote data set to : /data/jypmce/climafcache/remote_data (use $CLIMAF_REMOTE_CACHE if set)
Available macros read from ~/.climaf.macros are : []

>>> req_cmip6 = ds(project='CMIP6', variable='pr', period='*',
...                table='Amon', model='CNRM-CM6-1',
...                realization='*'
...               )

>>> req_cmip6.listfiles()
error    : "This dataset is ambiguous on attribute realization='*'; please choose among : ['r10i1p1f2', 'r11i1p1f2', 'r12i1p1f2', 'r13i1p1f2', 'r14i1p1f2', 'r15i1p1f2', 'r16i1p1f2', 'r17i1p1f2', 'r18i1p1f2', 'r19i1p1f2', 'r1i1p1f2', 'r20i1p1f2', 'r21i1p1f2', 'r22i1p1f2', 'r24i1p1f2', 'r25i1p1f2', 'r26i1p1f2', 'r27i1p1f2', 'r28i1p1f2', 'r29i1p1f2', 'r2i1p1f2', 'r30i1p1f2', 'r3i1p1f2', 'r4i1p1f2', 'r5i1p1f2', 'r6i1p1f2', 'r7i1p1f2', 'r8i1p1f2', 'r9i1p1f2'] or use either 'ensure_dataset=False' (with method baseFiles or listfiles) or 'option='choices' (with method explore)"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_V2.0.0/climaf/classes.py", line 774, in listfiles
    return self.baseFiles(force=force, ensure_dataset=ensure_dataset)
  File "/ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_V2.0.0/climaf/classes.py", line 762, in baseFiles
    self.explore()
  File "/ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_V2.0.0/climaf/classes.py", line 748, in explore
    (kw, dic[kw], entry))
climaf.utils.Climaf_Classes_Error: "This dataset is ambiguous on attribute realization='*'; please choose among : ['r10i1p1f2', 'r11i1p1f2', 'r12i1p1f2', 'r13i1p1f2', 'r14i1p1f2', 'r15i1p1f2', 'r16i1p1f2', 'r17i1p1f2', 'r18i1p1f2', 'r19i1p1f2', 'r1i1p1f2', 'r20i1p1f2', 'r21i1p1f2', 'r22i1p1f2', 'r24i1p1f2', 'r25i1p1f2', 'r26i1p1f2', 'r27i1p1f2', 'r28i1p1f2', 'r29i1p1f2', 'r2i1p1f2', 'r30i1p1f2', 'r3i1p1f2', 'r4i1p1f2', 'r5i1p1f2', 'r6i1p1f2', 'r7i1p1f2', 'r8i1p1f2', 'r9i1p1f2'] or use either 'ensure_dataset=False' (with method baseFiles or listfiles) or 'option='choices' (with method explore)"

>>> req_cmip6.listfiles()
error    : "This dataset is ambiguous on attribute realization='*'; please choose among : ['r10i1p1f2', 'r11i1p1f2', 'r12i1p1f2', 'r13i1p1f2', 'r14i1p1f2', 'r15i1p1f2', 'r16i1p1f2', 'r17i1p1f2', 'r18i1p1f2', 'r19i1p1f2', 'r1i1p1f2', 'r20i1p1f2', 'r21i1p1f2', 'r22i1p1f2', 'r24i1p1f2', 'r25i1p1f2', 'r26i1p1f2', 'r27i1p1f2', 'r28i1p1f2', 'r29i1p1f2', 'r2i1p1f2', 'r30i1p1f2', 'r3i1p1f2', 'r4i1p1f2', 'r5i1p1f2', 'r6i1p1f2', 'r7i1p1f2', 'r8i1p1f2', 'r9i1p1f2'] or use either 'ensure_dataset=False' (with method baseFiles or listfiles) or 'option='choices' (with method explore)"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_V2.0.0/climaf/classes.py", line 774, in listfiles
    return self.baseFiles(force=force, ensure_dataset=ensure_dataset)
  File "/ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_V2.0.0/climaf/classes.py", line 762, in baseFiles
    self.explore()
  File "/ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_V2.0.0/climaf/classes.py", line 748, in explore
    (kw, dic[kw], entry))
climaf.utils.Climaf_Classes_Error: "This dataset is ambiguous on attribute realization='*'; please choose among : ['r10i1p1f2', 'r11i1p1f2', 'r12i1p1f2', 'r13i1p1f2', 'r14i1p1f2', 'r15i1p1f2', 'r16i1p1f2', 'r17i1p1f2', 'r18i1p1f2', 'r19i1p1f2', 'r1i1p1f2', 'r20i1p1f2', 'r21i1p1f2', 'r22i1p1f2', 'r24i1p1f2', 'r25i1p1f2', 'r26i1p1f2', 'r27i1p1f2', 'r28i1p1f2', 'r29i1p1f2', 'r2i1p1f2', 'r30i1p1f2', 'r3i1p1f2', 'r4i1p1f2', 'r5i1p1f2', 'r6i1p1f2', 'r7i1p1f2', 'r8i1p1f2', 'r9i1p1f2'] or use either 'ensure_dataset=False' (with method baseFiles or listfiles) or 'option='choices' (with method explore)"

>>> summary(req_cmip6)
Multiple available values for attribute "realization" that is set to "*" in your ds() call:  ['r10i1p1f2', 'r11i1p1f2', 'r12i1p1f2', 'r13i1p1f2', 'r14i1p1f2', 'r15i1p1f2', 'r16i1p1f2', 'r17i1p1f2', 'r18i1p1f2', 'r19i1p1f2', 'r1i1p1f2', 'r20i1p1f2', 'r21i1p1f2', 'r22i1p1f2', 'r24i1p1f2', 'r25i1p1f2', 'r26i1p1f2', 'r27i1p1f2', 'r28i1p1f2', 'r29i1p1f2', 'r2i1p1f2', 'r30i1p1f2', 'r3i1p1f2', 'r4i1p1f2', 'r5i1p1f2', 'r6i1p1f2', 'r7i1p1f2', 'r8i1p1f2', 'r9i1p1f2']
Specify one of them (within ds() or with cdef())
{'project': 'CMIP6', 'simulation': '', 'variable': 'pr', 'period': '*', 'domain': 'global', 'root': '/bdd', 'model': 'CNRM-CM6-1', 'institute': '*', 'mip': '*', 'table': 'Amon', 'experiment': 'historical', 'realization': '*', 'grid': 'g*', 'version': 'latest'}

>>> req_cmip6.listfiles()
'r10i1p1f2 r11i1p1f2 r12i1p1f2 r13i1p1f2 r14i1p1f2 r15i1p1f2 r16i1p1f2 r17i1p1f2 r18i1p1f2 r19i1p1f2 r1i1p1f2 r20i1p1f2 r21i1p1f2 r22i1p1f2 r24i1p1f2 r25i1p1f2 r26i1p1f2 r27i1p1f2 r28i1p1f2 r29i1p1f2 r2i1p1f2 r30i1p1f2 r3i1p1f2 r4i1p1f2 r5i1p1f2 r6i1p1f2 r7i1p1f2 r8i1p1f2 r9i1p1f2'

>>> req_cmip6.listfiles()
'r10i1p1f2 r11i1p1f2 r12i1p1f2 r13i1p1f2 r14i1p1f2 r15i1p1f2 r16i1p1f2 r17i1p1f2 r18i1p1f2 r19i1p1f2 r1i1p1f2 r20i1p1f2 r21i1p1f2 r22i1p1f2 r24i1p1f2 r25i1p1f2 r26i1p1f2 r27i1p1f2 r28i1p1f2 r29i1p1f2 r2i1p1f2 r30i1p1f2 r3i1p1f2 r4i1p1f2 r5i1p1f2 r6i1p1f2 r7i1p1f2 r8i1p1f2 r9i1p1f2'

>>> req_cmip6.listfiles(force=True)
error    : "This dataset is ambiguous on attribute realization='*'; please choose among : ['r10i1p1f2', 'r11i1p1f2', 'r12i1p1f2', 'r13i1p1f2', 'r14i1p1f2', 'r15i1p1f2', 'r16i1p1f2', 'r17i1p1f2', 'r18i1p1f2', 'r19i1p1f2', 'r1i1p1f2', 'r20i1p1f2', 'r21i1p1f2', 'r22i1p1f2', 'r24i1p1f2', 'r25i1p1f2', 'r26i1p1f2', 'r27i1p1f2', 'r28i1p1f2', 'r29i1p1f2', 'r2i1p1f2', 'r30i1p1f2', 'r3i1p1f2', 'r4i1p1f2', 'r5i1p1f2', 'r6i1p1f2', 'r7i1p1f2', 'r8i1p1f2', 'r9i1p1f2'] or use either 'ensure_dataset=False' (with method baseFiles or listfiles) or 'option='choices' (with method explore)"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_V2.0.0/climaf/classes.py", line 774, in listfiles
    return self.baseFiles(force=force, ensure_dataset=ensure_dataset)
  File "/ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_V2.0.0/climaf/classes.py", line 762, in baseFiles
    self.explore()
  File "/ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_V2.0.0/climaf/classes.py", line 748, in explore
    (kw, dic[kw], entry))
climaf.utils.Climaf_Classes_Error: "This dataset is ambiguous on attribute realization='*'; please choose among : ['r10i1p1f2', 'r11i1p1f2', 'r12i1p1f2', 'r13i1p1f2', 'r14i1p1f2', 'r15i1p1f2', 'r16i1p1f2', 'r17i1p1f2', 'r18i1p1f2', 'r19i1p1f2', 'r1i1p1f2', 'r20i1p1f2', 'r21i1p1f2', 'r22i1p1f2', 'r24i1p1f2', 'r25i1p1f2', 'r26i1p1f2', 'r27i1p1f2', 'r28i1p1f2', 'r29i1p1f2', 'r2i1p1f2', 'r30i1p1f2', 'r3i1p1f2', 'r4i1p1f2', 'r5i1p1f2', 'r6i1p1f2', 'r7i1p1f2', 'r8i1p1f2', 'r9i1p1f2'] or use either 'ensure_dataset=False' (with method baseFiles or listfiles) or 'option='choices' (with method explore)"

>>> req_cmip6.listfiles()
'r10i1p1f2 r11i1p1f2 r12i1p1f2 r13i1p1f2 r14i1p1f2 r15i1p1f2 r16i1p1f2 r17i1p1f2 r18i1p1f2 r19i1p1f2 r1i1p1f2 r20i1p1f2 r21i1p1f2 r22i1p1f2 r24i1p1f2 r25i1p1f2 r26i1p1f2 r27i1p1f2 r28i1p1f2 r29i1p1f2 r2i1p1f2 r30i1p1f2 r3i1p1f2 r4i1p1f2 r5i1p1f2 r6i1p1f2 r7i1p1f2 r8i1p1f2 r9i1p1f2'
>>>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant