Normalizing columns: add some checks #138
Conversation
| @@ -61,7 +61,7 @@ class Tag(str, Enum): | |||
| uc_sets = "~UC_SETS" | |||
| uc_t = "~UC_T" | |||
| # This is used by Veda for unit conversion when displaying results | |||
| # unitconversion = "~UNITCONVERSION" | |||
| unitconversion = "~UNITCONVERSION" | |||
There was a problem hiding this comment.
@olejandro is it okay to uncomment this? This is needed for the check below.
There was a problem hiding this comment.
It is definitely okay to uncomment it. I am wondering how we should go about this one though, because we won't use the info provided by this type of an input table (so there is no need to normalise or do anything else about it). At the same time, it is good for it to appear in raw_tables.txt for diffing purposes.
times_reader/datatypes.py
Outdated
| @@ -262,6 +262,8 @@ def _read_veda_tags_info(veda_tags_file: str) -> Dict[Tag, Dict[str, str]]: | |||
| if "tag_fields" in tag_info: | |||
| # The file stores the tag name in lowercase, and without the ~ | |||
| tag_name = "~" + tag_info["tag_name"].upper() | |||
| if not Tag.has_tag(tag_name): | |||
There was a problem hiding this comment.
@siddharth-krishna I am thinking of veda_tags.json as of reference which may have more information that is needed / used by the tool. In this case I am wondering whether the direction of the warning should change? I.e. the tool should alert if a tag is defined, but the info about it is not present in veda_tags.json?
There was a problem hiding this comment.
Sounds good. Btw if I add a warning in the other direction, we are at present missing the following:
WARNING: datatypes.Tag has an unknown Tag Tag.comagg not in veda-tags.json
WARNING: datatypes.Tag has an unknown Tag Tag.comemi not in veda-tags.json
WARNING: datatypes.Tag has an unknown Tag Tag.tfm_fill_r not in veda-tags.json
WARNING: datatypes.Tag has an unknown Tag Tag.tfm_ins_txt not in veda-tags.json
WARNING: datatypes.Tag has an unknown Tag Tag.time_slices not in veda-tags.json
WARNING: datatypes.Tag has an unknown Tag Tag.tradelinks not in veda-tags.json
WARNING: datatypes.Tag has an unknown Tag Tag.tradelinks_dins not in veda-tags.json
WARNING: datatypes.Tag has an unknown Tag Tag.uc_sets not in veda-tags.json
There was a problem hiding this comment.
Ok, cool! At least one of them will be gone after merging.
This is some stuff I forgot to include in #135