-
Notifications
You must be signed in to change notification settings - Fork 192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dev/manage meta #518
Dev/manage meta #518
Conversation
…ev/llm_info_extract
…ev/llm_info_extract
minor fix
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about removing Fields.agg
and AggKeys
? Instead, we can unify them into Fields.meta
and MetaKeys
. This makes sense because they are implemented in a very similar way, and conceptually, agg
is still a type of meta
.
Additionally, currently, we only have two types of AggKeys
, and keeping both agg
and meta
may increase the cognitive load for users and introduce inconsistencies. Our 2.0 paper mentions that we have three main parts of the fields: 1. data payload, 2. stats, 3. meta. Unifying agg
into meta
would align better with this structure.
Others LGTM.
Have organized and consolidated the generated metadata uniformly.
Fields.meta
field. Corresponding keys are list inMetaKeys
.Fields.agg
field. Corresponding keys are list inAggKeys
. Here, we add a defaultFields.agg
field to all samples in all aggregator operators, without adding a decorator markerTAGGING_OPS
like we do with mappers.nested_set
function.