Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements/add testcode modify cadinality #81

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

bourbonkk
Copy link
Contributor

@bourbonkk bourbonkk commented Feb 11, 2023

Hello, I modified some code and added test code. #31

  1. Changed the type of cardinality's fieid([]string -> []interface{}). According to the documentation
    Each individual element of the "fields" list can be a String or DimensionSpec

  2. Remove omitempty of index of substring - If omitempty is present, it disappears from json when index is 0

  3. add SetDimension method of lookup - I wrote a test code with the example code in the official documentation, but it didn't work properly, so I modified it.

Additionally, I wrote test codes for cardinality, extraction, javascript, and lookup.

Each individual element of the "fields" list can be a String or DimensionSpec. A String dimension in the fields list is equivalent to a DefaultDimensionSpec (no transformations).
Copy link
Contributor

@jbguerraz jbguerraz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably it would deserve multiple PR so I would have merged the index & lookup dimension quickly since I have some reserve with cardinality change ("method polymorphism").
See https://github.com/apache/druid/blob/master/processing/src/main/java/org/apache/druid/query/aggregation/cardinality/CardinalityAggregatorFactory.java#L118
Probably we should enforce DimensionSpec type here.
Maybe we could add an extra method SetFieldNames that would be an adapter for convenience but I don't even think we should. I would always tend to go with the strongly typed way instead of the shorthand.
What do you think ?

@@ -37,6 +37,11 @@ func (l *Lookup) SetOutputName(outputName string) *Lookup {
return l
}

func (l *Lookup) SetDimension(dimensionName string) *Lookup {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better name it dimension instead of dimensionName since it's what we've done everywhere (just like druid own codebase @JsonProperty("dimension") String dimension, https://github.com/apache/druid/blob/master/processing/src/main/java/org/apache/druid/query/dimension/LookupDimensionSpec.java)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the detailed review, @jbguerraz

I completely agree with your thoughts. I hadn't thought of specifying it until I looked at the Java source code. I created a DimensionSpec interface and modified the Fields variable of cadinality. Please check if it matches your idea.

type DimensionSpec interface {
	Dimension
	GetDimension() string
	GetOutputName() string
	GetOutputType() types.OutputType
	GetExtractionFn() ExtractionFn // Deprecated
}

@bourbonkk bourbonkk requested a review from jbguerraz March 14, 2023 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants