Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Homogenise new column parameter in steps interfaces #464

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@
- Write percentage step result in a new column
- Write substring step result in a new column

### Breaking

- Homogenise new column parameter in steps: this parameter is now always named
`newColumn`

## [0.12.0] - 2020-02-26

### Added
Expand Down
36 changes: 18 additions & 18 deletions docs/_docs/tech/steps.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ An aggreation step has the following strucure:
on: ['column1', 'column2'],
aggregations: [
{
newcolumn: 'sum_value1',
newColumn: 'sum_value1',
aggfunction: 'sum',
column: 'value1'
}
Expand Down Expand Up @@ -58,7 +58,7 @@ An aggreation step has the following strucure:
on: ['Group'],
aggregations: [
{
newcolumn: 'Total',
newColumn: 'Total',
aggfunction: 'sum',
column: 'Value'
}
Expand Down Expand Up @@ -296,7 +296,7 @@ This step allows to concatenate several `columns` using a `separator`.
name: 'concatenate',
columns: ['Company', 'Group']
separator: ' - ' // can be a string of any length
new_column_name: 'Label' // The new column in which to write the concatenation
newColumn: 'Label' // The new column in which to write the concatenation
}
```

Expand Down Expand Up @@ -325,7 +325,7 @@ This step allows to concatenate several `columns` using a `separator`.
name: 'concatenate',
columns: ['Company', 'Group']
separator: ' - '
new_column_name: 'Label'
newColumn: 'Label'
}
```

Expand Down Expand Up @@ -464,7 +464,7 @@ Here's an example of such a step:
name: 'dateextract',
column: 'date',
operation: 'day',
new_column_name: 'date_day',
newColumn: 'date_day',
format: '%Y-%m-%d',
}
```
Expand Down Expand Up @@ -494,7 +494,7 @@ Here's an example of such a step:
name: 'dateextract',
column: 'Date',
operation: 'day',
new_column_name: 'Date_day',
newColumn: 'Date_day',
format: '%Y-%m-%d',
}
```
Expand Down Expand Up @@ -583,7 +583,7 @@ This step is meant to duplicate a column.
{
name: 'duplicate',
column: 'my-column'
new_column_name: 'my-duplicate'
newColumn: 'my-duplicate'
}
```

Expand All @@ -608,7 +608,7 @@ This step is meant to duplicate a column.
{
name: 'duplicate',
column: "Company",
new_column_name: "Company-copy",
newColumn: "Company-copy",
}
```

Expand Down Expand Up @@ -707,7 +707,7 @@ Column names must not be escaped. Strings have to be escaped with quotes.
{
{
name: 'formula',
new_column: 'result', // if the column already exists, overwrites it
newColumn: 'result', // if the column already exists, overwrites it
formula: '(Value1 + Value2) / Value3 - Value4 * 2'
}
}
Expand All @@ -728,7 +728,7 @@ Column names must not be escaped. Strings have to be escaped with quotes.
```javascript
{
name: 'formula',
new_column: 'Result',
newColumn: 'Result',
formula: '(Value1 + Value2) / Value3 - Value4 * 2'
}
```
Expand Down Expand Up @@ -1001,7 +1001,7 @@ by `group` if specified. The result is written in a new column.
```javascript

name: 'percentage',
new_column: 'Percentage_of_total',
newColumn: 'Percentage_of_total',
column: 'Value',
group: ['Group']
newColumn: 'Percentage'
Expand Down Expand Up @@ -1191,12 +1191,12 @@ specifying for every row the label, level and parent in dedicated columns.
hierarchy: ['continent', 'country', 'city'], // Hierarchical columns, from top to bottom level
aggregations: [ // The columns to be aggregated, and how
{
newcolumn: 'revenues',
newColumn: 'revenues',
aggfunction: 'sum', // aggregate using sum
column: 'revenues'
},
{
newcolumn: 'average_cost',
newColumn: 'average_cost',
aggfunction: 'avg', // aggregate using average
column: 'cost'
}
Expand Down Expand Up @@ -1244,7 +1244,7 @@ specifying for every row the label, level and parent in dedicated columns.
hierarchy: ['CONTINENT', 'COUNTRY', 'CITY'],
aggregations: [
{
newcolumn: 'revenues',
newColumn: 'revenues',
aggfunction: 'sum',
column: 'revenues'
},
Expand Down Expand Up @@ -1302,12 +1302,12 @@ specifying for every row the label, level and parent in dedicated columns.
hierarchy: ['CONTINENT', 'COUNTRY', 'CITY'],
aggregations: [
{
newcolumn: 'VALUE-sum',
newColumn: 'VALUE-sum',
aggfunction: 'sum',
column: 'VALUE'
},
{
newcolumn: 'VALUE-avg',
newColumn: 'VALUE-avg',
aggfunction: 'avg',
column: 'VALUE'
}
Expand Down Expand Up @@ -1680,7 +1680,7 @@ specified `text`.
{
{
name: 'text',
new_column: 'new', // if the column already exists, overwrites it
newColumn: 'new', // if the column already exists, overwrites it
text: 'some text'
}
}
Expand All @@ -1706,7 +1706,7 @@ specified `text`.
```javascript
{
name: 'text',
new_column: 'KPI',
newColumn: 'KPI',
formula: 'Sales'
}
```
Expand Down
27 changes: 16 additions & 11 deletions playground/app.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,26 +19,31 @@ const mongo40translator = getTranslator(TRANSLATOR);
// Example to set a custom editor:
// This one is quite simple. It customizes the placeholder.
setCodeEditor({
props:['value', 'placeholder'],
props: ['value', 'placeholder'],
render(createElement) {
return createElement("textarea", {
return createElement('textarea', {
domProps: {
value: this.value,
placeholder: "OMG I have to write code in here",
placeholder: 'OMG I have to write code in here',
},
attrs: {
type: "text"
type: 'text',
},
on: {
input: (event) => {this.$emit('input', event.target.value)},
blur: (event) => {this.$emit('blur')},
focus: (event) => {this.$emit('focus')},
input: event => {
this.$emit('input', event.target.value);
},
blur: event => {
this.$emit('blur');
},
focus: event => {
this.$emit('focus');
},
},
})
});
},
});


const CASTERS = {
date: val => new Date(val),
};
Expand Down Expand Up @@ -289,7 +294,7 @@ async function buildVueApp() {
{
name: 'formula',
formula: 'Value2 * 10',
new_column: 'ValueRight1',
newColumn: 'ValueRight1',
},
],
pipelineRight2: [
Expand All @@ -309,7 +314,7 @@ async function buildVueApp() {
{
name: 'formula',
formula: 'Value3 * 10',
new_column: 'ValueRight2',
newColumn: 'ValueRight2',
},
],
},
Expand Down
12 changes: 6 additions & 6 deletions src/components/stepforms/AddTextColumnStepForm.vue
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@
/>
<InputTextWidget
id="newColumnInput"
v-model="editedStep.new_column"
v-model="editedStep.newColumn"
name="New colum:"
placeholder="Enter a new column name"
data-path=".new_column"
data-path=".newColumn"
:errors="errors"
:warning="duplicateColumnName"
/>
Expand All @@ -39,14 +39,14 @@ import BaseStepForm from './StepForm.vue';
},
})
export default class AddTextColumnStepForm extends BaseStepForm<AddTextColumnStep> {
@Prop({ type: Object, default: () => ({ name: 'text', new_column: '', text: '' }) })
@Prop({ type: Object, default: () => ({ name: 'text', newColumn: '', text: '' }) })
initialStepValue!: AddTextColumnStep;

readonly title: string = 'Add Text Column';

get duplicateColumnName() {
if (this.columnNames.includes(this.editedStep.new_column)) {
return `A column name "${this.editedStep.new_column}" already exists. You will overwrite it.`;
if (this.columnNames.includes(this.editedStep.newColumn)) {
return `A column name "${this.editedStep.newColumn}" already exists. You will overwrite it.`;
} else {
return null;
}
Expand All @@ -55,7 +55,7 @@ export default class AddTextColumnStepForm extends BaseStepForm<AddTextColumnSte
submit() {
this.$$super.submit();
if (this.errors === null) {
this.setSelectedColumns({ column: this.editedStep.new_column });
this.setSelectedColumns({ column: this.editedStep.newColumn });
}
}
}
Expand Down
12 changes: 6 additions & 6 deletions src/components/stepforms/AggregateStepForm.vue
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ export default class AggregateStepForm extends BaseStepForm<AggregationStep> {
get defaultAggregation() {
const agg: AggFunctionStep = {
column: '',
newcolumn: '',
newColumn: '',
aggfunction: 'sum',
};
return agg;
Expand All @@ -76,16 +76,16 @@ export default class AggregateStepForm extends BaseStepForm<AggregationStep> {
submit() {
/**
* If different aggregations have to be performed on the same column, add a suffix
* to the automatically generated newcolumn name
* to the automatically generated new column name
*/
const newcolumnOccurences: { [prop: string]: number } = {};
for (const agg of this.editedStep.aggregations) {
agg.newcolumn = agg.column;
newcolumnOccurences[agg.newcolumn] = (newcolumnOccurences[agg.newcolumn] || 0) + 1;
agg.newColumn = agg.column;
newcolumnOccurences[agg.newColumn] = (newcolumnOccurences[agg.newColumn] || 0) + 1;
}
for (const agg of this.editedStep.aggregations) {
if (newcolumnOccurences[agg.newcolumn] > 1) {
agg.newcolumn = `${agg.newcolumn}-${agg.aggfunction}`;
if (newcolumnOccurences[agg.newColumn] > 1) {
agg.newColumn = `${agg.newColumn}-${agg.aggfunction}`;
}
}
this.$$super.submit();
Expand Down
10 changes: 5 additions & 5 deletions src/components/stepforms/ConcatenateStepForm.vue
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@
:errors="errors"
/>
<InputTextWidget
id="newColumnName"
v-model="editedStep.new_column_name"
id="newColumn"
v-model="editedStep.newColumn"
name="New column name:"
placeholder="Enter a columnn name"
data-path=".new_column_name"
data-path=".newColumn"
:errors="errors"
/>
<StepFormButtonbar />
Expand Down Expand Up @@ -56,7 +56,7 @@ import BaseStepForm from './StepForm.vue';
export default class ConcatenateStepForm extends BaseStepForm<ConcatenateStep> {
@Prop({
type: Object,
default: () => ({ name: 'concatenate', columns: [''], separator: '', new_column_name: '' }),
default: () => ({ name: 'concatenate', columns: [''], separator: '', newColumn: '' }),
})
initialStepValue!: ConcatenateStep;

Expand All @@ -70,7 +70,7 @@ export default class ConcatenateStepForm extends BaseStepForm<ConcatenateStep> {
name: 'concatenate' as 'concatenate',
columns: [this.selectedColumns[0]],
separator: '',
new_column_name: '',
newColumn: '',
};
} else {
// Otherwise, fallback on the default initial value
Expand Down
12 changes: 6 additions & 6 deletions src/components/stepforms/DateExtractStepForm.vue
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,11 @@
:errors="errors"
/>
<InputTextWidget
id="newColumnName"
v-model="editedStep.new_column_name"
id="newColumn"
v-model="editedStep.newColumn"
name="New column name:"
:placeholder="newColumnNamePlaceholder"
data-path=".new_column_name"
data-path=".newColumn"
:errors="errors"
/>
<StepFormButtonbar />
Expand Down Expand Up @@ -111,9 +111,9 @@ export default class DateExtractStepForm extends BaseStepForm<DateExtractPropert
}

submit() {
// make sure new_column_name doesn't overwrite an existing columm name
this.editedStep.new_column_name = generateNewColumnName(
this.editedStep.new_column_name ?? this.defaultNewColumnName,
// make sure newColumn doesn't overwrite an existing columm name
this.editedStep.newColumn = generateNewColumnName(
this.editedStep.newColumn ?? this.defaultNewColumnName,
this.columnNames,
);
this.$$super.submit();
Expand Down
6 changes: 3 additions & 3 deletions src/components/stepforms/DuplicateColumnStepForm.vue
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@
/>
<InputTextWidget
id="newColumnNameInput"
v-model="editedStep.new_column_name"
v-model="editedStep.newColumn"
name="New column name:"
placeholder="Enter a column name"
data-path=".new_column_name"
data-path=".newColumn"
:errors="errors"
/>
<StepFormButtonbar />
Expand All @@ -40,7 +40,7 @@ import BaseStepForm from './StepForm.vue';
},
})
export default class DuplicateColumnForm extends BaseStepForm<DuplicateColumnStep> {
@Prop({ type: Object, default: () => ({ name: 'duplicate', column: '', new_column_name: '' }) })
@Prop({ type: Object, default: () => ({ name: 'duplicate', column: '', newColumn: '' }) })
initialStepValue!: DuplicateColumnStep;

readonly title: string = 'Duplicate column';
Expand Down
Loading