You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem?
This feature request proposes the addition of an appendcol- command to OpenSearch’s Piped Processing Language (PPL). The appendcol command (modeled after Splunk’s command of the same name), allows users to append the result of one or more searches as additional columns to the existing search results, offering a powerful mechanism to enrich data by combining results horizontally.
What solution would you like?
The following query:
source=logs | stats count by status | appendcol [ stats avg(response_time) by status ]
In this example the appendcol command adds a new column containing the average response time for each status alongside the original column that shows the count by status. The original row structure is preserved.
Implementation could probably use rewriting into a join query.
Add Proposal Document
In many use cases, users need to compare or merge multiple result sets side-by-side without altering the existing row structure. The appendcol command will become a used feature for horizontally merging datasets, and bringing this feature to OpenSearch PPL would greatly enhance data processing flexibility.
By adding appendcol to PPL, users can:
Combine different queries into a unified result set without duplicating rows.
Enrich an existing dataset with additional metrics or fields from other searches.
Improve the efficiency and readability of complex queries.
Technical Details
Syntax
The appendcol command would accept a query inside square brackets [ ], representing the additional pipeline that produces the column(s) to append to the original result set.
Behavior
The new column(s) would be aligned with the rows of the original dataset based on their order of appearance. Each appended column must produce the same number of rows as the base dataset to ensure proper alignment. Any discrepancies in row counts could result in null values for mismatched rows.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem?
This feature request proposes the addition of an
appendcol
- command to OpenSearch’s Piped Processing Language (PPL). Theappendcol
command (modeled after Splunk’s command of the same name), allows users to append the result of one or more searches as additional columns to the existing search results, offering a powerful mechanism to enrich data by combining results horizontally.What solution would you like?
The following query:
In this example the
appendcol
command adds a new column containing the average response time for each status alongside the original column that shows the count by status. The original row structure is preserved.Implementation could probably use rewriting into a join query.
Add Proposal Document
In many use cases, users need to compare or merge multiple result sets side-by-side without altering the existing row structure. The
appendcol
command will become a used feature for horizontally merging datasets, and bringing this feature to OpenSearch PPL would greatly enhance data processing flexibility.By adding
appendcol
to PPL, users can:Technical Details
Syntax
The
appendcol
command would accept a query inside square brackets[ ]
, representing the additional pipeline that produces the column(s) to append to the original result set.Behavior
The new column(s) would be aligned with the rows of the original dataset based on their order of appearance. Each appended column must produce the same number of rows as the base dataset to ensure proper alignment. Any discrepancies in row counts could result in null values for mismatched rows.
The text was updated successfully, but these errors were encountered: