[flink](lookup) project row to avoid deserialize unnecessary fields #148
Conversation
wuchong
force-pushed
the
lookup-improve
branch
from
66c6abf to
c49e191
Compare
| // to avoid deserialize unnecessary fields | ||
| RowData flinkRow = flussRowToFlinkRowConverter.toFlinkRowData(row); | ||
| RowData flinkRow = | ||
| flussRowToFlinkRowConverter.toFlinkRowData( |
There was a problem hiding this comment.
flussRowToFlinkRowConverter works on full fields and it fails ArrayIndexOutOfBoundsException if passing projected row. Try run com.alibaba.fluss.connector.flink.source.FlinkTableSourceITCase#testLookup1PkTable.
flussRowToFlinkRowConverter should also be adapted for projection if we want it work on projected row.
There was a problem hiding this comment.
Thank you, it has been modified, and FlinkTableSourceITCase#testLookup1PkTable has been successfully run
|
I rebased the branch onto |
|
@wuchong comments addressed, PTAL, Thanks. |
wuchong
force-pushed
the
lookup-improve
branch
from
316bdfb to
453279b
Compare
| new FlussRowToFlinkRowConverter(FlinkConversions.toFlussRowType(flinkRowType)); | ||
| new FlussRowToFlinkRowConverter( | ||
| FlinkConversions.toFlussRowType( | ||
| FlinkLookupFunction.filterRowType(flinkRowType, projection))); |
There was a problem hiding this comment.
projection may be null, this can cause NPE
| for (int i = 0; i < pkIndex.length; i++) { | ||
| types[i] = rowType.getTypeAt(pkIndex[i]); | ||
| names[i] = rowType.getFieldNames().get(pkIndex[i]); | ||
| static RowType filterRowType(RowType rowType, int[] filterIndex) { |
There was a problem hiding this comment.
Generally, this is called projectRowType and we can move this to a common util.
| RowData flinkRow = flussRowToFlinkRowConverter.toFlinkRowData(row); | ||
| RowData flinkRow = | ||
| flussRowToFlinkRowConverter.toFlinkRowData( | ||
| ProjectedRow.from(projection).replaceRow(row)); |
There was a problem hiding this comment.
This can cause NPE as the projection can be null.
wuchong
force-pushed
the
lookup-improve
branch
from
453279b to
f805a7e
Compare
Purpose
project row to avoid deserialize unnecessary fields