You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm encountering an issue when using Doris' Stream Load feature to load Parquet files that contain INT96-formatted timestamps. The timestamps appear to be automatically converted to UTC+8 during the load process, which is not the expected behavior. These timestamps should be absolute date, but Doris seems to be applying an incorrect timezone offset, leading to discrepancies in the data.
Steps to Reproduce:
Create a Parquet file with INT96 timestamps using a tool like Parquet.Net.
Use Doris Stream Load feature to load the Parquet file.
Observe that the timestamps are converted to UTC+8 during the load process.
Expected Behavior:
The INT96 timestamps should be absolute date, or Doris should allow configuration of the timezone for timestamp fields during the load. Alternatively, the timestamps should be loaded without any automatic timezone conversion.
Actual Behavior:
The timestamps in the INT96 format are incorrectly converted to UTC+8 during the load process when using the Stream Load feature. This results in inaccurate data in the loaded Parquet files.
Metadata of the Parquet File:
The metadata for the Parquet file is as follows:
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Issue Description:
I'm encountering an issue when using Doris' Stream Load feature to load Parquet files that contain INT96-formatted timestamps. The timestamps appear to be automatically converted to UTC+8 during the load process, which is not the expected behavior. These timestamps should be absolute date, but Doris seems to be applying an incorrect timezone offset, leading to discrepancies in the data.
Steps to Reproduce:
Create a Parquet file with INT96 timestamps using a tool like Parquet.Net.
Use Doris Stream Load feature to load the Parquet file.
Observe that the timestamps are converted to UTC+8 during the load process.
Expected Behavior:
The INT96 timestamps should be absolute date, or Doris should allow configuration of the timezone for timestamp fields during the load. Alternatively, the timestamps should be loaded without any automatic timezone conversion.
Actual Behavior:
The timestamps in the INT96 format are incorrectly converted to UTC+8 during the load process when using the Stream Load feature. This results in inaccurate data in the loaded Parquet files.
Metadata of the Parquet File:
The metadata for the Parquet file is as follows:
Beta Was this translation helpful? Give feedback.
All reactions