Skip to content

Commit

Permalink
[doc] Update roadmap
Browse files Browse the repository at this point in the history
  • Loading branch information
JingsongLi committed Nov 14, 2024
1 parent bf8f559 commit 8b77207
Showing 1 changed file with 6 additions and 43 deletions.
49 changes: 6 additions & 43 deletions docs/content/project/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,6 @@ under the License.

# Roadmap

## Native Format IO

Integrate native Parquet & ORC reader & writer.

## Deletion Vectors (Merge On Write)

1. Primary Key Table Deletion Vectors Mode supports async compaction.
2. Append Table supports DELETE & UPDATE with Deletion Vectors Mode. (Now only Spark SQL)
3. Optimize lookup performance for HDD disk.

## Flink Lookup Join

Support Flink Custom Data Distribution Lookup Join to reach large-scale data lookup join.
Expand All @@ -44,51 +34,24 @@ Support Flink Custom Data Distribution Lookup Join to reach large-scale data loo

Introduce a mode to produce Iceberg snapshots.

## Branch

Branch production ready.

## Changelog life cycle decouple

Changelog life cycle decouple supports none changelog-producer.

## Partition Mark Done

Support partition mark done.

## Default File Format

- Default compression is ZSTD with level 1.
- Parquet supports filter push down.
- Parquet supports arrow with row type element.
- Parquet becomes default file format.

## Variant Type

Support Variant Type with Spark 4.0 and Flink 2.0. Unlocking support for semi-structured data.

## Bucketed Join

Support Bucketed Join with Spark SQL to reduce shuffler in Join.

## File Index

Add more index:
1. Bitmap
2. Inverse

## Column Family
1. Inverse

## Vector Compaction

Support Column Family for super Wide Table.
Support Vector Compaction for super Wide Table.

## View & Function support
## Function support

Paimon Catalog supports views and functions.
Paimon Catalog supports functions.

## Files Schema Evolution Ingestion

Introduce a files Ingestion with Schema Evolution.

## Foreign Key Join

Explore Foreign Key Join solution.

0 comments on commit 8b77207

Please sign in to comment.