Skip to content

Commit

Permalink
[doc][github] Update README and create issue/pr template (apache#631)
Browse files Browse the repository at this point in the history
  • Loading branch information
JingsongLi authored Mar 20, 2023
1 parent ec85b08 commit 6430b6f
Show file tree
Hide file tree
Showing 8 changed files with 145 additions and 33 deletions.
56 changes: 56 additions & 0 deletions .github/ISSUE_TEMPLATE/bug-report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: Bug report
description: Problems with the software
title: "[Bug] "
labels: ["bug"]
body:
- type: markdown
attributes:
value: |
Thank you very much for your feedback!
- type: checkboxes
attributes:
label: Search before asking
description: >
Please search [issues](https://github.com/apache/incubator-paimon/issues) to check if your issue has already been reported.
options:
- label: >
I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
required: true
- type: textarea
attributes:
label: Paimon version
description: >
Please provide the version of Paimon you are using. If you are using the master branch, please provide the commit id.
validations:
required: true
- type: textarea
attributes:
label: Compute Engine
description: >
Please provide the compute engine you use and its version. Flink or Spark or Hive or Trino or Presto or JavaAPI.
validations:
required: true
- type: textarea
attributes:
label: Minimal reproduce step
description: Please try to give reproducing steps to facilitate quick location of the problem.
validations:
required: true
- type: textarea
attributes:
label: What doesn't meet your expectations?
validations:
required: true
- type: textarea
attributes:
label: Anything else?
- type: checkboxes
attributes:
label: Are you willing to submit a PR?
description: >
We look forward to the community of developers or users helping solve Paimon problems together. If you are willing to submit a PR to fix this problem, please check the box.
options:
- label: I'm willing to submit a PR!
- type: markdown
attributes:
value: "Thanks for completing our form!"
8 changes: 8 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: Subscribe Mailing List
url: mailto:[email protected]
about: Subscribe User Mailing List
- name: DingTalk Group (Chinese)
url: https://qr.dingtalk.com/action/joingroup?code=v1,k1,+LzfVP12Bl5pdCtcy+G9YIDZjnXsKjhN/y3yLCQAffY=&_dt_no_comment=1&origin=11
about: Join DingTalk Group
40 changes: 40 additions & 0 deletions .github/ISSUE_TEMPLATE/feature.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Feature
description: Add new feature, improve code, and more
labels: [ "enhancement" ]
body:
- type: markdown
attributes:
value: |
Thank you very much for your feature proposal!
- type: checkboxes
attributes:
label: Search before asking
description: >
Please search [issues](https://github.com/apache/incubator-paimon/issues) to check if your issue has already been reported.
options:
- label: >
I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
required: true
- type: textarea
attributes:
label: Motivation
description: Describe the motivations for this feature, like how it fixes the problem you meet.
validations:
required: true
- type: textarea
attributes:
label: Solution
description: Describe the proposed solution and add related materials like links if any.
- type: textarea
attributes:
label: Anything else?
- type: checkboxes
attributes:
label: Are you willing to submit a PR?
description: >
We look forward to the community of developers or users helping develop Paimon features together. If you are willing to submit a PR to implement the feature, please check the box.
options:
- label: I'm willing to submit a PR!
- type: markdown
attributes:
value: "Thanks for completing our form!"
17 changes: 17 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
*(Please specify the module before the PR name: [core] ... or [flink] ...)*

### Purpose

*(What is the purpose of the change, or the associated issue)*

### Tests

*(List UT and IT cases to verify this change)*

### API and Format

*(Does this change affect API or storage format)*

### Documentation

*(Does this change introduce a new feature)*
25 changes: 11 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,22 @@
# Paimon
# Paimon (Incubating)

Paimon is a data lake storage for streaming updates/deletes changelog ingestion and high-performance queries in real time.
Paimon is a streaming data lake platform that supports high-speed data ingestion, change data tracking and efficient real-time analytics.

Paimon is developed under the umbrella of [Apache Flink](https://flink.apache.org/).
Background and documentation is available at https://paimon.apache.org

## Documentation & Getting Started
Paimon's former name was Flink Table Store, developed from the Flink community. The architecture refers to some design concepts of Iceberg.
Thanks to Apache Flink and Apache Iceberg.

Please check out the full [documentation](https://nightlies.apache.org/flink/paimon-docs-master/), hosted by the
[ASF](https://www.apache.org/), for detailed information and user guides.
## Collaboration

Check our [quick-start](https://nightlies.apache.org/flink/paimon-docs-master/docs/try-paimon/quick-start/) guide for simple setup instructions to get you started with the paimon.
Paimon tracks issues in GitHub and prefers to receive contributions as pull requests.

## Building

Run the `mvn clean package -DskipTests` command to build the project.
Community discussions happen primarily on the [dev mailing list](mailto:[email protected]) or on specific issues.

Then you will find a JAR file for Flink engine with all shaded dependencies: `paimon-flink/paimon-flink-**/target/paimon-flink-**-<version>.jar`.

## Contributing
## Building

You can learn more about how to contribute on the [Apache Flink website](https://flink.apache.org/contributing/how-to-contribute.html). For code contributions, please read carefully the [Contributing Code](https://flink.apache.org/contributing/contribute-code.html) section for an overview of ongoing community work.
- Run the `mvn clean package -DskipTests` command to build the project.
- Run the `mvn spotless:apply` to format the project (both Java and Scala).

## License

Expand Down
23 changes: 10 additions & 13 deletions docs/content/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,15 @@ under the License.

# Apache Paimon

Paimon is a unified storage to build dynamic tables for both streaming and
batch processing in Flink, supporting high-speed data ingestion and timely data query.
Apache Paimon(incubating) is a streaming data lake platform that supports high-speed data ingestion, change data tracking and efficient real-time analytics.

Paimon offers the following core capabilities:
- Support storage of large datasets and allow read/write in both batch and streaming mode.
- Support streaming queries with minimum latency down to milliseconds.
- Support Batch/OLAP queries with minimum latency down to the second level.
- Support incremental snapshots for stream consumption by default. So users do not need to combine different pipelines by themself.

- Unified Batch & Streaming: Paimon supports batch write and batch read, as well as streaming write changes and streaming read table changelogs.
- Data Lake: As a data lake storage, Paimon has the following advantages: low cost, high reliability, and scalable metadata.
- Merge Engines: Paimon supports rich Merge Engines. By default, the last entry of the primary key is reserved. You can also use the "partial-update" or "aggregation" engine.
- Changelog producer: Paimon supports rich Changelog producers, such as "lookup" and "full-compaction". The correct changelog can simplify the construction of a streaming pipeline.
- Append Only Tables: Paimon supports Append Only tables, automatically compact small files, and provides orderly stream reading. You can use this to replace message queues.

{{< columns >}}
## Try Paimon
Expand All @@ -43,12 +45,7 @@ step introduction to the APIs and guides you through real applications.

## Get Help with Paimon

If you get stuck, check out our [community support
resources](https://flink.apache.org/community.html). In particular, Apache
Flink’s user mailing list is consistently ranked as one of the most active of
any Apache project, and is a great way to get help quickly.
If you get stuck, you can subscribe User Mailing List ([email protected]),
Paimon tracks issues in GitHub and prefers to receive contributions as pull requests. You can also create an issue.

{{< /columns >}}

Paimon is developed under the umbrella of
[Apache Flink](https://flink.apache.org/).
9 changes: 3 additions & 6 deletions docs/content/concepts/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,7 @@ under the License.

# Overview

Paimon is a unified storage to build dynamic tables for both streaming and
batch processing in Flink, supporting high-speed data ingestion and timely data query.
Apache Paimon(incubating) is a streaming data lake platform that supports high-speed data ingestion, change data tracking and efficient real-time analytics.

## Architecture

Expand All @@ -46,10 +45,8 @@ As shown in the architecture above:
**Ecosystem:** In addition to Apache Flink, Paimon also supports read by other computation
engines like Apache Hive, Apache Spark and Trino.

**Internal:** Under the hood, Paimon uses a hybrid storage architecture with a lake format to store
historical data and a queue system to store incremental data. The former stores the columnar files on
the filesystem/object-store and uses the LSM tree structure to support a large volume of data updates
and high-performance queries. The latter uses Apache Kafka to capture data in real-time.
**Internal:** Under the hood, Paimon stores the columnar files on the filesystem/object-store and uses
the LSM tree structure to support a large volume of data updates and high-performance queries.

## Unified Storage

Expand Down
Binary file modified docs/static/img/architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 6430b6f

Please sign in to comment.