From 3f20c803e468c22b59f29d53038de75e8e070f7b Mon Sep 17 00:00:00 2001 From: Guilherme Noronha Date: Mon, 16 Oct 2023 13:06:55 -0300 Subject: [PATCH] Update README.md --- README.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 3d9dadd..03d629d 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,16 @@ Package to convert parquet files into a single hyper file. ## Benchmarking -To be announced soon. +The package was benchmarked using a Latitude 3420 Dell Notebook with 16GB of Memory RAM, a 250GB SSD, and an i7-1165G7 CPU. The time may vary using different architectures. The table used for benchmarking contained **60 columns**. Although the tests were carried with a maximum of 500 million of rows, the package supports higher amount of volume. The limitation is only to the size of a single parquet file (up to 30GB). For larger volumes, it's recommended to split them into multiple parquet files. Follow the results: + +| Rows (in millions) | Time (in seconds) | Parquet size (in MegaBytes) | +|--------------------|-------------------|-----------------------------| +| 1 | 4.05 | 54 | +| 10 | 36.8 | 520 | +| 100 | 412.6 | 4900 | +| 500 | 2669.25 | 25400 | + +![parquet-to-hyper](https://github.com/guilhermenoronha/parquet-to-hyper/assets/2208226/f8c54a68-e689-4fb3-9d09-05122d220fee) ## How to use