Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core-clp): Add LZMA Compressor implementation and LZMA dependency. #614

Merged
merged 68 commits into from
Dec 23, 2024
Merged
Changes from 1 commit
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
dceb564
Add lzma download and port lzma scripts
Bill-hbrhbr Nov 25, 2024
d5af274
Make unit test pass
Bill-hbrhbr Nov 25, 2024
b94ca26
Refactor lzma compressor to group common functionalities into helplers
Bill-hbrhbr Nov 26, 2024
707c412
Improve comments
Bill-hbrhbr Nov 27, 2024
6d1ab8f
Fix reference link
Bill-hbrhbr Nov 27, 2024
89b5707
Add install for CentOS
Bill-hbrhbr Nov 27, 2024
c646cea
Apply coderabbit suggestions
Bill-hbrhbr Nov 27, 2024
c91e5fb
Remove decompressor related files
Bill-hbrhbr Nov 27, 2024
26b0663
Address review concerns
Bill-hbrhbr Nov 30, 2024
740bc1c
Address review concern
Bill-hbrhbr Dec 2, 2024
e2be883
Simplify else-if
Bill-hbrhbr Dec 2, 2024
905367d
Fix else-if
Bill-hbrhbr Dec 2, 2024
8ae88b2
Add lzma (xz) dep to MacOS
Bill-hbrhbr Dec 2, 2024
0d0c20e
Refactor helper run_lzma()
Bill-hbrhbr Dec 2, 2024
559485d
Update function doc
Bill-hbrhbr Dec 2, 2024
7c69c69
Clarify unit test early termination
Bill-hbrhbr Dec 2, 2024
a6d68b8
Update components/core/tests/test-StreamingCompression.cpp
Bill-hbrhbr Dec 2, 2024
1519c21
Split LZMA_RUN from flush actions
Bill-hbrhbr Dec 3, 2024
655bb46
Refactor unit test
Bill-hbrhbr Dec 3, 2024
4fb6c01
Update components/core/src/clp/streaming_compression/lzma/Compressor.cpp
Bill-hbrhbr Dec 3, 2024
a8799b5
Merge edits
Bill-hbrhbr Dec 3, 2024
2b85f01
Fix import
Bill-hbrhbr Dec 3, 2024
eda7d6c
Apply suggestions from code review
Bill-hbrhbr Dec 4, 2024
4164a9d
Address review concern
Bill-hbrhbr Dec 4, 2024
8ab0653
Add a comment
Bill-hbrhbr Dec 5, 2024
c436f21
Apply suggestions from code review
Bill-hbrhbr Dec 6, 2024
7bd34d2
Update comment to 100-char length
Bill-hbrhbr Dec 6, 2024
efd2b27
Fix according to coding style guidelines
Bill-hbrhbr Dec 11, 2024
c530f92
Apply suggestions from code review
Bill-hbrhbr Dec 12, 2024
e751ee6
Update CMakeLists.txt
Bill-hbrhbr Dec 12, 2024
1c5efcd
Address review concern
Bill-hbrhbr Dec 12, 2024
856c7cb
Update TODO
Bill-hbrhbr Dec 12, 2024
43e22d2
Case fix
Bill-hbrhbr Dec 12, 2024
829a6b2
Remove unnecessary function inline comments
Bill-hbrhbr Dec 12, 2024
81e1807
Improve comment
Bill-hbrhbr Dec 12, 2024
09b73c7
Refactor lzma stream related functions into a nested helper class
Bill-hbrhbr Dec 17, 2024
7cedb25
Adress coderabbit suggestions
Bill-hbrhbr Dec 19, 2024
3dbe388
feat(clp-package): Add support for deleting archives that are exclusi…
haiqi96 Nov 27, 2024
5d90054
feat(clp-s): Add the write path for single-file archives. (#563)
wraymo Nov 27, 2024
2b88c6f
test: Allow multiple trials when unittesting http headers (#613)
anlowee Nov 28, 2024
290ede3
chore(log-viewer-webui): Update `yscope-log-viewer` to the latest ver…
junhaoliao Nov 29, 2024
2c0e053
test(clp-s): Add end-to-end test case for compression and extraction.…
AVMatthews Dec 2, 2024
cbf8bf9
docs(clp-json): Update list of characters that requires escaping in q…
gibber9809 Dec 2, 2024
44b0f2b
feat(core): Add `ErrorCode` template to standardize conversion of use…
LinZhihao-723 Dec 2, 2024
4d21d9b
revert(core): Remove temporary output directory option from `clp` and…
haiqi96 Dec 5, 2024
36892c1
refactor(clp-package): Unify the metadata schema for JSON and IR stre…
haiqi96 Dec 6, 2024
604bd75
feat(clp-s): Add command line options for stubbed out kv-pair-IR inge…
AVMatthews Dec 9, 2024
0a9322b
feat(ffi): Add initial implementation of `IrErrorCode` (using the `Er…
LinZhihao-723 Dec 9, 2024
78a535c
feat(ffi): Add support for auto/user-generated KV-pairs in `KeyValueP…
LinZhihao-723 Dec 9, 2024
42db88c
build(docs): Update dependencies to latest versions. (#631)
kirkrodrigues Dec 10, 2024
13c7528
ci(pr-title-checks): Remove default GH workflow permissions and docum…
kirkrodrigues Dec 12, 2024
8b34dac
feat(core-clp): Add `BoundedReader` to prevent out-of-bound reads in …
gibber9809 Dec 13, 2024
02d8956
build(core): Update Boost to v1.87.0 in order to pull in boost::urls;…
gibber9809 Dec 16, 2024
da66fbf
refactor(clp-s): Replace instances of `std::string const&` with `std:…
gibber9809 Dec 16, 2024
b7741c0
docs(core): Indicate dependency install scripts should be run with el…
jackluo923 Dec 16, 2024
1edc16e
feat(package)!: Add support for writing clp-s single file archives to…
haiqi96 Dec 19, 2024
e4c9dd3
fix(clp-package): Remove faulty error handling for parsing archive co…
haiqi96 Dec 19, 2024
32dc989
fix(core): Add missing `../` to fix relative header file includes. (#…
Bill-hbrhbr Dec 19, 2024
02f0b8f
Refactor lzma stream and add some doc strings.
davidlion Dec 20, 2024
fcfc73a
Fix accidental comment reflow.
davidlion Dec 20, 2024
3b89d4c
Merge branch 'main' into pr-614
davidlion Dec 20, 2024
50c8e89
Merge branch 'main' into pr-614
davidlion Dec 20, 2024
d139bd5
Apply suggestions from code review
davidlion Dec 20, 2024
68c4c36
Add missing fixes for PR suggestion.
davidlion Dec 20, 2024
dcb843e
Address review concern
Bill-hbrhbr Dec 20, 2024
b20162f
Address review concern
Bill-hbrhbr Dec 20, 2024
df41b22
nit fix
Bill-hbrhbr Dec 20, 2024
524fe1d
Change all instances of programming-error-induced error codes to Erro…
Bill-hbrhbr Dec 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 2 additions & 9 deletions components/core/tests/test-StreamingCompression.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@

#include <boost/filesystem/operations.hpp>
#include <Catch2/single_include/catch2/catch.hpp>
#include <lzma.h>
#include <zstd.h>

#include "../src/clp/Array.hpp"
Expand All @@ -35,20 +34,14 @@ using std::string_view;
namespace {
constexpr string_view cCompressedFilePath{"test_streaming_compressed_file.bin"};
constexpr size_t cBufferSize{128L * 1024 * 1024}; // 128MB
// Interleave no-ops to ensure the integrity of the compressor states at all times.
constexpr auto cCompressionChunkSizes = std::to_array<size_t>(
{cBufferSize / 100,
0,
{0, // no-op
Bill-hbrhbr marked this conversation as resolved.
Show resolved Hide resolved
cBufferSize / 100,
cBufferSize / 50,
0,
cBufferSize / 25,
0,
cBufferSize / 10,
0,
cBufferSize / 5,
0,
cBufferSize / 2,
0,
cBufferSize}
);

Expand Down
Loading