Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

G3SkyMapMask is not efficiently stored on disk #107

Closed
arahlin opened this issue Jun 1, 2023 · 0 comments · Fixed by #108
Closed

G3SkyMapMask is not efficiently stored on disk #107

arahlin opened this issue Jun 1, 2023 · 0 comments · Fixed by #108
Assignees
@nwhitehorn
Labels
bug

Comments

@arahlin
Copy link
Member

arahlin commented Jun 1, 2023 •
edited
Loading

G3SkyMap mask objects, which are essentially std::vector<bool> objects with some extra features, are packed on disk as one byte per element, rather than 1 bit per element as one would expect. This is rather inefficient, especially when masks are used as input arguments to pipeline modules (and therefore end up in memory-intensive G3PipelineInfo frames all over the place).
@arahlin arahlin added the bug label Jun 1, 2023
@nwhitehorn nwhitehorn mentioned this issue Jun 2, 2023
Merged
nwhitehorn added a commit that referenced this issue Jun 2, 2023
@nwhitehorn
Make serialized masks more compact. (#108)
75fc54b 
Cereal, by default, serializes std::vector<bool> with one byte per bool.
Since only one bit is relevant, this uses eight times the disk space that
is strictly needed. Pack the bits instead.

Closes #107.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nwhitehorn nwhitehorn

Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Make serialized masks more compact. CMB-S4/spt3g_software
2 participants
@arahlin @nwhitehorn