Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added zip_entry_noallocreadwithoffset #359

Merged
merged 5 commits into from
Nov 29, 2024
Merged

Conversation

JCash
Copy link
Contributor

@JCash JCash commented Nov 29, 2024

The new function zip_entry_noallocread_offset() will allow us to stream content from a zip file.

I know there was no discussion beforehand, so consider this as a draft and discussion material.

src/zip.c Show resolved Hide resolved
src/zip.c Outdated
return (ssize_t)ZIP_ENORITER;
}

mz_uint8 tmpbuf[ZIP_DEFAULT_ITER_BUF_SIZE];
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a define for users to control their stack usage.

src/zip.c Outdated Show resolved Hide resolved
src/zip.h Outdated
/**
* Default zip iterator stack size (in bytes)
*/
#define ZIP_DEFAULT_ITER_BUF_SIZE 32*1024
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's a good default size?

src/zip.c Outdated
read_size = to_read;
MZ_ASSERT(read_size <= tmpbuf_size);

memcpy(&((mz_uint8*)buf)[write_cursor], &tmpbuf[read_cursor], read_size);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need this tmpbuf, and memcpy here?
mz_zip_reader_extract_iter_read already calls memcpy. Maybe we can optimize it a litle bit, if we can get rid of tmpbuf.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, the question is, where should mz_zip_reader_extract_iter_read() copy to?

Perhaps you meant that the caller also provides the tempbuffer?
Something like:

ssize_t zip_entry_noallocread_offset(struct zip_t *zip,
                                    size_t offset, size_t size, void *buf,
                                    size_t tmpbufsize, void *tmpbuf)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get what you mean now.
I think it should be possible.
It will likely still need a memmove, to put any "unaligned" bytes from the middle of the buffer, into the start of the buffer.

src/zip.h Outdated
* @return the return code - the number of bytes actually read on success.
* Otherwise a negative number (< 0) on error (e.g. offset is too large).
*/
extern ZIP_EXPORT ssize_t zip_entry_noallocread_offset(struct zip_t *zip,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe to follow similar API, we can name it zip_entry_noallocreadwithoffset. I know it's not the prettiest name, but at least it's consistent with other API ...witherror, ...withindex, etc.

@JCash JCash requested a review from kuba-- November 29, 2024 13:24
@JCash JCash changed the title Added zip_entry_noallocread_offset Added zip_entry_noallocreadwithoffset Nov 29, 2024
@kuba--
Copy link
Owner

kuba-- commented Nov 29, 2024

Thanks, could you just reformat a code by running:

for file in $(git ls-files | \grep -E '\.(c|h)$' | \grep -v -- '#')
do
    clang-format -i $file
done

@JCash JCash requested a review from kuba-- November 29, 2024 15:06
@kuba-- kuba-- merged commit a91b5a3 into kuba--:master Nov 29, 2024
4 checks passed
@JCash JCash deleted the read-iter branch November 29, 2024 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants