Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support discontinuous memory maps #30

Closed
willglynn opened this issue Jan 15, 2017 · 3 comments
Closed

Support discontinuous memory maps #30

willglynn opened this issue Jan 15, 2017 · 3 comments

Comments

@willglynn
Copy link

willglynn commented Jan 15, 2017

I'm working with a file format that scatters its data around on disk. It's composed of multiple logical data streams which are intended to be memory mapped. The segments are all page-aligned, and records within a stream constantly refer to other records based on their positions within the stream entirely oblivious to the physical layout.

Ultimately I want a single continuous read-only &[u8] given a list of offsets and sizes:

0x462C000:   0x1000 bytes
0x473D000: 0x8C4000 bytes
0x5003000: 0xFFE000 bytes
0x6003000: 0x29E000 bytes

In the case of POSIX mmap(), I think this would be done by using mmap(NULL, <total size>, MAP_ANON, PROT_NONE, 0, 0) to ask for a chunk of address space without actually committing resources to it, followed by repeated mmap(<address>, <size>, MAP_FIXED, PROT_READ, <fd>, <offset>) to map each file piece to the appropriate location within that space.

On Windows, I think this is VirtualAllocEx(MEM_RESERVE) to get the address space, then MapViewOfFileEx() to map each piece.

@m4b
Copy link

m4b commented Jan 18, 2017

Here's how a unix dynamic linker might do this:

  1. first reserve the entire breadth of space: https://github.com/m4b/dryad/blob/585189f3986c284aad52701d6de944a06192ab01/src/loader.rs#L47-L70
  2. then mmap chunks with the FIXED flag into the reserved space: https://github.com/m4b/dryad/blob/585189f3986c284aad52701d6de944a06192ab01/src/loader.rs#L141-L156

E.g., the program headers need to be mmap'd into different locations, but the code + data reference each other assuming it's essentially continguous memory, I think this is similar to your requirements.

Some sugar to do this in this lib would be nice though :)

@retep998
Copy link
Contributor

retep998 commented May 2, 2017

This is not going to happen on Windows. To quote MSDN:

No other memory allocation can take place in the region that is used for mapping, including the use of the VirtualAlloc or VirtualAllocEx function to reserve memory.

@danburkert
Copy link
Owner

The only thing I think we can really do here is expose MAP_FIXED, which is already being tracked in #21 , so I'm going to go ahead and close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants