Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft: Implement ishidden for ZFS filesystems #20

Open
wants to merge 17 commits into
base: master
Choose a base branch
from
Open

Conversation

jakewilliami
Copy link
Owner

@jakewilliami jakewilliami commented Sep 26, 2022

ZFS has some special considerations for hidden files, namely the mount points of ZFS,[1], [2] or a ZFS-classified hidden file.[3], [4]

Addresses #3, #12, #13.

Tasks:

  • Implement iszfs function
  • Implement _ishidden_zfs
    • Check if the file is a mount point for ZFS (i.e., is .zfs)
    • Check whether the file is hidden, from either z_pflags from znode,[5] xat_hidden from xvattr (which uses z_pflags),[6], [7], [8] or by obtaining the znode struct from a vnode[9], [10] and then parsing z_pflags.

Unfortunately, I am very limited by what functions I have exported within the shared object files. Trying to keep it in pure Julia, but we will see. I've already spent far too long researching this, so I've made the decision that this feature will be implemented after this package is registered/released, in the interest of getting the package out there for use (as this feature is probably pretty niche).

In Julia 1.6, nice libuv statfs bindings were added to Julia, which
allows system-independent statfs calls.  In Julia 1.8, this was made
even nicer by tucking all of that logic into a struct, behind
diskstat.  While I would like to use this, it would mean not
supporting versions < 1.6 for this functionality.  I will have to
think on it/ask some friends for their opinions.

I also did some research/testing into ZFS type values for once we have
the f_type field from statfs.

Finally, I have (and will continue to) do testing into
system-dependent offsets of the f_type field from statfs, in case we
can indeed write some janky solution that will work < 1.6.
Added C programme in comment for offset research.

macOS:
$ gcc -o statfs statfs.c && ./statfs
offsetof(struct statfs, f_type): 0x3c
st.f_type: 0x1a

BSD:
$ gcc11 statfs.c -o statfs && ./statfs
offsetof(struct statfs, f_type): 0x4
st.f_type: 0xde
@jakewilliami
Copy link
Owner Author

To relay some investigation from c95cb4f:

macOS:
$ gcc -o statfs statfs.c && ./statfs
offsetof(struct statfs, f_type): 0x3c
st.f_type: 0x1a

BSD:
$ gcc11 statfs.c -o statfs && ./statfs
offsetof(struct statfs, f_type): 0x4
st.f_type: 0xde

However, this is not always consistent with Julia:

macOS:
julia> UInt8(findfirst(==(0x1a), iszfs()))
0x45  # != 0x3c + 1 (Julia indexes from 1)

BSD:
julia> UInt8(findfirst(==(0xde), iszfs()))
0x05  # 0x04 + 1

@jakewilliami
Copy link
Owner Author

It appears that at offset 0x44 (or 0x45 in Julia) is the field f_fssubtype (which isn't even mentioned for statfs in the man page):

offsetof(struct statfs, f_fssubtype): 0x44

Hence, perhaps on macOS I should check both offsets.

I also finished cleaning up the macOS and BSD versions of iszfs.
Interestingly, macOS appears to have both f_type and f_fssubtype
fields set from statfs (despite the latter not actually being
documented.

I am still deciding whether or not our < 1.6 version is a good idea.
@jakewilliami
Copy link
Owner Author

For future reference, this is how I determine the offsets of flags:

#include <stdio.h>
#include <stddef.h>
#include <sys/param.h>
#include <sys/mount.h>

int main() {
    struct statfs st;
    printf("offsetof(struct statfs, f_type): 0x%lx\n", offsetof(struct statfs, f_type));
    statfs("/", &st);
    printf("st.f_type: 0x%llx\n", (uint64_t) st.f_type);
    return 0;
}

@jakewilliami
Copy link
Owner Author

The equivalent of #20 (comment) for Linux:

#include <stdio.h>
#include <stddef.h>
#include <sys/vfs>    /* or <sys/statfs.h> */

int main() {
    printf("sizeof(struct statfs): %ld\n", sizeof(struct statfs));
    printf("offsetof(struct statfs, f_type): 0x%lx\n", offsetof(struct statfs, f_type));
    struct statfs st;
    statfs("/", &st);
    printf("st.f_type: 0x%lx\n", st.f_type);
    return 0;
}

This produced:

Linux:
$ gcc -o statfs statfs.c && ./statfs
sizeof(struct statfs): 120
offsetof(struct statfs, f_type): 0x0
st.f_type: 0xef53

Which appears to be consistent within Julia:

julia> statfs("/").ftype
0x000000000000ef53

julia> _iszfs("/")[0x01]
0x0000ef53

The statfs buffer I implemented previously had elements that were
UInt8s, but after implementing Linux's _iszfs function, I realised
that some statfs calls needed to store larger-sized integers.  As
such, we can now dynamically change the element type of the buffer and
this will dynamically change the size of the array, and the
appropriate offset (cld because Julia is indexed by one)
These values did not need to be defined for each OS branch if Julia
version < 1.6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant