Skip to content

Latest commit

 

History

History
250 lines (177 loc) · 7.99 KB

objfs.1.asciidoc

File metadata and controls

250 lines (177 loc) · 7.99 KB
Raw

objfs(1)

NAME

objfs - object storage file system

SYNOPSIS

objfs [-options] command args...

DESCRIPTION

The objfs program implements the “object storage file system”.

Objfs exposes objects from an object storage, such as a cloud drive, etc. as files in a file system that is fully integrated with the operating system. Programs that run on the operating system are able to access these files as if they are stored in a local "drive" (perhaps with some delay due to network operations).

Objfs accepts commands such as auth and mount, but also shell-like commands, such as ls, stat, etc.

Objfs configures itself on every run according to these sources of configuration in order of precedence:

  • Command line options.

  • A configuration file, which is found in a platform-specific location (unless overriden by the -config option).

  • Sensible defaults.

Default Storage

Objfs uses defaults to simplify command line invocation. In the default build of objfs, the default storage is onedrive.

Auth

Objfs supports multiple "auth" (authentication or authorization) mechanisms through the -credentials path option and the auth command.

In general before an object storage service can be used it requires auth. The specific auth mechanism used depends on the service and it ranges from no auth, to username/password, to Oauth2, etc. Auth mechanisms require credentials, which can be supplied using the -credentials path option.

In some cases the object storage service cannot readily accept the supplied credentials, they must be converted to other credentials first. As an authentication example, a particular service may require username/password credentials to be converted to some form of service-level token before they can be used. As an authorization example Oauth2 requires application-level credentials together with user consent to form a service-level token that can be used to access the service.

The auth command can be used for this purpose. It takes user-level or application-level credentials and converts them to service-level credentials.

Credentials can be stored in the local file system or the system keyring. The syntax /file/path is used to name credentials stored in the file system. The syntax keyring:service/user is used to name credentials stored in the system keyring.

Example - Oauth2 Flow

  • Prepare the Oauth2 client_secret credentials in a file or the system keyring:

    client_id="XXXXXXXX"
client_secret="XXXXXXXX"
redirect_uri="http://localhost:xxxxx"
scope="files.readwrite.all offline_access"

  • Issue the command:

    $ ./objfs -credentials=CLIENT_SECRET_PATH auth TOKEN_PATH

  • This will launch your browser and ask for authorization. If the access is authorized the Oauth2 access_token and refresh_token will be stored in the specified path.

  • The object storage can now be mounted using the command:

    $ ./objfs -credentials=TOKEN_PATH mount MOUNTPOINT

Mount

The objfs mount command is used to mount an object storage as a file system on a mountpoint. On Windows the mount point must be a non-existing drive or directory; it is recommended that an object storage is only mounted as a drive when the object storage is case-sensitive. On macOS and Linux the mount point must be an existing directory.

To mount on Windows:

> objfs -credentials=TOKEN_PATH mount -o uid=-1,gid=-1 mount X:

To mount on macOS and Linux:

$ ./objfs -credentials=TOKEN_PATH mount MOUNTPOINT

Objfs uses a local file cache to speed up file system operations. This caches files locally when they are first opened; subsequent I/O operations will be performed against the local file and are therefore fast. Modified files will be uploaded to the object storage when they are closed. File system operations such as creating and deleting files and listing directories are sent directly to the object storage and are therefore slow (although some of their results are cached).

The Objfs cache was inspired by an early version of the Andrew File System (AFS). For more information see the paper http://pages.cs.wisc.edu/~remzi/OSTEP/dist-afs.pdf.

Diagnostics

Objfs includes a tracing facility that can be used to troubleshoot problems, to gain insights into its internal workings, etc. This facility is enabled when the -v option is used.

The environment variable GOLIB_TRACE controls which traces are enabled. This variable accepts a comma separated list of file-style patterns containing wildcards such as * and ?.

$ export GOLIB_TRACE=pattern1,...,patternN

Examples:

$ export GOLIB_TRACE=github.com/billziss-gh/objfs/fs.*      # file system traces
$ export GOLIB_TRACE=github.com/billziss-gh/objfs/objio.*   # object storage traces
$ export GOLIB_TRACE=github.com/billziss-gh/objfs/fs.*,github.com/billziss-gh/objfs/objio.*
$ ./objfs -v -credentials=TOKEN_PATH mount MOUNTPOINT


COMMANDS

The following commands may be used:

version

get current version information

config

get or set configuration options

keyring

get or set keys

auth output-credentials

perform authentication/authorization

mount [-o option...] mountpoint

mount file system

statfs

get storage information

ls [-l][-n count] path...

list files

stat [-l] path...

display file information

mkdir path...

make directories

rmdir path...

remove directories

rm path...

remove files

mv oldpath newpath

move (rename) files

get [-r range][-s signature] path [local-path]

get (download) files

put [local-path] path

put (upload) files

cache-pending

list pending cache files

cache-reset

reset cache (upload and evict files)

GENERAL OPTIONS

The following options apply to all commands:

-accept-tls-cert

accept any TLS certificate presented by the server (insecure)

-auth name

auth name to use

-config path

path to configuration file

-credentials path

auth credentials path (keyring:service/user or /file/path)

-datadir path

path to supporting data and caches

-keyring string

keyring type to use: system, private (default "private")

-storage name

storage name to access (default "onedrive")

-storage-uri uri

storage uri to access

-v

verbose

CONFIGURATION FILE

During startup objfs consults a congifuration file from a platform-specific location (see the FILES section); this location can be overriden with the -config option.

The configuration file stores a list of properties (key/value) pairs, that may also be grouped into sections. The basic syntax of the configuration file is as follows:

name1=value1
name2=value2
...
[section]
name3=value3
name4=value4
...

The valid property names are a subset of the command-line options: auth, credentials, storage, storage-uri. They specify the same value as the equivalent command-line option.

The command line option or property storage may specify the name of a storage service (e.g. onedrive), but it may also specify a section within the configuration file, which should be used to retrieve additional configuration options. For example, given the configuration file below and a command line option -storage=onedrive2, it will instruct objfs to act on the OneDrive storage identified by the credentials keyring:objfs/onedrive2:

[onedrive1]
storage=onedrive
credentials=keyring:objfs/onedrive1

[onedrive2]
storage=onedrive
credentials=keyring:objfs/onedrive2


FILES

Windows

  • config: %USERPROFILE%\AppData\Roaming\objfs.conf

  • datadir: %USERPROFILE%\AppData\Roaming\objfs

macOS

  • config: ~/Library/Preferences/objfs.conf

  • datadir: ~/Library/Application Support/objfs

Linux

  • config: ~/.config/objfs.conf

  • datadir: ~/.local/share/objfs


COPYRIGHT

© 2018 Bill Zissimopoulos