Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

initial use case/feature updates, needs discussion #113

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

initial use case/feature updates, needs discussion #113

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f16733c
initial use case/feature updates, needs discussion
mjpritchard Oct 22, 2024
427accf
updates per meeting with Neil
mjpritchard Oct 23, 2024
f7a986e
updates per meeting with Neil
mjpritchard Oct 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 53 additions & 35 deletions docs/spec/GWS_use_case.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,35 +1,53 @@
JASMIN USER copies a file from their GWS disk to the NLDS.

JASMIN USER recursively copies a directory from their GWS disk to the NLDS.

JASMIN USER copies a specified set of files and/or directories to the NLDS by specifying an include file.

JASMIN USER retrieves a file from the NLDS to their GWS disk.

JASMIN USER retrieves a directory from the NLDS to their GWS disk.

JASMIN USER copies a file from their GWS disk, and already present in the NLDS, as an incremental update to that file.

JASMIN USER retrieves a particular copy of a file from the NLDS to their GWS disk.

JASMIN USER lists their holdings and files on the NLDS. A holding is a collection of files (at the user view).

JASMIN USER monitors their transactions to/from NLDS.

JASMIN USER adds or amends metadata to a holding on the NLDS. e.g. tags, label.

JASMIN USER verifies their data (checksum).

JASMIN USER specifies which quota to use when copying to NLDS.

JASMIN USER checks the value of the remaining quota and the usage against the quota.

JASMIN MANAGER copies a file belonging to another user from their GWS to the NLDS.

JASMIN MANAGER deletes a file belonging to another from their GWS, from the NLDS.

JASMIN MANAGER (can do all the above actions for USER, for data belonging to another USER in the same GWS)

Storing data on the tape requires quota remaining, allocated to GWS. Quota is from the project portal.

? Other types of quota as well as GWS quota, for example a personal quota ?
## GWS Use cases

Showing version number when this feature will be available from a client a) on JASMIN b) from remote location

Feature | v JASMIN | v remote | notes
---|---|---|---
**User PUT**|||
JASMIN USER copies a file from their GWS disk to the NLDS. | 1.0 | 1.3 |
JASMIN USER recursively copies a directory from their GWS disk to the NLDS. | 1.0 | 1.3 |
JASMIN USER copies a specified set of files and/or directories to the NLDS by specifying an include file. | 1.0 | 1.3 |
JASMIN USER copies a file from their GWS disk already present in the NLDS, as an incremental update to that file. | 1.0 | 1.3 |
JASMIN USER specifies the GWS to which their PUT will be assigned, when PUTting to NLDS, | 1.0 | 1.3 | quotas enforced in 1.1
**User GET**|||
JASMIN USER retrieves a file from the NLDS to their GWS disk. | 1.0 | 1.3 |
JASMIN USER retrieves a directory from the NLDS to their GWS disk. | 1.0 | 1.3 |
JASMIN USER retrieves a specified set of files and/or directories from the NLDS by specifying an include file. | 1.0 | 1.3 |
JASMIN USER retrieves data belonging to another user in the same GWS | 1.0 | 1.3 |
JASMIN USER retrieves a particular copy of a file identified by holding ID or label, from the NLDS to their GWS disk | 1.0 | 1.3 |
**User STAGE**|||
JASMIN USER stages a file from the NLDS to the cache. | 1.2 | 1.3 | Does this require a user-level cache quota, in addition to the GWS:cache quota?
JASMIN USER stages a directory from the NLDS to the cache. | 1.2 | 1.3 | Can they set policy (residence time in cache?)
JASMIN USER stages a specified set of files and/or directories by specifying an include file. | 1.2 | 1.3 |
JASMIN USER 2 in the same GWS as JASMIN USER 1, accesses files staged by JASMIN USER 1 | 1.2 | 1.3 |
File staged by a user are purged according to policy.
JASMIN USER accesses files in persistent staging location, for successive stage/unstage cycles | 1.2 | 1.3 |
Access control list is created on PUT, giving access to ALL users to staged data (initial placeholder functionality) | 1.2 | 1.3 | Rationale: if it's there, users can edit it to restrict as they wish (via S3 portal)
Access control lists (ACLs) created on PUT, that restrict URL access to staged data, to members of the GWS | 1.2 | 1.3 |
**User DELETE**|||
JASMIN USER deletes a specified file from the NLDS. | 1.1 | 1.3 | requires roles
JASMIN USER recursively deletes a directory from the NLDS. | 1.1 | 1.3 | requires roles
JASMIN USER deletes a specified set of files and/or directories from the NLDS by specifying an include file. | 1.1 | 1.3 | requires roles
**User Misc functions**|||
JASMIN USER lists their holdings and files on the NLDS. A holding is a collection of files (at the user view). | 1.0 | 1.3 |
JASMIN USER monitors their transactions to/from NLDS. | 1.0 | 1.3 | requires API for polling ingest status
JASMIN USER adds or amends metadata to a holding on the NLDS. e.g. tags, label. | 1.0 | 1.3 |
JASMIN USER verifies their data (checksum). | 1.1 | 1.3 |
JASMIN USER checks usage against quota for GWSs they belong to (GWS:tape) | 1.1 | 1.3 | requires GWS:tape quota
JASMIN USER cancels a command currently in progress | 1.2 | 1.3 |
**Management funtions** (for users with MANAGER or DEPUTY role on a GWS)|||
JASMIN MANAGER copies a file belonging to another user from their GWS to the NLDS. | 1.0 | 1.3 | Requires read access to file in question, but that (MANAGER) user would then own that holding in NLDS.
JASMIN MANAGER deletes a file belonging to another USER from their GWS, from the NLDS. | 1.1 | 1.3 | Requires DELETE and roles
JASMIN USER PUT has a default tape pool determined by contacting the projects portal to get the name of the default pool for that GWS | 1.1 | 1.3 | requires default tape pool as additional field on the GWS service in the PP

## Quotas

Types of quota:

- `GWS:tape` - quota for tape (aka archive) storage in the NLDS, per GWS. How much tape media the project is allowed to consume. Covers all holdings belinging to that GWS.
- Time-based policy for purging "cold" data from the cache. Possibility of preferentially purging data from **own** GWS data in cache, TBD.

### staging (TBD)
It **may** transpire that we need data staged for some minimum period, in which case an additional quota may be needed for this, at the GWS level:
- `GWS:cache` - quota for cache storage in the NLDS, per GWS.
Loading