diff --git a/docs/data/osdf/install-cache-rpm.md b/docs/data/osdf/install-cache-rpm.md new file mode 100644 index 000000000..09123f35d --- /dev/null +++ b/docs/data/osdf/install-cache-rpm.md @@ -0,0 +1,177 @@ +title: Installing the OSDF Cache by RPM + +Installing the OSDF Cache +========================= + +This document describes how to install a Pelican-based Open Science Data Federation (OSDF) Cache service via RPMs. +This service allows a site or regional network to cache data frequently used in Open Science Pool jobs, +reducing data transfer over the wide-area network and decreasing access latency. + + +Before Starting +--------------- + +Before starting the installation process, consider the following requirements: + +* __Operating system:__ A RHEL 8 or RHEL 9 or [compatible operating system](../../release/supported_platforms.md). +* __User IDs:__ If it does not exist already, the installation will create the Linux user named `xrootd` for running daemons. +* __File Systems:__ The cache should have a partition of its own for storing data and metadata. +* __Host certificate:__ Required for authentication. See note below. +* __Network ports:__ The cache service requires the following ports open: + * Inbound TCP port 8443 for file access via the HTTP(S) and XRoot protocols. + * (Optional) Inbound TCP port 8444 for access to the web interface for monitoring and configuration; + if enabled, access to this port should be restricted to the LAN. +* __Service requirements:__ + * A cache serving the OSDF federation as a regional cache should have at least: + * 8 cores + * 40 Gbps connectivity + * 50-200 TB of NVMe disk for the cache partition; you may distribute the disk, e.g., by using an NVMe-backed Ceph pool, + if you cannot fit that much disk into a single chassis + * 24 GB of RAM + * A cache being used to serve data from the OSDF to a single site should have at least: + * 8 cores + * 40 Gbps connectivity + * 2 TB of NVMe disk for the cache partition + * 24 GB of RAM + We suggest that several gigabytes of local disk space be available for log files, + although some logging verbosity can be reduced. + +As with all OSG software installations, there are some one-time steps to prepare in advance: + +* Obtain root access to the host +* Prepare [the required Yum repositories](../../common/yum.md) + + +!!! note "OSG 23" + In OSG 23, the Pelican-based OSDF RPMs are only available in the "osg-upcoming" repositories. + +!!! note "Host certificates" + Caches should use a CA that is accepted by major browsers and operating systems, + such as InCommon RSA or [Let's Encrypt](../../security/host-certs/lets-encrypt). + IGTF certs are not recommended because clients are not configured to accept them by default. + Note that you will need the full certificate chain, not just the certificate. + + The following locations should be used (note that they are in separate directories): + + * **Host Certificate Chain**: `/etc/pki/tls/certs/pelican.crt` + * **Host Key**: `/etc/pki/tls/private/pelican.key` + + +Installing the Cache +-------------------- + +The cache service is provided by the `osdf-cache` RPM. +Install it using one of the following commands: + + +OSG 24: +```console +root@host # yum install osdf-cache --enablerepo=osg-testing +``` + +OSG 23: +```console +root@host # yum install --enablerepo=osg-upcoming-testing osdf-cache +``` + + +Configuring the Cache Server +---------------------------- + +!!! note "osdf-cache 7.11.1" + This configuration requires version 7.11.1 or newer of the `osdf-cache` + and `pelican` RPMs. + +In `/etc/pelican/config.d/20-cache.yaml`, set `Cache.LocalRoot`, `Cache.DataLocation` and `Cache.MetaLocation` as follows, +replacing `` with the mount point of the partition you will use for the cache. +``` +Cache: + LocalRoot: "/namespaces" + DataLocation: "/data" + MetaLocation: "/meta" +``` + + +Preparing for Initial Startup +----------------------------- + +1. The cache identifies itself to the federation via public key authentication; +before starting the cache for the first time, it is generate a keypair. + + :::console + root@host$ cd /etc/pelican + root@host$ pelican generate keygen + + + The newly created files, `issuer.jwk` and `issuer-pub.jwks` are the private and public keys, respectively. + +1. **Save these files**; if you lose them, your cache will need to be re-approved. + + +Validating the Cache Installation +--------------------------------- + +Do the following steps to verify that the cache is functional: + +1. Start the cache using the following command: + + :::console + root@host$ systemctl start osdf-cache + +1. Download a test file from the OSDF through your cache (replacing `CACHE_HOSTNAME` with the host name of your cache) + + :::console + root@host$ osdf object get -c CACHE_HOSTNAME:8443 /ospool/uc-shared/public/OSG-Staff/validation/test.txt /tmp/test.txt + root@host$ cat /tmp/test.txt + + Hello, World! + + If the download fails, rerun the above `osdf object get` command with the `-d` flag added; + additional debugging information is located in `/var/log/pelican/osdf-cache.log`. + See [this page](../../common/help.md) for requesting assistance; please include the log file + and the `osdf object get -d` output in your request. + + +Joining the Cache to the Federation +----------------------------------- + +The cache must be registered with the OSG prior to joining the data federation. +Send mail to requesting registration; provide the following information: + +* Cache hostname +* Administrative and security contact(s) +* Institution that the cache belongs to + +OSG Staff will register the cache and respond with the Resource Name that the cache was registered as. + +Once you have that information, edit `/etc/pelican/config.d/15-osdf.yaml`, and set `XRootD.Sitename`: +``` +XRootD: + Sitename: +``` + +Then, restart the cache by running + +```console +root@host$ systemctl restart osdf-cache +``` + +Let OSG Staff know that you have restarted the cache with the updated sitename, +so they can approve the new cache. + + +Managing the Cache Service +--------------------------- +Use the following SystemD commands as root to start, stop, enable, and disable the OSDF Cache. + +| To... | Run the command... | +| :--------------------------------------- | :--------------------------------- | +| Start the cache | `systemctl start osdf-cache` | +| Stop the cache | `systemctl stop osdf-cache` | +| Enable the cache to start on boot | `systemctl enable osdf-cache` | +| Disable the cache from starting on boot | `systemctl disable osdf-cache` | + + +Getting Help +------------ +To get assistance, please use the [this page](../../common/help.md). diff --git a/docs/data/osdf/install-origin-rpm.md b/docs/data/osdf/install-origin-rpm.md new file mode 100644 index 000000000..4870e65f1 --- /dev/null +++ b/docs/data/osdf/install-origin-rpm.md @@ -0,0 +1,172 @@ +title: Installing the OSDF Origin by RPM + +Installing the OSDF Origin +========================== + +This document describes how to install a Pelican-based Open Science Data Federation (OSDF) Origin service via RPMs. +This service allows an organization to export its data to the Data Federation. + +!!! note + The origin must be registered with the OSG prior to joining the data federation. + You may start the registration process prior to finishing the installation by [using this link](#registering-the-origin) + along with information like: + + * Resource name and hostname + * VO associated with this origin server (which will be used to determine the origin's namespace prefix) + * Administrative and security contact(s) + * Who (or what) will be allowed to access the VO's data + * Which caches will be allowed to cache the VO data + + +Before Starting +--------------- + +Before starting the installation process, consider the following requirements: + +* __Operating system:__ A RHEL 8 or RHEL 9 or compatible operating systems. +* __User IDs:__ If they do not exist already, the installation will create the Linux user ID `xrootd` for running daemons. +* __Host certificate:__ Required for authentication. See note below. +* __Network ports:__ The origin service requires the following ports open: + * Inbound TCP port 8443 for file access via the HTTP(S) and XRoot protocols. + * (Optional) Inbound TCP port 8444 for access to the web interface for monitoring and configuration; + if enabled, this should be restricted to the LAN or management network. +* __Hardware requirements:__ We recommend that an origin has at least 1Gbps connectivity and 12GB of RAM. + We suggest that several gigabytes of local disk space be available for log files, + although some logging verbosity can be reduced. + +As with all OSG software installations, there are some one-time steps to prepare in advance: + +* Obtain root access to the host +* Prepare [the required Yum repositories](../../common/yum.md) + + +!!! note "OSG 23" + In OSG 23, the Pelican-based OSDF RPMs are only available in the "osg-upcoming" repositories. + +!!! note "Host certificates" + Origins should use a CA that is accepted by major browsers and operating systems, + such as InCommon RSA or [Let's Encrypt](../../security/host-certs/lets-encrypt). + IGTF certs are not recommended because clients are not configured to accept them by default. + Note that you will need the full certificate chain, not just the certificate. + + The following locations should be used (note that they are in separate directories): + + * **Host Certificate Chain**: `/etc/pki/tls/certs/pelican.crt` + * **Host Key**: `/etc/pki/tls/private/pelican.key` + + +Installing the Origin +--------------------- + +The origin service is provided by the `osdf-origin` RPM. +Install it using the following command: + + +```console +root@host # yum install --enablerepo=osg-upcoming osdf-origin +``` + + +Configuring the Origin Server +----------------------------- + +Configuration for a Pelican-based OSDF Origin is located in `/etc/pelican/osdf-origin.yaml`. + +You must configure the following: +``` +XRootD: + Sitename: +``` + +In addition, you must tell Pelican the data to export to the federation. +An origin may export one or more directory trees, or one or more S3 buckets -- follow one of the sections below. +(An single origin cannot export both a bucket and a directory tree.) + + + +### Configuring POSIX (directory) export + +Set these options to export one or more directory trees to the federation. + +``` +Origin: + StorageType: "posix" + Exports: + # You may have one or more of the following block: + - FederationPrefix: "" + StoragePrefix: "" + Capabilities: # Add or remove as desired + - Reads # Enable authenticated reading of objects from under the directory tree through a cache + - PublicReads # Enable unauthenticated reading of objects from under the directory tree through a cache + - DirectReads # Enable reading objects from under the directory tree + # without going through a cache + - Listings # Enable directory listings + - Writes # Enable writing to files in the directory tree +``` + +### Configuring S3 export + +Set these options to export one or more S3 buckets to the federation + +``` +Origin: + StorageType: "s3" + S3Region: "" + S3ServiceUrl: "" + S3UrlStyle: "path" + Exports: + # You may have one or more of the following block: + - FederationPrefix: "" + S3Bucket: "" + S3AccessKeyfile: "" + S3SecretKeyfile: "" + Capabilities: # Add or remove as desired + - Reads # Enable authenticated reading of objects from the bucket through a cache + - PublicReads # Enable unauthenticated reading of objects from the bucket through a cache + - DirectReads # Enable reading objects from the bucket + # without going through a cache + - Listings # Enable listing bucket items + - Writes # Enable writing to objects in the bucket +``` + + +Preparing for Initial Startup +----------------------------- + +1. The origin identifies itself to the federation via public key authentication; +before starting the origin for the first time, it is recommended to generate a keypair. + + :::command + root@host$ cd /etc/pelican + root@host$ pelican generate keygen + + + The newly created files, `issuer.jwk` and `issuer-pub.jwks` are the private and public keys, respectively. + **Save these files**; if you lose them, you will have to re-register the origin. + +1. Contact OSG Staff and let them know that you are about to start your origin, + and what namespace(s) the origin will serve. + OSG Staff will need to approve the origin's registration. + + +Managing the Origin Service +--------------------------- +Use the following SystemD commands as root to start, stop, enable, and disable the OSDF Origin. + +| To... | Run the command... | +| :--------------------------------------- | :--------------------------------- | +| Start the origin | `systemctl start osdf-origin` | +| Stop the origin | `systemctl stop osdf-origin` | +| Enable the origin to start on boot | `systemctl enable osdf-origin` | +| Disable the origin from starting on boot | `systemctl disable osdf-origin` | + + +Registering the Origin +---------------------- +To be part of the Open Science Data Federation, your origin must be +[registered with the OSG](../../common/registration.md). The service type is `Pelican origin`. + + +Getting Help +------------ +To get assistance, please use the [this page](../../common/help.md). diff --git a/docs/data/stashcache/install-cache.md b/docs/data/stashcache/install-cache.md index 93e8442ed..cc2299126 100644 --- a/docs/data/stashcache/install-cache.md +++ b/docs/data/stashcache/install-cache.md @@ -27,6 +27,7 @@ Before starting the installation process, consider the following requirements: * __Operating system:__ Ensure the host has [a supported operating system](../../release/supported_platforms.md) * __User IDs:__ If they do not exist already, the installation will create the Linux user IDs `condor` and `xrootd` +* __File Systems:__ The cache should have a partition of its own for storing data and metadata. * __Host certificate:__ Required for authentication. See our [host certificate documentation](../../security/host-certs.md) for instructions on how to request and install host certificates. * __Network ports:__ Your host may run a public cache instance (for serving public data only), an authenticated cache instance (for serving protected data), or both. @@ -38,8 +39,19 @@ Before starting the installation process, consider the following requirements: * An authenticated cache instance requires the following ports open: * Inbound TCP port 8443 for authenticated file access via HTTPS * Outbound UDP port 9930 for reporting to `xrd-report.osgstorage.org` and `xrd-mon.osgstorage.org` for monitoring -* __Hardware requirements:__ We recommend that a cache has at least 10Gbps connectivity, 1TB of - disk space for the cache directory, and 12GB of RAM. +* __Hardware requirements:__ + * A cache serving the OSDF federation as a regional cache should have at least: + * 8 cores + * 40 Gbps connectivity + * 50-200 TB of NVMe disk for the cache partition; you may distribute the disk, e.g., by using an NVMe-backed Ceph pool, + if you cannot fit that much disk into a single chassis + * 24 GB of RAM + * A cache being used to serve data from the OSDF to a single site should have at least: + * 8 cores + * 40 Gbps connectivity + * 2 TB of NVMe disk for the cache partition + * 24 GB of RAM + As with all OSG software installations, there are some one-time steps to prepare in advance: diff --git a/docs/data/stashcache/run-stashcache-container.md b/docs/data/stashcache/run-stashcache-container.md index 9e3e46160..a5c3398fa 100644 --- a/docs/data/stashcache/run-stashcache-container.md +++ b/docs/data/stashcache/run-stashcache-container.md @@ -32,13 +32,21 @@ Before starting the installation process, consider the following requirements: * Inbound TCP port 8000 for unauthenticated file access via HTTP(S) and/or * Inbound TCP port 8443 for authenticated file access via HTTPS * Outbound UDP port 9930 for reporting to `xrd-report.osgstorage.org` and `xrd-mon.osgstorage.org` for monitoring -1. **File Systems:** The cache needs host partitions to store user data. - For improved performance and storage, we recommend multiple partitions for handling namespaces (HDD, SSD, or NVMe), - data (HDDs), and metadata (SSDs or NVMe). +1. **File Systems:** The cache should have a partition of its own for storing data and metadata. 1. **Host certificate:** Required for authentication. See our [host certificate documentation](../../security/host-certs.md) for instructions on how to request host certificates. -1. **Hardware requirements:** We recommend that a cache has at least 10Gbps connectivity, - 1 TB of disk space for the cache directory, and 12GB of RAM. +1. **Hardware requirements:** + * A cache serving the OSDF federation as a regional cache should have at least: + * 8 cores + * 40 Gbps connectivity + * 50-200 TB of NVMe disk for the cache partition; you may distribute the disk, e.g., by using an NVMe-backed Ceph pool, + if you cannot fit that much disk into a single chassis + * 24 GB of RAM + * A cache being used to serve data from the OSDF to a single site should have at least: + * 8 cores + * 40 Gbps connectivity + * 2 TB of NVMe disk for the cache partition + * 24 GB of RAM diff --git a/docs/security/host-certs/lets-encrypt.md b/docs/security/host-certs/lets-encrypt.md index 52e18d796..79257a5e6 100644 --- a/docs/security/host-certs/lets-encrypt.md +++ b/docs/security/host-certs/lets-encrypt.md @@ -34,11 +34,22 @@ Installation and Obtaining the Initial Certificate 1. Set up hostcert/hostkey links: + If using host certificates for Pelican/OSDF: + :::console + root@host # ln -sf /etc/letsencrypt/live/*/fullchain.pem /etc/pki/tls/certs/pelican.crt + root@host # ln -sf /etc/letsencrypt/live/*/privkey.pem /etc/pki/tls/private/pelican.key + root@host # chmod 0600 /etc/letsencrypt/archive/*/privkey*.pem + + If using host certificates for other software: :::console root@host # ln -sf /etc/letsencrypt/live/*/cert.pem /etc/grid-security/hostcert.pem root@host # ln -sf /etc/letsencrypt/live/*/privkey.pem /etc/grid-security/hostkey.pem root@host # chmod 0600 /etc/letsencrypt/archive/*/privkey*.pem + + Note that Pelican requires the full certificate chain, not just the certificate, + so the pelican.crt symlink needs to point to fullchain.pem, not cert.pem. + 1. Restart services running on port 80 if there were any.