From b5ff3738b1af6f75e1dae01c8f498b22230af6a6 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 11 Oct 2024 15:19:12 -0500 Subject: [PATCH 01/12] WIP: data/osdf/install-cache-rpm.md --- docs/data/osdf/install-cache-rpm.md | 171 ++++++++++++++++++++++++++++ 1 file changed, 171 insertions(+) create mode 100644 docs/data/osdf/install-cache-rpm.md diff --git a/docs/data/osdf/install-cache-rpm.md b/docs/data/osdf/install-cache-rpm.md new file mode 100644 index 000000000..7fadcfd39 --- /dev/null +++ b/docs/data/osdf/install-cache-rpm.md @@ -0,0 +1,171 @@ +title: Installing the OSDF Origin by RPM + +Installing the OSDF Origin +========================== + +This document describes how to install a Pelican-based Open Science Data Federation (OSDF) Origin service via RPMs. +This service allows an organization to export its data to the Data Federation. + +!!! note + The origin must be registered with the OSG prior to joining the data federation. + You may start the registration process prior to finishing the installation by [using this link](#registering-the-origin) + along with information like: + + * Resource name and hostname + * VO associated with this origin server (which will be used to determine the origin's namespace prefix) + * Administrative and security contact(s) + * Who (or what) will be allowed to access the VO's data + * Which caches will be allowed to cache the VO data + + +Before Starting +--------------- + +Before starting the installation process, consider the following requirements: + +* __Operating system:__ A RHEL 8 or RHEL 9 or compatible operating systems. +* __User IDs:__ If they do not exist already, the installation will create the Linux user ID `xrootd` for running daemons. +* __Host certificate:__ Required for authentication. See note below. +* __Network ports:__ The origin service requires the following ports open: + * Inbound TCP port 8443 for file access via the HTTP(S) and XRoot protocols. + * (Optional) Inbound TCP port 8444 for access to the web interface for monitoring and configuration; + if enabled, this should be restricted to the LAN or management network. +* __Hardware requirements:__ We recommend that an origin has at least 1Gbps connectivity and 12GB of RAM. + We suggest that several gigabytes of local disk space be available for log files, + although some logging verbosity can be reduced. + +As with all OSG software installations, there are some one-time steps to prepare in advance: + +* Obtain root access to the host +* Prepare [the required Yum repositories](../../common/yum.md) + + +!!! note "OSG 23" + In OSG 23, the Pelican-based OSDF RPMs are only available in the "osg-upcoming" repositories. + +!!! note "Host certificates" + Origins should use a CA that is accepted by major browsers and operating systems, + such as InCommon RSA or [Let's Encrypt](../../security/host-certs/lets-encrypt). + IGTF certs are not recommended because clients are not configured to accept them by default. + + The following locations should be used (note that they are in separate directories): + + * **Host Certificate**: `/etc/pki/tls/certs/pelican.crt` + * **Host Key**: `/etc/pki/tls/private/pelican.key` + + +Installing the Origin +--------------------- + +The origin service is provided by the `osdf-origin` RPM. +Install it using the following command: + + +```console +root@host # yum install --enablerepo=osg-upcoming osdf-origin +``` + + +Configuring the Origin Server +----------------------------- + +Configuration for a Pelican-based OSDF Origin is located in `/etc/pelican/osdf-origin.yaml`. + +You must configure the following: +``` +XRootD: + Sitename: +``` + +In addition, you must tell Pelican the data to export to the federation. +An origin may export one or more directory trees, or one or more S3 buckets -- follow one of the sections below. +(An single origin cannot export both a bucket and a directory tree.) + + + +### Configuring POSIX (directory) export + +Set these options to export one or more directory trees to the federation. + +``` +Origin: + StorageType: "posix" + Exports: + # You may have one or more of the following block: + - FederationPrefix: "" + StoragePrefix: "" + Capabilities: # Add or remove as desired + - Reads # Enable authenticated reading of objects from under the directory tree through a cache + - PublicReads # Enable unauthenticated reading of objects from under the directory tree through a cache + - DirectReads # Enable reading objects from under the directory tree + # without going through a cache + - Listings # Enable directory listings + - Writes # Enable writing to files in the directory tree +``` + +### Configuring S3 export + +Set these options to export one or more S3 buckets to the federation + +``` +Origin: + StorageType: "s3" + S3Region: "" + S3ServiceUrl: "" + S3UrlStyle: "path" + Exports: + # You may have one or more of the following block: + - FederationPrefix: "" + S3Bucket: "" + S3AccessKeyfile: "" + S3SecretKeyfile: "" + Capabilities: # Add or remove as desired + - Reads # Enable authenticated reading of objects from the bucket through a cache + - PublicReads # Enable unauthenticated reading of objects from the bucket through a cache + - DirectReads # Enable reading objects from the bucket + # without going through a cache + - Listings # Enable listing bucket items + - Writes # Enable writing to objects in the bucket +``` + + +Preparing for Initial Startup +----------------------------- + +1. The origin identifies itself to the federation via public key authentication; +before starting the origin for the first time, it is recommended to generate a keypair. + + :::command + root@host$ cd /etc/pelican + root@host$ pelican generate keygen + + + The newly created files, `issuer.jwk` and `issuer-pub.jwks` are the private and public keys, respectively. + **Save these files**; if you lose them, you will have to re-register the origin. + +1. Contact OSG Staff and let them know that you are about to start your origin, + and what namespace(s) the origin will serve. + OSG Staff will need to approve the origin's registration. + + +Managing the Origin Service +--------------------------- +Use the following SystemD commands as root to start, stop, enable, and disable the OSDF Origin. + +| To... | Run the command... | +| :--------------------------------------- | :--------------------------------- | +| Start the origin | `systemctl start osdf-origin` | +| Stop the origin | `systemctl stop osdf-origin` | +| Enable the origin to start on boot | `systemctl enable osdf-origin` | +| Disable the origin from starting on boot | `systemctl disable osdf-origin` | + + +Registering the Origin +---------------------- +To be part of the Open Science Data Federation, your origin must be +[registered with the OSG](../../common/registration.md). The service type is `Pelican origin`. + + +Getting Help +------------ +To get assistance, please use the [this page](../../common/help.md). From 9d86fea176634c65f811e36fd385488a88a6d022 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 11 Oct 2024 17:05:01 -0500 Subject: [PATCH 02/12] WIP: Installing a Pelican/OSDF cache by RPM This is based on the Pelican/OSDF origin by RPM doc (PR #193) so changes added there should generally be added here too. --- docs/data/osdf/install-cache-rpm.md | 132 +++++++++------------------- 1 file changed, 43 insertions(+), 89 deletions(-) diff --git a/docs/data/osdf/install-cache-rpm.md b/docs/data/osdf/install-cache-rpm.md index 7fadcfd39..0e39b1058 100644 --- a/docs/data/osdf/install-cache-rpm.md +++ b/docs/data/osdf/install-cache-rpm.md @@ -1,21 +1,18 @@ -title: Installing the OSDF Origin by RPM +title: Installing the OSDF Cache by RPM -Installing the OSDF Origin -========================== +Installing the OSDF Cache +========================= -This document describes how to install a Pelican-based Open Science Data Federation (OSDF) Origin service via RPMs. -This service allows an organization to export its data to the Data Federation. +This document describes how to install a Pelican-based Open Science Data Federation (OSDF) Cache service via RPMs. +This service allows a site or regional network to cache data !!! note - The origin must be registered with the OSG prior to joining the data federation. - You may start the registration process prior to finishing the installation by [using this link](#registering-the-origin) + The cache must be registered with the OSG prior to joining the data federation. + You may start the registration process prior to finishing the installation by [using this link](#registering-the-cache) along with information like: * Resource name and hostname - * VO associated with this origin server (which will be used to determine the origin's namespace prefix) * Administrative and security contact(s) - * Who (or what) will be allowed to access the VO's data - * Which caches will be allowed to cache the VO data Before Starting @@ -24,11 +21,12 @@ Before Starting Before starting the installation process, consider the following requirements: * __Operating system:__ A RHEL 8 or RHEL 9 or compatible operating systems. -* __User IDs:__ If they do not exist already, the installation will create the Linux user ID `xrootd` for running daemons. +* __User IDs:__ If they do not exist already, the installation will create the Linux user named `xrootd` for running daemons. +* __File Systems:__ The cache should have a partition of its own for storing data and metadata. * __Host certificate:__ Required for authentication. See note below. -* __Network ports:__ The origin service requires the following ports open: +* __Network ports:__ The cache service requires the following ports open: * Inbound TCP port 8443 for file access via the HTTP(S) and XRoot protocols. - * (Optional) Inbound TCP port 8444 for access to the web interface for monitoring and configuration; + * (Optional) Inbound TCP port cache for access to the web interface for monitoring and configuration; if enabled, this should be restricted to the LAN or management network. * __Hardware requirements:__ We recommend that an origin has at least 1Gbps connectivity and 12GB of RAM. We suggest that several gigabytes of local disk space be available for log files, @@ -44,96 +42,51 @@ As with all OSG software installations, there are some one-time steps to prepare In OSG 23, the Pelican-based OSDF RPMs are only available in the "osg-upcoming" repositories. !!! note "Host certificates" - Origins should use a CA that is accepted by major browsers and operating systems, + Caches should use a CA that is accepted by major browsers and operating systems, such as InCommon RSA or [Let's Encrypt](../../security/host-certs/lets-encrypt). IGTF certs are not recommended because clients are not configured to accept them by default. - + Note that you will need the full certificate chain, not just the certificate. + The following locations should be used (note that they are in separate directories): - * **Host Certificate**: `/etc/pki/tls/certs/pelican.crt` + * **Host Certificate Chain**: `/etc/pki/tls/certs/pelican.crt` * **Host Key**: `/etc/pki/tls/private/pelican.key` -Installing the Origin ---------------------- +Installing the Cache +-------------------- -The origin service is provided by the `osdf-origin` RPM. +The cache service is provided by the `osdf-cache` RPM. Install it using the following command: ```console -root@host # yum install --enablerepo=osg-upcoming osdf-origin +root@host # yum install --enablerepo=osg-upcoming osdf-cache ``` -Configuring the Origin Server ------------------------------ +Configuring the Cache Server +---------------------------- -Configuration for a Pelican-based OSDF Origin is located in `/etc/pelican/osdf-origin.yaml`. +Configuration for a Pelican-based OSDF Cache is located in `/etc/pelican/osdf-cache.yaml`. You must configure the following: ``` XRootD: Sitename: +Cache: + DataLocation: "" ``` -In addition, you must tell Pelican the data to export to the federation. -An origin may export one or more directory trees, or one or more S3 buckets -- follow one of the sections below. -(An single origin cannot export both a bucket and a directory tree.) - - - -### Configuring POSIX (directory) export - -Set these options to export one or more directory trees to the federation. - -``` -Origin: - StorageType: "posix" - Exports: - # You may have one or more of the following block: - - FederationPrefix: "" - StoragePrefix: "" - Capabilities: # Add or remove as desired - - Reads # Enable authenticated reading of objects from under the directory tree through a cache - - PublicReads # Enable unauthenticated reading of objects from under the directory tree through a cache - - DirectReads # Enable reading objects from under the directory tree - # without going through a cache - - Listings # Enable directory listings - - Writes # Enable writing to files in the directory tree -``` - -### Configuring S3 export - -Set these options to export one or more S3 buckets to the federation - -``` -Origin: - StorageType: "s3" - S3Region: "" - S3ServiceUrl: "" - S3UrlStyle: "path" - Exports: - # You may have one or more of the following block: - - FederationPrefix: "" - S3Bucket: "" - S3AccessKeyfile: "" - S3SecretKeyfile: "" - Capabilities: # Add or remove as desired - - Reads # Enable authenticated reading of objects from the bucket through a cache - - PublicReads # Enable unauthenticated reading of objects from the bucket through a cache - - DirectReads # Enable reading objects from the bucket - # without going through a cache - - Listings # Enable listing bucket items - - Writes # Enable writing to objects in the bucket -``` +If you are using a separate partition for the cached data, which is strongly recommended, +then use the mount point of the cache partition as `Cache.DataLocation`. Preparing for Initial Startup ----------------------------- -1. The origin identifies itself to the federation via public key authentication; -before starting the origin for the first time, it is recommended to generate a keypair. +1. The cache identifies itself to the federation via public key authentication; +before starting the cache for the first time, it is recommended to generate a keypair. :::command root@host$ cd /etc/pelican @@ -141,29 +94,30 @@ before starting the origin for the first time, it is recommended to generate a k The newly created files, `issuer.jwk` and `issuer-pub.jwks` are the private and public keys, respectively. - **Save these files**; if you lose them, you will have to re-register the origin. + **Save these files**; if you lose them, your cache will need to be re-approved. -1. Contact OSG Staff and let them know that you are about to start your origin, - and what namespace(s) the origin will serve. - OSG Staff will need to approve the origin's registration. +1. Contact OSG Staff and let them know that you are about to start your cache, + and what the hostname of the cache is. + OSG Staff will need to approve the cache's registration. -Managing the Origin Service +Managing the Cache Service --------------------------- -Use the following SystemD commands as root to start, stop, enable, and disable the OSDF Origin. +Use the following SystemD commands as root to start, stop, enable, and disable the OSDF Cache. | To... | Run the command... | | :--------------------------------------- | :--------------------------------- | -| Start the origin | `systemctl start osdf-origin` | -| Stop the origin | `systemctl stop osdf-origin` | -| Enable the origin to start on boot | `systemctl enable osdf-origin` | -| Disable the origin from starting on boot | `systemctl disable osdf-origin` | +| Start the cache | `systemctl start osdf-cache` | +| Stop the cache | `systemctl stop osdf-cache` | +| Enable the cache to start on boot | `systemctl enable osdf-cache` | +| Disable the cache from starting on boot | `systemctl disable osdf-cache` | -Registering the Origin ----------------------- -To be part of the Open Science Data Federation, your origin must be -[registered with the OSG](../../common/registration.md). The service type is `Pelican origin`. +Registering the Cache in OSG Topology +------------------------------------- +To be part of the Open Science Data Federation, your cache must be +[registered in the OSG Topology system](../../common/registration.md). +The service type is `Pelican cache`. Getting Help From bea7c756adcce773e398b4c2d9cb46baceb7fbec Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Wed, 6 Nov 2024 15:55:50 -0600 Subject: [PATCH 03/12] Update hardware requirements; add install instructions for OSG 23 and 24; add cache configuration based on Pelican 7.11.1 config.d (SOFTWARE-6013) --- docs/data/osdf/install-cache-rpm.md | 48 ++++++++++++++++++++++------- 1 file changed, 37 insertions(+), 11 deletions(-) diff --git a/docs/data/osdf/install-cache-rpm.md b/docs/data/osdf/install-cache-rpm.md index 0e39b1058..335160b15 100644 --- a/docs/data/osdf/install-cache-rpm.md +++ b/docs/data/osdf/install-cache-rpm.md @@ -4,7 +4,8 @@ Installing the OSDF Cache ========================= This document describes how to install a Pelican-based Open Science Data Federation (OSDF) Cache service via RPMs. -This service allows a site or regional network to cache data +This service allows a site or regional network to cache data frequently used in Open Science Pool jobs, +reducing data transfer over the wide-area network and decreasing access latency. !!! note The cache must be registered with the OSG prior to joining the data federation. @@ -27,8 +28,19 @@ Before starting the installation process, consider the following requirements: * __Network ports:__ The cache service requires the following ports open: * Inbound TCP port 8443 for file access via the HTTP(S) and XRoot protocols. * (Optional) Inbound TCP port cache for access to the web interface for monitoring and configuration; - if enabled, this should be restricted to the LAN or management network. -* __Hardware requirements:__ We recommend that an origin has at least 1Gbps connectivity and 12GB of RAM. + if enabled, this should be restricted to the LAN. +* __Hardware requirements:__ + * A cache serving the OSDF federation as a regional cache should have at least: + * 8 cores + * 40 Gbps connectivity + * 50-200 TB of NVMe disk for the cache partition; you may distribute the disk, e.g., by using an NVMe-backed Ceph pool, + if you cannot fit that much disk into a single chassis + * 24 GB of RAM + * A cache being used to serve data from the OSDF to a single site should have at least: + * 8 cores + * 40 Gbps connectivity + * 2 TB of NVMe disk for the cache partition + * 24 GB of RAM We suggest that several gigabytes of local disk space be available for log files, although some logging verbosity can be reduced. @@ -57,29 +69,43 @@ Installing the Cache -------------------- The cache service is provided by the `osdf-cache` RPM. -Install it using the following command: - +Install it using one of the following commands: +OSG 23: ```console root@host # yum install --enablerepo=osg-upcoming osdf-cache ``` +OSG 24: +```console +root@host # yum install osdf-cache +``` Configuring the Cache Server ---------------------------- -Configuration for a Pelican-based OSDF Cache is located in `/etc/pelican/osdf-cache.yaml`. +!!! note "osdf-cache 7.11.1" + This configuration requires version 7.11.1 or newer of the `osdf-cache` + and `pelican` RPMs. + +Configuration for a Pelican-based OSDF Cache is located in files in `/etc/pelican/config.d`. + +You must set the following config options: -You must configure the following: +In `/etc/pelican/config.d/15-osdf.yaml`, set `XRootD.Sitename`: ``` XRootD: Sitename: -Cache: - DataLocation: "" ``` -If you are using a separate partition for the cached data, which is strongly recommended, -then use the mount point of the cache partition as `Cache.DataLocation`. +In `/etc/pelican/config.d/20-cache.yaml`, set `Cache.LocalRoot`, `Cache.DataLocation` and `Cache.MetaLocation` as follows, +replacing `` with the mount point of the partition you will use for the cache. +``` +Cache: + LocalRoot: "/namespaces" + DataLocation: "/data" + MetaLocation: "/meta" +``` Preparing for Initial Startup From 92ce45dd0813bae75f723543313e1b58d1c46b38 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Wed, 6 Nov 2024 16:07:28 -0600 Subject: [PATCH 04/12] typo --- docs/data/osdf/install-cache-rpm.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/data/osdf/install-cache-rpm.md b/docs/data/osdf/install-cache-rpm.md index 335160b15..5d79e575b 100644 --- a/docs/data/osdf/install-cache-rpm.md +++ b/docs/data/osdf/install-cache-rpm.md @@ -27,7 +27,7 @@ Before starting the installation process, consider the following requirements: * __Host certificate:__ Required for authentication. See note below. * __Network ports:__ The cache service requires the following ports open: * Inbound TCP port 8443 for file access via the HTTP(S) and XRoot protocols. - * (Optional) Inbound TCP port cache for access to the web interface for monitoring and configuration; + * (Optional) Inbound TCP port 8444 for access to the web interface for monitoring and configuration; if enabled, this should be restricted to the LAN. * __Hardware requirements:__ * A cache serving the OSDF federation as a regional cache should have at least: From 2d4020f228ea67f87c3c8e6d0f5e548b16a00246 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Wed, 6 Nov 2024 17:36:50 -0600 Subject: [PATCH 05/12] Use "Service requirements" instead of "Hardware requirements" This should make the lopsided requirements stand out less --- docs/data/osdf/install-cache-rpm.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/data/osdf/install-cache-rpm.md b/docs/data/osdf/install-cache-rpm.md index 5d79e575b..9c462da66 100644 --- a/docs/data/osdf/install-cache-rpm.md +++ b/docs/data/osdf/install-cache-rpm.md @@ -29,7 +29,7 @@ Before starting the installation process, consider the following requirements: * Inbound TCP port 8443 for file access via the HTTP(S) and XRoot protocols. * (Optional) Inbound TCP port 8444 for access to the web interface for monitoring and configuration; if enabled, this should be restricted to the LAN. -* __Hardware requirements:__ +* __Service requirements:__ * A cache serving the OSDF federation as a regional cache should have at least: * 8 cores * 40 Gbps connectivity @@ -81,6 +81,7 @@ OSG 24: root@host # yum install osdf-cache ``` + Configuring the Cache Server ---------------------------- From 3ac7ce807f4b2692eb012db686009f76adac7541 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Wed, 6 Nov 2024 17:38:16 -0600 Subject: [PATCH 06/12] Add a validation test; move the registration step to the end and call it "Joining the Cache to the Federation" --- docs/data/osdf/install-cache-rpm.md | 71 ++++++++++++++++++----------- 1 file changed, 45 insertions(+), 26 deletions(-) diff --git a/docs/data/osdf/install-cache-rpm.md b/docs/data/osdf/install-cache-rpm.md index 9c462da66..f07e9521e 100644 --- a/docs/data/osdf/install-cache-rpm.md +++ b/docs/data/osdf/install-cache-rpm.md @@ -7,14 +7,6 @@ This document describes how to install a Pelican-based Open Science Data Federat This service allows a site or regional network to cache data frequently used in Open Science Pool jobs, reducing data transfer over the wide-area network and decreasing access latency. -!!! note - The cache must be registered with the OSG prior to joining the data federation. - You may start the registration process prior to finishing the installation by [using this link](#registering-the-cache) - along with information like: - - * Resource name and hostname - * Administrative and security contact(s) - Before Starting --------------- @@ -93,12 +85,6 @@ Configuration for a Pelican-based OSDF Cache is located in files in `/etc/pelica You must set the following config options: -In `/etc/pelican/config.d/15-osdf.yaml`, set `XRootD.Sitename`: -``` -XRootD: - Sitename: -``` - In `/etc/pelican/config.d/20-cache.yaml`, set `Cache.LocalRoot`, `Cache.DataLocation` and `Cache.MetaLocation` as follows, replacing `` with the mount point of the partition you will use for the cache. ``` @@ -113,9 +99,9 @@ Preparing for Initial Startup ----------------------------- 1. The cache identifies itself to the federation via public key authentication; -before starting the cache for the first time, it is recommended to generate a keypair. +before starting the cache for the first time, it is generate a keypair. - :::command + :::console root@host$ cd /etc/pelican root@host$ pelican generate keygen @@ -123,9 +109,49 @@ before starting the cache for the first time, it is recommended to generate a ke The newly created files, `issuer.jwk` and `issuer-pub.jwks` are the private and public keys, respectively. **Save these files**; if you lose them, your cache will need to be re-approved. -1. Contact OSG Staff and let them know that you are about to start your cache, - and what the hostname of the cache is. - OSG Staff will need to approve the cache's registration. + +Validating the Cache Installation +--------------------------------- + +Do the following steps to verify that the cache is functional: + +1. Start the cache using the following command: + + :::console + root@host$ systemctl start osdf-cache + +1. Download a test file from the OSDF through your cache (replacing `CACHE_HOSTNAME` with the host name of your cache) + + :::console + root@host$ osdf object get -c CACHE_HOSTNAME:8443 /ospool/uc-shared/public/OSG-Staff/validation/test.txt /tmp/test.txt + root@host$ cat /tmp/test.txt + + Hello, World! + + +Joining the Cache to the Federation +----------------------------------- + +The cache must be registered with the OSG prior to joining the data federation. +Send mail to requesting registration; provide the following information: + +* Cache hostname +* Administrative and security contact(s) +* Institution that the cache belongs to + +OSG Staff will register the cache and respond with the Resource Name that the cache was registered as. + +Once you have that information, edit `/etc/pelican/config.d/15-osdf.yaml`, and set `XRootD.Sitename`: +``` +XRootD: + Sitename: +``` + +Then, restart the cache by running + +```console +root@host$ systemctl restart osdf-cache +``` Managing the Cache Service @@ -140,13 +166,6 @@ Use the following SystemD commands as root to start, stop, enable, and disable t | Disable the cache from starting on boot | `systemctl disable osdf-cache` | -Registering the Cache in OSG Topology -------------------------------------- -To be part of the Open Science Data Federation, your cache must be -[registered in the OSG Topology system](../../common/registration.md). -The service type is `Pelican cache`. - - Getting Help ------------ To get assistance, please use the [this page](../../common/help.md). From 0576f0c1aa0be3fad43261bd75683e0b2c2625bb Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Wed, 6 Nov 2024 17:41:38 -0600 Subject: [PATCH 07/12] fix syntax --- docs/data/osdf/install-cache-rpm.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/data/osdf/install-cache-rpm.md b/docs/data/osdf/install-cache-rpm.md index f07e9521e..29c58e5f5 100644 --- a/docs/data/osdf/install-cache-rpm.md +++ b/docs/data/osdf/install-cache-rpm.md @@ -101,9 +101,9 @@ Preparing for Initial Startup 1. The cache identifies itself to the federation via public key authentication; before starting the cache for the first time, it is generate a keypair. - :::console - root@host$ cd /etc/pelican - root@host$ pelican generate keygen + :::console + root@host$ cd /etc/pelican + root@host$ pelican generate keygen The newly created files, `issuer.jwk` and `issuer-pub.jwks` are the private and public keys, respectively. From 7427d52586fc300d747479ea150e3baf8409c0b2 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 8 Nov 2024 13:54:13 -0600 Subject: [PATCH 08/12] Apply suggestions from code review Co-authored-by: Brian Lin --- docs/data/osdf/install-cache-rpm.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/docs/data/osdf/install-cache-rpm.md b/docs/data/osdf/install-cache-rpm.md index 29c58e5f5..0bdb9ecfd 100644 --- a/docs/data/osdf/install-cache-rpm.md +++ b/docs/data/osdf/install-cache-rpm.md @@ -13,14 +13,14 @@ Before Starting Before starting the installation process, consider the following requirements: -* __Operating system:__ A RHEL 8 or RHEL 9 or compatible operating systems. -* __User IDs:__ If they do not exist already, the installation will create the Linux user named `xrootd` for running daemons. +* __Operating system:__ A RHEL 8 or RHEL 9 or [compatible operating system](../../release/supported_platforms.md). +* __User IDs:__ If it does not exist already, the installation will create the Linux user named `xrootd` for running daemons. * __File Systems:__ The cache should have a partition of its own for storing data and metadata. * __Host certificate:__ Required for authentication. See note below. * __Network ports:__ The cache service requires the following ports open: * Inbound TCP port 8443 for file access via the HTTP(S) and XRoot protocols. * (Optional) Inbound TCP port 8444 for access to the web interface for monitoring and configuration; - if enabled, this should be restricted to the LAN. + if enabled, access to this port should be restricted to the LAN. * __Service requirements:__ * A cache serving the OSDF federation as a regional cache should have at least: * 8 cores @@ -70,7 +70,7 @@ root@host # yum install --enablerepo=osg-upcoming osdf-cache OSG 24: ```console -root@host # yum install osdf-cache +root@host # yum install osdf-cache --enablerepo=osg-testing ``` @@ -83,8 +83,6 @@ Configuring the Cache Server Configuration for a Pelican-based OSDF Cache is located in files in `/etc/pelican/config.d`. -You must set the following config options: - In `/etc/pelican/config.d/20-cache.yaml`, set `Cache.LocalRoot`, `Cache.DataLocation` and `Cache.MetaLocation` as follows, replacing `` with the mount point of the partition you will use for the cache. ``` From 66f6f22037e7224cfb6dc2b55f49bfd9ae56aae7 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 8 Nov 2024 14:55:20 -0600 Subject: [PATCH 09/12] Prefer OSG 24 --- docs/data/osdf/install-cache-rpm.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/data/osdf/install-cache-rpm.md b/docs/data/osdf/install-cache-rpm.md index 0bdb9ecfd..63b0fddd3 100644 --- a/docs/data/osdf/install-cache-rpm.md +++ b/docs/data/osdf/install-cache-rpm.md @@ -63,16 +63,17 @@ Installing the Cache The cache service is provided by the `osdf-cache` RPM. Install it using one of the following commands: -OSG 23: -```console -root@host # yum install --enablerepo=osg-upcoming osdf-cache -``` OSG 24: ```console root@host # yum install osdf-cache --enablerepo=osg-testing ``` +OSG 23: +```console +root@host # yum install --enablerepo=osg-upcoming-testing osdf-cache +``` + Configuring the Cache Server ---------------------------- @@ -105,7 +106,8 @@ before starting the cache for the first time, it is generate a keypair. The newly created files, `issuer.jwk` and `issuer-pub.jwks` are the private and public keys, respectively. - **Save these files**; if you lose them, your cache will need to be re-approved. + +1. **Save these files**; if you lose them, your cache will need to be re-approved. Validating the Cache Installation From 5f53d06871c8ade00cba97448cffe8be9c16f128 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 8 Nov 2024 16:17:57 -0600 Subject: [PATCH 10/12] tweaks --- docs/data/osdf/install-cache-rpm.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/data/osdf/install-cache-rpm.md b/docs/data/osdf/install-cache-rpm.md index 63b0fddd3..fb90beb1e 100644 --- a/docs/data/osdf/install-cache-rpm.md +++ b/docs/data/osdf/install-cache-rpm.md @@ -82,8 +82,6 @@ Configuring the Cache Server This configuration requires version 7.11.1 or newer of the `osdf-cache` and `pelican` RPMs. -Configuration for a Pelican-based OSDF Cache is located in files in `/etc/pelican/config.d`. - In `/etc/pelican/config.d/20-cache.yaml`, set `Cache.LocalRoot`, `Cache.DataLocation` and `Cache.MetaLocation` as follows, replacing `` with the mount point of the partition you will use for the cache. ``` @@ -153,6 +151,9 @@ Then, restart the cache by running root@host$ systemctl restart osdf-cache ``` +Let OSG Staff know that you have restarted the cache with the updated sitename, +so they can approve the new cache. + Managing the Cache Service --------------------------- From 37a3d79a74e109b57f43247c8f51778e14a65f46 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 8 Nov 2024 16:37:53 -0600 Subject: [PATCH 11/12] Tell people to email us with the logs if validation fails --- docs/data/osdf/install-cache-rpm.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/data/osdf/install-cache-rpm.md b/docs/data/osdf/install-cache-rpm.md index fb90beb1e..09123f35d 100644 --- a/docs/data/osdf/install-cache-rpm.md +++ b/docs/data/osdf/install-cache-rpm.md @@ -126,6 +126,11 @@ Do the following steps to verify that the cache is functional: Hello, World! + If the download fails, rerun the above `osdf object get` command with the `-d` flag added; + additional debugging information is located in `/var/log/pelican/osdf-cache.log`. + See [this page](../../common/help.md) for requesting assistance; please include the log file + and the `osdf object get -d` output in your request. + Joining the Cache to the Federation ----------------------------------- From 30709f532dc860c23aef10fb2c7c59339904a751 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 8 Nov 2024 17:07:05 -0600 Subject: [PATCH 12/12] Fix link --- docs/data/osdf/install-cache-rpm.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/data/osdf/install-cache-rpm.md b/docs/data/osdf/install-cache-rpm.md index 09123f35d..e038edf88 100644 --- a/docs/data/osdf/install-cache-rpm.md +++ b/docs/data/osdf/install-cache-rpm.md @@ -47,7 +47,7 @@ As with all OSG software installations, there are some one-time steps to prepare !!! note "Host certificates" Caches should use a CA that is accepted by major browsers and operating systems, - such as InCommon RSA or [Let's Encrypt](../../security/host-certs/lets-encrypt). + such as InCommon RSA or [Let's Encrypt](../../security/host-certs/lets-encrypt.md). IGTF certs are not recommended because clients are not configured to accept them by default. Note that you will need the full certificate chain, not just the certificate.