quattor · jrha · May 2, 2018 · Jan 26, 2018
diff --git a/docs/CAF/Application.md b/docs/CAF/Application.md
@@ -64,7 +64,7 @@ Applications can extend or overwrite the default methods.
     Returns the option value coming from the command line and/or
     configuration file. Scalar can be a string, or a reference to a hash
     or an array containing the option's value. option() is a wrapper
-    on top of AppConfig->get($opt). 
+    on top of AppConfig->get($opt).
 
     If the option doesn't exist, returns `undef`, except if the `default`
     argument has been specified: in this case this value is returned but

diff --git a/docs/CAF/Exception.md b/docs/CAF/Exception.md
@@ -0,0 +1,57 @@
+
+### NAME
+
+CAF::Exception - provides basic methods for failure and exception handling
+
+#### Private methods
+
+- \_get\_noaction
+
+    Return NoAction setting:
+
+    - Return 0 is `keeps_state` is true
+
+        Any other value of `keeps_state` is ignored. (In particular,
+        you cannot use `keeps_state` to enable NoAction).
+
+    - Return value of `noAction` method (when defined)
+    - `CAF::Object::NoAction` otherwise
+
+    Supports an optional `msg` that is prefixed to reporter.
+
+- \_reset\_exception\_fail
+
+    Reset previous fail attribute and/or exception.
+
+    `msg` is a suffix when reporting the old `fail` attribute
+    and/or exception error (with debug level 1).
+
+    `EC` is a `LC::Exception::Context` instance that is checked for an
+    existing error, which is set to ignore if it exists.
+
+    Always returns SUCCESS.
+
+- \_function\_catch
+
+    Execute function reference `funcref` with arrayref `$args` and hashref `$opts`.
+
+    Method resets any existing fail attribute and error from `LC::Exception::Context` instance `EC`.
+
+    When an exception thrown is thrown, it is catched and reset. No error is reported
+    and undef is returned in this case and the fail attribute is set with the exception
+    error text.
+
+- \_safe\_eval
+
+    Run function reference `funcref` with arrayref `argsref` and hashref `optsref`.
+
+    Return and set fail attribute with `failmsg` (`$@` is added when set) on die
+    or in case of an error (`undef` returned by `funcref`).
+    In case of success, report `msg` (stringified result is added unless `sensitive` attribute is set)
+    at verbose level.
+
+    Note that `_safe_eval` doesn't work with functions
+    that don't return a defined value when they succeed.
+
+    Resets previous fail attribute and or exceptions
+    (via the `LC::Exception::Context` instance `EC`).
diff --git a/docs/CAF/FileWriter.md b/docs/CAF/FileWriter.md
@@ -72,7 +72,9 @@ This is a wrapper class for `IO::String` with customised close based on
 
     - `backup`
 
-        Path for the backup file, if this one has to be re-written.
+        Create a backup file when the file already exists and will be modified.
+        The value is used as a suffix to create the backup filename
+        (e.g. `.old`).
 
     - `keeps_state`
 

diff --git a/docs/CAF/Path.md b/docs/CAF/Path.md
@@ -42,41 +42,6 @@ undef on failure and store the error message in the `fail` attribute.
 
 #### Methods
 
-- \_get\_noaction
-
-    Return NoAction setting:
-
-    - Return 0 is `keeps_state` is true
-
-        Any other value of `keeps_state` is ignored. (In particular,
-        you cannot use `keeps_state` to enable NoAction).
-
-    - Return value of `CAF::Object::NoAction` otherwise.
-
-    Supports an optional `msg` that is prefixed to reporter.
-
-- \_reset\_exception\_fail
-
-    Reset previous exceptions and/or fail attribute.
-
-- \_function\_catch
-
-    Execute function reference `funcref` with arrayref `$args` and hashref `$opts`.
-
-    Method resets/ignores any existing errors and fail attribute, and catches any exception thrown.
-    No error is reported, it returns undef in this case and the fail attribute is set.
-
-- \_safe\_eval
-
-    Run function reference `funcref` with arrayref `argsref` and hashref `optsref`.
-
-    Return and set fail attribute with `failmsg` on die or an error (`undef` returned
-    by `funcref`), or print (at verbose level) `msg` on success (respectively $@ and
-    stringified result are appended). Note that `_safe_eval` doesn't work with functions
-    that don't return a defined value when they succeed.
-
-    Resets previous exceptions and/or fail attribute
-
 - LC\_Check
 
     Execute function `<LC::Check::<function`>> with arrayref `$args` and hashref `$opts`.

diff --git a/docs/CAF/Process.md b/docs/CAF/Process.md
@@ -72,6 +72,17 @@ secure.
             By default, commands modify the state and thus `keeps_state` is
             false.
 
+        - `sensitive`
+
+            A boolean specifying whether the arguments contain sensitive information
+            (like passwords). If `sensitive` is true, the commandline will not be reported
+            (by default when `log` option is used, the commandline is reported
+            with verbose level).
+
+            This does not cover command output. If the output (stdout and/or stderr) contains
+            sensitve information, make sure to handle it yourself via `stdout` and/or `stderr`
+            options (or by using the `output` method).
+
         These options will only be used by the execute method.
 
 - \_LC\_Process
@@ -81,7 +92,7 @@ secure.
     `noaction_value` is is the value to return with `NoAction`.
 
     `msg` and `postmsg` are used to construct log message
-    `<<msg` command: <COMMAND>\[ &lt;postmsg>\]>>.
+    `<<msg` command: <COMMAND>\[ <postmsg>\]>>.
 
 #### Public methods
 

diff --git a/docs/CCM/CacheManager::Element.md b/docs/CCM/CacheManager::Element.md
@@ -153,10 +153,17 @@ The class <EDG::WP4::CCM::CacheManager::Element> is a base class for
     - convert\_nlist
 
         Array ref of anonymous methods to convert the argument
-        (nlist of elements) to another representation/format.
+        (dict of elements) to another representation/format.
 
         Each element is already processed before the conversion.
 
+    - convert\_key
+
+        Array ref of anonymous methods to convert the key(s) of the dicts
+        to another representation/format.
+
+        At the end, a stringification of the result is used as key.
+
     The arrayref of anonymous methods are applied as follows:
     convert methods `[a, b, c]` will produce `$new = c(b(a($old)))`.
     (An exception is thrown if these methods are not code references).
diff --git a/docs/CCM/TextRender.md b/docs/CCM/TextRender.md
@@ -45,18 +45,18 @@ difference the support of a `EDG::WP4::CCM::CacheManager::Element` instance as c
 
                 generates a block with format
                     <"key">
-                        &lt;recursive rendering of the value>
-                    &lt;/"key">
+                        &lt;recursive rendering of the value> &lt;/"key">
+                    </"key">
 
             - arrayref of hashref
 
                 generates series of blocks
                     <"key">
-                        &lt;recursive rendering of the element0>
-                    &lt;/"key">
+                        &lt;recursive rendering of the element0> &lt;/"key">
+                    </"key">
                     <"key">
-                        &lt;recursive rendering of the element1>
-                    &lt;/"key">
+                        &lt;recursive rendering of the element1> &lt;/"key">
+                    </"key">
                     ...
 
             (Whitespace in the block name is enforced with double quotes.)
@@ -149,6 +149,18 @@ difference the support of a `EDG::WP4::CCM::CacheManager::Element` instance as c
 
             Caveat: is preceded by `joincomma` option.
 
+        - unescapekey
+
+            Unescape all dict keys.
+
+        - lowerkey
+
+            Convert all dict keys to lowercase.
+
+        - upperkey
+
+            Convert all dict keys to uppercase.
+
         Other `getTree` options
 
         - depth

diff --git a/docs/Unittesting/Quattor.md b/docs/Unittesting/Quattor.md
@@ -121,6 +121,13 @@ that control the following variables:
 
     You can add paths using the `set_immutable` function.
 
+- `%status`
+
+    The content of this hash (keys are the absolute path names) indicates
+    current `CAF::Path::status` (`mode`, `mtime`, `owner` and/or `group`).
+
+    You can add paths using the `set_status` function.
+
 #### Redefined functions
 
 In order to achieve this, the following functions are redefined
@@ -238,6 +245,10 @@ automatically:
 
     `remove_any` and store args in `caf_path` using `add_caf_path`.
 
+- `CAF::Path::status`
+
+    Set and compare status.
+
 - `CAF::Path::_listdir`
 
     Mock underlying \_listdir method that does the actual opendir/readdir/closedir.
@@ -256,6 +267,7 @@ The following functions are exported by default:
 - `set_file_contents`
 
     For file `$filename`, sets the initial `$contents` the component should see.
+    It also sets the default [FileWriter](../CAF/FileWriter.md) permissions (`mode` 644).
 
     Returns the contents on success, undef otherwise.
 
@@ -344,6 +356,10 @@ The following functions are exported by default:
     Make [path](../components/path.md) immutable. Pass a false `bool` to make the path mutable again
     (not &lt;undef>, default is to make the path immutable).
 
+- set\_status
+
+    (Re)set status of [path](../components/path.md) to the options (`mode`, `mtime`, `owner` and/or `group`).
+
 - is\_mutable
 
     Check if the path and parent path are mutable.

diff --git a/docs/components-grid/condorconfig.md b/docs/components-grid/condorconfig.md
@@ -1,16 +1,16 @@
 
 ### NAME
 
-The _condorconfig_ component manages the configuration file of 
-Condor.  
+The _condorconfig_ component manages the configuration file of
+Condor.
 
 ### DESCRIPTION
 
 The _condorconfig_ component manages the configuration file (default
 is `/opt/condor/etc/condor.conf`) for Condor.  All of the condor
 parameters are available with exactly the same name in Quattor.  See
 the condor documentation for the names and descriptions of the
-parameters. 
+parameters.
 
 ### RESOURCES
 

diff --git a/docs/components-grid/dpmlfc.md b/docs/components-grid/dpmlfc.md
@@ -5,14 +5,14 @@ ncm-dpmlfc : NCM component to manage DPM and LFC configuration.
 
 ### DESCRIPTION
 
-This component allows to manage configuration of DPM and LFC services, with the exception of DPM xrootd protocol which is managed by 
+This component allows to manage configuration of DPM and LFC services, with the exception of DPM xrootd protocol which is managed by
 the [xrootd](../components-grid/xrootd.md) configuration module.
 
-Configuration module **ncm-dpmlfc** requires that the DPM and/or LFC configuration describes all nodes participating to the service and their respective 
+Configuration module **ncm-dpmlfc** requires that the DPM and/or LFC configuration describes all nodes participating to the service and their respective
 role (in term of daemon running on each node). Each daemon/host combination is called a daemon instance in this documentation.
 
-Using the whole DPM and/or LFC description, **ncm-dpmlfc** takes care of action needed on every node to configure it as requested 
-(you MUST use the same configuration description on every node participating to DPM and/or LFC). This includes restarting 
+Using the whole DPM and/or LFC description, **ncm-dpmlfc** takes care of action needed on every node to configure it as requested
+(you MUST use the same configuration description on every node participating to DPM and/or LFC). This includes restarting
 a service after configuration changes if needed.
 
 There are 2 sets of configuration options:
@@ -49,31 +49,31 @@ DPM and LFC accept the same global options but there is a separate set for each
 
 - `/software/components/dpmlfc/options/PRODUCT/gridmapfile`
 
-    This option defines the local gridmap file used by products daemons. 
+    This option defines the local gridmap file used by products daemons.
 
     Default: None (default configuration provided by RPM will be used)
 
 - `/software/components/dpmlfc/options/PRODUCT/gridmapdir`
 
-    This option defines the gridmap dir used by products daemons. 
+    This option defines the gridmap dir used by products daemons.
 
     Default: None (default configuration provided by RPM will be used)
 
 - `/software/components/dpmlfc/options/PRODUCT/group`
 
-    This option defines the userid used by product daemons. 
+    This option defines the userid used by product daemons.
 
     Default: None (default configuration provided by RPM will be used)
 
 - `/software/components/dpmlfc/options/PRODUCT/user`
 
-    This option defines the userid used by product daemons. 
+    This option defines the userid used by product daemons.
 
     Default: dpmmgr for DPM, lfcmgr for LFC
 
 ### DATABASE CONNECTION OPTIONS (DPM and LFC)
 
-DPM and LFC accepts the same set of options to describe the database connection. In the following option names, 
+DPM and LFC accepts the same set of options to describe the database connection. In the following option names,
 replace `PRODUCT` by either `dpm` or `lfc`. Both sets can coexist.
 
 - `/software/components/dpmlfc/options/PRODUCT/db/configfile`
@@ -113,8 +113,8 @@ replace `PRODUCT` by either `dpm` or `lfc`. Both sets can coexist.
 - `/software/components/dpmlfc/options/PRODUCT/db/server` (string, optional)
 
     This option defines the server running the database. This component checks that
-    DPM and LFC database server run on different node (DPNS and LFC use the same database name). 
-    `localhost` is considered different as DPNS and LFC are not allowed to run on the same node. 
+    DPM and LFC database server run on different node (DPNS and LFC use the same database name).
+    `localhost` is considered different as DPNS and LFC are not allowed to run on the same node.
 
     Default : localhost.
 

diff --git a/docs/components-grid/gip2.md b/docs/components-grid/gip2.md
@@ -58,12 +58,12 @@ Default : none.
 
 ### LDIF entries : nlist (optional)
 
-nlist of LDIF entries (key is the DN, value is a nlist of attribute/value pairs) to put in the resulting file 
+nlist of LDIF entries (key is the DN, value is a nlist of attribute/value pairs) to put in the resulting file
 if staticInfoCmd is not specified or sets of key value/pairs (key is the set name and and value is a nlist of key/value pairs).
 
 Key is interpreted as an escaped value.
 
-If ommitted and confFile is defined, must be defined in `/software/components/gip2/ldifConfEntries` 
+If ommitted and confFile is defined, must be defined in `/software/components/gip2/ldifConfEntries`
 key matching confFile.
 
 Default : none.
@@ -100,7 +100,7 @@ Default : none.
 
 #### `/software/components/gip2/staticInfoCmd` : string (optional)
 
-Path of the command to execute to transform entries into a LDIF file if none is defined in the 
+Path of the command to execute to transform entries into a LDIF file if none is defined in the
 `/software/components/gip2/ldif` entry. It is here for backward compatibility but it is recommended
 to define it as part of the ldif entries. If undefined in both locations, the configuration file
 is read directly without any processing.

diff --git a/docs/components-grid/glitestartup.md b/docs/components-grid/glitestartup.md
@@ -68,14 +68,14 @@ List of paths where to look for a script matching service name.
 
 Default : `/opt/glite/etc/init.d`
 
-#### `/software/components`/@COMP/services : nlist of string 
+#### `/software/components`/@COMP/services : nlist of string
 
 Nlist with one entry per service to start. Key is the service name,
 value is an optional nlist. This nlist can contain the following element:
 
 - args startup script arguments
 
-Default : none 
+Default : none
 
 ### DEPENDENCIES
 

diff --git a/docs/components-grid/glitestartup::schema.md b/docs/components-grid/glitestartup::schema.md
@@ -5,7 +5,7 @@
     - `/software/glitestartup/glitestartup_component_service/args`
         - Optional
         - Type: string
-        - Default value: 
+        - Default value:
  - `/software/glitestartup/glitestartup_component_post_restart`
     - `/software/glitestartup/glitestartup_component_post_restart/cmd`
         - Optional