Merge pull request #3111 from nickanderson/ENT-10903/master

ENT-10930: Added notes about lock purging

cheatsheet.markdown

@@ -59,7 +59,7 @@ _Most_ (`¯\_(ツ)_/¯`) special characters are _okay_. For example:
 * Link targets with `/` (forward slashes) work
    * ```[Export/import][Settings#Export/import]``` == [Export/import][Settings#Export/import]
 
-Anchors with _underscores_ are problematic, they need to be escaped.
+Anchors with _underscores_ are problematic, *may* need to be escaped.
 
 For example ```services_autorun``` in the MPF documentation the underscore needs to be escaped with a ```\```.
 
@@ -69,6 +69,16 @@ For example ```services_autorun``` in the MPF documentation the underscore needs
 
 **See also:** [`services_autorun` in the Masterfiles Policy Framework][Masterfiles Policy Framework#services\_autorun]
 
+But not always! For example
+
+```
+**See also:** [cf_lock.lmdb][CFEngine directory structure#state/cf_lock.lmdb]
+```
+
+**See also:** [cf_lock.lmdb][CFEngine directory structure#state/cf_lock.lmdb]
+
+Backticks are problematic. It seems impossible to link to anchors that contain backticks.
+
 ### Link to CFEngine keyword
 
 The documentation pre-processor will create those automatically.

examples/tutorials/writing-and-serving-policy/controlling-frequency.markdown

@@ -5,9 +5,55 @@ published: true
 sorting: 90
 ---
 
-When checking a series of expensive functions and verifying complex promises,
-you may want to make sure that CFEngine is not checking too frequently. One
-way of doing this is classes and class expression, another is using locks.
+By default CFEngine runs relatively frequently (every 5 minutes) but you may not
+want every promise to be evaluated each agent execution. Classes and promise
+locks are the two primary ways in which a promises frequency can be controlled.
+Classes are the canonical way of controlling if a promise is in context and
+should be evaluated. Promise locks control frequency based on the number of
+minutes since the last promise actuation.
+
+## Controlling frequency using classes
+
+Classes are the canonical way of controlling promise executions in CFEngine.
+
+Use time based classes to restrict promises to run during a specific period of time. For example, here `sshd` promises to be the latest version available, but only on Tuesdays during the first 15 minutes of the 5:00 hour.
+
+```cf3
+bundle agent __main__
+{
+  packages:
+    Tuesday.Hr05_Q1::
+    "sshd"
+      version => "latest",
+      comment => "Make sure sshd is at the latest version, but only Tuesday between 5:00 and 5:15am";
+}
+```
+
+Persistent classes can exist for a period of time, across multiple executions of
+`cf-agent`. Persistent classes can be used to avoid re-execution of a promise.
+For example, here `/tmp/heartbeat.dat` promises to update it's timestamp when
+`heartbeat_repaired` is not defined. When the file is repaired the class
+`heartbeat_repaired` is defined for 10 minutes causing the promise to be out of
+context during subsequent executions for the next 10 minutes.
+
+```cf3
+bundle agent __main__
+{
+  files:
+    !heartbeat_repaired::
+      "/tmp/heartbeat.dat"
+        create => "true",
+        touch => "true",
+        classes => persistent_results( "heartbeat", 10 );
+}
+body classes persistent_results( prefix, time )
+{
+    inherit_from => results( "namespace", "$(prefix)" );
+    persist_time => "$(time)";
+}
+```
+
+## Controlling frequency using promise locks
 
 CFEngine incorporates a series of locks which prevent it from checking
 promises too often, and which prevent it from spending too long trying to
@@ -16,19 +62,17 @@ a way that you can start several CFEngine components simultaneously without
 them interfering with each other. You can control two things about each kind
 of action in CFEngine:
 
-    ifelapsed
-
-The minimum time (in minutes) which should have passed since the last time
-that promise was verified. It will not be executed again until this amount of
-time has elapsed. Default time is 1 minute.
+* `ifelapsed` - The minimum time (in minutes) which should have passed since the
+  last time that promise was verified. It will not be executed again until this
+  amount of time has elapsed. If the value is `0` the promise has no lock and
+  will always be executed when in context. Additionally, a value of `0` disables
+  function caching. Default time is `1` minute.
 
-    expireafter
-
-The maximum amount (in minutes) of time `cf-agent` should wait for an old
-instantiation to finish before killing it and starting again. You can think
-about [`expireafter`][cf-agent#expireafter] as a timeout to use when a promise verification may
-involve an operation that could wait indefinitely. Default time is 120
-minutes.
+* `expireafter` - The maximum amount (in minutes) of time `cf-agent` should wait
+  for an old instantiation to finish before killing it and starting again. You
+  can think about [`expireafter`][cf-agent#expireafter] as a timeout to use when
+  a promise verification may involve an operation that could wait indefinitely.
+  Default time is `120` minutes.
 
 You can set these values either globally (for all actions) or for each action
 separately. If you set global and local values, the local values override the
@@ -62,3 +106,27 @@ atomic promise checks on the same objects (packages, users, files,
 etc.). Several different `cf-agent` instances can run concurrently.
 The locks ensure that promises will not be verified by two cf-agents
 at the same time or too soon after a verification.
+
+For example, here the `sshd` package promises to be at the latest version. It
+has the `if_elapsed_day` action body attached which sets `ifelapsed` to `1440`
+causing the promise lock to persist for a day effectively restricting the
+promise to run just once a day.
+
+```cf3
+bundle agent __main__
+{
+  packages:
+    "sshd"
+      version => "latest",
+      action => if_elapsed_day,
+      comment => "Make sure sshd is at the latest version, but just once a day.";
+}
+```
+
+Note: Promise locks are ignored when CFEngine is run with the `--no-lock` or
+`-K` option, e.g. a common **manual** execution of the agent, `cf-agent -KI`
+would not respect promises that are locked from a recent execution. Furthermore,
+locks are purged in order to maintain the integrity and health of the underlying
+lock database.
+
+**See also:** [cf_lock.lmdb][CFEngine directory structure#state/cf_lock.lmdb]

overview/directory-structure.markdown

@@ -129,25 +129,34 @@ each run.
 
 ## Database files in /var/cfengine
 
-* bundles.lmdb
-* `cf_classes.lmdb`
+### state/cf_classes.lmdb
 
-A database of classes that have been defined on the current host,
-including their relative frequencies, scaled like a probability.
+A database of classes that have been defined on the current host, including
+their relative frequencies, scaled like a probability.
 
-* `cf_lastseen.lmdb`
+### state/cf_lastseen.lmdb
 
-A database of hosts that last contacted this host, or were contacted by
-this host, and includes the times at which they were last observed.
+A database of hosts that last contacted this host, or were contacted by this
+host, and includes the times at which they were last observed.
 
-* `cf_changes.lmdb`
+### state/cf_lock.lmdb
+
+A database of active and inactive promise locks and their expiry times. Deleting
+this database will reset all lock protections in CFEngine.
+
+**Note:** Locks are purged in order to maintain the integrity and health of the
+underlying lock database. When the lock database utilization grows to 25%
+locks 4 weeks or older are purged. At 50% locks 2 weeks or older are purged
+and at 75% locks older than 1 week are purged.
+
+### state/cf_changes.lmdb
 
 The database of hash values used in CFEngine's change management
 functions.
 
-* `nova_agent_execution.lmdb`
-* `nova_track.lmdb`
-* `performance.lmdb`
+### state/nova_agent_execution.lmdb
+### state/nova_track.lmdb
+### state/performance.lmdb
 
 A database of last, average and deviation times of jobs recorded by
 `cf-agent`. Most promises take an immeasurably short time to check, but
@@ -227,11 +236,6 @@ IP address of the policy server
 
 ## Not verified
 
-* `state/cf_lock.lmdb`
-
-A database of active and inactive locks and their expiry times. Deleting
-this database will reset all lock protections in CFEngine.
-
 * `state/history.lmdb`
 
 CFEngine Enterprise maintains this long-term trend database.

reference/components/cf-agent.markdown

@@ -884,6 +884,7 @@ body agent control
 ```
 
 **Notes:**
+* A value of `0` means no locking, all promises will be executed each execution if in context. This also disables function caching.
 * This is not a reliable way to control frequency over a long period of time.
 * Locks provide simple but weak frequency control.
 * Locks older than 4 weeks are automatically purged.