system-operational-cache: system daemon that provides cache of operational information

[ishidawataru]

Moved the discussion from #63 (comment)

Considering adding system-operational-cache to serve the following YANG model.
system-operational-cache provides an operational info-caching mechanism for all internal components.

// goldstone-operational-cache.yang

rpc get-operational-cache {
  input {
    leaf xpath {
      type string;
    }
    leaf acceptable-cache-age {
      ...
    }
  }
  output {
    anydata data {
      ...
    }
    leaf cache-hit {
      type boolean;
    }
    leaf cache-age {
      ...
    }
  }
}

intended use:

• north-snmp: periodically fetches operational information by querying Goldstone YANG models.
• system-telemetry: periodically fetches operational information of the subscribed XPATH

Consideration points: (copied from #63 (comment))

• Should it get operational states for a cache miss?
  • Yes: It is useful for uses. But get-operational-cache may be called recursively. Can the framework handle it?
    • e.g. system-telemetry -get-operational-cache-> xlate-oc -get-operational-cache-> south-tai
  • No: It should provide set-operational-cache RPC and users call it to save retrieved operational states in the cache.
• Should it return subtree data specified by xpath? e.g. If data for /openconfig-platform:components is cached, cached data is returned for request with /openconfig-platform:components/component/name
  • Yes: It is useful for uses and reduces load on the system. But, I think, it is difficult. Now we are dealing it by using the central datastore.
  • No: Uses may find data inconsistency.

[ishidawataru]

Should it get operational states for a cache miss?

Only allow caching operational info of Goldstone native models?

---

Another idea is to add rpc get-operational-cache to all Goldstone native YANG models and let south (and xlate) daemons handle it.

We could improve lib/core.py to have a generic caching mechanism to avoid ad-hoc caching implementation in each daemon.

Then we could also improve lib/connector/sysrepo.py to provide user-friendly caching support.

# check if the model of the XPATH has `rpc get-operational-cache`,
# if it has, use it with the given acceptable-cache-age. if it doesn't have, just query operational datastore
conn.get_operational("/goldstone-interfaces:interfaces", acceptable_cache_age=60)

Should it return subtree data specified by xpath?

Handling this might become easier because now the cache query will be handled inside the daemon that handles the actual operational information.

@noguchiko WDYT?

[noguchiko]

Should it get operational states for a cache miss?

Only allow caching operational info of Goldstone native models?

It sounds good for now. system-operational-cache can reject an RPC for a non-supported model.

Should it return subtree data specified by xpath?

Handling this might become easier because now the cache query will be handled inside the daemon that handles the actual operational information.

We should handle XPath::Abbreviated Syntax to do it. I think it's the main reason for the difficulty.

Then we could also improve lib/connector/sysrepo.py to provide user-friendly caching support.

# check if the model of the XPATH has `rpc get-operational-cache`,
# if it has, use it with the given acceptable-cache-age. if it doesn't have, just query operational datastore
conn.get_operational("/goldstone-interfaces:interfaces", acceptable_cache_age=60)

It's useful even if we choose which option.

We have three options:

• single implementation, single runtime entity: system-operational-cache
• single implementation, multiple runtime entity: south daemons with lib
• multiple implementation, multiple runtime entity: south daemons with ad-hoc caching implementation

I believe ad-hoc implementations should be avoided.

Two remains:

• single implementation, single runtime entity: system-operational-cache
• single implementation, multiple runtime entity: south daemons with lib

We need further consideration.

[ishidawataru]

We should handle XPath::Abbreviated Syntax to do it.

What is the expected usage of this?
Does system-telemetry need to use it? I think north-snmp doesn't need to use any abbreviated syntax.

[noguchiko]

What is the expected usage of this?
Does system-telemetry need to use it? I think north-snmp doesn't need to use any abbreviated syntax.

system-telemetry will accept an xpath with abbreviated syntax as /goldstone-telemetry:subscribe-requests/subscribe-request/subscription/subscription/config/path if its user requests so.

Even if we don't supprt the syntax for now, I think we should design the architecture to be able to support it. So, I think the implementation of the subtree data retrieval should be highly cohesive. (lib or system-operational-cache)