Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add _stats API to retreive statistics on plugin. #302

Closed
adnapibar opened this issue May 1, 2020 · 2 comments
Closed

Add _stats API to retreive statistics on plugin. #302

adnapibar opened this issue May 1, 2020 · 2 comments

Comments

@adnapibar
Copy link
Contributor

adnapibar commented May 1, 2020

Currently, the plugin exposes a _cachestats API to get some information on the cache usage. It could be a good idea to provide a stats API which will provide an overall combined usage and performance statistics on the plugin,

node level statistics including the cache statistics with more details (as mentioned in #140), e.g,

 "nodes" : {
    "IgWDUfzFRzW0FWAXM5FGJw" : {
      "timestamp": 1585250953737,
      "name": "elasticsearch1",
      "cache": {
        "feature": {
            "eviction_count": 0,
            "miss_count": 1,
            "hit_count": 2,
            "entry_count": 10,
            "memory_size_in_bytes": 200
        },
        "featureset": {
            "eviction_count": 0,
            "miss_count": 1,
            "hit_count": 2,
            "entry_count": 1,
            "memory_size_in_bytes": 300
        },
        "model": {
            "eviction_count": 1,
            "miss_count": 9,
            "hit_count": 10,
            "entry_count": 9,
            "memory_size_in_bytes": 2000
        }
      }
    },
    "y7YUQWukQEWOYbfdEq13hQ" : {
      ...
    }
  }

cluster wide information on the plugin such as overall health and information on stores, e.g.,

"status": "green",
"stores": {
    ".ltrstore": {
        "status": "green",
        "feature_count": 2,
        "feature_set_count": 1,
        "model_count" : 9
    },
  }

I'd like to know your views on this and any suggestions on other metrics (such as model performance) that might be helpful to add as part of the API.

@nathancday
Copy link
Member

nathancday commented May 21, 2020

I like this idea a lot. I think it would be a huge improvement for visibility.

I'm not immediately sure what other metrics should be added, but would love input for other folks on what they'd like to see. Just exposing the eviction count would be a win to address #140

adnapibar pushed a commit to adnapibar/elasticsearch-learning-to-rank that referenced this issue May 27, 2020
 Fixes o19s#302, adds the `_ltr\_stats` api to retrieve aggregate stats on the plugin.
@nathancday
Copy link
Member

Can not say thank you enough for the awesome contribution! Very much appreciated @adnapibar 🙇‍♂️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants