Skip to content
This repository has been archived by the owner on Dec 1, 2021. It is now read-only.

Prediction Analytics Dashboard not showing metrics #42

Open
jmdinis opened this issue Jul 17, 2019 · 0 comments
Open

Prediction Analytics Dashboard not showing metrics #42

jmdinis opened this issue Jul 17, 2019 · 0 comments

Comments

@jmdinis
Copy link

jmdinis commented Jul 17, 2019

I deployed a keras mnist model using seldon core and I'm trying to monitor the model using seldon-core-analytics with grafana dashaboards. The cluster_monitoring dashboard looks fine but the Prediction Analytics doesn't find the my deployed model and all the panels are empty. I have installed kubeflow, the seldon-core and seldon-core-analytics in the same namespace and I have my model deployed in that same namespace.

After checking prometheus service logs in the same namespace I found the following message recurrently:

level=warn ts=2019-07-17T16:24:27.691294684Z caller=scrape.go:836 component="scrape manager" scrape_pool=kubernetes-pods target=http://192.168.49.198:16686/metrics msg="append failed" err=""INVALID" is not a valid start token"

Can you please advise on this?

Thanks.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant