19 May 2022 |
_slack_kubeflow_U9UFLSBM4 | I'm working on a slightly strange deployment. KServe-raw on openshift without istio. I am able to serve models and get predictions no problem. I can hit /metrics on the kserve-controller-manager-metrics-service and I see metrics, but they all seem to be only for the controller itself rather than anything about the models being served (ie: number of predictions, time per prediction, etc). Is there something else I need to enable or somewhere else I need to be looking to get the metrics about the InferenceService itself? | 15:29:53 |
| @californiatok:matrix.org joined the room. | 22:02:59 |
20 May 2022 |
| 레몬버터구이 joined the room. | 07:14:28 |
| @californiatok:matrix.org left the room. | 09:34:04 |
Dan Sun | croberts that depends on the model server you are running, most of the model servers are producing metrics. | 19:25:09 |
Shri Javadekar | I'm sure curious about this too.
One way I was thinking about this was to write my own logger that exported metrics such as # of predictions, time, prediction confidence score, etc.
Dan Sun: Is there any specific model server that already does this? Triton? | 21:50:13 |
Dan Sun | if you are running knative then queue proxy already produces a set of standard metrics for request count, latency etc, confidence score is a bit tricky as KServe has no idea about the model accuracy | 22:11:08 |
Dan Sun | Triton itself does produce a set of metrics too
https://github.com/triton-inference-server/server/blob/main/docs/metrics.md | 22:11:39 |
| _slack_kubeflow_U03FZJA1GBZ joined the room. | 23:44:39 |
21 May 2022 |
| _slack_kubeflow_U03GF0NC9QS joined the room. | 14:12:13 |
Dan Sun | If anyone from Asia interested in doing a monthly KServe community call in Asia friendly time, please help give a thumbs up here and DM me your email address, thanks! | 16:27:49 |
| Amalesh joined the room. | 16:45:01 |
22 May 2022 |
| mobin nikkhesal joined the room. | 11:46:25 |
| mobin nikkhesal changed their display name from _slack_kubeflow_U03GUM55T0R to mobin nikkhesal. | 11:58:34 |
| mobin nikkhesal set a profile picture. | 11:58:36 |
Shri Javadekar | Is the current schedule of KServe community calls published somewhere? | 15:25:15 |
| mobin nikkhesal changed their profile picture. | 17:00:14 |
23 May 2022 |
zorba(손주형) | what will be done in community call? | 02:50:04 |
| Lize Cai joined the room. | 08:25:53 |
| Abhishek Sharma joined the room. | 11:34:49 |
Jan Buijnsters | https://kserve.github.io/website/0.8/blog/articles/2022-02-18-KServe-0.8-release/#join-the-community
Biweekly Wednesday 9am PST | 14:22:45 |
| Thomas Ounnas joined the room. | 16:11:39 |
24 May 2022 |
| Benjamin Tan changed their profile picture. | 15:25:17 |
_slack_kubeflow_U9UFLSBM4 | Trying to debug a kserve-raw deployment. Can anyone think of reasons why the storage-initializer init container is not getting attached to the tensorflow pod? | 15:57:02 |
Dan Sun | which namespace it is deployed to? it does not work in the kserve control plane namespace | 16:07:24 |
_slack_kubeflow_U9UFLSBM4 | The person trying it originally put it in the kserve namespace, but later tried in another namespace and had the same result | 16:22:01 |
_slack_kubeflow_U9UFLSBM4 | We even tried uninstalling "everything" and re-installing, but we can't seem to get the init-container to happen....no errors in any place we've looked so far | 16:22:39 |
_slack_kubeflow_U9UFLSBM4 | Oh...interesting...her project does have the control-plane: controller-manager label | 16:28:55 |
_slack_kubeflow_U9UFLSBM4 | Thanks Dan Sun that was it. Somehow, she tried in the one project that would NOT work. | 16:32:28 |
Dan Sun | ye the pod mutator webhook checks the namespace label
https://github.com/kserve/kserve/blob/master/config/webhook/manifests.yaml#L40 | 16:37:15 |