3 Jun 2022 |
zorba(손주형) | yeah but this is not about terminationGracePeriodSecond.
it’s timeout of calling predictor service (In seconds).
As in the document.
apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
name: "pytorch-cifar10"
spec:
predictor:
timeout: 60
minReplicas: 1
batcher:
maxBatchSize: 32
maxLatency: 5000
pytorch:
storageUri: " gs://kfserving-examples/models/torchserve/image-classifier "
• maxBatchSize : the max batch size for triggering a prediction.
• maxLatency : the max latency for triggering a prediction (In milliseconds).
• timeout : timeout of calling predictor service (In seconds).
| 11:39:20 |
Christian Lehre | Nevermind, I figured it out. Had to annotate the service account in the namespace with the gcp service principal. Thank you so much for your help, you really guided me into the right track 😄 | 11:54:12 |
Benjamin Tan | Yayyyyy | 11:57:14 |
Benjamin Tan | Awesome 👌 👏 👍 | 11:57:19 |
@californiatl:matrix.org | I'll help 10 individuals how to earn $20,000 in just 72 hours from the crypto market. But you will pay me 10% commission when you receive your profit. if interested send me a direct message via WhatsApp by asking me HOW for more details on how to get started
+1 (2297781881 | 11:58:06 |
Sebastian Lehrig | any admin here to get rid of this spam? | 12:01:49 |
Dan Sun | The termination grace period is controlled by timeout because you need to wait for the request to drain and finish processing before shutting down the pod. | 12:36:22 |
Dan Sun | That’s why it is not allowed setting termination grace period directly | 12:36:50 |
zorba(손주형) | very nice. thanks | 13:24:57 |
| Demetrios joined the room. | 14:41:41 |
Rachit Chauhan | Shri Javadekar: I followed this https://kserve.github.io/website/0.8/developer/developer/#install-knative-on-a-kubernetes-cluster to test it out. You can checkout the branch for PR https://github.com/kserve/kserve/pull/1910 and follow this https://kserve.github.io/website/0.8/developer/developer/#deploy-kserve-with-your-own-version | 15:58:41 |
Robert Irvine | hi, I use kserve batching which is great but it means you cannot pass in parameters others than the inference text. For debugging I want to be able to bypass this and just call with parameters directly (e.g diff temperature values for nlp model). Is this possible? | 16:19:18 |
Robert Irvine | if its not possible then I have to deploy a second instance without batching just for debugging with additional parameters (which kinda sucks) | 16:20:47 |
Thomas Ounnas | Kenneth Koski it seems you installed this app, could you remove this spams ? | 17:50:09 |
| Pradeep Thalasta joined the room. | 18:39:20 |
Pradeep Thalasta | Redacted or Malformed Event | 18:39:26 |
Thomas Ounnas | Hello 👋 ,
Just a quick feedback, it may have change in new KServe version, but for those on Kubeflow v1.4 (KFServing/KServe v0.6.1) like me, I had an issue when trying to deploy an sklearn inference service, and got a 404...🤔 Everything was ok (pod, authpolicy, iam role service account, etc..) I kinda went crazy since 404 means Not Found but I had the logs of the pod with the requests’ hit arriving...
It turns out... hidden in the logs, I found an error with the load of the model, indeed, KServe v0.6.1 rely on sklearn 0.20.3 and there was a version mismatch.... 🤦♂️
Kinda silly error, that I would have found more quickly if the status was a 502 (since I would have look at the right place, server side error) 😅 | 20:07:04 |
Vivian Pan | Shri Javadekar I already have the v0.8.0 version of kserve installed in a cluster, then I built and pushed the latest version of the kserve controller and router from the graph feature branch to a registry. Then reconfigure the existing kserve set up from v0.8.0 images to the feature branch images.
You should be able to test out the inference graph features. Otherwise you can do the local dev setup that is documented above | 21:11:35 |
@californiatl:matrix.org | I'll help 10 individuals how to earn $20,000 in just 72 hours from the crypto market. But you will pay me 10% commission when you receive your profit. if interested send me a direct message via WhatsApp by asking me HOW for more details on how to get started
+1 (2297781881 | 21:18:44 |
Shri Javadekar | Got it.. thanks for the reply! | 21:18:51 |
Dan Sun | Where can you see the installed app? | 22:26:50 |
Dan Sun | It means that the model is not found, I think in later version we actually failed the start up in case model is not loaded | 22:38:53 |
Dan Sun | Mathew Wicks can you help remove the spam app? | 22:41:50 |
4 Jun 2022 |
Mathew Wicks | James Wu can you please remove the "element bridge" slack app ASAP? It is spamming this channel with commercial ads. | 01:21:12 |
@californiatl:matrix.org | I'll help 10 individuals how to earn $20,000 in just 72 hours from the crypto market. But you will pay me 10% commission when you receive your profit. if interested send me a direct message via WhatsApp by asking me HOW for more details on how to get started
> +1 (2297781881
| 09:49:01 |
| Andrej Albrecht changed their display name from _slack_kubeflow_U02QG484SPM to Andrej Albrecht. | 18:35:44 |
| Andrej Albrecht set a profile picture. | 18:35:46 |
Maciek Stopa | Hey Cesar Flores did you manage to solve this problem back then?
I'm facing a similar issue when I try to follow https://kserve.github.io/website/0.7/get_started/first_isvc/ , I get 404 even when I send requests from a pod within my cluster or run performance tests according to docs.
In my case kubectl describe inferenceservice sklearn-iris -n kserve-test outputs
Warning InternalError 5m26s v1beta1Controllers fails to reconcile predictor: fails to update knative service: Operation cannot be fulfilled on services.serving.knative.dev "sklearn-iris-predictor-default": the object has been modified; please apply your changes to the latest version and try again . Were you getting the same warning? | 23:50:07 |
5 Jun 2022 |
| Jack Jin joined the room. | 19:47:02 |
Jack Jin | Hello.
I installed Kubeflow 1.5, and follow up the instruction in https://www.kubeflow.org/docs/external-add-ons/kserve/webapp/ to install the Metrics with below yaml, not seeing any error during the yaml installation, and all the pod in knative-monitoring namespace are working fine, but the Kubeflow GUI models detail page doesn't show the METRICS tab.
I had a mnist model with trition inference server, all the inference components are green check.
Does any one have experience on this?
Thanks
configmap.yaml
monitoring-metrics-prometheus.yaml
authorizationpolicy.yaml
monitoring-core.yaml
virtualservice.yaml | 19:54:30 |