In reply to@_slack_kubeflow_U02JY0VHJ0Y:matrix.org
Hello community members, We are trying to setup kubeflow pipelines with kserve(kfserving as well). As we found out that kserve moved out of kubeflow so rather than setting up kserve (as it creates its own namesapce and we are not clear of the integration with kubeflow pipelines), we installed kfserving_kubeflow.yaml from here (https://github.com/kserve/kserve/releases/tag/v0.6.1) which installed kfserving in the kubeflow namespace. But the installation resulted in a couple of errors as follows: unable to recognize "https://github.com/kubeflow/kfserving/releases/download/v0.6.1/kfserving_kubeflow.yaml": no matches for kind "VirtualService" in version "networking.istio.io/v1beta1" unable to recognize "https://github.com/kubeflow/kfserving/releases/download/v0.6.1/kfserving_kubeflow.yaml": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1" Is there some documentation or tutorial where we can use kserve(or kfserving) to deploy via kubeflow pipelines? Caveats: 1. We don't have istio, we are using nginx

In reply to@_slack_kubeflow_U9673D1KJ:matrix.org
Kubeflow Pipelines componet needs to be updated for KServe release. Timeline will follow when KServe is made available with Kubeflow release. cc Tommy Li Paul Van Eck

In reply to@_slack_kubeflow_U9FNKAAD9:matrix.org
Kubeflow 1.4 still comes with KFServing. The default deployment of Kserve/KFServing requires istio to be installed first, so you see the above errors if you don’t have istio.

In reply to@_slack_kubeflow_U01FC4Y6QBB:matrix.org
Installing KServe 0.7.0 from the stand alone manifests. Getting an issue on creating the Certificate and Issuer artifacts. failed calling webhook "webhook.cert-manager.io": the server could not find the requested resource. I am using the existing cert-manager install from the KF 1.3 installation (this deployment has been running fine for ~3 months). I've verified that cert-manager is healthy (including the apiservice) and that the api versions match. Wondering if anyone has any pointers on debugging from here. This issue is all I've found and looks similar but not the same: https://github.com/kserve/kserve/issues/1867

In reply to@_slack_kubeflow_U02JY0VHJ0Y:matrix.org
Hi Mark Winter, so we tried using kserve ()installed it using the raw deployment tutorial) to run the basic inferenceservice example https://kserve.github.io/website/get_started/first_isvc/ But after installing the inferenceservice, the service is not up and the ready state is False with out any url and when we describe the inferenceservice we get the following output (Check image) We found a similar issue on kserve - https://github.com/kserve/kserve/issues/1839 and tried deleting the kserve-controller-manager-0 pod but this also doesnt solve the issue. We think once this is fixed we can then use the kubeflow pipeline for deploying kserve!

In reply to@_slack_kubeflow_U01HS89M1U6:matrix.org
Do I understand correctly, that “Model Experimentation” from the roadmap will give us the possibility of shadow deployments?

In reply to@_slack_kubeflow_UM56LA7N3:matrix.org
tbh I think it could be possible with traffic mirroring

In reply to@_slack_kubeflow_U01HS89M1U6:matrix.org
which technology will allow for that? Googling for kserve/kfserving traffic mirroring does not yield any useful results

In reply to@_slack_kubeflow_UM56LA7N3:matrix.org
Knative might be able to do it (Kfsering is based on it)

In reply to@_slack_kubeflow_UM56LA7N3:matrix.org
Maybe: https://iter8.tools/0.6/tutorials/traffic-engineering/mirroring/

In reply to@_slack_kubeflow_U01HS89M1U6:matrix.org
Thanks! I guess now my problem will be that I actually want to communicate with model via Kafka, not HTTP.

In reply to@_slack_kubeflow_U01T25HRREK:matrix.org
Traffic mirroring can be done with Istio

In reply to@_slack_kubeflow_U01T25HRREK:matrix.org
The Iter8 tool is very helpful and worth checking out

In reply to@_slack_kubeflow_UFVUV2UFP:matrix.org
Ian Miller The KServe 0.7 is not certified with Kubeflow yet, hopefully that will land in Kubeflow 1.5. Do you still have kfserving running with your Kubeflow 1.3 installation ?

In reply to@_slack_kubeflow_U02AF95BSUF:matrix.org
Can you help to reproduce the issue?

In reply to@_slack_kubeflow_US7RRCDL2:matrix.org
Sorry for the delay Suresh Nakkiran We encountered this problem while upgrading from KFServing v0.6.0 to KServe v0.7.0 we've since "retreated" to v0.6.0 because of the issue --- mmm but looks like that is broken as well. We deployed the manifest here: https://github.com/kserve/kserve/releases/download/v0.7.0/kserve.yaml Knative serving.version 0.25.0 Istio 1.8.5

In reply to@_slack_kubeflow_U02JY0VHJ0Y:matrix.org
iamlovingit Continuing our conversation from https://github.com/kserve/kserve/issues/1872 First of all thanks a lot to you and Dan Sun for replying to our queries so quickly. Now we just need to know 1 thing, we have updated the v1 to v1beta1 as you mentioned, but we are not sure whether making dev-deploy would install Kserve in the "Serverless" mode or "RawDeployment" mode. As mentioned in the ticket, we want to use RawDeployment mode as we don't have istio and knative serving installed in our cluster and we are using nginx for ingress.

In reply to@_slack_kubeflow_U0104H1616Z:matrix.org
I see, before you run make deploy-dev , you should change the default deployment to RawDeployment

In reply to@_slack_kubeflow_U0104H1616Z:matrix.org
https://github.com/kserve/kserve/blob/fb239bde6395e107a9e884ff36671cdf23a0571d/config/configmap/inferenceservice.yaml#L190

"defaultDeploymentMode": "RawDeployment"

In reply to@_slack_kubeflow_U0104H1616Z:matrix.org

"defaultDeploymentMode": "RawDeployment"