| 19 Oct 2021 |
 Dan Sun | In reply to@_slack_kubeflow_US7RRCDL2:matrix.org
hmmm
apiVersion: cert-manager.io/v1alpha3
kind: Certificate
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: ,
{"apiVersion":"cert-manager.io/v1alpha2","kind":"Certificate","metadata":{"annotations":{},"labels":{"argocd.argoproj.io/instance":"kfserving-ms-stg"},"name":"serving-cert","namespace":"kserve"},"spec":{"commonName":"kserve-webhook-ser
ver-service.kserve.svc","dnsNames":["kserve-webhook-server-service.kserve.svc"],"issuerRef":{"kind":"Issuer","name":"selfsigned-issuer"},"secretName":"kserve-webhook-server-cert"}}
creationTimestamp: "2021-10-18T20:55:47Z"
generation: 1
labels:
argocd.argoproj.io/instance: kfserving-ms-stg
name: serving-cert
namespace: kserve
resourceVersion: "274549132"
uid: 670ad7e7-2b79-4b26-986f-59f83faba216
spec:
commonName: kserve-webhook-server-service.kserve.svc
dnsNames:
- kserve-webhook-server-service.kserve.svc
issuerRef:
kind: Issuer
name: selfsigned-issuer
secretName: kserve-webhook-server-cert
status:
conditions:
- lastTransitionTime: "2021-10-18T20:55:47Z"
message: Certificate is up to date and has not expired
reason: Ready
status: "True"
type: Ready
notAfter: "2022-01-16T20:55:47Z" thanks for the report, Suresh Nakkiran @_slack_kubeflow_U029WEK7ZQQ:matrix.org Can you help take a look? | 00:42:50 |
|  Andrews Arokiam joined the room. | 00:42:58 |
| Andrews Arokiam changed their display name from _slack_kubeflow_U029WEK7ZQQ to Andrews Arokiam. | 00:42:58 |
| Andrews Arokiam set a profile picture. | 00:43:00 |
|  _slack_kubeflow_U02J9EP8EUU joined the room. | 02:36:33 |
|  Ian Miller joined the room. | 02:42:17 |
| Ian Miller | Installing KServe 0.7.0 from the stand alone manifests. Getting an issue on creating the Certificate and Issuer artifacts. failed calling webhook "webhook.cert-manager.io": the server could not find the requested resource. I am using the existing cert-manager install from the KF 1.3 installation (this deployment has been running fine for ~3 months). I've verified that cert-manager is healthy (including the apiservice) and that the api versions match. Wondering if anyone has any pointers on debugging from here. This issue is all I've found and looks similar but not the same: https://github.com/kserve/kserve/issues/1867 | 02:46:17 |
 Suresh Nakkiran | In reply to@_slack_kubeflow_UFVUV2UFP:matrix.org
thanks for the report, Suresh Nakkiran @_slack_kubeflow_U029WEK7ZQQ:matrix.org Can you help take a look? sure Dan Sun | 03:38:12 |
 Yulong | In reply to@_slack_kubeflow_U0253V66WCT:matrix.org
Dan Sun Thank you for your kind response. That is a possible solution to make it. Tested, it works. Thanks a lot. Dan Sun | 04:41:16 |
| Yulong | In reply to@_slack_kubeflow_U0253V66WCT:matrix.org
Tested, it works. Thanks a lot. Dan Sun I added a timestamp in annotations.
isvc = V1beta1InferenceService(api_version=api_version,
kind=constants.KFSERVING_KIND,
metadata=client.V1ObjectMeta(
name=predictor_name,
namespace=namespace,
annotations={"isvc_timestamp": datetime.datetime.now().strftime("%Y%m%d-%H%M%S"),
"sidecar.istio.io/inject": "false"}),
spec=V1beta1InferenceServiceSpec(
predictor=V1beta1PredictorSpec(
canary_traffic_percent=100,
min_replicas=1,
max_replicas=3,
tensorflow=(V1beta1TFServingSpec(
storage_uri=storage_uri_2,
image=image))))
)
| 04:42:46 |
| Yulong | In reply toundefined
(edited) ... a timestamp in annotations.
```isvc = V1beta1InferenceService(api_version=api_version,
kind=constants.KFSERVING_KIND,
metadata=client.V1ObjectMeta(
name=predictor_name,
namespace=namespace,
annotations={"isvc_timestamp": datetime.datetime.now().strftime("%Y%m%d-%H%M%S"),
"<http://sidecar.istio.io/inject|sidecar.istio.io/inject>": "false"}),
spec=V1beta1InferenceServiceSpec(
predictor=V1beta1PredictorSpec(
canary_traffic_percent=100,
min_replicas=1,
max_replicas=3,
tensorflow=(V1beta1TFServingSpec(
storage_uri=storage_uri_2,
image=image))))
)```
=> ... a datetime in annotations.
```isvc = V1beta1InferenceService(api_version=api_version,
kind=constants.KFSERVING_KIND,
metadata=client.V1ObjectMeta(
name=predictor_name,
namespace=namespace,
annotations={"isvc_timestamp": datetime.datetime.now().strftime("%Y%m%d-%H%M%S"),
"<http://sidecar.istio.io/inject|sidecar.istio.io/inject>": "false"}),
spec=V1beta1InferenceServiceSpec(
predictor=V1beta1PredictorSpec(
canary_traffic_percent=100,
min_replicas=1,
max_replicas=3,
tensorflow=(V1beta1TFServingSpec(
storage_uri=storage_uri_2,
image=image))))
)``` | 04:43:27 |
| Suresh Nakkiran | In reply to@_slack_kubeflow_U02AF95BSUF:matrix.org
sure Dan Sun Timothy Laurent May I know the prev kfserving, knative version, new target knative version and kubernetes version? | 06:41:00 |
|  Biplob Biswas joined the room. | 08:41:44 |
| Biplob Biswas changed their display name from _slack_kubeflow_U02JY0VHJ0Y to Biplob Biswas. | 09:05:57 |
| Biplob Biswas set a profile picture. | 09:06:00 |
| Biplob Biswas | Keith Adler Did you manage to solve this issue? | 09:06:00 |
| Biplob Biswas | Hello community members, We are trying to setup kubeflow pipelines with kserve(kfserving as well). As we found out that kserve moved out of kubeflow so rather than setting up kserve (as it creates its own namesapce and we are not clear of the integration with kubeflow pipelines), we installed kfserving_kubeflow.yaml from here (https://github.com/kserve/kserve/releases/tag/v0.6.1) which installed kfserving in the kubeflow namespace. But the installation resulted in a couple of errors as follows: | 09:32:03 |
| Biplob Biswas | (edited) ... as follows: => ... as follows:
`unable to recognize "<https://github.com/kubeflow/kfserving/releases/download/v0.6.1/kfserving_kubeflow.yaml>": no matches for kind "VirtualService" in version "<http://networking.istio.io/v1beta1|networking.istio.io/v1beta1>"`
`unable to recognize "<https://github.com/kubeflow/kfserving/releases/download/v0.6.1/kfserving_kubeflow.yaml>": no matches for kind "AuthorizationPolicy" in version "<http://security.istio.io/v1beta1|security.istio.io/v1beta1>"`
Is there some documentation or tutorial where we can use kserve(or kfserving) to deploy via kubeflow pipelines?
Caveats:
1. We don't have istio, we are using nginx | 09:37:24 |
|  Matej joined the room. | 09:40:14 |
|  Mark Winter joined the room. | 10:43:38 |
| Mark Winter | KFServing 0.6.1 requires istio. There is a raw deployment mode in alpha in the new 0.7 that doesn't require Knative or Istio | 10:44:17 |
| Mark Winter | https://kserve.github.io/website/admin/kubernetes_deployment/ | 10:44:41 |
| Mark Winter | (edited) KFServing 0.6.1 ... => Biplob Biswas KFServing 0.6.1 ... | 10:45:08 |
| Suresh Nakkiran | In reply to@_slack_kubeflow_U02AF95BSUF:matrix.org
Timothy Laurent May I know the prev kfserving, knative version, new target knative version and kubernetes version? Timothy Laurent I could not reproduce this issue. Below are the steps that I followed to reproduce this issue.
1. Install KFServing v0.6.0 using quick install which uses Istio v1.9.0 and Knative v0.22.0
2. Redeploy Knative v0.24.0
3. Deploy sample tensorflow model
| 11:43:45 |
| Biplob Biswas | In reply to@_slack_kubeflow_U01T25HRREK:matrix.org
https://kserve.github.io/website/admin/kubernetes_deployment/ Hi Mark Winter, so we tried using kserve ()installed it using the raw deployment tutorial) to run the basic inferenceservice example https://kserve.github.io/website/get_started/first_isvc/
But after installing the inferenceservice, the service is not up and the ready state is False with out any url and when we describe the inferenceservice we get the following output (Check image)
We found a similar issue on kserve - https://github.com/kserve/kserve/issues/1839 | 13:03:37 |
| Biplob Biswas | In reply toundefined
(edited) ... - <https://github.com/kserve/kserve/issues/1839> => ... - <https://github.com/kserve/kserve/issues/1839> and tried deleting the `kserve-controller-manager-0` pod but this also doesnt solve the issue.
We think once this is fixed we can then use the kubeflow pipeline for deploying kserve! | 13:04:46 |
| Suresh Nakkiran | In reply to@_slack_kubeflow_U02AF95BSUF:matrix.org
Timothy Laurent I could not reproduce this issue. Below are the steps that I followed to reproduce this issue.
1. Install KFServing v0.6.0 using quick install which uses Istio v1.9.0 and Knative v0.22.0
2. Redeploy Knative v0.24.0
3. Deploy sample tensorflow model Can you help to reproduce the issue? | 14:31:52 |
 Vedant Padwal | https://www.youtube.com/watch?v=0H-HvK8zIUI | 14:51:18 |
|  ANIMESH SINGH joined the room. | 16:12:15 |
| ANIMESH SINGH | In reply to@_slack_kubeflow_U02JY0VHJ0Y:matrix.org
Hello community members, We are trying to setup kubeflow pipelines with kserve(kfserving as well). As we found out that kserve moved out of kubeflow so rather than setting up kserve (as it creates its own namesapce and we are not clear of the integration with kubeflow pipelines), we installed kfserving_kubeflow.yaml from here (https://github.com/kserve/kserve/releases/tag/v0.6.1) which installed kfserving in the kubeflow namespace. But the installation resulted in a couple of errors as follows:
unable to recognize "https://github.com/kubeflow/kfserving/releases/download/v0.6.1/kfserving_kubeflow.yaml": no matches for kind "VirtualService" in version "networking.istio.io/v1beta1"
unable to recognize "https://github.com/kubeflow/kfserving/releases/download/v0.6.1/kfserving_kubeflow.yaml": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
Is there some documentation or tutorial where we can use kserve(or kfserving) to deploy via kubeflow pipelines?
Caveats:
1. We don't have istio, we are using nginx Kubeflow Pipelines componet needs to be updated for KServe release. Timeline will follow when KServe is made available with Kubeflow release.
cc Tommy Li Paul Van Eck | 16:12:15 |
