| 15 Oct 2021 |
|  Kevin Hu set a profile picture. | 21:15:51 |
| 16 Oct 2021 |
|  Nagaraj Janardhana joined the room. | 04:26:21 |
|  Marek Opęchowski joined the room. | 07:37:37 |
|  Andrey Velichkevich joined the room. | 08:38:23 |
| 18 Oct 2021 |
 Yulong | In reply to@_slack_kubeflow_UFVUV2UFP:matrix.org
Any change should be surfaced on inference service yaml which is the whole point of declarative deployment Dan Sun Thank you for your kind response. That is a possible solution to make it. | 03:09:54 |
|  _slack_kubeflow_U025D42L7J9 joined the room. | 06:58:59 |
|  _slack_kubeflow_U0140TXAN4T joined the room. | 08:05:42 |
|  metafeather joined the room. | 12:09:05 |
|  _slack_kubeflow_U02HQ5HFBLP joined the room. | 14:48:45 |
|  Timothy Laurent joined the room. | 19:59:49 |
| Timothy Laurent changed their display name from _slack_kubeflow_US7RRCDL2 to Timothy Laurent. | 20:04:11 |
| Timothy Laurent set a profile picture. | 20:04:11 |
| Timothy Laurent | We're currently upgrading our stg cluster to kserve 0.7 and I'm seeing
Error creating: Internal error occurred: failed calling webhook "inferenceservice.kserve-webhook-server.pod-mutator": Post "https://kserve-webhook-server-service.kserve.
svc:443/mutate-pods?timeout=10s": x509: certificate is not valid for any names, but wanted to match kserve-webhook-server-service.kserve.svc
At the replicaset . I tried to delete the TLS secret but that hasn't resolved the issue.
Just wanted to see if anyone could help point us to how to correctly configure the secret.
Thanks, | 20:04:12 |
| Timothy Laurent | In reply to@_slack_kubeflow_US7RRCDL2:matrix.org
We're currently upgrading our stg cluster to kserve 0.7 and I'm seeing
Error creating: Internal error occurred: failed calling webhook "inferenceservice.kserve-webhook-server.pod-mutator": Post "https://kserve-webhook-server-service.kserve.
svc:443/mutate-pods?timeout=10s": x509: certificate is not valid for any names, but wanted to match kserve-webhook-server-service.kserve.svc
At the replicaset . I tried to delete the TLS secret but that hasn't resolved the issue.
Just wanted to see if anyone could help point us to how to correctly configure the secret.
Thanks, Looks related to this issue:
https://github.com/kserve/kserve/issues/1749 | 20:41:00 |
| Timothy Laurent | In reply to@_slack_kubeflow_US7RRCDL2:matrix.org
Looks related to this issue:
https://github.com/kserve/kserve/issues/1749 hmmm
apiVersion: cert-manager.io/v1alpha3
kind: Certificate
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: ,
{"apiVersion":"cert-manager.io/v1alpha2","kind":"Certificate","metadata":{"annotations":{},"labels":{"argocd.argoproj.io/instance":"kfserving-ms-stg"},"name":"serving-cert","namespace":"kserve"},"spec":{"commonName":"kserve-webhook-ser
ver-service.kserve.svc","dnsNames":["kserve-webhook-server-service.kserve.svc"],"issuerRef":{"kind":"Issuer","name":"selfsigned-issuer"},"secretName":"kserve-webhook-server-cert"}}
creationTimestamp: "2021-10-18T20:55:47Z"
generation: 1
labels:
argocd.argoproj.io/instance: kfserving-ms-stg
name: serving-cert
namespace: kserve
resourceVersion: "274549132"
uid: 670ad7e7-2b79-4b26-986f-59f83faba216
spec:
commonName: kserve-webhook-server-service.kserve.svc
dnsNames:
- kserve-webhook-server-service.kserve.svc
issuerRef:
kind: Issuer
name: selfsigned-issuer
secretName: kserve-webhook-server-cert
status:
conditions:
- lastTransitionTime: "2021-10-18T20:55:47Z"
message: Certificate is up to date and has not expired
reason: Ready
status: "True"
type: Ready
notAfter: "2022-01-16T20:55:47Z"
| 20:59:02 |
|  Luc Cary joined the room. | 22:27:24 |
| Luc Cary changed their display name from _slack_kubeflow_U017XEVD8AD to Luc Cary. | 22:27:43 |
| Luc Cary set a profile picture. | 22:27:44 |
| 19 Oct 2021 |
 Dan Sun | In reply to@_slack_kubeflow_US7RRCDL2:matrix.org
hmmm
apiVersion: cert-manager.io/v1alpha3
kind: Certificate
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: ,
{"apiVersion":"cert-manager.io/v1alpha2","kind":"Certificate","metadata":{"annotations":{},"labels":{"argocd.argoproj.io/instance":"kfserving-ms-stg"},"name":"serving-cert","namespace":"kserve"},"spec":{"commonName":"kserve-webhook-ser
ver-service.kserve.svc","dnsNames":["kserve-webhook-server-service.kserve.svc"],"issuerRef":{"kind":"Issuer","name":"selfsigned-issuer"},"secretName":"kserve-webhook-server-cert"}}
creationTimestamp: "2021-10-18T20:55:47Z"
generation: 1
labels:
argocd.argoproj.io/instance: kfserving-ms-stg
name: serving-cert
namespace: kserve
resourceVersion: "274549132"
uid: 670ad7e7-2b79-4b26-986f-59f83faba216
spec:
commonName: kserve-webhook-server-service.kserve.svc
dnsNames:
- kserve-webhook-server-service.kserve.svc
issuerRef:
kind: Issuer
name: selfsigned-issuer
secretName: kserve-webhook-server-cert
status:
conditions:
- lastTransitionTime: "2021-10-18T20:55:47Z"
message: Certificate is up to date and has not expired
reason: Ready
status: "True"
type: Ready
notAfter: "2022-01-16T20:55:47Z" thanks for the report, Suresh Nakkiran @_slack_kubeflow_U029WEK7ZQQ:matrix.org Can you help take a look? | 00:42:50 |
|  Andrews Arokiam joined the room. | 00:42:58 |
| Andrews Arokiam changed their display name from _slack_kubeflow_U029WEK7ZQQ to Andrews Arokiam. | 00:42:58 |
| Andrews Arokiam set a profile picture. | 00:43:00 |
|  _slack_kubeflow_U02J9EP8EUU joined the room. | 02:36:33 |
|  Ian Miller joined the room. | 02:42:17 |
| Ian Miller | Installing KServe 0.7.0 from the stand alone manifests. Getting an issue on creating the Certificate and Issuer artifacts. failed calling webhook "webhook.cert-manager.io": the server could not find the requested resource. I am using the existing cert-manager install from the KF 1.3 installation (this deployment has been running fine for ~3 months). I've verified that cert-manager is healthy (including the apiservice) and that the api versions match. Wondering if anyone has any pointers on debugging from here. This issue is all I've found and looks similar but not the same: https://github.com/kserve/kserve/issues/1867 | 02:46:17 |
 Suresh Nakkiran | In reply to@_slack_kubeflow_UFVUV2UFP:matrix.org
thanks for the report, Suresh Nakkiran @_slack_kubeflow_U029WEK7ZQQ:matrix.org Can you help take a look? sure Dan Sun | 03:38:12 |
| Yulong | In reply to@_slack_kubeflow_U0253V66WCT:matrix.org
Dan Sun Thank you for your kind response. That is a possible solution to make it. Tested, it works. Thanks a lot. Dan Sun | 04:41:16 |
| Yulong | In reply to@_slack_kubeflow_U0253V66WCT:matrix.org
Tested, it works. Thanks a lot. Dan Sun I added a timestamp in annotations.
isvc = V1beta1InferenceService(api_version=api_version,
kind=constants.KFSERVING_KIND,
metadata=client.V1ObjectMeta(
name=predictor_name,
namespace=namespace,
annotations={"isvc_timestamp": datetime.datetime.now().strftime("%Y%m%d-%H%M%S"),
"sidecar.istio.io/inject": "false"}),
spec=V1beta1InferenceServiceSpec(
predictor=V1beta1PredictorSpec(
canary_traffic_percent=100,
min_replicas=1,
max_replicas=3,
tensorflow=(V1beta1TFServingSpec(
storage_uri=storage_uri_2,
image=image))))
)
| 04:42:46 |
| Yulong | In reply toundefined
(edited) ... a timestamp in annotations.
```isvc = V1beta1InferenceService(api_version=api_version,
kind=constants.KFSERVING_KIND,
metadata=client.V1ObjectMeta(
name=predictor_name,
namespace=namespace,
annotations={"isvc_timestamp": datetime.datetime.now().strftime("%Y%m%d-%H%M%S"),
"<http://sidecar.istio.io/inject|sidecar.istio.io/inject>": "false"}),
spec=V1beta1InferenceServiceSpec(
predictor=V1beta1PredictorSpec(
canary_traffic_percent=100,
min_replicas=1,
max_replicas=3,
tensorflow=(V1beta1TFServingSpec(
storage_uri=storage_uri_2,
image=image))))
)```
=> ... a datetime in annotations.
```isvc = V1beta1InferenceService(api_version=api_version,
kind=constants.KFSERVING_KIND,
metadata=client.V1ObjectMeta(
name=predictor_name,
namespace=namespace,
annotations={"isvc_timestamp": datetime.datetime.now().strftime("%Y%m%d-%H%M%S"),
"<http://sidecar.istio.io/inject|sidecar.istio.io/inject>": "false"}),
spec=V1beta1InferenceServiceSpec(
predictor=V1beta1PredictorSpec(
canary_traffic_percent=100,
min_replicas=1,
max_replicas=3,
tensorflow=(V1beta1TFServingSpec(
storage_uri=storage_uri_2,
image=image))))
)``` | 04:43:27 |
| Suresh Nakkiran | In reply to@_slack_kubeflow_U02AF95BSUF:matrix.org
sure Dan Sun Timothy Laurent May I know the prev kfserving, knative version, new target knative version and kubernetes version? | 06:41:00 |
