| 26 May 2022 |
|  John Paulett joined the room. | 10:59:09 |
 Alexandre Brown | Hello, maybe try modifying the domain template to replace the . with /
kubectl edit configmap config-network -n knative-serving | 11:19:05 |
| John Paulett changed their display name from _slack_kubeflow_U01B8DPEY01 to John Paulett. | 11:19:41 |
| John Paulett set a profile picture. | 11:19:43 |
 Kuba Dawczynski | there was my idea also, but no luck 😕
i've putted
domainTemplate: "{{.Domain}}/{{.Namespace}}/{{.Name}}"
also i've tried to add - instead of / but this not change anything | 12:50:14 |
| Alexandre Brown | Have you tried deploying a new inference service after applying the changes ? (not patching an existing deployment) | 12:51:25 |
| Kuba Dawczynski | yes, i created new inference service couple of times, and i also recreated kserver itself
and it like for domain change in domain config file everyting went well that for this i don't see any changes
unless i'm doing something wrong in config-network cm
apiVersion: v1
data:
mydomain.com: ,
domainTemplate: "{{.Domain}}/{{.Namespace}}/{{.Name}}"
where in mydomain.com i've putted domain which i'm using in config-domain cm | 12:56:33 |
| Alexandre Brown | On my side I didn't put the mydomain.com inside this config map, I put inside
kubectl edit configmap config-domain -n knative-serving
Perhaps you can try moving mydomain.com to this config map and only modifying the domain template in the config-network configmap ?
kubectl edit configmap config-network -n knative-serving | 13:02:28 |
| Kuba Dawczynski | ok but as default in config-network configimap
apiVersion: v1
data:
_example: , | 13:04:43 |
| Alexandre Brown | Yes I believe what is under _example is not applied and only there for documentation purposes | 13:05:47 |
| Alexandre Brown | Maybe try this :
config-network
apiVersion: v1
data:
domainTemplate: "{{.Domain}}/{{.Namespace}}/{{.Name}}"
_example: ,
config-domain
apiVersion: v1
data:
mydomain.com: ""
_example: , | 13:07:41 |
| Kuba Dawczynski | ok give me the second | 13:08:14 |
| Kuba Dawczynski | ```error: configmaps "config-network" could not be patched: admission webhook "config.webhook.serving.knative.dev" denied the request: validation failed: domain template has url path: /bar/foo
You can run `kubectl replace -f /var/folders/m1/c6dp897x6jb4p9k0j32_mqbh0000gq/T/kubectl-edit-2589593085.yaml` to try this update```
validation failed | 13:33:05 |
| Alexandre Brown | Now this makes more sense, it clearly states that having url path /bar/foo is not allowed and so I'd be inclined to think it's simply not feasible | 13:34:15 |
| Kuba Dawczynski | ok, thx for information and your time | 13:41:18 |
 Rachit Chauhan | Due to some limitations at our org, we have to use namespace other than knative-serving for knative’s control plane. Will it have any problems with kserve ? | 19:45:23 |
 Dan Sun | you might need to create additional virtual service for the path rewrite | 19:47:29 |
| Dan Sun | Kunming checkout this example https://github.com/kserve/kserve/blob/master/docs/samples/gcp-iap/virtual-service.yaml | 19:48:10 |
| Dan Sun | should be fine, probably need a few configuration changes | 19:48:48 |
| Rachit Chauhan | ok. I see there are references to knative-serving namespace (for ingresses) in kserve.yaml :
1. https://github.com/kserve/kserve/blob/master/install/v0.8.0/kserve.yaml#L14995
2. https://github.com/kserve/kserve/blob/master/install/v0.8.0/kserve.yaml#L14850
3. https://github.com/kserve/kserve/blob/master/install/v0.8.0/kserve.yaml#L14852 | 19:52:55 |
| Rachit Chauhan | I am going to give it a try and will let here know what problems i see. | 19:53:42 |
| 27 May 2022 |
| Rachit Chauhan | so, the problem that i am seeing when using a different namespace for knative-serving is that for webhooks domainmapping-webhook and webhook , the tls certs are missing.
{"severity":"WARNING","timestamp":"2022-05-27T00:12:49.070225477Z","logger":"domainmapping-webhook","caller":"webhook/webhook.go:154","message":"server key missing","knative.dev/pod":"domainmapping-webhook-775c44fdd7-nrfx4"}
2022/05/27 00:12:49 http: TLS handshake error from 10.1.0.1:62164: tls: no certificates configured
| 00:14:23 |
| Rachit Chauhan | this is happening in a strange way. I deleted the NS and created it fresh and then tried installation and everything worked fine. | 00:46:59 |
 Mark Winter | Triton's fil_backend has support for categorical features recently https://github.com/triton-inference-server/fil_backend#categorical-feature-support | 03:44:08 |
| Mark Winter | You can using triton via kserve | 03:44:39 |
| Mark Winter | Note that it only supports FP32 input/output at the moment | 03:44:47 |
|  รัชพล เเขมภูเขียว joined the room. | 05:46:26 |
| รัชพล เเขมภูเขียว | how to use pvc mount ? fails to reconcile predictor: admission webhook "validation.webhook.serving.knative.dev" denied the request: validation failed: Persistent volume claim support is disabled, but found persistent volume claim task-pv-claim: Persistent volume write support is disabled, but found persistent volume claim task-pv-claim that is not read-only: must not set the field(s): spec.template.spec.volumes[0].persistentVolumeClaim
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
name: nb-logistic-regression
spec:
predictor:
volumes:
- name: volume
persistentVolumeClaim:
claimName: task-pv-claim
containers:
- name: classifier
image: visairegistry.azurecr.io/xxxxxxxx
imagePullPolicy: Always
env:
- name: PROTOCOL
value: v2
volumeMounts:
- name: volume
mountPath: /opt/mlserver/outputs
subPath: inputs-bucket/inputs-sentiment
ports:
- containerPort: 8080
protocol: TCP | 05:48:30 |
| Mark Winter | You should load pvc via STORAGE_URI instead of using volumeMounts directly
env:
- name: STORAGE_URI
value: pvc://pvc-name | 09:11:06 |
|  Nithin R joined the room. | 10:06:23 |
