| 20 Oct 2021 |
 Timothy Laurent | In reply to@_slack_kubeflow_US7RRCDL2:matrix.org
Sorry for the delay Suresh Nakkiran
We encountered this problem while upgrading from KFServing v0.6.0 to KServe v0.7.0 we've since "retreated" to v0.6.0 because of the issue --- mmm but looks like that is broken as well.
We deployed the manifest here: https://github.com/kserve/kserve/releases/download/v0.7.0/kserve.yaml
Knative serving.version 0.25.0
Istio 1.8.5 We think it's related to Istio sidecar injection. | 20:07:43 |
| 21 Oct 2021 |
|  _slack_kubeflow_U02JNS1HCE5 joined the room. | 04:01:48 |
 Biplob Biswas | iamlovingit Continuing our conversation from https://github.com/kserve/kserve/issues/1872
First of all thanks a lot to you and Dan Sun for replying to our queries so quickly.
Now we just need to know 1 thing, we have updated the v1 to v1beta1 as you mentioned, but we are not sure whether making dev-deploy would install Kserve in the "Serverless" mode or "RawDeployment" mode. As mentioned in the ticket, we want to use RawDeployment mode as we don't have istio and knative serving installed in our cluster and we are using nginx for ingress. | 07:53:44 |
 iamlovingit | In reply to@_slack_kubeflow_U02JY0VHJ0Y:matrix.org
iamlovingit Continuing our conversation from https://github.com/kserve/kserve/issues/1872
First of all thanks a lot to you and Dan Sun for replying to our queries so quickly.
Now we just need to know 1 thing, we have updated the v1 to v1beta1 as you mentioned, but we are not sure whether making dev-deploy would install Kserve in the "Serverless" mode or "RawDeployment" mode. As mentioned in the ticket, we want to use RawDeployment mode as we don't have istio and knative serving installed in our cluster and we are using nginx for ingress. I see, before you run make deploy-dev , you should change the default deployment to RawDeployment | 08:01:27 |
| iamlovingit | In reply to@_slack_kubeflow_U0104H1616Z:matrix.org
I see, before you run make deploy-dev , you should change the default deployment to RawDeployment https://github.com/kserve/kserve/blob/fb239bde6395e107a9e884ff36671cdf23a0571d/config/configmap/inferenceservice.yaml#L190 | 08:02:04 |
| iamlovingit | In reply to@_slack_kubeflow_U0104H1616Z:matrix.org
https://github.com/kserve/kserve/blob/fb239bde6395e107a9e884ff36671cdf23a0571d/config/configmap/inferenceservice.yaml#L190 "defaultDeploymentMode": "RawDeployment" | 08:03:06 |
| Biplob Biswas | In reply to@_slack_kubeflow_U0104H1616Z:matrix.org
"defaultDeploymentMode": "RawDeployment" iamlovingit Thanks a ton, we would try this out and get back to you. But seriously, thanks! | 08:05:52 |
|  Arun George joined the room. | 12:34:49 |
| Arun George | https://github.com/machine-learning-exchange/mlx/blob/main/docs/install-mlx-on-kubeflow.md | 12:56:55 |
| Arun George | In reply to@_slack_kubeflow_UTURPK4N4:matrix.org
https://github.com/machine-learning-exchange/mlx/blob/main/docs/install-mlx-on-kubeflow.md Does it work in aws? ANIMESH SINGH | 12:57:07 |
| Biplob Biswas | In reply to@_slack_kubeflow_U02JY0VHJ0Y:matrix.org
iamlovingit Thanks a ton, we would try this out and get back to you. But seriously, thanks! iamlovingit So we tried building after the changes but we are getting the following error:
``./hack/image_patch_dev.sh development
2021/10/21 15:26:48 Using base gcr.io/distroless/static:nonroot for github.com/kserve/kserve/cmd/manager
2021/10/21 15:26:49 Building github.com/kserve/kserve/cmd/manager for linux/amd64
2021/10/21 15:27:16 Unexpected error running "go build": exit status 2
# github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/reconcilers/ingress
pkg/controller/v1beta1/inferenceservice/reconcilers/ingress/kube_ingress_reconciler.go:73:8: unknown field 'Service' in struct literal of type "k8s.io/api/networking/v1beta1".IngressBackend
Error: error processing import paths in "config/manager/manager.yaml": error resolving image references: exit status 2
2021/10/21 15:27:16 error during command execution:error processing import paths in "config/manager/manager.yaml": error resolving image references: exit status 2
# Remove the certmanager certificate if KSERVE_ENABLE_SELF_SIGNED_CA is not false
cd config/default && if [ false != false ]; then \
kustomize edit remove resource certmanager/certificate.yaml; \
else kustomize edit add resource certmanager/certificate.yaml; fi;
2021/10/21 15:27:16 certmanager/certificate.yaml has no match
kustomize build config/overlays/development , kubectl apply --validate=false -f -
Error: trouble configuring builtin PatchStrategicMergeTransformer with config:
paths:
- configmap/inferenceservice_patch.yaml
- manager_image_patch.yaml
: evalsymlink failure on '/Users/username/kserve/config/overlays/development/configmap/inferenceservice_patch.yaml' : lstat /Users/username/kserve/config/overlays/development/configmap/inferenceservice_patch.yaml: no such file or directory
error: no objects passed to apply
make: *** [deploy-dev] Error 1``
| 13:50:38 |
 Suresh Nakkiran | In reply to@_slack_kubeflow_US7RRCDL2:matrix.org
We think it's related to Istio sidecar injection. Timothy Laurent I tried with the above config but I could not reproduce the issue. Below are the steps that I followed to reproduce this issue.
1. Install KFServing v0.6.0 with Istio v1.8.5 and Knative v0.25.0
2. Deploy Kserve v0.7.0 with manifest file
3. Deploy sample tensorflow inference service
Please correct me if I am wrong. | 14:30:44 |
| Timothy Laurent | In reply to@_slack_kubeflow_U02AF95BSUF:matrix.org
Timothy Laurent I tried with the above config but I could not reproduce the issue. Below are the steps that I followed to reproduce this issue.
1. Install KFServing v0.6.0 with Istio v1.8.5 and Knative v0.25.0
2. Deploy Kserve v0.7.0 with manifest file
3. Deploy sample tensorflow inference service
Please correct me if I am wrong. We're chatting about it now -- When we turned off Istio injection the certificate error went away but we noticed that no queue-proxy was coming up in the services.
I'll surely report our resolution here once we get it working. | 14:36:39 |
|  Almog Baku joined the room. | 15:55:21 |
| Almog Baku changed their display name from _slack_kubeflow_U0177T2LE5Q to Almog Baku. | 15:55:23 |
| Almog Baku set a profile picture. | 15:55:24 |
| Almog Baku | Starting in a few more minutes!
https://www.eventbrite.com/e/serving-ml-at-scale-with-kserve-animesh-singh-ibm-cto-tickets-187680616777 | 15:55:24 |
| Suresh Nakkiran | In reply to@_slack_kubeflow_US7RRCDL2:matrix.org
We're chatting about it now -- When we turned off Istio injection the certificate error went away but we noticed that no queue-proxy was coming up in the services.
I'll surely report our resolution here once we get it working. Okay. Thanks! | 16:02:04 |
| Arun George | In reply to@_slack_kubeflow_U0177T2LE5Q:matrix.org
Starting in a few more minutes!
https://www.eventbrite.com/e/serving-ml-at-scale-with-kserve-animesh-singh-ibm-cto-tickets-187680616777 Unable to get in. It says waiting. | 16:03:55 |
| iamlovingit | In reply to@_slack_kubeflow_U02JY0VHJ0Y:matrix.org
iamlovingit So we tried building after the changes but we are getting the following error:
``./hack/image_patch_dev.sh development
2021/10/21 15:26:48 Using base gcr.io/distroless/static:nonroot for github.com/kserve/kserve/cmd/manager
2021/10/21 15:26:49 Building github.com/kserve/kserve/cmd/manager for linux/amd64
2021/10/21 15:27:16 Unexpected error running "go build": exit status 2
# github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/reconcilers/ingress
pkg/controller/v1beta1/inferenceservice/reconcilers/ingress/kube_ingress_reconciler.go:73:8: unknown field 'Service' in struct literal of type "k8s.io/api/networking/v1beta1".IngressBackend
Error: error processing import paths in "config/manager/manager.yaml": error resolving image references: exit status 2
2021/10/21 15:27:16 error during command execution:error processing import paths in "config/manager/manager.yaml": error resolving image references: exit status 2
# Remove the certmanager certificate if KSERVE_ENABLE_SELF_SIGNED_CA is not false
cd config/default && if [ false != false ]; then \
kustomize edit remove resource certmanager/certificate.yaml; \
else kustomize edit add resource certmanager/certificate.yaml; fi;
2021/10/21 15:27:16 certmanager/certificate.yaml has no match
kustomize build config/overlays/development , kubectl apply --validate=false -f -
Error: trouble configuring builtin PatchStrategicMergeTransformer with config:
paths:
- configmap/inferenceservice_patch.yaml
- manager_image_patch.yaml
: evalsymlink failure on '/Users/username/kserve/config/overlays/development/configmap/inferenceservice_patch.yaml' : lstat /Users/username/kserve/config/overlays/development/configmap/inferenceservice_patch.yaml: no such file or directory
error: no objects passed to apply
make: *** [deploy-dev] Error 1`` Biplob Biswas Hi, I have checked the v1beta1, the IngressBackend struct is:
```type IngressBackend struct {
// Specifies the name of the referenced service.
// +optional
ServiceName string `json:"serviceName,omitempty" protobuf:"bytes,1,opt,name=serviceName"`
// Specifies the port of the referenced service.
// +optional
ServicePort intstr.IntOrString `json:"servicePort,omitempty" protobuf:"bytes,2,opt,name=servicePort"`
// Resource is an ObjectRef to another Kubernetes resource in the namespace
// of the Ingress object. If resource is specified, serviceName and servicePort
// must not be specified.
// +optional
Resource *v1.TypedLocalObjectReference `json:"resource,omitempty" protobuf:"bytes,3,opt,name=resource"
}`
so, we should change this code block to
`Backend: netv1.IngressBackend{
ServiceName: componentName,
ServicePort: constants.CommonDefaultHttpPort,
},``
| 16:05:13 |
| Almog Baku | In reply to@_slack_kubeflow_UTURPK4N4:matrix.org
Unable to get in. It says waiting. try this link
https://us06web.zoom.us/j/81319395480?pwd=L2xabUNDaC9oeUFnR3B3NU9BdHM2QT09 | 16:05:16 |
| Almog Baku | https://us06web.zoom.us/j/81319395480?pwd=L2xabUNDaC9oeUFnR3B3NU9BdHM2QT09 | 16:05:22 |
| iamlovingit | In reply toundefined
(edited) ... the `IngressBackend` struct is:
```type IngressBackend struct {
// Specifies the name of the referenced service.
// +optional
ServiceName string `json:"serviceName,omitempty" protobuf:"bytes,1,opt,name=serviceName"`
// Specifies the port of the referenced service.
// +optional
ServicePort intstr.IntOrString `json:"servicePort,omitempty" protobuf:"bytes,2,opt,name=servicePort"`
// Resource is an ObjectRef to another Kubernetes resource in the namespace
// of the Ingress object. If resource is specified, serviceName and servicePort
// must not be specified.
// +optional
Resource *v1.TypedLocalObjectReference `json:"resource,omitempty" protobuf:"bytes,3,opt,name=resource"`
}```
so, we should change this code block to
```Backend: netv1.IngressBackend{
ServiceName: componentName,
ServicePort: constants.CommonDefaultHttpPort,
},```
=> ... the v1beta1 `IngressBackend` struct is:
```type IngressBackend struct {
// Specifies the name of the referenced service.
// +optional
ServiceName string `json:"serviceName,omitempty" protobuf:"bytes,1,opt,name=serviceName"`
// Specifies the port of the referenced service.
// +optional
ServicePort intstr.IntOrString `json:"servicePort,omitempty" protobuf:"bytes,2,opt,name=servicePort"`
// Resource is an ObjectRef to another Kubernetes resource in the namespace
// of the Ingress object. If resource is specified, serviceName and servicePort
// must not be specified.
// +optional
Resource *v1.TypedLocalObjectReference `json:"resource,omitempty" protobuf:"bytes,3,opt,name=resource"`
}```
so, we should change this code block to
```Backend: netv1.IngressBackend{
ServiceName: componentName,
ServicePort: constants.CommonDefaultHttpPort,
},``` | 16:05:36 |
 ANIMESH SINGH | In reply to@_slack_kubeflow_UTURPK4N4:matrix.org
Does it work in aws? ANIMESH SINGH yes | 17:43:04 |
| ANIMESH SINGH | In reply to@_slack_kubeflow_U9673D1KJ:matrix.org
yes cc Tommy Li @_slack_kubeflow_UP37HDE4B:matrix.org | 17:43:32 |
| ANIMESH SINGH | In reply to@_slack_kubeflow_U9673D1KJ:matrix.org
cc Tommy Li @_slack_kubeflow_UP37HDE4B:matrix.org the main website is hosted off AWS | 17:43:39 |
|  Christian Kadner joined the room. | 17:43:43 |
| Christian Kadner changed their display name from _slack_kubeflow_UP37HDE4B to Christian Kadner. | 17:43:43 |
| Christian Kadner set a profile picture. | 17:43:44 |
|  Ted Chang joined the room. | 18:14:36 |
