24 May 2022 |
Gautam Kumar | Thomas Ounnas can u also share the details on ur use case as well as why would you prefer seldon over kserve? | 02:36:23 |
Thomas Ounnas | I didn’t choose one over the other yet, and thus I wanted to test both.
But I do need a deep focus on monitoring and feedback events, which seems quite elegant with Seldon. By the way I didn’t find, yet, any exemple of a simple model performance monitoring and feedbacks loop for KServe, so if you know one it’d be great 😉 | 05:14:28 |
Thomas Ounnas | Here is an issue, which treats about the model monitoring part (but in my case I also need a strong feedback loop mechanism, like the Seldon one 🙂)
https://github.com/kserve/kserve/issues/1608 | 05:36:07 |
Thomas Ounnas | Redacted or Malformed Event | 05:43:39 |
| vgrovg joined the room. | 10:43:26 |
vgrovg | Redacted or Malformed Event | 10:50:27 |
| Shahwar Saleem joined the room. | 19:32:56 |
| Shahwar Saleem changed their display name from _slack_kubeflow_U03GTT32J9H to Shahwar Saleem. | 19:40:32 |
| Shahwar Saleem set a profile picture. | 19:40:33 |
Shahwar Saleem | Hi Team, Is Kubeflow v1.4-branch compatible with Amazon EKS v1.20 ? | 19:40:33 |
Shahwar Saleem | I am trying to install Kubeflow 1.4 on AWS EKS 1.20 using this documentation: https://awslabs.github.io/kubeflow-manifests/docs/deployment/vanilla/guide/
The problem I face is that Single command never ends. It keeps on retrying all different kinds of resources, there are a few errors.
If I check on EKS, Kubeflow Central Dashboard is created, but the dashboard relies on all other resources created properly. So, the interface I see never opens anything under dashboard. | 19:43:16 |
Kartik Kalamadi | Yes
https://awslabs.github.io/kubeflow-manifests/docs/about/eks-compatibility/ | 21:05:57 |
Kartik Kalamadi | the while command exits automatically on successful installation
see what resource is causing the issue | 21:09:22 |
Kartik Kalamadi | you could try deleting all the resources and then try installing
kustomize build docs/deployment/vanilla/ , kubectl delete -f - | 21:27:12 |
Shahwar Saleem | Hmm, I did that as well. When I access the Central Dashboard it does weird behaviour: | 21:28:01 |
Kartik Kalamadi | doesn’t look like the installation is proper
check if all the pods are up
kubectl get pods -A | 21:29:41 |
Shahwar Saleem | Also, is it necessary to only access Dashboard after logging in via Dex? | 21:30:28 |
Shahwar Saleem | I am using Lens, and just clicked on Port forwarding in Central Dashboard Pod | 21:31:01 |
Kartik Kalamadi | I haven’t used lens
you can try
kubectl port-forward svc/istio-ingressgateway -n istio-system 8080:80
then open http://localhost:8080/ in the browser
to go to dex
default username and password
user@example.com
12341234 | 21:34:36 |
Kartik Kalamadi | Also make sure you have correct version of Prerequisites
https://awslabs.github.io/kubeflow-manifests/docs/deployment/prerequisites/
Especially kustomize version 3.2.0 | 21:36:10 |
Shahwar Saleem | I have the correct versions of pre-reqs. Installed them carefully.
I think the part where I am stuck is this:
Error from server ([denied by deny-istio-system-envoy-filter] Cell EnvoyFilters must be created in the cell namespace and not in istio-system): error when creating "STDIN": admission webhook "validation.gatekeeper.sh" denied the request: [denied by deny-istio-system-envoy-filter] Cell EnvoyFilters must be created in the cell namespace and not in istio-system
Error from server ([denied by deny-istio-system-envoy-filter] Cell EnvoyFilters must be created in the cell namespace and not in istio-system): error when creating "STDIN": admission webhook "validation.gatekeeper.sh" denied the request: [denied by deny-istio-system-envoy-filter] Cell EnvoyFilters must be created in the cell namespace and not in istio-system
| 21:57:45 |
25 May 2022 |
Shahwar Saleem | Hi Team, Has anyone done the Kubeflow v1.4 installation on EKS without Single Command ?
I am wondering what are the minimum core components required to get Kubeflow working installation.
So far I have tried installing:
• Knative, Kubeflow namespace, Kubeflow roles, Kubeflow Istio Resources, Kubeflow Pipelines, KFServing, Central Dashboard, Notebooks, Volumes Webapp, User namespace.
A few pods in deployment are always crashing. | 17:01:04 |
| Wenjin Situ joined the room. | 18:23:06 |
| Wenjin Situ changed their display name from _slack_kubeflow_U03GX12C44D to Wenjin Situ. | 18:23:07 |
Kartik Kalamadi | looks like you have some webhooks that are preventing you
when are you seeing this error ? | 18:47:02 |
Kartik Kalamadi | Saw your older messages too
Not immediately obvious to me what is going wrong
Feel free to create an Issue here with all the details
https://github.com/awslabs/kubeflow-manifests/issues
We will take a look | 18:49:37 |
| Shan joined the room. | 22:06:28 |
| Payman Touliat changed their display name from _slack_kubeflow_U034BN4GGDB to Payman Touliat. | 23:48:11 |
| Payman Touliat set a profile picture. | 23:48:12 |
27 May 2022 |
| _slack_kubeflow_U01FHK2TZ8Q joined the room. | 02:31:07 |