16 Feb 2022 |
charlesa101 | Thanks Keith Mattix! nice to meet everyone | 19:53:01 |
Ed Vielmetti (Equinix Metal) | Hey charlesa101! Happy to sync up some time with you and Keith Mattix and figure out how we can support the efforts here at Equinix Metal. | 20:10:48 |
charlesa101 | Sure Ed Vielmetti (Equinix Metal) we can workout something | 20:53:19 |
20 Feb 2022 |
| Mohit Huria changed their display name from _slack_kubeflow_U031F9RCFSR to Mohit Huria. | 16:38:34 |
| Mohit Huria set a profile picture. | 16:38:36 |
24 Feb 2022 |
| Alexey Volkov joined the room. | 02:09:49 |
Alexey Volkov | I want the Cloud Pipelines Editor app to be usable with on-prem installations. (Pipeline Editor is a drag-and-drop pipeline editor with 1st-class support for Kubeflow Pipelines: https://cloud-pipelines.net/pipeline-editor/)
What would be the most common way to authenticate and connect to an on-prem API server?
Think of this from the UX perspective: What is the minimum amount of information the user must enter (e.g. the endpoint URL plus token) to be able to authenticate with the API server and submit a pipeline. | 02:15:01 |
jeff | sig-onprem meeting today at 10AM Central, I will past a url here at that time | 13:32:12 |
jeff | https://us02web.zoom.us/j/84560606352?pwd=c0JqR09odzExWHZaSDI1S0VhMVNBZz09 | 16:04:21 |
25 Feb 2022 |
| Matthew joined the room. | 15:47:02 |
Matthew | In reply to@_slack_kubeflow_UE4TG9TJ8:matrix.org I want the Cloud Pipelines Editor app to be usable with on-prem installations. (Pipeline Editor is a drag-and-drop pipeline editor with 1st-class support for Kubeflow Pipelines: https://cloud-pipelines.net/pipeline-editor/)
What would be the most common way to authenticate and connect to an on-prem API server?
Think of this from the UX perspective: What is the minimum amount of information the user must enter (e.g. the endpoint URL plus token) to be able to authenticate with the API server and submit a pipeline. An interesting option would be to deploy Pipeline Editor within a Kubeflow cluster and authenticate via ServiceAccountToken like you would typically do from a Notebook Server. This wouldn't need any information beyond the (internal) kfp endpoint.
By the way, I really like the idea of the Pipeline Editor with its focus on reusable components 👍 | 15:47:02 |
26 Feb 2022 |
| _slack_kubeflow_U0171QHSUQJ joined the room. | 02:16:48 |
3 Mar 2022 |
jeff | On prem meeting is 4 mins | 15:26:39 |
jeff | https://us02web.zoom.us/j/85099328496?pwd=ajJXZXowb09JTG5YTU43Q2ZLak5NQT09 | 15:26:39 |
4 Mar 2022 |
Alexey Volkov | In reply to@_slack_kubeflow_U01CELR6Q9E:matrix.org An interesting option would be to deploy Pipeline Editor within a Kubeflow cluster and authenticate via ServiceAccountToken like you would typically do from a Notebook Server. This wouldn't need any information beyond the (internal) kfp endpoint.
By the way, I really like the idea of the Pipeline Editor with its focus on reusable components 👍 Is that how KFP is normally used on-prem? Vis in-cluster notebook, not from outside of the cluster via proxies? Does the ServiceAccountToken end up being transformed and passed as Bearer token with the REST request?
> By the way, I really like the idea of the Pipeline Editor with its focus on reusable components
Thank you, Matthew. The reusable component ecosystem was my passion since 2018 when I developed the format. The Pipeline Editor was my vision from the start, but I did not know enough frontend and there were no good drag and drop UX libraries. I jumped in as soon as I saw a usable drag-and-drop web UX library. | 21:46:00 |
5 Mar 2022 |
| Alexey Volkov joined the room. | 03:07:38 |
Matthew | In reply to@_slack_kubeflow_UE4TG9TJ8:matrix.org Is that how KFP is normally used on-prem? Vis in-cluster notebook, not from outside of the cluster via proxies? Does the ServiceAccountToken end up being transformed and passed as Bearer token with the REST request?
> By the way, I really like the idea of the Pipeline Editor with its focus on reusable components
Thank you, Matthew. The reusable component ecosystem was my passion since 2018 when I developed the format. The Pipeline Editor was my vision from the start, but I did not know enough frontend and there were no good drag and drop UX libraries. I jumped in as soon as I saw a usable drag-and-drop web UX library. Exactly, the ServiceAccountToken ist projected into the pod and then send as a bearer token. There is a good explanation about it here: https://github.com/kubeflow/pipelines/issues/5138
I think this would make PipelineEditor usable regardless of how authentication is set up. In addition, it would be great to access PipelineEditor as a tab in Kubeflow. When running outside the cluster, things get more complicated. In that case it's necessary to obtain a session token. This could be done programmatically as shown here: https://github.com/kubeflow/kfctl/issues/140
Unfortunately this depends on your authentication setup. By default his will be Dex (https://www.arrikto.com/blog/kubeflow/news/kubeflow-authentication-with-istio-dex/) but others use keycloak instead. The authentication routine would be different for every setup.
Taking elyra as an example, they implemented authentication for a couple of different Dex configurations: https://elyra.readthedocs.io/en/latest/user_guide/runtime-conf.html#kubeflow-authentication-type-auth-type | 06:54:49 |
Matthew | In reply to@_slack_kubeflow_U01CELR6Q9E:matrix.org Exactly, the ServiceAccountToken ist projected into the pod and then send as a bearer token. There is a good explanation about it here: https://github.com/kubeflow/pipelines/issues/5138
I think this would make PipelineEditor usable regardless of how authentication is set up. In addition, it would be great to access PipelineEditor as a tab in Kubeflow. When running outside the cluster, things get more complicated. In that case it's necessary to obtain a session token. This could be done programmatically as shown here: https://github.com/kubeflow/kfctl/issues/140
Unfortunately this depends on your authentication setup. By default his will be Dex (https://www.arrikto.com/blog/kubeflow/news/kubeflow-authentication-with-istio-dex/) but others use keycloak instead. The authentication routine would be different for every setup.
Taking elyra as an example, they implemented authentication for a couple of different Dex configurations: https://elyra.readthedocs.io/en/latest/user_guide/runtime-conf.html#kubeflow-authentication-type-auth-type But if you're just looking for the absolute minimum amount of information for a Proof of Concept, the API endpoint and (manually obtained) session token should suffice | 07:59:58 |
14 Mar 2022 |
| Keith Adler joined the room. | 20:52:27 |
Keith Adler | Question: I was looking into multi-tenancy for onprem, and these steps are straightforward:
https://www.kubeflow.org/docs/components/multi-tenancy/getting-started/
However, this appears to use the dex configmap in the auth namespace, which is pre-configured to run its own user list including a hash of user passwords.
1. We have our own okta provider onprem. Is there a good guide or example of how you could connect this dex configmap to okta?
2. This is currently Kubeflow 1.3.1 which does not yet support the 1.5 feature of "automatic profile creation". It looks like we would need to manually add profiles for each user even if okta was configured. Am I understanding this correctly? Would it be more intuitive to the current kubeflow version, and enable automatic profile generation alongside okta user auth? | 20:59:17 |
15 Mar 2022 |
| Josh Tan joined the room. | 15:46:59 |
16 Mar 2022 |
| _slack_kubeflow_U036U7Y8415 joined the room. | 06:55:06 |
17 Mar 2022 |
| petteri changed their display name from _slack_kubeflow_U02SQF4RNKA to petteri. | 06:39:01 |
| petteri set a profile picture. | 06:39:03 |
20 Mar 2022 |
Matthew | In reply to@_slack_kubeflow_U01CELR6Q9E:matrix.org But if you're just looking for the absolute minimum amount of information for a Proof of Concept, the API endpoint and (manually obtained) session token should suffice On second thought: It's probably also necessary to specify in which namespace the run should be started. | 11:33:07 |
23 Mar 2022 |
| _slack_kubeflow_U038P6TQEBB joined the room. | 13:30:14 |
28 Mar 2022 |
Matthew | In reply to@_slack_kubeflow_U01CELR6Q9E:matrix.org On second thought: It's probably also necessary to specify in which namespace the run should be started. Alexey Volkov I finally had the time to try this approach. When deploying Pipeline Editor as a pod within a kubeflow cluster and adding it to the CentralDashboard, you don't need any extra authentication. Since the user is already logged in when accessing the CentralDashboard and everything is done Client-Side, the user's browser will automatically send the authservice_session header when starting a run in Pipeline Editor. The only thing we had to add was another input field to select the experiment for the run. The namespace is then automatically determined by the experiment.
In other words, there is no need to use ServiceAccoutToken authentication since (almost) everything works out of the box :partying_face:
Thank you again for your work on Pipeline Editor. I think that it fits perfectly into the existing kubeflow pipelines ecosystem 👍 | 10:50:07 |
29 Mar 2022 |
| Atra Akandeh joined the room. | 20:22:38 |
5 Apr 2022 |
| Eddy Decena changed their profile picture. | 14:38:04 |
13 Apr 2022 |
| _slack_kubeflow_U03B2UX4BAS joined the room. | 01:49:06 |