2 May 2022 |
Haris Farooqui | Thanks Rahul Kharse Gautam Kumar. minioServiceRegion was set and it was working fine with 1.7.1 Had to apply above patch for `1.8.11 | 19:42:48 |
| _slack_kubeflow_U03DTPDRGQK joined the room. | 22:17:12 |
3 May 2022 |
Gautam Kumar | jcostanzo Are u only worried about costs? We have some blogs/doc, which describes how to setup kubecost on EKS in multi-tenant environment. | 04:26:49 |
jcostanzo | yeah ive seen those and am currently trialing kubecost actually. just curious what others are doing and what they’re happy with (or what gaps they think they have) | 12:10:05 |
jcostanzo | dont know that kubecost could tell me how many api requests to s3 someones random pipeline is making | 12:10:24 |
| Martin Abeleda joined the room. | 16:10:45 |
| Martin Abeleda changed their display name from _slack_kubeflow_U03ELT5356C to Martin Abeleda. | 16:10:56 |
| Martin Abeleda set a profile picture. | 16:10:59 |
4 May 2022 |
| Sam Gaunt joined the room. | 00:43:06 |
| _slack_kubeflow_UM4D6LFEF joined the room. | 03:30:25 |
| Ryan McCaffrey changed their display name from _slack_kubeflow_U03D067RTJN to Ryan McCaffrey. | 03:59:41 |
| Ryan McCaffrey set a profile picture. | 03:59:43 |
Ryan McCaffrey | Hi, I'm new to Kubeflow and I'm trying to install Kubeflow v1.4 on EKS. I'm following the automated setup option of step 2.0 of this guide.
_Create an IAM user with permissions to get bucket location and allow read and write access to objects in an S3 bucket where you want to store the Kubeflow artifacts._
I just wanted to confirm -- it seems like user permissions go well beyond just getting bucket location and read/write access in S3. As I'm running the auto-rds-s3-setup.py script I'm trudging through _a lo_t of IAM user permission errors spanning anything from subnet groups to random password generators for secrets manager and just wanted to confirm that I'm not missing anything. I'm slowly adding them as I as go but the process seems arduous. Am I missing anything? | 13:57:53 |
| _slack_kubeflow_U03DSUVTQM8 joined the room. | 18:48:11 |
| Sam Gaunt changed their display name from _slack_kubeflow_U03DZDPD7DH to Sam Gaunt. | 22:13:37 |
| Sam Gaunt set a profile picture. | 22:13:40 |
Sam Gaunt | The documentation is pretty frustrating and I had to do the exact same thing. Honestly, it seems like no one has actually run through the process.
I also had a syntax issue with the utils/rds-s3/auto-rds-s3-setup.py file in that step. Running that with python2 like `PYTHONPATH=.. python utils/rds-s3/auto-rds-s3-setup.py…`would give my a syntax error on line 42. Running it with python3 like PYTHONPATH=.. python3 utils/rds-s3/auto-rds-s3-setup.py… would throw a No module named 'importlib_metadata error. This was fixed by changing line 7 to from importlib.metadata import metadata
I have had so many issues that I am just about to start fresh and do the manual setup process. Let me know how you go with step 3.0. The while loop gets stuck in a loop and never finishes for me. I had varies pods stuck in error states. | 22:13:40 |
Ryan McCaffrey | Thanks for confirming. I didn't see any other comments about the permissions stuff and I thought I was the only one. If you run with Python 3.8 the script syntax should be better. I did have to modify a line referencing the subnet tags but then after the IAM user changes I was eventually able to run the script to completion. | 23:06:14 |
Sam Gaunt | Seems like this is very dependant on specific versions installed in the prerequisites.
• installing eksctl (at least via homebrew) also installs a version of kubectl, v1.23, that is currently incompatible with the version of the cluster you are then told to create, v1.20.
• kustomize v3.2.0 is required, but is frustratingly difficult to install as only the latest release is on homebrew. | 23:29:03 |
5 May 2022 |
Sam Gaunt | Also I am now on python 3.8.10 and still get the importlib_metadata error | 00:05:43 |
Ryan McCaffrey | Okay, yeah I made sure that the client and serve kubectl versions were the same. And I'm on a Mac so installing kustomize v3.2.0 was much more straightforward. My colleague @_slack_kubeflow_U03DYQXFCBX:matrix.org recently installed v3.2.0 in a non-Mac environment. I think he's running his dev environment in a docker container — maybe he can share his approach with you if you're still struggling with that. | 00:17:12 |
Sam Gaunt | How did you install v3.2.0? I am on Mac as well. | 00:18:37 |
Sam Gaunt | And what are your client and server kubectl versions? | 00:20:18 |
Ryan McCaffrey | For kustomize I think I used this (though I'm not in front of my computer right now to verify):
https://coder-question.com/cq-blog/1078 | 00:24:02 |
Ryan McCaffrey | Kubectl versions were 1.19.0 | 00:24:29 |
Sam Gaunt | Thanks for your help, sorry for all the questions. | 00:24:44 |
Sam Gaunt | Is it possible to setup cognito authentication without a custom domain? | 05:28:45 |
| Alexander Measure joined the room. | 12:32:24 |
| Alexander Measure changed their display name from _slack_kubeflow_U03DYQXFCBX to Alexander Measure. | 12:33:30 |
| Alexander Measure set a profile picture. | 12:33:33 |