Sender | Message | Time |
---|---|---|
10 May 2022 | ||
JoshBottum | n unexpected exception has occurred Line: /usr/local/lib/python3.6/dist-packages/git/cmd.py:928 Type: GitCommandError Reason: Cmd('git') failed due to: exit code(129) cmdline: git diff --cached --abbrev=40 --full-index --raw stderr: 'Not a git repository To compare two paths outside a working tree: usage: git diff [--no-index] path path ' The logfile `/home/minikf/provision.log' may contain more information on this error, including a backtrace. *** MiniKF provisioning tool terminated. Press Enter to restart, Ctrl-C to exit... *** | 02:36:24 |
JoshBottum | As an fyi, I tried to deploy on AWS, it failed, then tried GCP and MiniKF deployed after 2nd try. I set-up the demo for presentation and then stopped it. When I went back in to restart, I am getting this error. | 02:37:38 |
rawkintrevo | Hiya JoshBottum while I do recommend minikf as a quick on ramp to kubeflow for trial purposes, i don't really play with it much myself anymore. Alex Aidun still monitors this channel- he may be able to help. best of luck. ... that said- what info was in that log file? the error is saying whatever git you're trying to pull from does not exist so if that's a configurable field I would check there- o/w someone bipped a repo they shouldn't have. | 10:41:31 |
Alex Aidun | JoshBottum rawkintrevo hey gents, asked internally to see if we recognize the error, that being said I would head over to kubeflow.arrikto.com and check out the new Kubeflow as a Service! https://www.arrikto.com/blog/introducing-kubeflow-as-a-service-giving-data-scientists-instant-access-to-a-complete-mlops-platform/ | 12:59:23 |
Alex Aidun | JoshBottum does this address your issue? https://support.arrikto.com/hc/en-us/articles/5449137426449 | 13:26:09 |
JoshBottum | Hi Alex Aidun thanks for your prompt response. I have deleted my MiniKF and I am trying KaaS. I have a presentation in Austin tomorrow and wanted to provide a KF demo, and my plan is to use KaaS and the distributed pytorch tutorial. Josh | 14:28:28 |
Bilal Obeidat | Hi Maciek Stopa I am trying to use a custom image where I need to add only add an inter whl library to gcr.io/arrikto/jupyter-kale-py38:release-1.3-l0-release-1.3-rc6 | 14:29:55 |
Bilal Obeidat | I followed your approach here but still Minikf notebook server complain about the custom image | 14:31:04 |
Bilal Obeidat | here is my dockerfile : | 14:31:34 |
Bilal Obeidat | FROM gcr.io/arrikto/jupyter-kale-py38:release-1.3-l0-release-1.3-rc6 COPY snowflake_snowpark_python-0.5.0-py3-none-any.whl /tmp/ WORKDIR /usr/local/ USER root RUN pip install '/tmp/snowflake_snowpark_python-0.5.0-py3-none-any.whl[pandas]' -t /usr/local/lib/python3.8/dist-packages | 14:31:46 |
Bilal Obeidat | any idea? | 14:31:52 |
Alex Aidun | Good to know thanks JoshBottum for the context | 14:52:37 |
11 May 2022 | ||
yasha joined the room. | 00:24:28 | |
Evans Kiplagat | how do you allow locally deployed MiniKF to access the GPU resources? | 06:45:31 |
Zoltán R. Jánki changed their display name from _slack_kubeflow_U02JYD39G57 to Zoltán R. Jánki. | 11:24:06 | |
Zoltán R. Jánki set a profile picture. | 11:24:09 | |
Alex Aidun | Evans Kiplagat I think what you are looking for is in this tips and tricks post: https://www.arrikto.com/blog/kubeflow-tips-tricks-february-23rd-2022/. How do I enable GPUs on MiniKF local? | 14:25:11 |
Evans Kiplagat | Thanks Alex Aidun will be checking it out. | 14:29:29 |
Alex Aidun | Evans Kiplagat sounds good, please share any feedback good luck! | 14:31:41 |
Maciek Stopa | hey Bilal Obeidat, can you show the error message? | 15:55:23 |
12 May 2022 | ||
Bilal Obeidat | Hi Maciek Stopa thank you for the response. simply the notebook server creation never complete. I believe the whl library was causing some weird gcc issue so I fixed it. for other reasons i had to use the latest arkito image gcr.io/arrikto/jupyter-kale-py38:release-1.5-l0-release-1.5-rc1-17-gef3a04d6a | 07:34:37 |
Bilal Obeidat | Redacted or Malformed Event | 07:37:40 |
Bilal Obeidat | Download Screen Shot 2022-05-12 at 12.28.04 AM.png | 17:42:25 |
Bilal Obeidat | I am getting the attached rpc error in the screen shot every time I click compile and run or save a pipeline. I am using a custom image based on the latest arkito image gcr.io/arrikto/jupyter-kale-py38:release-1.5-l0-release-1.5-rc1-17-gef3a04d6aI know it is not my code and i think it is the image because I used the titanic notebook example and I still get the 400 error. any idea? | 17:42:25 |
Alex Aidun | Hey Bilal Obeidat, I'm Dr. Education here at Arrikto - I've raised the question internally, can I ask are you using MiniKF on GCP or AWS? Are you aware that we have a free Kubeflow as a Service offering now so you don't have to worry about MiniKF installs or managing your resources? That can be found here if you are curious: kubeflow.arrikto.com | 17:53:26 |
Bilal Obeidat | Thanks Alex Aidun I am using it on AWS | 17:56:50 |
Bilal Obeidat | I will take a look. I am using a custom image because I need an internal library | 18:00:07 |
13 May 2022 | ||
Alex Aidun | Hey Bilal Obeidat sorry for the delay here, the response I got back from engineering is below:
"MiniKF though doesn't have the latest Kubeflow version with all the required updates in the frontend/backend in order to support private(namespaced) pipelines.
Kale by default in the latest images uploads private pipeline definitions (pipeline definition in specific namespace that cannot be accessed by others living in different ones).
This procedure requires latest Kubeflow version. From the UI I'm not sure if they can avoid this actually. However, with Kale's CLI they can use the --shared-pipeline flag to overcome this issue."
Does this advice help? Again sorry for delay! | 12:24:05 |
Bilal Obeidat | Thank you Alex Aidun this is helpful in explaining the issue. I still workaround it by generating the pipeline.yaml file and continue running it that way. I will see if using the Kale's cli is helpful here | 16:06:54 |
17 May 2022 | ||
_slack_kubeflow_U03FLV8QGG6 joined the room. | 15:37:44 |