!caaCTdhYlLGpbJVvly:matrix.org

kubeflow-pipelines

741 Members
2 Servers

Load older messages


SenderMessageTime
1 Jun 2022
@_slack_kubeflow_U01AGRLQ8QY:matrix.orgRituraj Kumar joined the room.17:28:35
@_slack_kubeflow_U01AGRLQ8QY:matrix.orgRituraj Kumar changed their display name from _slack_kubeflow_U01AGRLQ8QY to Rituraj Kumar.17:30:23
@_slack_kubeflow_U01AGRLQ8QY:matrix.orgRituraj Kumar set a profile picture.17:30:24
@_slack_kubeflow_U01AGRLQ8QY:matrix.orgRituraj Kumar Is there a any solution for v1 and v2 compatibility of load_component_from_file ? As I see there is no kfp.v2.components.load_component_from_file method in kfp.v2 versions 17:30:24
@_slack_kubeflow_U02JN8ZRU2E:matrix.orgYingding Wang @Nicholas Kosteski can you open the browser developertool and see if there is any javascript error in the console? For multi-tenant setting in KF1.4, i have also the issue visualization is not shown. I was hoping KF1.5 will solve this issue. 18:12:56
@_slack_kubeflow_U01C62YLURX:matrix.orgNicholas Kosteski Yingding Wang I’m not seeing anything unusual. What is the javascript error you’re seeing? 18:23:45
@_slack_kubeflow_U02JN8ZRU2E:matrix.orgYingding Wang I ran the example code from Joseph Olaide just a moment ago on my on-prem KF 1.4 and got the error in browser console Failed to load resource: the server responded with a status of 503 (Service Unavailable) , i think it is a known issue for kf1.4, but should go away in kf1.5 18:29:41
@_slack_kubeflow_U01C62YLURX:matrix.orgNicholas Kosteski Yeah I definitely didn’t get any specific errors, it seems like the js function that is supposed to grab the artifacts is just kinda coming back with 0 things when I select the Run output tab. So I’m not sure if we’re having the same issue or not but it certainly could be that they’re related? ¯\(ツ)/¯ I don’t know if in a multi-user env this problem persists or not. I’m just using the standalone feature (since its all we need/use) right now. But I might try to look at KF1.5 next since this is taking up much more time than I was expecting… What’s super bizarre is that I can sometimes get it to display in the Run output. It requires that I go to the run before the execution is done, click into the pod/task visualizations until they show up. Then as long as I don’t refresh the page, everything appears exactly how I would expect in both the pod visualizations and the run output tab. Once I refresh the page however, the visualizations disappear in the Run output tab only. It seems something is changing/clearing some state that it shouldn’t be 😖 18:58:46
@_slack_kubeflow_U02LW7FHWLS:matrix.orgRahul Mehta I think the idea is that you don't delete anything (I encountered this too when I found there isn't an out-of-the-box way to enforce a retention policy) -- my understanding is that it's helpful for experiment tracking/archival and audit purposes if you only archives are permitted instead of deletions 20:51:45
@_slack_kubeflow_U02LW7FHWLS:matrix.orgRahul Mehta For example, ml-metadata is designed to be immutable 20:52:21
@_slack_kubeflow_UKM89RPD0:matrix.orgdroctothorpe Interesting! Thanks, Rahul. I guess that helps contextualize the design choice. You got around this with the retention policy you introduced though, right? 21:20:54
@_slack_kubeflow_U02LW7FHWLS:matrix.orgRahul Mehta Yes, basically a k8s CronJob that periodically deletes rows that are older than 30 days from the various tables in the MySQL DB 21:21:33
@_slack_kubeflow_U02LW7FHWLS:matrix.orgRahul Mehta Though, we don't enforce anything like that for experiments (just runs & assoc. metadata) 21:21:48
2 Jun 2022
@_slack_kubeflow_U03HVCQDWVB:matrix.orgBalu joined the room.03:24:48
@_slack_kubeflow_U03HVGTGQHK:matrix.org_slack_kubeflow_U03HVGTGQHK joined the room.08:18:16
@_slack_kubeflow_U03J0DGHKFD:matrix.org_slack_kubeflow_U03J0DGHKFD joined the room.13:21:08
@_slack_kubeflow_U03HYSZAD45:matrix.org_slack_kubeflow_U03HYSZAD45 joined the room.16:35:49
@_slack_kubeflow_U03J2CCP2FM:matrix.org_slack_kubeflow_U03J2CCP2FM joined the room.18:30:32
3 Jun 2022
@_slack_kubeflow_U03J221BNLD:matrix.org_slack_kubeflow_U03J221BNLD joined the room.06:52:40
@_slack_kubeflow_U03HVCQDWVB:matrix.orgBalu changed their display name from _slack_kubeflow_U03HVCQDWVB to Balu.08:53:24
@_slack_kubeflow_U03HVCQDWVB:matrix.orgBalu set a profile picture.08:53:26
@_slack_kubeflow_U03HVCQDWVB:matrix.orgBalu Can someone help me on How to add label to dsl.ResourceOp. That label value must be dynamic one. 08:53:27
@_slack_kubeflow_U032CK41Z8V:matrix.orgNSLog0 any experience with GCP Transfer jobs service? I have a requirement to build pipeline which related with Transfer jobs. the job will be sync data from s3 to gcs daily. after data sync done then next pipeline will be run after. 1. sync data -> wait until sync done 2. (train data after sync done) Does the pipeline can work with Transfer jobs like above scenario ? 11:02:01
@_slack_kubeflow_U01A4GE3FFB:matrix.orgBrett Koonce i have done that 13:24:43
@_slack_kubeflow_U01A4GE3FFB:matrix.orgBrett Koonce i believe you can have gcp emit an event after the transfer is done 13:24:54
@_slack_kubeflow_U01A4GE3FFB:matrix.orgBrett Koonce you could use that to trigger your job 13:25:03
@_slack_kubeflow_U01A4GE3FFB:matrix.orgBrett Koonce if you don't have a ton of data changes 13:25:17
@_slack_kubeflow_U01A4GE3FFB:matrix.orgBrett Koonce then you might do a simple cron 13:25:25
@_slack_kubeflow_U01A4GE3FFB:matrix.orgBrett Koonce eg sync at 12am 13:25:30
@_slack_kubeflow_U01A4GE3FFB:matrix.orgBrett Koonce run job at 12 pm 13:25:34

Show newer messages


Back to Room ListRoom Version: 6