| 31 May 2022 |
 Joseph Olaide | Yes | 15:26:41 |
 Irvin Tang | hi all. currently, when querying for pipelines using the KFP client, the pipeline object that gets returned has an attribute called parameters which is a list of kfp_server_api.models.api_parameter.ApiParameter objects. These objects only specify the parameter name and value. Is there a way to retrieve the expected type for the pipeline parameter as well? | 15:27:31 |
 Nicholas Kosteski | It does mention the visualizations should be available on the Run Outputs though:
The Run output tab shows the visualizations for all pipeline steps in the selected run. To open the tab in the Kubeflow Pipelines UI: | 15:28:41 |
| Nicholas Kosteski | I just realized there was a link to the code that is meant to load up the visualizations in that issue. I'll try taking a look to see if something happened to where its not being labeled correctly in the instance I have up | 15:31:19 |
| 1 Jun 2022 |
|  _slack_kubeflow_U03HM661ZPY joined the room. | 11:33:55 |
 droctothorpe | Is there any particular reason you can’t delete, as opposed to archive, experiments in the web GUI? | 14:38:07 |
| Nicholas Kosteski | Looks like I finally found something that’s different between the two versions I have. The actual argo_artifact is being created differently in the version that works the artifact is:
{
"name": "mlpipeline-ui-metadata",
"optional": true,
"path": "/tmp/outputs/mlpipeline_ui_metadata/data",
"s3": {
"accessKeySecret": {
"key": "accesskey",
"name": "mlpipeline-minio-artifact"
},
"bucket": "mlpipeline",
"endpoint": "minio-service.kubeflow:9000",
"insecure": true,
"key": "artifacts/allocation-tx-5gpvh/allocation-tx-5gpvh-170870440/mlpipeline-ui-metadata.tgz",
"secretKeySecret": {
"key": "secretkey",
"name": "mlpipeline-minio-artifact"
}
}
}
in the one that doesn’t work its created as the argo_artifact:
{
"name": "mlpipeline-ui-metadata",
"optional": true,
"path": "/tmp/outputs/mlpipeline_ui_metadata/data",
"s3": {
"key": "artifacts/markdown-pipeline-ldvxr/2022/06/01/markdown-pipeline-ldvxr-4134324342/mlpipeline-ui-metadata.tgz"
}
}
| 15:58:17 |
| Nicholas Kosteski | It seems like maybe my workflow-controller-configmap might have some kind of issue | 16:00:20 |
| Joseph Olaide | from typing import NamedTuple
import kfp
from kfp.components import create_component_from_func
@create_component_from_func
def produce_markdown() -> NamedTuple('Outputs', [('MLPipeline_UI_metadata', 'UI_metadata')]):
import sys, json, subprocess
subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])
import pandas as pd
matrix = [
['y', 'y', 10],
['y', 'n', 9],
['n', 'y', 6],
['n', 'y', 7]
]
df = pd.DataFrame(matrix,columns=['target','predicted','count'])
metadata = {
"outputs": [
{
"type": "confusion_matrix",
"format": "csv",
"schema": [
{
"name": "target",
"type": "CATEGORY"
},
{
"name": "predicted",
"type": "CATEGORY"
},
{
"name": "count",
"type": "NUMBER"
}
],
"source": df.to_csv(header=False, index=False),
"storage": "inline",
"labels": [
"yummy",
"not yummy"
]
}
]
}
return [json.dumps(metadata)]
def my_pipeline():
produce_markdown()
kfp.Client().create_run_from_pipeline_func(my_pipeline, arguments={}) | 16:30:39 |
| Joseph Olaide | Hi Nicholas Kosteski, this sample code above creates a run output artifact and a visualization in the pod.
I hope this is helpful | 16:33:10 |
|  Rituraj Kumar joined the room. | 17:28:35 |
| Rituraj Kumar changed their display name from _slack_kubeflow_U01AGRLQ8QY to Rituraj Kumar. | 17:30:23 |
| Rituraj Kumar set a profile picture. | 17:30:24 |
| Rituraj Kumar | Is there a any solution for v1 and v2 compatibility of load_component_from_file ?
As I see there is no kfp.v2.components.load_component_from_file method in kfp.v2 versions | 17:30:24 |
 Yingding Wang | @Nicholas Kosteski can you open the browser developertool and see if there is any javascript error in the console? For multi-tenant setting in KF1.4, i have also the issue visualization is not shown. I was hoping KF1.5 will solve this issue. | 18:12:56 |
| Nicholas Kosteski | Yingding Wang I’m not seeing anything unusual. What is the javascript error you’re seeing? | 18:23:45 |
| Yingding Wang | I ran the example code from Joseph Olaide just a moment ago on my on-prem KF 1.4 and got the error in browser console Failed to load resource: the server responded with a status of 503 (Service Unavailable) , i think it is a known issue for kf1.4, but should go away in kf1.5 | 18:29:41 |
| Nicholas Kosteski | Yeah I definitely didn’t get any specific errors, it seems like the js function that is supposed to grab the artifacts is just kinda coming back with 0 things when I select the Run output tab. So I’m not sure if we’re having the same issue or not but it certainly could be that they’re related? ¯\(ツ)/¯
I don’t know if in a multi-user env this problem persists or not. I’m just using the standalone feature (since its all we need/use) right now. But I might try to look at KF1.5 next since this is taking up much more time than I was expecting…
What’s super bizarre is that I can sometimes get it to display in the Run output. It requires that I go to the run before the execution is done, click into the pod/task visualizations until they show up. Then as long as I don’t refresh the page, everything appears exactly how I would expect in both the pod visualizations and the run output tab. Once I refresh the page however, the visualizations disappear in the Run output tab only. It seems something is changing/clearing some state that it shouldn’t be 😖 | 18:58:46 |
 Rahul Mehta | I think the idea is that you don't delete anything (I encountered this too when I found there isn't an out-of-the-box way to enforce a retention policy) -- my understanding is that it's helpful for experiment tracking/archival and audit purposes if you only archives are permitted instead of deletions | 20:51:45 |
| Rahul Mehta | For example, ml-metadata is designed to be immutable | 20:52:21 |
| droctothorpe | Interesting! Thanks, Rahul. I guess that helps contextualize the design choice. You got around this with the retention policy you introduced though, right? | 21:20:54 |
| Rahul Mehta | Yes, basically a k8s CronJob that periodically deletes rows that are older than 30 days from the various tables in the MySQL DB | 21:21:33 |
| Rahul Mehta | Though, we don't enforce anything like that for experiments (just runs & assoc. metadata) | 21:21:48 |
| 2 Jun 2022 |
|  Balu joined the room. | 03:24:48 |
|  _slack_kubeflow_U03HVGTGQHK joined the room. | 08:18:16 |
|  _slack_kubeflow_U03J0DGHKFD joined the room. | 13:21:08 |
|  _slack_kubeflow_U03HYSZAD45 joined the room. | 16:35:49 |
|  _slack_kubeflow_U03J2CCP2FM joined the room. | 18:30:32 |
| 3 Jun 2022 |
|  _slack_kubeflow_U03J221BNLD joined the room. | 06:52:40 |
| Balu changed their display name from _slack_kubeflow_U03HVCQDWVB to Balu. | 08:53:24 |
