!GVzqqhOOirSFrkrnzX:matrix.org

platform-aws

229 Members
3 Servers

Load older messages


SenderMessageTime
28 Oct 2021
@_slack_kubeflow_UJSC5KQ4D:matrix.orgLuca set a profile picture.14:52:26
@_slack_kubeflow_UJSC5KQ4D:matrix.orgLuca Hi all! I’m new to this channel (just found out its existance 😅 ) I’m trying to update our Kubeflow deployment (based on EKS), but I don’t want to lose all the volumes that our data scientists created for notebooks (they rely on EFS storage class dynamic provisioning). How would you perform a backup of the volumes providing a seemless upgrade experience to the end users (e.g. if they already had a notebook volume they can reuse it without significant changes even after kubeflow is updated)? Thanks for any suggestion! 14:52:26
29 Oct 2021
@_slack_kubeflow_U0241BFPK6J:matrix.orgmetafeather joined the room.08:14:02
@_slack_kubeflow_U02BYV8KYEP:matrix.orgHari Kantipudi Hello All, we have installed Kubeflow 1.2 on AWS EKS but want to install individual level components and configure according to our Enterprise needs . The documentation out there is not great. Is there a more simplified version of installating and setting up Kubeflow please ?? 12:11:23
@_slack_kubeflow_U02KLE55JNP:matrix.org_slack_kubeflow_U02KLE55JNP joined the room.19:59:15
2 Nov 2021
@_slack_kubeflow_U014PVBKWEQ:matrix.orgSuraj Kota
In reply to@_slack_kubeflow_U02BYV8KYEP:matrix.org
Hello All, we have installed Kubeflow 1.2 on AWS EKS but want to install individual level components and configure according to our Enterprise needs . The documentation out there is not great. Is there a more simplified version of installating and setting up Kubeflow please ??
We(AWS) are working on moving away from kfctl to using kustomize. Not for 1.2 but 1.3 onwards. Does that work?
17:50:11
@_slack_kubeflow_U01H0MB53AS:matrix.orgRustam Gimadiev changed their display name from _slack_kubeflow_U01H0MB53AS to Rustam Gimadiev.19:58:11
@_slack_kubeflow_U01H0MB53AS:matrix.orgRustam Gimadiev set a profile picture.19:58:12
3 Nov 2021
@_slack_kubeflow_U02LDG220RX:matrix.orgAndrea Toro joined the room.11:10:27
@_slack_kubeflow_U02LDG220RX:matrix.orgAndrea Toro Hi there! I have an EKS cluster (1.18 version)in the pre-production environment. Now I would like to use this cluster to install Kubeflow (1.4 version). Unfortunately, when I try to install it with the kfctl apply -V -f kfctl_aws.yaml I get this error:
WARN[0024] Encountered error applying application cert-manager-crds:  (kubeflow.error): Code 500 with message: Apply.Run : [error when applying patch:
{"metadata":{"annotations":{"cert-manager.io/inject-ca-from-secret":null,"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apiextensions.k8s.io/v1beta1\",\"kind\":\"CustomResourceDefinition\",\"metadata\":{\"annotations\":{},\"name\":\"certificaterequests.cert-manager.io\"},\"spec\":{\"additionalPrinterColumns\":[{\"JSONPath\":\".status.conditions[?(@.type==\\\"Ready\\\")].status\",\"name\":\"Ready\",\"type\":\"string\"},{\"JSONPath\":\".spec.issuerRef.name\",\"name\":\"Issuer\",\"priority\":1,\"type\":\"string\"},{\"JSONPath\":\".status.conditions[?(@.type==\\\"Ready\\\")].message\",\"name\":\"Status\",\"priority\":1,\"type\":\"string\"},{\"JSONPath\":\".metadata.creationTimestamp\",\"description\":\"CreationTimestamp is a timestamp representing the server time when this object was created. It is not guaranteed to be set in happens-before order across separate operations. Clients may not set this value. It is represented in RFC3339 form and is in UTC.\",\"name\":\"Age\",\"type\":\"date\"}],\"group\":\"cert-manager.io\",\"names\":{\"kind\":\"CertificateRequest\",\"listKind\":\"CertificateRequestList\",\"plural\":\"certificaterequests\",\"shortNames\":[\"cr\",\"crs\"],\"singular\":\"certificaterequest\"},\"scope\":\"Namespaced\",\"subresources\":{\"status\":{}},\"validation\":{\"openAPIV3Schema\":{\"description\":\"CertificateRequest is a type to represent a Certificate Signing Request\",\"properties\":{\"apiVersion\":{\"description\":\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\"type\":\"string\"},\"kind\":{\"description\":\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated.....
Could it be due because I already have cert-manager installed? If so how I can skip this step from the installation process? Thanks a lot!
11:12:21
@_slack_kubeflow_U01H0MB53AS:matrix.orgRustam Gimadiev
In reply to@_slack_kubeflow_U02LDG220RX:matrix.org
Hi there! I have an EKS cluster (1.18 version)in the pre-production environment. Now I would like to use this cluster to install Kubeflow (1.4 version). Unfortunately, when I try to install it with the kfctl apply -V -f kfctl_aws.yaml I get this error:
WARN[0024] Encountered error applying application cert-manager-crds:  (kubeflow.error): Code 500 with message: Apply.Run : [error when applying patch:
{"metadata":{"annotations":{"cert-manager.io/inject-ca-from-secret":null,"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apiextensions.k8s.io/v1beta1\",\"kind\":\"CustomResourceDefinition\",\"metadata\":{\"annotations\":{},\"name\":\"certificaterequests.cert-manager.io\"},\"spec\":{\"additionalPrinterColumns\":[{\"JSONPath\":\".status.conditions[?(@.type==\\\"Ready\\\")].status\",\"name\":\"Ready\",\"type\":\"string\"},{\"JSONPath\":\".spec.issuerRef.name\",\"name\":\"Issuer\",\"priority\":1,\"type\":\"string\"},{\"JSONPath\":\".status.conditions[?(@.type==\\\"Ready\\\")].message\",\"name\":\"Status\",\"priority\":1,\"type\":\"string\"},{\"JSONPath\":\".metadata.creationTimestamp\",\"description\":\"CreationTimestamp is a timestamp representing the server time when this object was created. It is not guaranteed to be set in happens-before order across separate operations. Clients may not set this value. It is represented in RFC3339 form and is in UTC.\",\"name\":\"Age\",\"type\":\"date\"}],\"group\":\"cert-manager.io\",\"names\":{\"kind\":\"CertificateRequest\",\"listKind\":\"CertificateRequestList\",\"plural\":\"certificaterequests\",\"shortNames\":[\"cr\",\"crs\"],\"singular\":\"certificaterequest\"},\"scope\":\"Namespaced\",\"subresources\":{\"status\":{}},\"validation\":{\"openAPIV3Schema\":{\"description\":\"CertificateRequest is a type to represent a Certificate Signing Request\",\"properties\":{\"apiVersion\":{\"description\":\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\"type\":\"string\"},\"kind\":{\"description\":\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated.....
Could it be due because I already have cert-manager installed? If so how I can skip this step from the installation process? Thanks a lot!
Yes, this issue happen because you have already installed cert-manager, and you can exclude it from kfctl_aws.yaml file but need to keep in place the creation of self signed issuer that used in kubeflow
11:54:39
@_slack_kubeflow_U02H7AEDE4S:matrix.orgMargit Zwemer joined the room.16:19:30
4 Nov 2021
@_slack_kubeflow_U02LDG220RX:matrix.orgAndrea Toro
In reply to@_slack_kubeflow_U01H0MB53AS:matrix.org
Yes, this issue happen because you have already installed cert-manager, and you can exclude it from kfctl_aws.yaml file but need to keep in place the creation of self signed issuer that used in kubeflow
Thanks a lot Rustam Gimadiev do you know how to exclude it from kfctl_aws.yaml? (if it is easy for you to find otherwise I will study it.. 🙂)
11:56:04
@_slack_kubeflow_U02LASX03LL:matrix.orgRyan Steakley joined the room.20:20:20
@_slack_kubeflow_U02LASX03LL:matrix.orgRyan Steakley changed their display name from _slack_kubeflow_U02LASX03LL to Ryan Steakley.20:21:43
@_slack_kubeflow_U02LASX03LL:matrix.orgRyan Steakley set a profile picture.20:21:43
@_slack_kubeflow_U02LASX03LL:matrix.orgRyan Steakley Today we are proud to announce that SageMaker Components for Kubeflow Pipelines officially supports the 1.7.0 version of KubeFlow Pipelines, which uses the latest 1.4. KubeFlow version. If you are interested there is a written guide for setup https://docs.aws.amazon.com/sagemaker/latest/dg/amazon-sagemaker-components-for-kubeflow-pipelines.html and examples on the official Kubeflow Pipelines github https://github.com/kubeflow/pipelines/tree/master/components/aws/sagemaker   Cheers 🎉 20:21:44
5 Nov 2021
@_slack_kubeflow_U02G8MWJBTN:matrix.orgBank
In reply to@_slack_kubeflow_U02LASX03LL:matrix.org
Today we are proud to announce that SageMaker Components for Kubeflow Pipelines officially supports the 1.7.0 version of KubeFlow Pipelines, which uses the latest 1.4. KubeFlow version. If you are interested there is a written guide for setup https://docs.aws.amazon.com/sagemaker/latest/dg/amazon-sagemaker-components-for-kubeflow-pipelines.html and examples on the official Kubeflow Pipelines github https://github.com/kubeflow/pipelines/tree/master/components/aws/sagemaker   Cheers 🎉
Hope to see AWS integrated with others KF Components on ver 1.4 soon! 🎉 I'm waiting for Cognito and IAM roles for KF Notebook 🙂
09:07:32
@_slack_kubeflow_U02G8MWJBTN:matrix.orgBank
In reply toundefined
(edited) ... AWS integrated with ... => ... AWS integrate with ...
09:08:17
@_slack_kubeflow_U02G8MWJBTN:matrix.orgBank
In reply toundefined
(edited) Hope to ... => Great work! Hope to ...
09:10:33
@_slack_kubeflow_U02G8MWJBTN:matrix.orgBank
In reply toundefined
(edited) ... with others KF ... => ... with other KF ...
09:17:13
@_slack_kubeflow_U02G8MWJBTN:matrix.orgBank
In reply toundefined
(edited) ... on ver 1.4 soon! ... => ... on v1.4 soon! ...
09:40:51
@_slack_kubeflow_U021865GVA6:matrix.orgLars Heimes joined the room.15:21:40
@_slack_kubeflow_U021865GVA6:matrix.orgLars Heimes
In reply to@_slack_kubeflow_U02G8MWJBTN:matrix.org
Great work! Hope to see AWS integrate with other KF Components on v1.4 soon! 🎉 I'm waiting for Cognito and IAM roles for KF Notebook 🙂
Awesome! Looking forward to get kubeflow1.4 on eks.
15:21:40
@_slack_kubeflow_U014PVBKWEQ:matrix.orgSuraj Kota
In reply to@_slack_kubeflow_U021865GVA6:matrix.org
Awesome! Looking forward to get kubeflow1.4 on eks.
Bank could you expand more on IAM roles for KF Notebook is there a github issue for this?
19:33:05
@_slack_kubeflow_U014PVBKWEQ:matrix.orgSuraj Kota
In reply to@_slack_kubeflow_U02LDG220RX:matrix.org
Thanks a lot Rustam Gimadiev do you know how to exclude it from kfctl_aws.yaml? (if it is easy for you to find otherwise I will study it.. 🙂)
delete the kustmize directory created by kfctl build command
19:34:18
@_slack_kubeflow_U014PVBKWEQ:matrix.orgSuraj Kota
In reply to@_slack_kubeflow_U014PVBKWEQ:matrix.org
delete the kustmize directory created by kfctl build command
comment out the entries related to cert-manager in kfdef manifest, build again and reapply
19:34:39
@_slack_kubeflow_U014PVBKWEQ:matrix.orgSuraj Kota
In reply to@_slack_kubeflow_U014PVBKWEQ:matrix.org
comment out the entries related to cert-manager in kfdef manifest, build again and reapply
comment out a component like this https://github.com/kubeflow/manifests/blob/v1.3-branch/distributions/kfdef/kfctl_aws.v1.3.0.yaml#L27-L31
19:35:27
6 Nov 2021
@_slack_kubeflow_UERJ7PE3D:matrix.orgJiaxin Shan joined the room.02:32:27
@_slack_kubeflow_U02G8MWJBTN:matrix.orgBank
In reply to@_slack_kubeflow_U014PVBKWEQ:matrix.org
Bank could you expand more on IAM roles for KF Notebook is there a github issue for this?
Thanks, here it is IAM Role for pipelines in-progress https://www.kubeflow.org/docs/distributions/aws/pipeline/#s3-access-from-kubeflow-pipelines Auth inside Notebook still be cookie https://www.kubeflow.org/docs/distributions/aws/pipeline/#authenticate-kubeflow-pipeline-using-sdk-inside-cluster
04:20:08

There are no newer messages yet.


Back to Room ListRoom Version: 6