!LuUSGaeArTeoOgUpwk:matrix.org

kubeflow-kfserving

433 Members
2 Servers

Load older messages


SenderMessageTime
17 May 2022
@_slack_kubeflow_UM56LA7N3:matrix.orgBenjamin Tan https://github.com/kserve/kserve/tree/master/docs/samples/istio-dex 04:00:45
@_slack_kubeflow_U03CN7QAHN3:matrix.orgzorba(손주형) What happens if I put more than 1 container in Transformer yaml??
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: torch-transformer
spec:
  predictor:
    model:
      modelFormat:
        name: pytorch
      storageUri:  gs://kfserving-examples/models/torchserve/image_classifier 
  transformer:
    containers:
      - image: kserve/image-transformer:latest
        name: kserve-container
        command:
          - "python"
          - "-m"
          - "model"
        args:
          - --model_name
          - mnist
08:40:29
@_slack_kubeflow_UFVUV2UFP:matrix.orgDan Sun You can horizontally scale up ingress gateway as you want, it is a reverse proxy so it can process ~1000/s for each replica 08:58:10
@_slack_kubeflow_U032CM1LH3N:matrix.orgSaurabh Agarwalimage.png
Download image.png
12:11:44
@_slack_kubeflow_U032CM1LH3N:matrix.orgSaurabh Agarwal facing the issue where the inferenceservice is created but never gets into running(respective pod is not initializing) 12:11:44
@_slack_kubeflow_UAYJJUQJZ:matrix.orgtheofpa https://twitter.com/techatbloomberg/status/1526554734680850432?s=21&t=_gw_HEdPK0TMgWhEnvtYxg 14:23:10
@_slack_kubeflow_U02Q0DARM8B:matrix.org_slack_kubeflow_U02Q0DARM8B joined the room.15:46:18
@_slack_kubeflow_U0315UY2WRM:matrix.orgShri Javadekar I've see the transformer (or even the predictor) components get translated into Kubernetes deployments. So, it could become a deployment with two containers (unless there are checks in place to prevent it). Care to try it out and let us know? 🙂 16:01:07
@_slack_kubeflow_U02PHBULPDZ:matrix.orgDiego Kiner It's also not clear how to actually make a request to a graph - there doesn't appear to be any extra virtual service stood up to provide an entrypoint? 19:29:15
@_slack_kubeflow_U036ZCFAFLP:matrix.orgJonny Browning changed their display name from _slack_kubeflow_U036ZCFAFLP to Jonny Browning.22:20:28
@_slack_kubeflow_U036ZCFAFLP:matrix.orgJonny Browning set a profile picture.22:20:33
@_slack_kubeflow_U022U7KG24W:matrix.orgRachit Chauhan (commenting to follow the thread) 22:58:55
18 May 2022
@_slack_kubeflow_UFVUV2UFP:matrix.orgDan Sun Diego Kiner We create graph orchestrator as a service to chain the requests, so the entry point is the graph orchestrator service 00:27:36
@_slack_kubeflow_U02PHBULPDZ:matrix.orgDiego Kiner Ok, we're trying to work off of the changes in the PR to get it working - how is the orchestrator created? 00:28:32
@_slack_kubeflow_UFVUV2UFP:matrix.orgDan Sun what’s the use case of having two containers on transformer? 00:29:57
@_slack_kubeflow_UFVUV2UFP:matrix.orgDan Sun there is a docker file on the PR to build the graph orchestrator image 00:32:10
@_slack_kubeflow_U02PHBULPDZ:matrix.orgDiego Kiner I see the router.Dockerfile - is this what you mean? But even if built, how is it meant to be deployed? 00:35:01
@_slack_kubeflow_UFVUV2UFP:matrix.orgDan Sun just run make deploy-dev that will deploy the graph orchestrator image and when you create the inference graph it will expose the endpoint for you 00:42:46
@_slack_kubeflow_U02PHBULPDZ:matrix.orgDiego Kiner Ok thanks! will give it a try 00:43:05
@_slack_kubeflow_U02PHBULPDZ:matrix.orgDiego Kiner ``kustomize build config/overlays/development , kubectl apply -f - Error: trouble configuring builtin PatchStrategicMergeTransformer with config: paths: - configmap/inferenceservice_patch.yaml - manager_image_patch.yaml : evalsymlink failure on '.../kserve/config/overlays/development/configmap/inferenceservice_patch.yaml' : lstat .../kserve/config/overlays/development/configmap/inferenceservice_patch.yaml: no such file or directory`` 01:28:59
@_slack_kubeflow_U03CN7QAHN3:matrix.orgzorba(손주형) Dan Sun I don’t know.. but it’s possible since containers yaml input list List of container 02:40:04
@_slack_kubeflow_U0315UY2WRM:matrix.orgShri Javadekar I can imagine one container for the predictions and a sidecar container for logs/metrics, etc. 04:18:29
@_slack_kubeflow_U03CN7QAHN3:matrix.orgzorba(손주형) Shri Javadekar I thought that too. However how kserve know which container is transformer and which is sidecar? 05:24:27
@_slack_kubeflow_UFVUV2UFP:matrix.orgDan Sun well kserve already can inject a model agent sidecar, if any features need to be added I recommend contributing there 07:42:13
@_slack_kubeflow_UFVUV2UFP:matrix.orgDan Sun KServe introduction talk on Kubecon AI Days is now on youtube ! https://www.youtube.com/watch?v=FX6naJLaq2Y&list=PLj6h78yzYM2PJdsIBxtDOyiFqP3wIbOcc&index=9 08:25:56
@_slack_kubeflow_UM56LA7N3:matrix.orgBenjamin Tan Hurray! Was looking forward to this ❤️ 09:38:53
@_slack_kubeflow_U03CN7QAHN3:matrix.orgzorba(손주형) oh it will be in 0.9.0 11:20:03
@_slack_kubeflow_U02UYNBU951:matrix.orgAlexa Griffith changed their display name from _slack_kubeflow_U02UYNBU951 to Alexa Griffith.13:40:58
@_slack_kubeflow_U02UYNBU951:matrix.orgAlexa Griffith set a profile picture.13:41:01
@_slack_kubeflow_U02UYNBU951:matrix.orgAlexa Griffith Yay!! Had so much fun 13:41:12

Show newer messages


Back to Room ListRoom Version: 6