!LuUSGaeArTeoOgUpwk:matrix.org

kubeflow-kfserving

433 Members
2 Servers

Load older messages


SenderMessageTime
2 May 2022
@_slack_kubeflow_U0127AUTPMH:matrix.orgNick Good questions 🙂 these also aren't supported currently, but are things that we've thought about too. I think we should be able to support them soon via the inference router feature which is being added, but it would also be possible/nice to incorporate into the modelmesh routing layer itself for best performance / lowest overhead. 21:03:39
@_slack_kubeflow_U0315UY2WRM:matrix.orgShri Javadekar PR for updated docs: https://github.com/kserve/kserve/pull/2171/ Comments/suggestions welcome. 23:54:31
3 May 2022
@_slack_kubeflow_U0315UY2WRM:matrix.orgShri Javadekar I am following this doc for trying to create my own transformer. I have a super simple pass-through transformer. But that doesn't seem to work. • I ran into a problem that kserve.KFModel is no longer an object available. I had to change it to kserve.Model . • But even with this, the transformer pod is always in a Crashloopbackoff. I don't see any logs of the kserve-container. The queue-proxy sidecar has logs complaining about not being able to connect to 127.0.0.1:8080. Can someone share what I might be missing here?
from typing import Dict
import kserve
import logging
logging.basicConfig(level=kserve.constants.KSERVE_LOGLEVEL)

class MyBasicTransformer(kserve.Model):
    def __init__(self, name: str, predictor_host: str):
        super().__init__(name)
        self.predictor_host = predictor_host

    def preprocess(self, inputs: Dict) -> Dict:
        print(f"Transformer received inputs: {inputs}")
        return inputs

    def postprocess(self, inputs: Dict) -> Dict:
        return inputs
03:46:39
@_slack_kubeflow_U0315UY2WRM:matrix.orgShri Javadekar Nevermind.. there is a need to package the __main__.py and __init__.py files into the docker container as is shown here: https://kserve.github.io/website/modelserving/v1beta1/transformer/torchserve_image_transformer/image_transformer/ 06:10:00
@_slack_kubeflow_U02SF1C1Y67:matrix.orgTimos If you are running Kserve 0.7.0 then you probably need the python SDK of same version. If you pip install then you will get 0.7.0. You need to do sth like pip install kserve==0.7.0 08:37:20
@_slack_kubeflow_U02SF1C1Y67:matrix.orgTimos kserve.Model comes in version v0.8.0 08:37:51
@_slack_kubeflow_U027MP4SJAU:matrix.orgSurya Iyer Hi All, I am trying to implement multi model serving with models stored in s3. The agent service seems to be making the host http://{bucket}.host. Is there a way I can enable path style access? I know the aws config has s3ForcePathStyle , Is there a way I can pass it to the deployment? 14:47:29
@_slack_kubeflow_U03DQ1D2WKG:matrix.org_slack_kubeflow_U03DQ1D2WKG joined the room.14:59:03
@_slack_kubeflow_U027MP4SJAU:matrix.orgSurya Iyer I do see that in the agent uses this S3_USER_VIRTUAL_BUCKET env variable. How can I pass it? 15:04:40
@_slack_kubeflow_U0315UY2WRM:matrix.orgShri Javadekar Got it! Thanks. I was able to get this to work (for trying it out) once I also copied init and main python files. 15:21:40
@_slack_kubeflow_U03ELT5356C:matrix.orgMartin Abeleda joined the room.16:10:31
@_slack_kubeflow_U03ELT5356C:matrix.orgMartin Abeleda changed their display name from _slack_kubeflow_U03ELT5356C to Martin Abeleda.16:10:55
@_slack_kubeflow_U03ELT5356C:matrix.orgMartin Abeleda set a profile picture.16:11:00
@_slack_kubeflow_U013VE77D62:matrix.orgChris Chase joined the room.20:06:48
@_slack_kubeflow_U013VE77D62:matrix.orgChris Chase changed their display name from _slack_kubeflow_U013VE77D62 to Chris Chase.20:14:21
@_slack_kubeflow_U013VE77D62:matrix.orgChris Chase set a profile picture.20:14:22
@_slack_kubeflow_U013VE77D62:matrix.orgChris Chase Hey Kserve folks. I deployed an example modelmesh-serving deployment using the quickstart install for someone to demo model mesh. Unfortunately, the cluster I deployed to upgraded over the weekend, meaning pods were evicted. As a result my sample installation was dead today before the demo because both the minio installation and the etcd installation had both disappeared. Is there a non-ephemeral deployment example I can use? I would think etcd disappearing would be very unfortunate since all the services died at that point. 20:14:23
@_slack_kubeflow_U02C41UBRT3:matrix.orgJohn Daciuk Does KServe give a FIFO guarantee for some subsection of the request path? 23:43:42
4 May 2022
@_slack_kubeflow_U022U7KG24W:matrix.orgRachit Chauhan kimwnasptd: in case of standalone installation, i had to mention namespace: kserve in kustomization.yaml otherwise the config map , kserve-models-web-app-config, was getting created in default namespace and was not available to the deployment 01:06:34
@_slack_kubeflow_U027MP4SJAU:matrix.orgSurya Iyer Was able to use serving.kubeflow.org/s3-usevirtualbucket to resolve the issue 17:29:51
@_slack_kubeflow_U027MP4SJAU:matrix.orgSurya IyerRedacted or Malformed Event17:37:59
@_slack_kubeflow_U02LE3KB53M:matrix.orgVivian Pan Hi kserve team, my team is doing a POC on kServe v0.8.0. We have most of the components configured with knative. However it is not clear to how the Storage Initializer is configured for the namespace? The knative deployments are in knative and knative-serving , whereas the kserve deployments are in kserve namespace. The only way we have gotten the InferenceService to work is from within knative-serving namespace due to the storage-initializer not being available in other namespaces. Could someone help clarify how the storage-initializer is configured? 18:47:52
@_slack_kubeflow_U03DSUVTQM8:matrix.org_slack_kubeflow_U03DSUVTQM8 joined the room.18:48:48
@_slack_kubeflow_UFVUV2UFP:matrix.orgDan Sun There is no specific configuration it should work in all namespace except kserve control plane namespace 23:22:09
@_slack_kubeflow_UFVUV2UFP:matrix.orgDan Sun Surya Iyer is there any doc we should update for this ? 23:24:14
@_slack_kubeflow_UFVUV2UFP:matrix.orgDan Sun It does not work in kserve namespace because that’s for control plane, and the best practice is to deploy in user namespaces 23:25:34
@_slack_kubeflow_UFVUV2UFP:matrix.orgDan Sun Vivian Pan 23:25:54
5 May 2022
@_slack_kubeflow_U02LE3KB53M:matrix.orgVivian Pan ok thank you clarifying 00:24:12
@_slack_kubeflow_U02S0UGJWCV:matrix.orgwenyang zhou I have deploy a model named realtime-losingorder-predict-xgb-neg-clean which name length is 43. When I deployed the first revision, it works. And then I deployed the second revision, it showed Invalid value: "prev-realtime-losingorder-predict-xgb-neg-clean-predictor-default": must be no more than 63 characters 06:15:16
@_slack_kubeflow_U02S0UGJWCV:matrix.orgwenyang zhou Is it a normal phenomenon? 06:16:17

Show newer messages


Back to Room ListRoom Version: 6