2 May 2022 |
Nick | Good questions 🙂 these also aren't supported currently, but are things that we've thought about too. I think we should be able to support them soon via the inference router feature which is being added, but it would also be possible/nice to incorporate into the modelmesh routing layer itself for best performance / lowest overhead. | 21:03:39 |
Shri Javadekar | PR for updated docs: https://github.com/kserve/kserve/pull/2171/
Comments/suggestions welcome. | 23:54:31 |
3 May 2022 |
Shri Javadekar | I am following this doc for trying to create my own transformer. I have a super simple pass-through transformer. But that doesn't seem to work.
• I ran into a problem that kserve.KFModel is no longer an object available. I had to change it to kserve.Model .
• But even with this, the transformer pod is always in a Crashloopbackoff. I don't see any logs of the kserve-container . The queue-proxy sidecar has logs complaining about not being able to connect to 127.0.0.1:8080 .
Can someone share what I might be missing here?
from typing import Dict
import kserve
import logging
logging.basicConfig(level=kserve.constants.KSERVE_LOGLEVEL)
class MyBasicTransformer(kserve.Model):
def __init__(self, name: str, predictor_host: str):
super().__init__(name)
self.predictor_host = predictor_host
def preprocess(self, inputs: Dict) -> Dict:
print(f"Transformer received inputs: {inputs}")
return inputs
def postprocess(self, inputs: Dict) -> Dict:
return inputs | 03:46:39 |
Shri Javadekar | Nevermind.. there is a need to package the __main__.py and __init__.py files into the docker container as is shown here: https://kserve.github.io/website/modelserving/v1beta1/transformer/torchserve_image_transformer/image_transformer/ | 06:10:00 |
Timos | If you are running Kserve 0.7.0 then you probably need the python SDK of same version. If you pip install then you will get 0.7.0. You need to do sth like
pip install kserve==0.7.0 | 08:37:20 |
Timos | kserve.Model comes in version v0.8.0 | 08:37:51 |
Surya Iyer | Hi All, I am trying to implement multi model serving with models stored in s3. The agent service seems to be making the host http://{bucket}.host. Is there a way I can enable path style access? I know the aws config has s3ForcePathStyle , Is there a way I can pass it to the deployment? | 14:47:29 |
| _slack_kubeflow_U03DQ1D2WKG joined the room. | 14:59:03 |
Surya Iyer | I do see that in the agent uses this S3_USER_VIRTUAL_BUCKET env variable. How can I pass it? | 15:04:40 |
Shri Javadekar | Got it! Thanks.
I was able to get this to work (for trying it out) once I also copied init and main python files. | 15:21:40 |
| Martin Abeleda joined the room. | 16:10:31 |
| Martin Abeleda changed their display name from _slack_kubeflow_U03ELT5356C to Martin Abeleda. | 16:10:55 |
| Martin Abeleda set a profile picture. | 16:11:00 |
| Chris Chase joined the room. | 20:06:48 |
| Chris Chase changed their display name from _slack_kubeflow_U013VE77D62 to Chris Chase. | 20:14:21 |
| Chris Chase set a profile picture. | 20:14:22 |
Chris Chase | Hey Kserve folks. I deployed an example modelmesh-serving deployment using the quickstart install for someone to demo model mesh. Unfortunately, the cluster I deployed to upgraded over the weekend, meaning pods were evicted. As a result my sample installation was dead today before the demo because both the minio installation and the etcd installation had both disappeared. Is there a non-ephemeral deployment example I can use? I would think etcd disappearing would be very unfortunate since all the services died at that point. | 20:14:23 |
John Daciuk | Does KServe give a FIFO guarantee for some subsection of the request path? | 23:43:42 |
4 May 2022 |
Rachit Chauhan | kimwnasptd: in case of standalone installation, i had to mention namespace: kserve in kustomization.yaml otherwise the config map , kserve-models-web-app-config , was getting created in default namespace and was not available to the deployment | 01:06:34 |
Surya Iyer | Was able to use serving.kubeflow.org/s3-usevirtualbucket to resolve the issue | 17:29:51 |
Surya Iyer | Redacted or Malformed Event | 17:37:59 |
Vivian Pan | Hi kserve team, my team is doing a POC on kServe v0.8.0. We have most of the components configured with knative. However it is not clear to how the Storage Initializer is configured for the namespace?
The knative deployments are in knative and knative-serving , whereas the kserve deployments are in kserve namespace.
The only way we have gotten the InferenceService to work is from within knative-serving namespace due to the storage-initializer not being available in other namespaces. Could someone help clarify how the storage-initializer is configured? | 18:47:52 |
| _slack_kubeflow_U03DSUVTQM8 joined the room. | 18:48:48 |
Dan Sun | There is no specific configuration it should work in all namespace except kserve control plane namespace | 23:22:09 |
Dan Sun | Surya Iyer is there any doc we should update for this ? | 23:24:14 |
Dan Sun | It does not work in kserve namespace because that’s for control plane, and the best practice is to deploy in user namespaces | 23:25:34 |
Dan Sun | Vivian Pan | 23:25:54 |
5 May 2022 |
Vivian Pan | ok thank you clarifying | 00:24:12 |
wenyang zhou | I have deploy a model named realtime-losingorder-predict-xgb-neg-clean which name length is 43.
When I deployed the first revision, it works. And then I deployed the second revision, it showed Invalid value: "prev-realtime-losingorder-predict-xgb-neg-clean-predictor-default": must be no more than 63 characters | 06:15:16 |
wenyang zhou | Is it a normal phenomenon? | 06:16:17 |