| 2 May 2022 |
 Nick | Hi croberts we don't have this kind of thing in modelmesh yet, but have discussed it in the past. There's some in-progress work (mostly complete I think) to support kserve transformers with modelmesh predictors, I expect something similar could be done with explainers. | 20:54:45 |
 croberts | Thanks Nick. Is there support for batch inference in mm-serving? The other thing I couldn't find anything on was the possibility of canary rollout/a-b testing. Anything on those? | 20:56:15 |
| Nick | Good questions 🙂 these also aren't supported currently, but are things that we've thought about too. I think we should be able to support them soon via the inference router feature which is being added, but it would also be possible/nice to incorporate into the modelmesh routing layer itself for best performance / lowest overhead. | 21:03:39 |
 Shri Javadekar | PR for updated docs: https://github.com/kserve/kserve/pull/2171/
Comments/suggestions welcome. | 23:54:31 |
| 3 May 2022 |
| Shri Javadekar | I am following this doc for trying to create my own transformer. I have a super simple pass-through transformer. But that doesn't seem to work.
• I ran into a problem that kserve.KFModel is no longer an object available. I had to change it to kserve.Model .
• But even with this, the transformer pod is always in a Crashloopbackoff. I don't see any logs of the kserve-container. The queue-proxy sidecar has logs complaining about not being able to connect to 127.0.0.1:8080.
Can someone share what I might be missing here?
from typing import Dict
import kserve
import logging
logging.basicConfig(level=kserve.constants.KSERVE_LOGLEVEL)
class MyBasicTransformer(kserve.Model):
def __init__(self, name: str, predictor_host: str):
super().__init__(name)
self.predictor_host = predictor_host
def preprocess(self, inputs: Dict) -> Dict:
print(f"Transformer received inputs: {inputs}")
return inputs
def postprocess(self, inputs: Dict) -> Dict:
return inputs | 03:46:39 |
| Shri Javadekar | Nevermind.. there is a need to package the __main__.py and __init__.py files into the docker container as is shown here: https://kserve.github.io/website/modelserving/v1beta1/transformer/torchserve_image_transformer/image_transformer/ | 06:10:00 |
 Timos | If you are running Kserve 0.7.0 then you probably need the python SDK of same version. If you pip install then you will get 0.7.0. You need to do sth like
pip install kserve==0.7.0 | 08:37:20 |
| Timos | kserve.Model comes in version v0.8.0 | 08:37:51 |
 Surya Iyer | Hi All, I am trying to implement multi model serving with models stored in s3. The agent service seems to be making the host http://{bucket}.host. Is there a way I can enable path style access? I know the aws config has s3ForcePathStyle , Is there a way I can pass it to the deployment? | 14:47:29 |
|  _slack_kubeflow_U03DQ1D2WKG joined the room. | 14:59:03 |
| Surya Iyer | I do see that in the agent uses this S3_USER_VIRTUAL_BUCKET env variable. How can I pass it? | 15:04:40 |
| Shri Javadekar | Got it! Thanks.
I was able to get this to work (for trying it out) once I also copied init and main python files. | 15:21:40 |
|  Martin Abeleda joined the room. | 16:10:31 |
| Martin Abeleda changed their display name from _slack_kubeflow_U03ELT5356C to Martin Abeleda. | 16:10:55 |
| Martin Abeleda set a profile picture. | 16:11:00 |
|  Chris Chase joined the room. | 20:06:48 |
