| 22 Oct 2021 |
 clokep | In reply to @dmrobertson:matrix.org
why should there be a single distinguished main process at all? what does it do that is special and unique? Ideally there wouldn't be, but there currently is. | 15:04:19 |
| clokep | There's data streams that can only have a single writer, by default those are in the "main" process. We've put in effort to allow you to split some of those out to other processes. | 15:04:58 |
| clokep | Some of those we've also put in effort to let you "shard" them so you can have multiple workers which work on different parts of those streams. | 15:05:18 |
| clokep | And +1 to making an issue that we can have a longer, slower, discussion about this. 🙂 | 15:05:35 |
| clokep | dmr: Please do question whether any of that makes sense, just cause it is that way doesn't mean that it should be. 😀 Mostly trying to give background. | 15:07:36 |
| clokep | I think your point is more of "instead of putting effort into splitting this out properly maybe we should fix the problem of needing a main process"? | 15:07:57 |
 Olivier 'reivilibre' | In reply to @clokep:matrix.org
And +1 to making an issue that we can have a longer, slower, discussion about this. 🙂 also hoping that it might help me because I'm out of the loop and don't want to be suggesting things that everyone else already thought of 5 years ago 😅 | 15:08:17 |
 dmr | It sounds like "tl;dr we used to be a monolith and it's tricky/slow to break it up" | 15:08:24 |
| dmr | * It sounds like "tl;dr we used to be a monolith and it's tricky/slow to break it up" | 15:08:31 |
| clokep | Pretty much. 🙂 | 15:08:37 |
| dmr | In reply to @clokep:matrix.org
I think your point is more of "instead of putting effort into splitting this out properly maybe we should fix the problem of needing a main process"? A bit of wanting to challenge our implicit assumptions, but I was thinking in that direction too, yeah. | 15:10:53 |
| clokep | As long as you have some methods which should only be called from particular writer workers you end up with this problem though, I think? | 15:11:45 |
| clokep | It actually becomes harder because it isn't "the main process" but "whoever is allowed to write to blah". | 15:12:00 |
| clokep | In some places I think we proxy over replication, but I think we do that at the handler level, not the storage level? | 15:12:14 |
| dmr |
There's data streams that can only have a single writer
I think I'd like to understand this better. The example that springs to mind is
N.B.: ASSUMES IT IS THE ONLY THING THAT MODIFIES THE USER DIRECTORY
from the user directory handler, but that seems to be a layer above
| 15:15:17 |
| clokep | https://matrix.org/blog/2020/11/03/how-we-fixed-synapses-scalability is probably the best description that exists? | 15:16:29 |
 Brendan Abolivier | I've come to the point where I've fiddled so much with Synapse's module system that I know whenever I see __init__.py in my pycharm tabs there's a 90% chance that's synapse/module_api/__init__.py | 15:32:28 |
 squah | In reply to @reivilibre.element:librepush.net
Do we have an issue to keep track of this? I've filed the data store discussion as #11165. | 15:59:26 |
 Github | https://github.com/matrix-org/synapse/issues/11165 : Improve type hints for data store usage | 15:59:27 |
| Olivier 'reivilibre' | In reply to @squah:matrix.org
I've filed the data store discussion as #11165. thank you! | 16:17:16 |
| Github | https://github.com/matrix-org/synapse/issues/11165 : Improve type hints for data store usage | 16:17:19 |
| clokep | Shay: Probably worth looking at https://github.com/matrix-org/synapse/pull/11129#pullrequestreview-787081712 before the specific comments I left there, sorry about that! | 17:08:11 |
| clokep | I realized it after leaving other comments... | 17:08:21 |
 Shay | Okay cool thanks! | 17:09:35 |
| squah | In reply to @andrewm:amorgan.xyz
cl0kep, squ4h: https://github.com/matrix-org/synapse/pull/11095 seems to be ready to merge? It's merged now! | 17:16:30 |
| clokep | disallow_untyped_decorators is something we might want to turn on! | 17:38:43 |
| clokep | Brings out 115 errors. | 17:38:47 |
| clokep | I guess this all fits in the eventually bucket, but seems useful in light of the log_function discussion. | 17:39:14 |
 [anoa] | test | 21:41:53 |
| 23 Oct 2021 |
|  Ashwinn changed their display name from Ashwin Nair to Ashwinn. | 02:24:43 |
