!XaqDhxuTIlvldquJaV:matrix.org

#synapse-dev:matrix.org

418 Members
If you are not a dev contributing code to Synapse, please talk in #synapse:matrix.org instead. This room is *only* for coordination of dev work. If you want to grab the attention of a dev, do it in #synapse:matrix.org. 157 Servers

Load older messages


SenderMessageTime
22 Oct 2021
@clokep:matrix.orgclokep
In reply to @dmrobertson:matrix.org
why should there be a single distinguished main process at all? what does it do that is special and unique?
Ideally there wouldn't be, but there currently is.
15:04:19
@clokep:matrix.orgclokepThere's data streams that can only have a single writer, by default those are in the "main" process. We've put in effort to allow you to split some of those out to other processes.15:04:58
@clokep:matrix.orgclokepSome of those we've also put in effort to let you "shard" them so you can have multiple workers which work on different parts of those streams.15:05:18
@clokep:matrix.orgclokepAnd +1 to making an issue that we can have a longer, slower, discussion about this. 🙂 15:05:35
@clokep:matrix.orgclokep dmr: Please do question whether any of that makes sense, just cause it is that way doesn't mean that it should be. 😀 Mostly trying to give background. 15:07:36
@clokep:matrix.orgclokepI think your point is more of "instead of putting effort into splitting this out properly maybe we should fix the problem of needing a main process"?15:07:57
@reivilibre.element:librepush.netOlivier 'reivilibre'
In reply to @clokep:matrix.org
And +1 to making an issue that we can have a longer, slower, discussion about this. 🙂
also hoping that it might help me because I'm out of the loop and don't want to be suggesting things that everyone else already thought of 5 years ago 😅
15:08:17
@dmrobertson:matrix.orgdmrIt sounds like "tl;dr we used to be a monolith and it's tricky/slow to break it up"15:08:24
@dmrobertson:matrix.orgdmr * It sounds like "tl;dr we used to be a monolith and it's tricky/slow to break it up"15:08:31
@clokep:matrix.orgclokepPretty much. 🙂 15:08:37
@dmrobertson:matrix.orgdmr
In reply to @clokep:matrix.org
I think your point is more of "instead of putting effort into splitting this out properly maybe we should fix the problem of needing a main process"?
A bit of wanting to challenge our implicit assumptions, but I was thinking in that direction too, yeah.
15:10:53
@clokep:matrix.orgclokepAs long as you have some methods which should only be called from particular writer workers you end up with this problem though, I think?15:11:45
@clokep:matrix.orgclokepIt actually becomes harder because it isn't "the main process" but "whoever is allowed to write to blah".15:12:00
@clokep:matrix.orgclokepIn some places I think we proxy over replication, but I think we do that at the handler level, not the storage level?15:12:14
@dmrobertson:matrix.orgdmr

There's data streams that can only have a single writer

I think I'd like to understand this better. The example that springs to mind is

N.B.: ASSUMES IT IS THE ONLY THING THAT MODIFIES THE USER DIRECTORY

from the user directory handler, but that seems to be a layer above

15:15:17
@clokep:matrix.orgclokephttps://matrix.org/blog/2020/11/03/how-we-fixed-synapses-scalability is probably the best description that exists?15:16:29
@babolivier:vector.modular.imBrendan Abolivier I've come to the point where I've fiddled so much with Synapse's module system that I know whenever I see __init__.py in my pycharm tabs there's a 90% chance that's synapse/module_api/__init__.py 15:32:28
@squah:matrix.orgsquah
In reply to @reivilibre.element:librepush.net
Do we have an issue to keep track of this?
I've filed the data store discussion as #11165.
15:59:26
@_neb_github:matrix.orgGithubhttps://github.com/matrix-org/synapse/issues/11165 : Improve type hints for data store usage15:59:27
@reivilibre.element:librepush.netOlivier 'reivilibre'
In reply to @squah:matrix.org
I've filed the data store discussion as #11165.
thank you!
16:17:16
@_neb_github:matrix.orgGithubhttps://github.com/matrix-org/synapse/issues/11165 : Improve type hints for data store usage16:17:19
@clokep:matrix.orgclokep Shay: Probably worth looking at https://github.com/matrix-org/synapse/pull/11129#pullrequestreview-787081712 before the specific comments I left there, sorry about that! 17:08:11
@clokep:matrix.orgclokepI realized it after leaving other comments...17:08:21
@shayshay:matrix.orgShayOkay cool thanks!17:09:35
@squah:matrix.orgsquah
In reply to @andrewm:amorgan.xyz
cl0kep, squ4h: https://github.com/matrix-org/synapse/pull/11095 seems to be ready to merge?
It's merged now!
17:16:30
@clokep:matrix.orgclokep disallow_untyped_decorators is something we might want to turn on! 17:38:43
@clokep:matrix.orgclokepBrings out 115 errors.17:38:47
@clokep:matrix.orgclokep I guess this all fits in the eventually bucket, but seems useful in light of the log_function discussion. 17:39:14
@anoa:matrix.org[anoa]test21:41:53
23 Oct 2021
@ashwinair:matrix.orgAshwinn changed their display name from Ashwin Nair to Ashwinn.02:24:43

There are no newer messages yet.


Back to Room List