!XaqDhxuTIlvldquJaV:matrix.org

#synapse-dev:matrix.org

120 Members
If you are not a dev contributing code to Synapse, please talk in #synapse:matrix.org instead. This room is *only* for coordination of dev work. If you want to grab the attention of a dev, do it in #synapse:matrix.org. 57 Servers

Load older messages


Timestamp Message
18 Feb 2020
18:07:43@_neb_github_=40travis=3at2l.io:matrix.orgGithub [@travis:t2l.io] [matrix-org/synapse] clokep synchronize pull request #6941: Do not send alias events when creating / upgrading a room [open] - https://github.com/matrix-org/synapse/pull/6941
17:36:29@erikj:jki.reErik(I think where we're falling over here is that we have rooms that we join large rooms and get 10000s of new events at once, causing us to trip over the limit)
17:36:55@erikj:jki.reErikanother option is to add a timestamp and say we're only allowed to e.g. get 15mins behind
18:07:56@travis:t2l.ioTravisR
In reply to @erikj:jki.re
another option is to add a timestamp and say we're only allowed to e.g. get 15mins behind
This would be better, as shoving tens of thousands of events down a queue isn't an error until the thing can't keep up.
18:08:13@travis:t2l.ioTravisRJust means your queue size is massive for a period of time
18:33:29@_neb_github_=40travis=3at2l.io:matrix.orgGithub [@travis:t2l.io] [matrix-org/synapse] clokep closed issue #6942: SyTest test "Device deletion propagates over federation" is flaky [closed] - https://github.com/matrix-org/synapse/issues/6942
18:08:25@erikj:jki.reErikyeah
18:33:32@_neb_github_=40travis=3at2l.io:matrix.orgGithub [@travis:t2l.io] [matrix-org/synapse] clokep reopened issue #6942: SyTest test "Device deletion propagates over federation" is flaky [open] - https://github.com/matrix-org/synapse/issues/6942
18:08:47@erikj:jki.reErikthe idea that queue size ~ how fare behind you are isn't really true for us
21:10:34@richvdh:sw1v.orgrichvdhI think a timelimit would be an improvement and less prone to false positives, but still, unless there's something that the process can do to try and fix itself, I'm not sure what the point of disconnecting the worker is
21:11:05@richvdh:sw1v.orgrichvdhultimately it just becomes a way of flagging that something is wrong, and kicking off the worker isn't something that gets noticed unless you have monitoring on reconnection rates
21:12:07@richvdh:sw1v.orgrichvdhso you might as well alert directly on the replication delay rather than using "how many disconnections" as an obscure proxy for it
21:12:50@richvdh:sw1v.orgrichvdhor maybe better: monitor on "size of transmission buffer for the worker stream"
21:15:09@travis:t2l.ioTravisRyea, a metric for how far behind (timewise) it is would be better
21:15:29@travis:t2l.ioTravisRso that admins can decide if they want to fix it after 30 seconds or 30 days.
21:16:47@richvdh:sw1v.orgrichvdhI think the reason it all falls apart is that if the master ends up buffering up an hour's worth of replication updates for each worker, that is huge amounts of memory usage with few clues as to what is using the memory
21:19:45@travis:t2l.ioTravisRmore replication metrics would help with that though, stuff like also recording the number of backlogged events/updates and which workers are falling behind.
21:20:24@travis:t2l.ioTravisRHigh memory usage isn't really a problem until you run out, in which case the planet has been restarted and the backlog goes away
22:39:49@_neb_github_=40travis=3at2l.io:matrix.orgGithub [@travis:t2l.io] [matrix-org/synapse] richvdh opened pull request #6948: Move MSC2432 stuff onto unstable prefix [open] - https://github.com/matrix-org/synapse/pull/6948
22:40:16@_neb_github_=40travis=3at2l.io:matrix.orgGithub [@travis:t2l.io] [matrix-org/synapse] richvdh edited pull request #6948: Move MSC2432 stuff onto unstable prefix [open] - https://github.com/matrix-org/synapse/pull/6948
22:40:25@_neb_github_=40travis=3at2l.io:matrix.orgGithub [@travis:t2l.io] [matrix-org/synapse] richvdh review_requested pull request #6948: Move MSC2432 stuff onto unstable prefix [open] - https://github.com/matrix-org/synapse/pull/6948
22:45:51@_neb_github_=40travis=3at2l.io:matrix.orgGithub [@travis:t2l.io] [matrix-org/synapse] richvdh opened pull request #6949: Make room alias lists peekable [open] - https://github.com/matrix-org/synapse/pull/6949
22:52:33@_neb_github_=40travis=3at2l.io:matrix.orgGithub [@travis:t2l.io] [matrix-org/synapse] richvdh synchronize pull request #6949: Make room alias lists peekable [open] - https://github.com/matrix-org/synapse/pull/6949
22:53:55@_neb_github_=40travis=3at2l.io:matrix.orgGithub [@travis:t2l.io] [matrix-org/synapse] richvdh edited pull request #6949: Make room alias lists peekable [open] - https://github.com/matrix-org/synapse/pull/6949
23:16:59@_neb_github_=40travis=3at2l.io:matrix.orgGithub [@travis:t2l.io] [matrix-org/synapse] richvdh synchronize pull request #6949: Make room alias lists peekable [open] - https://github.com/matrix-org/synapse/pull/6949
23:19:41@_neb_github_=40travis=3at2l.io:matrix.orgGithub [@travis:t2l.io] [matrix-org/synapse] richvdh edited pull request #6949: Make room alias lists peekable [open] - https://github.com/matrix-org/synapse/pull/6949
23:19:52@_neb_github_=40travis=3at2l.io:matrix.orgGithub [@travis:t2l.io] [matrix-org/synapse] richvdh review_requested pull request #6949: Make room alias lists peekable [open] - https://github.com/matrix-org/synapse/pull/6949
23:21:56@_neb_github_=40travis=3at2l.io:matrix.orgGithub [@travis:t2l.io] [matrix-org/synapse] richvdh synchronize pull request #6949: Make room alias lists peekable [open] - https://github.com/matrix-org/synapse/pull/6949
23:26:56@_neb_github_=40travis=3at2l.io:matrix.orgGithub [@travis:t2l.io] [matrix-org/synapse] richvdh opened pull request #6950: Tiny optimisation for _get_handler_for_request [open] - https://github.com/matrix-org/synapse/pull/6950
23:27:06@_neb_github_=40travis=3at2l.io:matrix.orgGithub [@travis:t2l.io] [matrix-org/synapse] richvdh review_requested pull request #6950: Tiny optimisation for _get_handler_for_request [open] - https://github.com/matrix-org/synapse/pull/6950

There are no newer messages yet.


Back to Room List