30 May 2022 |
tezlm | is there any way to import the dendrite media store or database into conduit? | 08:39:00 |
Timo ⚡️ | No | 08:39:13 |
tezlm | ok | 08:39:17 |
Timo ⚡️ | Logs:
ruma_state_res: conflicting events: 35469
Timo: 😭
| 08:44:18 |
petko ⚡️ | In reply to @jef:rustybever.be the trusted servers setting is a common misconception, it has nothing to do with with which servers you will federate if it is so common i assume it should be documented somewhere people can easily find it; like conduit.toml for example, no buried down somewhere in the Matrix API documentation; | 08:53:32 |
petko ⚡️ | In reply to @jef:rustybever.be
whatever the case i could not join any of their channels
chances are their room version might just not be supported by conduit their room nuber was/is 6 and i'm running conduit-next | 08:54:42 |
Timo ⚡️ | https://gitlab.com/famedly/conduit/-/issues/274 | 08:55:14 |
petko ⚡️ | In reply to @timokoesters:fachschaften.org https://gitlab.com/famedly/conduit/-/issues/274 thanks 👍 | 08:56:53 |
Jef | In reply to @petko:matrix.gago.life their room nuber was/is 6 and i'm running conduit-next hmmm then i'm not sure what the problem is, you're not getting any error messages? | 09:07:07 |
Timo ⚡️ | In reply to @timokoesters:fachschaften.org
Logs:
ruma_state_res: conflicting events: 35469
Timo: 😭
For some reason fetching those events is very slow. 30mins and still hasn't fetched them all | 09:09:08 |
Timo ⚡️ | I wonder if rocksdb doesn't like getting request from multiple threads. But I don't understand how IO can be this slow | 09:10:21 |
petko ⚡️ | In reply to @jef:rustybever.be hmmm then i'm not sure what the problem is, you're not getting any error messages? the problem is i get tons of log messages, not sure which of them are related to joining another room | 09:11:05 |
Jef | usually when i can't join a room, i get an error message in my client, i don't have to check the logs | 09:11:40 |
Timo ⚡️ | ![Screenshot_20220530_111340.png](https://matrix.org/_matrix/media/r0/thumbnail/fachschaften.org/yRcUHIOWNigjClIkNhCvpWjH?height=360&method=scale&width=360) Download Screenshot_20220530_111340.png | 09:13:48 |
Ticho | rocksdb is still single-writer-multiple-readers, right? | 09:14:54 |
Timo ⚡️ | Yeah | 09:15:10 |
Ticho | you'd probably need some sort of tokio-specific profiling/introspection tools, but my guess is most of the time is spent waiting for the db lock, and each thread gets it every now and then | 09:16:14 |
Timo ⚡️ | But they are almost exclusively reading | 09:16:45 |
Timo ⚡️ | And that wouldn't count as IO load | 09:16:53 |
Timo ⚡️ | * And that wouldn't count as IO wait | 09:17:01 |
Timo ⚡️ | This is a HDD | 09:17:34 |
Ticho | hm, right | 09:17:41 |
Timo ⚡️ | Is random access just that slow and there's nothing I can do? Maybe organize the db in a way where those pdus are more coupled | 09:19:27 |
Linerly | In reply to @timokoesters:fachschaften.org I wonder how element displays pings in old rooms that were upgraded Something like
You have 1 mention in the old room
or something, I kinda forgot
| 09:19:47 |
petko ⚡️ | In reply to @jef:rustybever.be usually when i can't join a room, i get an error message in my client, i don't have to check the logs gomuks for me... | 09:21:26 |
petko ⚡️ | i'll give it a try in the afternoon with element and will spam you guys again :) | 09:21:43 |
Ticho | In reply to @timokoesters:fachschaften.org Is random access just that slow and there's nothing I can do? Maybe organize the db in a way where those pdus are more coupled or cache what each thread reads (unless changed), to hopefully reduce disk IO | 09:22:24 |
| kraem joined the room. | 09:22:36 |
petko ⚡️ | In reply to @ticho:cyberdi.sk or cache what each thread reads (unless changed), to hopefully reduce disk IO heh, caching is so much fun... to get right... :) | 09:23:57 |
Ticho | this is probably already handled by the rocksdb cache, but if you're running on memory-constrained hardware (e.g. a RPi), there's just not enough room to reliably keep all the interesting PDUs in there - especially if your server has lots of users and rooms | 09:26:07 |