| 18 Jul 2024 |
 MrKaplan | if latency between LW and your instance is high then you might be running into a common issue where throughput is limited by lemmys sequential sending of activities | 10:20:59 |
| MrKaplan | https://grafana.lem.rocks/d/bdid38k9p0t1cf/federation-health-single-instance-overview?orgId=1&var-instance=lemmy.world&var-remote_instance=vegantheoryclub.org | 10:21:08 |
| MrKaplan | the activity progression rate seems relatively stable | 10:21:29 |
 HamidPayaamAbbasi | It's in a data center in Seattle | 10:52:41 |
| MrKaplan | that's probably enough to build up some lag during our busier times if your instance is subscribed to our more active communities, although it should still be low enough to consistently recover | 10:53:44 |
| HamidPayaamAbbasi | Yeah and it's been fine for months until the 14th, very weird | 10:54:21 |
| MrKaplan | see https://github.com/LemmyNet/lemmy/issues/4529 and https://github.com/LemmyNet/lemmy/pull/4623 | 10:54:51 |
| HamidPayaamAbbasi | I tried doubling the pool size in /apps/lemmy/config/lemmy.hjson and restarted Lemmy maybe its a limit on db connections on my side from subscribers | 14:01:37 |
| HamidPayaamAbbasi | hasn't seemed to make a difference | 14:01:46 |
| MrKaplan | maybe on 14th there were subscriptions to active LW communities that previously didn't have any subscribers on your instance? | 14:06:09 |
| HamidPayaamAbbasi | thats possible for sure, I had a few sign ups. It is people I know so I can ask them | 14:25:31 |
| HamidPayaamAbbasi | I'll check the communities tab, thanks for that idea | 14:25:44 |
| MrKaplan | there were a couple communities with their first subscriber on 11th | 14:30:17 |
| 19 Jul 2024 |
| HamidPayaamAbbasi | I increased my postgre sql connection limit and this seems to have caused the biggest improvement since this started. I was really worried that I had some kind of hack or something that was causing cloudflare to react but no, seems like it might just have been not great monitoring on my part | 03:09:40 |
| HamidPayaamAbbasi | thanks for your help | 03:09:46 |
| HamidPayaamAbbasi | looks like aussie.zone had the same problem I did with LW but they're now weeks out of sync | 09:44:39 |
| MrKaplan | aussie.zone has too high latency and physically can't keep up with sequential sending of activities | 09:45:24 |
| MrKaplan | there are at least another instance in australia and an instance in new zealand that are using a custom external activitypub queue to deal with this issue by cutting out most of the latency that gets added to very single request | 09:46:23 |
| HamidPayaamAbbasi | its interesting their graph exactly matches mine from pinning out the postgresql connections | 09:47:56 |
| HamidPayaamAbbasi | * its interesting their graph exactly matches mine from pinning out the postgresql connections on mine | 09:48:07 |
| HamidPayaamAbbasi | neat, I guess different queue problems can create similar graphs | 09:48:28 |
| HamidPayaamAbbasi | but theirs started a while ago | 09:48:48 |
|  elm joined the room. | 19:54:34 |
|  shatteredscales set a profile picture. | 21:55:17 |
| 20 Jul 2024 |
|  Rooki (fedihosting.foundation) changed their display name from Rooki (lemmy.world) to Rooki (fedihosting.foundation). | 19:09:24 |
| 22 Jul 2024 |
|  stolen laptop, mcdonalds wifi, tor browser changed their display name from regalia to stolen laptop, mcdonalds wifi, tor browser. | 05:47:32 |
|  Tyler (AtHeartEngineer) set a profile picture. | 16:05:34 |
| 24 Jul 2024 |
 jelloeater | FYI everyone we're looking into the server lag issue.
https://mastodon.world/@LemmyWorld/112839439951220047 | 05:19:46 |
|  elm_lw joined the room. | 11:06:56 |
| 25 Jul 2024 |
|  wetfish joined the room. | 09:40:13 |
