Sender | Message | Time |
---|---|---|
1 Nov 2024 | ||
Hank | I wonder if this has been erroring out for a while which is making the problem way worse | 19:40:18 |
Hank | FYI that count query took 8 min 16.884 sec to complete | 19:42:33 |
Hank | I'm doing an EXPLAIN on that bigger query which as far as I can tell comes from the deleteUnusedItemUri function in the ExpirePosts.php code | 19:43:16 |
Hank | explain says:
| 20:08:50 |
Hank | I've shut off cleanup temporarily and running this command my the CLI tool which I don't think ever times out (maybe I'm wrong) | 20:09:23 |
@reh678:matrix.org joined the room. | 22:11:06 | |
Hank | OK it took just over 28 minutes and came up with only 28K entries to delete. :( | 23:23:13 |
Hank | * OK it took just over 25 minutes and came up with only 28K entries to delete. :( | 23:23:26 |
2 Nov 2024 | ||
Hank | I manually did the query and wrote a quick delete script...the re-query takes just as long though. Does anyone know where the timeouts for worker.php would be found? Perhaps /etc/php/8.3/cli/php.ini ? | 01:06:19 |
alias @bkil:grin.hu banned @reh678:matrix.org. | 09:33:45 | |
3 Nov 2024 | ||
@vangelis369:matrix.org joined the room. | 01:08:55 | |
heluecht | We have got a limit for the expiry. It is set to 1000 by default. You can increase it to 100,000 it this doesn't kill your database. Also you should set the maintenance window to a time when you aren't working on the system. | 07:55:46 |
4 Nov 2024 | ||
@bicicleta12:matrix.org joined the room. | 07:01:47 | |
@bicicleta12:matrix.org left the room. | 07:02:07 | |
@vangelis369:matrix.org joined the room. | 10:18:02 | |
6 Nov 2024 | ||
serge90 joined the room. | 12:50:50 | |
7 Nov 2024 | ||
Hank | With the three active users on my server really pounding the keyboard and mouse clicks yesterday my "little" Digital Ocean droplet (4 cores, 8 GB of RAM with the database on a secondary volume) couldn't keep up. I thought I turned off cleanup operations but the ExpirePosts started and was really slamming the system mid-day causing several hours delay in processing AP delivery stuff. But even after I turned it off in late afternoon it still hadn't caught up by 11 pm. And that was with me going out to do stuff so basically shutting off my spigot of new activities five hours before. | 12:52:56 |
Hank | I have a project I just picked up that will focus me in the short term but I think something I'm going to try to double back to focus on is optimizing the data storage/database. I know a lot of effort has been going into it for years and it's gotten much better than it has in the past. I also know it's far outside my expertise range. But I feel having those resources for just a handful of admittedly very active users shouldn't be a problem for a machine with that much horsepower. | 13:11:19 |
Feeds | New post in Reddit: What happened to friendica.opensocial.space? | 23:09:22 |
10 Nov 2024 | ||
pepecyb joined the room. | 09:37:06 | |
Hank | So I'm continuing to try to diagnose my performance issues. My post table, which also has activities and comments, has about 18 million records and is 35 GB. I don't think that should be enough to explain a lot of this slowness. | 22:02:31 |
Hank | However over 98% of the content in that table is remote. Each month that ratio tips. The earlier stuff is like 93% remote. Last month was 99% remote. | 22:03:20 |
Hank | I thought I/we had it cleaning up older remote content but the fact I still have millions of entries going all the way back to the origin of my server makes me think that is not the case. | 22:03:51 |
Hank | Download MonthlyStats.png | 22:04:00 |
Hank | Those are off of the created entry for each row in the post table. | 22:04:22 |
Hank | Should this be a mostly empty graph past the cleanup window? | 22:04:53 |
krvai joined the room. | 22:26:20 | |
pepecyb | No connection to Hubzilla channels possible. I host two Hubzilla hubs and one Friendica instance. The hub ‘hub.hubzilla.hu’ and the Friendica instance ‘barat.fedi-verse.hu’ are running on a VPS under YunoHost. The second hub ‘klacker.org’ runs on another VPS where it is installed manually. It is not possible to create connections between my Friendica channel and a Hubzilla channel that is on one of my two hubs. Connecting (with handle or also with the Proful URL) from Friendica leads to ‘504 Gateway Time-out’ after some time. If I check the channels in the admin area ‘Check address’, the error ‘504’ appears even after a long wait. If I establish a connection to my Friendica channel from one of the Hubzilla channels, the connection is displayed as established in Hubzilla. In Friendica, however, the connection does not appear under ‘Contacts’. I also do not receive a connection request from Friendica and the channel does not appear under ‘Pending’. With my channels on the two hubs, I have functioning connections to channels on other Friendica instances without any problems. There are also no problems with connections to other Fediverse services. I also have working connections from my Friendica channel to Hubzilla channels on hubs other than my own, which work fine. As a test, I also made connections from another Friendica account (at anonsys) to channels on my two hubs. This also worked without any problems. Connections from the other (third-party) Friendica channel to my own Friendica channel can also be established without any problems. Address-Probe attempts lead to error messages in the nginx log:
I am desperate because I have no idea what the reason for this could be and how I can eliminate the malfunction. Does anyone have any ideas? | 23:31:44 |
Hank | I have no experience with the Hubzilla stuff :( | 23:33:16 |
11 Nov 2024 | ||
Hank | PS for reference, the stats for the local entries for the life of my server | 02:24:20 |