!BNMPbgPACTjCnwCtJb:grin.hu

Friendica (no IRC) unofficial

306 Members
(if you are fed up with the IRC bot kicking you out) - social network web & communications mobile app https://friendi.ca https://github.com/friendica/friendica/tags - directory: https://dir.friendica.social74 Servers

Load older messages


SenderMessageTime
1 Nov 2024
@hank:myportal.socialHankI wonder if this has been erroring out for a while which is making the problem way worse19:40:18
@hank:myportal.socialHankFYI that count query took 8 min 16.884 sec to complete19:42:33
@hank:myportal.socialHank I'm doing an EXPLAIN on that bigger query which as far as I can tell comes from the deleteUnusedItemUri function in the ExpirePosts.php code 19:43:16
@hank:myportal.socialHank

explain says:

+------+--------------------+------------------+----------------+---------------------------------+-----------------+---------+------+----------+--------------------------+
| id   | select_type        | table            | type           | possible_keys                   | key             | key_len | ref  | rows     | Extra                    |
+------+--------------------+------------------+----------------+---------------------------------+-----------------+---------+------+----------+--------------------------+
|    1 | PRIMARY            | item-uri         | range          | PRIMARY                         | PRIMARY         | 4       | NULL | 12510257 | Using where; Using index |
|   19 | MATERIALIZED       | mail             | range          | thr-parent-id                   | thr-parent-id   | 5       | NULL | 1363     | Using where; Using index |
|   18 | MATERIALIZED       | mail             | range          | parent-uri-id                   | parent-uri-id   | 5       | NULL | 1363     | Using where; Using index |
|   17 | MATERIALIZED       | post-delivery    | index          | inbox-id_created                | uid             | 4       | NULL | 1        | Using index              |
|   16 | MATERIALIZED       | post-delivery    | index          | PRIMARY                         | uid             | 4       | NULL | 1        | Using index              |
|   15 | MATERIALIZED       | inbox-status     | range          | uri-id                          | uri-id          | 5       | NULL | 549390   | Using where; Using index |
|   14 | MATERIALIZED       | diaspora-contact | index          | PRIMARY                         | gsid            | 5       | NULL | 18742    | Using index              |
|   13 | MATERIALIZED       | apcontact        | range          | uri-id                          | uri-id          | 5       | NULL | 315107   | Using where; Using index |
|   12 | MATERIALIZED       | contact          | range          | uri-id                          | uri-id          | 5       | NULL | 358020   | Using where; Using index |
|   11 | MATERIALIZED       | user-contact     | range          | uri-id_uid                      | uri-id_uid      | 5       | NULL | 6463     | Using where; Using index |
|   10 | MATERIALIZED       | event            | range          | uri-id                          | uri-id          | 5       | NULL | 483      | Using where; Using index |
|    9 | MATERIALIZED       | mail             | range          | uri-id                          | uri-id          | 5       | NULL | 1363     | Using where; Using index |
|    8 | DEPENDENT SUBQUERY | post-thread      | index_subquery | conversation-id                 | conversation-id | 5       | func | 1        | Using index; Using where |
|    7 | MATERIALIZED       | post-thread      | range          | context-id                      | context-id      | 5       | NULL | 18624    | Using where; Using index |
|    6 | MATERIALIZED       | post-user        | range          | replies-id                      | replies-id      | 5       | NULL | 9678376  | Using where; Using index |
|    5 | MATERIALIZED       | post-user        | range          | external-id                     | external-id     | 5       | NULL | 17342    | Using where; Using index |
|    4 | DEPENDENT SUBQUERY | post-user        | index_subquery | thr-parent-id                   | thr-parent-id   | 5       | func | 2        | Using index; Using where |
|    3 | DEPENDENT SUBQUERY | post-user        | index_subquery | parent-uri-id,parent-uri-id_uid | parent-uri-id   | 5       | func | 4        | Using index; Using where |
|    2 | DEPENDENT SUBQUERY | post-user        | index_subquery | uri-id                          | uri-id          | 4       | func | 1        | Using index              |
+------+--------------------+------------------+----------------+---------------------------------+-----------------+---------+------+----------+--------------------------+
20:08:50
@hank:myportal.socialHankI've shut off cleanup temporarily and running this command my the CLI tool which I don't think ever times out (maybe I'm wrong)20:09:23
@reh678:matrix.org@reh678:matrix.org joined the room.22:11:06
@hank:myportal.socialHankOK it took just over 28 minutes and came up with only 28K entries to delete. :(23:23:13
@hank:myportal.socialHank * OK it took just over 25 minutes and came up with only 28K entries to delete. :(23:23:26
2 Nov 2024
@hank:myportal.socialHank I manually did the query and wrote a quick delete script...the re-query takes just as long though. Does anyone know where the timeouts for worker.php would be found? Perhaps /etc/php/8.3/cli/php.ini? 01:06:19
@bkil:matrix.orgalias @bkil:grin.hu banned @reh678:matrix.org@reh678:matrix.org.09:33:45
3 Nov 2024
@vangelis369:matrix.org@vangelis369:matrix.org joined the room.01:08:55
@heluecht:myportal.socialheluechtWe have got a limit for the expiry. It is set to 1000 by default. You can increase it to 100,000 it this doesn't kill your database. Also you should set the maintenance window to a time when you aren't working on the system.07:55:46
4 Nov 2024
@bicicleta12:matrix.org@bicicleta12:matrix.org joined the room.07:01:47
@bicicleta12:matrix.org@bicicleta12:matrix.org left the room.07:02:07
@vangelis369:matrix.org@vangelis369:matrix.org joined the room.10:18:02
6 Nov 2024
@serge90:matrix.orgserge90 joined the room.12:50:50
7 Nov 2024
@hank:myportal.socialHankWith the three active users on my server really pounding the keyboard and mouse clicks yesterday my "little" Digital Ocean droplet (4 cores, 8 GB of RAM with the database on a secondary volume) couldn't keep up. I thought I turned off cleanup operations but the ExpirePosts started and was really slamming the system mid-day causing several hours delay in processing AP delivery stuff. But even after I turned it off in late afternoon it still hadn't caught up by 11 pm. And that was with me going out to do stuff so basically shutting off my spigot of new activities five hours before. 12:52:56
@hank:myportal.socialHankI have a project I just picked up that will focus me in the short term but I think something I'm going to try to double back to focus on is optimizing the data storage/database. I know a lot of effort has been going into it for years and it's gotten much better than it has in the past. I also know it's far outside my expertise range. But I feel having those resources for just a handful of admittedly very active users shouldn't be a problem for a machine with that much horsepower.13:11:19
@feeds:integrations.ems.hostFeeds New post in Reddit: What happened to friendica.opensocial.space? 23:09:22
10 Nov 2024
@pepecyb:matrix.orgpepecyb joined the room.09:37:06
@hank:myportal.socialHank So I'm continuing to try to diagnose my performance issues. My post table, which also has activities and comments, has about 18 million records and is 35 GB. I don't think that should be enough to explain a lot of this slowness. 22:02:31
@hank:myportal.socialHank However over 98% of the content in that table is remote. Each month that ratio tips. The earlier stuff is like 93% remote. Last month was 99% remote. 22:03:20
@hank:myportal.socialHankI thought I/we had it cleaning up older remote content but the fact I still have millions of entries going all the way back to the origin of my server makes me think that is not the case.22:03:51
@hank:myportal.socialHankMonthlyStats.png
Download MonthlyStats.png
22:04:00
@hank:myportal.socialHank Those are off of the created entry for each row in the post table. 22:04:22
@hank:myportal.socialHankShould this be a mostly empty graph past the cleanup window? 22:04:53
@krvai:grin.hukrvai joined the room.22:26:20
@pepecyb:matrix.orgpepecyb

No connection to Hubzilla channels possible.

I host two Hubzilla hubs and one Friendica instance. The hub ‘hub.hubzilla.hu’ and the Friendica instance ‘barat.fedi-verse.hu’ are running on a VPS under YunoHost. The second hub ‘klacker.org’ runs on another VPS where it is installed manually.

It is not possible to create connections between my Friendica channel and a Hubzilla channel that is on one of my two hubs. Connecting (with handle or also with the Proful URL) from Friendica leads to ‘504 Gateway Time-out’ after some time. If I check the channels in the admin area ‘Check address’, the error ‘504’ appears even after a long wait.

If I establish a connection to my Friendica channel from one of the Hubzilla channels, the connection is displayed as established in Hubzilla. In Friendica, however, the connection does not appear under ‘Contacts’. I also do not receive a connection request from Friendica and the channel does not appear under ‘Pending’.

With my channels on the two hubs, I have functioning connections to channels on other Friendica instances without any problems. There are also no problems with connections to other Fediverse services.

I also have working connections from my Friendica channel to Hubzilla channels on hubs other than my own, which work fine.

As a test, I also made connections from another Friendica account (at anonsys) to channels on my two hubs. This also worked without any problems.

Connections from the other (third-party) Friendica channel to my own Friendica channel can also be established without any problems.

Address-Probe attempts lead to error messages in the nginx log:

2024/11/10 00:09:04 [error] 3869222#3869222: *11133498 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 5.204.231.9, server: barat.fedi-verse.hu, request: "GET /probe?addr=pepecyb%40hub.hubzilla.hu HTTP/2.0", upstream: "fastcgi://unix:/var/run/php/php8.2-fpm-friendica.sock", host: "barat.fedi-verse.hu", referrer: "https://barat.fedi-verse.hu/probe"

2024/11/10 09:57:09 [error] 3869222#3869222: *11201928 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 5.204.231.9, server: barat.fedi-verse.hu, request: "GET /probe?addr=hvetinari%40klacker.org HTTP/2.0", upstream: "fastcgi://unix:/var/run/php/php8.2-fpm-friendica.sock", host: "barat.fedi-verse.hu", referrer: "https://barat.fedi-verse.hu/probe"

I am desperate because I have no idea what the reason for this could be and how I can eliminate the malfunction. Does anyone have any ideas?

23:31:44
@hank:myportal.socialHankI have no experience with the Hubzilla stuff :( 23:33:16
11 Nov 2024
@hank:myportal.socialHankPS for reference, the stats for the local entries for the life of my server02:24:20

Show newer messages


Back to Room ListRoom Version: 6