| 31 Oct 2024 |
|  haheute set a profile picture. | 19:31:43 |
|  johnassel joined the room. | 22:07:27 |
| 1 Nov 2024 |
 fikabot | 💬 (faeempress [XMPP])
It seems Friendica loves to complain about storage oer.issions in the logs. I've set everything below storage dir to 777 and chowned to www-data, but it still complains. | 04:11:45 |
| fikabot | 💬 (faeempress [XMPP])
storage *permissions | 04:11:58 |
 alias @bkil:grin.hu | You should treat Friendica as your baby: give her your constant love & attention so she could grow. It will cry from time to time and you will need to clean up its duty when full, but she will be grateful later on. | 07:25:37 |
|  @saruman:celeborn.de left the room. | 07:53:15 |
| alias @bkil:grin.hu | faeempress: By the way, Friendica does need write permissions on some unexpected directories as well. Please pay close attention to which directories the message mentions. | 09:48:27 |
|  @young3:matrix.org joined the room. | 10:01:04 |
| @young3:matrix.org left the room. | 10:01:19 |
 Hank | So far I'm not seeing any performance improvement and maybe even performance degradation with Redis :( | 10:50:18 |
| Hank | * So far I'm not seeing any performance improvement and maybe even performance degradation with Redis/Valkey :( | 10:52:44 |
| Hank | I've confirmed it is using it by shutting down redis and watching the Friendica site not render, along with looking at some of the Redis stats | 10:53:11 |
| Hank | I wonder if the problem is the large number of connections I have (~3500) and/or heavy use of circles | 11:08:04 |
 heluecht | There's a lot of possible improvements. Could be the number of open connections in the web server or the database. Also normally it is always the database when the system is slow. | 13:50:45 |
| Hank | It is definitely the database :( | 14:17:33 |
| Hank | I was just hoping the caching layer would help with that | 14:17:49 |
| Hank | Something might be broken with the DM system too. I keep having insufficient database connections errors as I'm trying to DM some people on my system | 15:03:37 |
 AndiS | After two days running Redis I'd also say performance is database bound - or more specificly the available memory. CPU usage seems to be a bit lower running Redis, but this is not the bottleneck as it seems. | 16:36:22 |
| AndiS | But since Friendica constantly uses less than 200 keys compared to almost 100.000 in use by Spamassassin - I'll just keep it configured | 16:42:07 |
| Hank | Yeah...making a DM apparently spikes the number of apache2 processes from ~40 to about ~150 on my Ubuntu 24.04 (recently upgraded) system, sans Redis now. Eventually it decays though. AT least that seems to be the catalyst for the process number explosion | 18:27:42 |
| Hank | Nah pretty much everything is causing the system to spike to 152 (the apache2 max processes) apparently :(. The biggest change that is persistent is the upgrade to 24.04 and PHP 8.3 from 7.4 | 19:06:46 |
| Hank | I wonder if there was a setting in php.ini that I need to copy over to 8.3 | 19:07:59 |
| Hank | Yeah max execution time, max args, and max memory were different...let's see if that fixes things | 19:13:31 |
| AndiS | I'm interested in what you find there! | 19:20:10 |
| Hank | Nope...I have two queries blocking all other 150 queries:
SELECT `id` FROM `item-uri` WHERE (`id` < 59247322 AND NOT EXISTS(SELECT `uri-id` FROM `post-user` WHERE `uri-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `parent-uri-id` FROM `post-user` WHERE `parent-uri-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `thr-parent-id` FROM `post-user` WHERE `thr-parent-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `external-id` FROM `post-user` WHERE `external-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `replies-id` FROM `post-user` WHERE `replies-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `context-id` FROM `post-thread` WHERE `context-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `conversation-id` FROM `post-thread` WHERE `conversation-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `uri-id` FROM `mail` WHERE `uri-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `uri-id` FROM `event` WHERE `uri-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `uri-id` FROM `user-contact` WHERE `uri-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `uri-id` FROM `contact` WHERE `uri-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `uri-id` FROM `apcontact` WHERE `uri-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `uri-id` FROM `diaspora-contact` WHERE `uri-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `uri-id` FROM `inbox-status` WHERE `uri-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `uri-id` FROM `post-delivery` WHERE `uri-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `uri-id` FROM `post-delivery` WHERE `inbox-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `parent-uri-id` FROM `mail` WHERE `parent-uri-id` = `item-uri`.`id`) AND NOT EXISTS(SELECT `thr-parent-id` FROM `mail` WHERE `thr-parent-id` = `item-uri`.`id`)) LIMIT 1000
| 19:20:56 |
| Hank | and another one which just finally cleared, so I guess this one is the one locking up everything else | 19:21:07 |
| Hank | still going 700 seconds later | 19:21:28 |
| Hank | Well whatever generates that command is doing it again. Is it the ExpirePosts call perhaps? | 19:28:18 |
| Hank | Something seems off if even this query is taking over two minutes:
select count(*) from `item-uri` where `id` < 59247322;
| 19:31:59 |
| Hank | This may explain why my DB has grown so large with most of it being content from other servers. There are 26,332,075 entries from that count command | 19:39:43 |
