| 27 Nov 2022 |
 realtakase | never | 22:28:03 |
 Jan | How lucky that posix has a thing for that :P | 22:28:30 |
| realtakase | lite-xl doesn't have anything more advanced than stdio's api aside from whatever functions we happen to extend in system | 22:28:41 |
| Jan | well, maybe it should | 22:29:06 |
| realtakase | for size checking, its probably possible to use trunc | 22:29:10 |
| realtakase | well maybe while we're at it moving most of the fs functions into its own module for sanity | 22:29:39 |
 Guldoman | before that we should not overwrite the file directly | 22:29:41 |
| Jan | it depends | 22:29:54 |
| Jan | I agree that simply overwriting the file is dirty and could leave damaged files if the program has a problem during it | 22:30:18 |
| Jan | but putting the file into a temporary location before moving it into the correct place double the size requirements | 22:30:45 |
| Guldoman | which for a text editor is fine | 22:31:03 |
| realtakase | maybe not do that after a threshold? | 22:31:03 |
| Jan | I guess so | 22:31:10 |
| Jan | another option would be writing in-place | 22:31:22 |
| Jan | idk, we'll see | 22:31:46 |
| Jan | my first goal will be improving memory consumption for bigger files | 22:31:59 |
| Guldoman | the only thing that bloats things is the tokenizer keeping every single string token saved | 22:32:41 |
| Guldoman | replacing each token string with two ints (start, end) would improve memory | 22:33:45 |
| Guldoman | * replacing each token string with two ints (start, end) would improve memory consumption | 22:33:54 |
| Guldoman | then extracting the token string when needed from the two ints | 22:34:17 |
| Guldoman | shouldn't be too demanding | 22:34:26 |
| Guldoman | hopefully | 22:34:28 |
| realtakase | strings are still interned, you'd still end up with chunks of string either way unless the tokenizer has direct access to string and doesn't pass back to renderer through lua | 22:35:46 |
| Guldoman | what do you mean | 22:36:16 |
| Guldoman | at the moment each single string token is saved | 22:36:30 |
| Guldoman | while lua dedupes them, in big files this adds up a lot | 22:36:53 |
| Guldoman | highlighter keeps the original text line (which shouldn't increase memory consumption as it's a copy of the one in doc) | 22:37:45 |
| Guldoman | so the tokenizer function should return only two numbers for each token instead of the partial string | 22:38:20 |
| Guldoman | extracting the strings on demand means that we don't keep them | 22:38:57 |
| 28 Nov 2022 |
|  daeraxa joined the room. | 03:06:24 |
