1 Mar 2020 |
emily | (also, hi! I'm curious how you found the repo since I haven't really advertised it around anywhere ^^ ) | 21:44:28 |
@freenode_energizer`:matrix.org | you helped me a few days ago in #nixos and i took a look at your github | 21:45:37 |
@freenode_energizer`:matrix.org | and got interested :-) | 21:45:52 |
emily | hopefully I'll find the time/motivation to make it more code and less ideas soon :) | 21:46:57 |
@freenode_energizer`:matrix.org | i think nix could be a good data science tool (like http://dvc.org/) if it were easy to keep 100GB files in the store. in that domain, data privacy rules may disallow publication of raw data sets, but permit publication of summary tables. under "differential privacy" the threat of deanonymization (analogous to decompilation) is mitigated | 21:48:52 |
emily | that's an interesting use case! I definitely think that systems like Nix are really important for ensuring long-term reproducibility of scientific results. storing objects of hundreds of gigabytes in the store would be a pretty big technical challenge in a few ways, though I guess you'd probably be processing them on a big enough machine that you can bear hashing them | 21:52:40 |
emily | the store changes I have planned for mew are more along the lines of making it viable to have more and smaller objects in the store (because of caching a lot more during evaluation) -- so much "DB-y" -- rather than supporting larger files, which would be closer to the FS end of things | 21:54:11 |
emily | (but outsourcing large blobs to the FS would be easy too, just add more complexity than I wanted to deal with for an initial draft) | 21:54:28 |
@freenode_energizer`:matrix.org | caching a lot more would be cool. i can imagine compilers putting their data like https://blog.rust-lang.org/2016/09/08/incremental.html into the store | 21:58:44 |
emily | yep; the basic idea is to structure the build language to integrate with the object store, and you can just slip a cache anywhere to have it seamlessly reused when the dependencies of an expression hasn't changed, etc. | 22:00:01 |
emily | nixpkgs evaluation time is a fairly substantial overhead and an impediment to recursive Nix use right now, so I want to address that upfront | 22:00:17 |
emily | and more fundamentally get rid of the distinction between "evaluation" vs. "instantiation" Nix has | 22:00:39 |
emily | instead pulling compilers into the language as if they're native functions | 22:00:55 |
@freenode_energizer`:matrix.org | similar to https://github.com/nmattia/snack i guess | 22:01:00 |
emily | mhm | 22:02:38 |
emily | you can quite practically do incremental Rust/Haskell builds with Nix now basically because the compilers are already so slow that the per-derivation overhead doesn't matter :p | 22:02:59 |
emily | (well, it's also partly having a larger unit of compilation, people are just used to Rust recompiling the entire crate) | 22:03:10 |
@freenode_energizer`:matrix.org | haha | 22:03:12 |
emily | but less practical if you want to e.g. hook every call the Linux kernel build does to cc to call back into your build system and hash its inputs for (s)ccache-type magic | 22:03:51 |
2 Mar 2020 |
| @freenode_energizer`:matrix.org left the room. | 03:31:59 |
| @creepy-skeleton:matrix.org joined the room. | 10:11:46 |
| @creepy-skeleton:matrix.org left the room. | 10:15:58 |
3 Mar 2020 |
| @freenode_emilazy[m]:matrix.org left the room. | 14:26:45 |
30 Mar 2020 |
| @mewtrino:matrix.org joined the room. | 01:05:51 |
@mewtrino:matrix.org | i'm suing for copyright infringement. | 01:06:14 |
6 May 2020 |
| @vika_nezrimaya:cs.xa0.uk joined the room. | 16:34:32 |
@vika_nezrimaya:cs.xa0.uk | mew | 16:37:52 |
@vika_nezrimaya:cs.xa0.uk | that's a very cute name for something as powerful and omnipresent as a build system | 16:38:13 |
20 May 2020 |
| @yegortimoshenko:matrix.org changed their profile picture. | 04:10:12 |
29 May 2020 |
| @arc:arcn.mx left the room. | 10:16:54 |