!MgxXRKIYSYvETkEOgo:matrix.org

mew

8 Members
the mew build system · https://mew.build/ · logs: https://view.matrix.org/alias/%23mew-build:matrix.org https://freenode.logbot.info/mew1 Servers

Load older messages


SenderMessageTime
1 Mar 2020
@freenode_energizer`:matrix.orgenergizer`you helped me a few days ago in #nixos and i took a look at your github21:45:37
@freenode_energizer`:matrix.orgenergizer`and got interested :-)21:45:52
@emilazy:matrix.orgemilyhopefully I'll find the time/motivation to make it more code and less ideas soon :)21:46:57
@freenode_energizer`:matrix.orgenergizer` i think nix could be a good data science tool (like http://dvc.org/) if it were easy to keep 100GB files in the store. in that domain, data privacy rules may disallow publication of raw data sets, but permit publication of summary tables. under "differential privacy" the threat of deanonymization (analogous to decompilation) is mitigated 21:48:52
@emilazy:matrix.orgemilythat's an interesting use case! I definitely think that systems like Nix are really important for ensuring long-term reproducibility of scientific results. storing objects of hundreds of gigabytes in the store would be a pretty big technical challenge in a few ways, though I guess you'd probably be processing them on a big enough machine that you can bear hashing them21:52:40
@emilazy:matrix.orgemilythe store changes I have planned for mew are more along the lines of making it viable to have more and smaller objects in the store (because of caching a lot more during evaluation) -- so much "DB-y" -- rather than supporting larger files, which would be closer to the FS end of things21:54:11
@emilazy:matrix.orgemily(but outsourcing large blobs to the FS would be easy too, just add more complexity than I wanted to deal with for an initial draft)21:54:28
@freenode_energizer`:matrix.orgenergizer`caching a lot more would be cool. i can imagine compilers putting their data like https://blog.rust-lang.org/2016/09/08/incremental.html into the store21:58:44
@emilazy:matrix.orgemilyyep; the basic idea is to structure the build language to integrate with the object store, and you can just slip a cache anywhere to have it seamlessly reused when the dependencies of an expression hasn't changed, etc.22:00:01
@emilazy:matrix.orgemilynixpkgs evaluation time is a fairly substantial overhead and an impediment to recursive Nix use right now, so I want to address that upfront22:00:17
@emilazy:matrix.orgemilyand more fundamentally get rid of the distinction between "evaluation" vs. "instantiation" Nix has22:00:39
@emilazy:matrix.orgemilyinstead pulling compilers into the language as if they're native functions22:00:55
@freenode_energizer`:matrix.orgenergizer`similar to https://github.com/nmattia/snack i guess22:01:00
@emilazy:matrix.orgemilymhm22:02:38
@emilazy:matrix.orgemilyyou can quite practically do incremental Rust/Haskell builds with Nix now basically because the compilers are already so slow that the per-derivation overhead doesn't matter :p22:02:59
@emilazy:matrix.orgemily(well, it's also partly having a larger unit of compilation, people are just used to Rust recompiling the entire crate)22:03:10
@freenode_energizer`:matrix.orgenergizer`haha22:03:12
@emilazy:matrix.orgemilybut less practical if you want to e.g. hook every call the Linux kernel build does to cc to call back into your build system and hash its inputs for (s)ccache-type magic22:03:51
2 Mar 2020
@freenode_energizer`:matrix.orgenergizer` left the room.03:31:59
@creepy-skeleton:matrix.orgcreepy-skeleton joined the room.10:11:46
@creepy-skeleton:matrix.orgcreepy-skeleton left the room.10:15:58
3 Mar 2020
@freenode_emilazy[m]:matrix.orgfreenode_emilazy[m] left the room.14:26:45
30 Mar 2020
@mewtrino:matrix.orgmewtrino joined the room.01:05:51
@mewtrino:matrix.orgmewtrinoi'm suing for copyright infringement.01:06:14
6 May 2020
@vika_nezrimaya:cs.xa0.uk@vika_nezrimaya:cs.xa0.uk joined the room.16:34:32
@vika_nezrimaya:cs.xa0.uk@vika_nezrimaya:cs.xa0.ukmew16:37:52
@vika_nezrimaya:cs.xa0.uk@vika_nezrimaya:cs.xa0.ukthat's a very cute name for something as powerful and omnipresent as a build system16:38:13
20 May 2020
@yegortimoshenko:matrix.orgyegortimoshenko changed their profile picture.04:10:12
29 May 2020
@arc:arcn.mxarc left the room.10:16:54
16 Jun 2020
@vika_nezrimaya:cs.xa0.uk@vika_nezrimaya:cs.xa0.uk left the room.16:59:48

Show newer messages


Back to Room List