!MgxXRKIYSYvETkEOgo:matrix.org

mew

8 Members
the mew build system · https://mew.build/ · logs: https://view.matrix.org/alias/%23mew-build:matrix.org https://freenode.logbot.info/mew2 Servers

Load older messages


Timestamp Message
1 Mar 2020
21:44:28@emilazy:matrix.orgemily(also, hi! I'm curious how you found the repo since I haven't really advertised it around anywhere ^^ )
21:45:37@freenode_energizer`:matrix.orgenergizer`you helped me a few days ago in #nixos and i took a look at your github
21:45:52@freenode_energizer`:matrix.orgenergizer`and got interested :-)
21:46:57@emilazy:matrix.orgemilyhopefully I'll find the time/motivation to make it more code and less ideas soon :)
21:48:52@freenode_energizer`:matrix.orgenergizer` i think nix could be a good data science tool (like http://dvc.org/) if it were easy to keep 100GB files in the store. in that domain, data privacy rules may disallow publication of raw data sets, but permit publication of summary tables. under "differential privacy" the threat of deanonymization (analogous to decompilation) is mitigated
21:52:40@emilazy:matrix.orgemilythat's an interesting use case! I definitely think that systems like Nix are really important for ensuring long-term reproducibility of scientific results. storing objects of hundreds of gigabytes in the store would be a pretty big technical challenge in a few ways, though I guess you'd probably be processing them on a big enough machine that you can bear hashing them
21:54:11@emilazy:matrix.orgemilythe store changes I have planned for mew are more along the lines of making it viable to have more and smaller objects in the store (because of caching a lot more during evaluation) -- so much "DB-y" -- rather than supporting larger files, which would be closer to the FS end of things
21:54:28@emilazy:matrix.orgemily(but outsourcing large blobs to the FS would be easy too, just add more complexity than I wanted to deal with for an initial draft)
21:58:44@freenode_energizer`:matrix.orgenergizer`caching a lot more would be cool. i can imagine compilers putting their data like https://blog.rust-lang.org/2016/09/08/incremental.html into the store
22:00:01@emilazy:matrix.orgemilyyep; the basic idea is to structure the build language to integrate with the object store, and you can just slip a cache anywhere to have it seamlessly reused when the dependencies of an expression hasn't changed, etc.
22:00:17@emilazy:matrix.orgemilynixpkgs evaluation time is a fairly substantial overhead and an impediment to recursive Nix use right now, so I want to address that upfront
22:00:39@emilazy:matrix.orgemilyand more fundamentally get rid of the distinction between "evaluation" vs. "instantiation" Nix has
22:00:55@emilazy:matrix.orgemilyinstead pulling compilers into the language as if they're native functions
22:01:00@freenode_energizer`:matrix.orgenergizer`similar to https://github.com/nmattia/snack i guess
22:02:38@emilazy:matrix.orgemilymhm
22:02:59@emilazy:matrix.orgemilyyou can quite practically do incremental Rust/Haskell builds with Nix now basically because the compilers are already so slow that the per-derivation overhead doesn't matter :p
22:03:10@emilazy:matrix.orgemily(well, it's also partly having a larger unit of compilation, people are just used to Rust recompiling the entire crate)
22:03:12@freenode_energizer`:matrix.orgenergizer`haha
22:03:51@emilazy:matrix.orgemilybut less practical if you want to e.g. hook every call the Linux kernel build does to cc to call back into your build system and hash its inputs for (s)ccache-type magic
2 Mar 2020
03:31:59@freenode_energizer`:matrix.orgenergizer` left the room.
10:11:46@creepy-skeleton:matrix.orgcreepy-skeleton joined the room.
10:15:58@creepy-skeleton:matrix.orgcreepy-skeleton left the room.
3 Mar 2020
14:26:45@freenode_emilazy[m]:matrix.orgfreenode_emilazy[m] left the room.
30 Mar 2020
01:05:51@mewtrino:matrix.orgmewtrino joined the room.
01:06:14@mewtrino:matrix.orgmewtrinoi'm suing for copyright infringement.
6 May 2020
16:34:32@vika_nezrimaya:cs.xa0.uk@vika_nezrimaya:cs.xa0.uk joined the room.
16:37:52@vika_nezrimaya:cs.xa0.uk@vika_nezrimaya:cs.xa0.ukmew
16:38:13@vika_nezrimaya:cs.xa0.uk@vika_nezrimaya:cs.xa0.ukthat's a very cute name for something as powerful and omnipresent as a build system
20 May 2020
04:10:12@yegortimoshenko:matrix.orgyegortimoshenko changed their profile picture.
29 May 2020
10:16:54@arc:arcn.mxarc left the room.

There are no newer messages yet.


Back to Room List