!MgxXRKIYSYvETkEOgo:matrix.org

mew

11 Members
the mew build system · https://mew.build/ · logs: https://view.matrix.org/alias/%23mew-build:matrix.org http://libera.irclog.whitequark.org/mew/3 Servers

Load older messages


SenderMessageTime
29 Feb 2020
@emilazy:matrix.orgemilyone of these days I'm going to have to get over my aversion for arrow notation I guess16:02:34
1 Mar 2020
@freenode_energizer`:matrix.orgenergizer` joined the room.21:17:38
@freenode_energizer`:matrix.orgenergizer`hello. i saw mew docs emphasize object capabilities. i am curious if something like https://github.com/NixOS/nix/issues/8#issuecomment-145911503 is in scope for mew21:25:20
@emilazy:matrix.orgemilyI've definitely thought about whether it would be practical to make the store not world-readable, yeah21:25:57
@emilazy:matrix.orgemilyI'm not sure it's ultimately the correct approach to take for secrets but it's something I'm interested in exploring21:26:10
@emilazy:matrix.orgemilythe plan is to not use the filesystem as backing storage so there'll need to be some amount of filesystem namespacing / FUSE / ... going on to begin with21:26:42
@emilazy:matrix.orgemily(there are some uses outside of general secrets -- e.g. if part of your code is trade secret and you want to restrict access while still allowing general builds, google actually has this -- though admittedly I don't know of any examples I'd consider particularly compelling)21:27:58
@freenode_energizer`:matrix.orgenergizer` meaning someone else is allowed to build my code but doesn't get access to its source? 21:34:30
@freenode_energizer`:matrix.orgenergizer`or just that someone else is not allowed to see my stuff even though we share the same bash21:38:25
@freenode_energizer`:matrix.orgenergizer`(and libc, etc)21:38:41
@emilazy:matrix.orgemilyyeah, along the lines of the former, secret libraries that get linked to. but it's kind of marginal and of course you can't defend against decompilation, hence why I said I don't feel like it's a particularly compelling usecase21:40:19
@freenode_energizer`:matrix.orgenergizer`mhmm21:41:32
@emilazy:matrix.orgemily(also, hi! I'm curious how you found the repo since I haven't really advertised it around anywhere ^^ )21:44:28
@freenode_energizer`:matrix.orgenergizer`you helped me a few days ago in #nixos and i took a look at your github21:45:37
@freenode_energizer`:matrix.orgenergizer`and got interested :-)21:45:52
@emilazy:matrix.orgemilyhopefully I'll find the time/motivation to make it more code and less ideas soon :)21:46:57
@freenode_energizer`:matrix.orgenergizer` i think nix could be a good data science tool (like http://dvc.org/) if it were easy to keep 100GB files in the store. in that domain, data privacy rules may disallow publication of raw data sets, but permit publication of summary tables. under "differential privacy" the threat of deanonymization (analogous to decompilation) is mitigated 21:48:52
@emilazy:matrix.orgemilythat's an interesting use case! I definitely think that systems like Nix are really important for ensuring long-term reproducibility of scientific results. storing objects of hundreds of gigabytes in the store would be a pretty big technical challenge in a few ways, though I guess you'd probably be processing them on a big enough machine that you can bear hashing them21:52:40
@emilazy:matrix.orgemilythe store changes I have planned for mew are more along the lines of making it viable to have more and smaller objects in the store (because of caching a lot more during evaluation) -- so much "DB-y" -- rather than supporting larger files, which would be closer to the FS end of things21:54:11
@emilazy:matrix.orgemily(but outsourcing large blobs to the FS would be easy too, just add more complexity than I wanted to deal with for an initial draft)21:54:28
@freenode_energizer`:matrix.orgenergizer`caching a lot more would be cool. i can imagine compilers putting their data like https://blog.rust-lang.org/2016/09/08/incremental.html into the store21:58:44
@emilazy:matrix.orgemilyyep; the basic idea is to structure the build language to integrate with the object store, and you can just slip a cache anywhere to have it seamlessly reused when the dependencies of an expression hasn't changed, etc.22:00:01
@emilazy:matrix.orgemilynixpkgs evaluation time is a fairly substantial overhead and an impediment to recursive Nix use right now, so I want to address that upfront22:00:17
@emilazy:matrix.orgemilyand more fundamentally get rid of the distinction between "evaluation" vs. "instantiation" Nix has22:00:39
@emilazy:matrix.orgemilyinstead pulling compilers into the language as if they're native functions22:00:55
@freenode_energizer`:matrix.orgenergizer`similar to https://github.com/nmattia/snack i guess22:01:00
@emilazy:matrix.orgemilymhm22:02:38
@emilazy:matrix.orgemilyyou can quite practically do incremental Rust/Haskell builds with Nix now basically because the compilers are already so slow that the per-derivation overhead doesn't matter :p22:02:59
@emilazy:matrix.orgemily(well, it's also partly having a larger unit of compilation, people are just used to Rust recompiling the entire crate)22:03:10
@freenode_energizer`:matrix.orgenergizer`haha22:03:12

Show newer messages


Back to Room ListRoom Version: 5