Sender | Message | Time |
---|---|---|
16 Mar 2023 | ||
rklaehn | @mindtree your best bet is https://github.com/n0-computer/beetle . `iroh` is no longer aiming for unixfs compatibility. | 04:24:05 |
rklaehn | Ipfs-embed itself is solid and is being used in production for years. But it has minimal support for unixfs (just reading). | 04:25:43 |
rklaehn | (Beetle is the former go-ipfs compatible iroh) | 04:26:51 |
paulali | mindtree: I think it depends on your usecase, if the current state of Iroh meets it that is okay. The rust implementation is a bit behind compared to where kubo support currently is. You could check out https://docs.rs/ipfs-unixfs/latest/ipfs_unixfs/ to see if you could use it to fit your needa | 04:30:13 |
paulali | * mindtree: I think it depends on your usecase, if the current state of Iroh meets it that is okay. The rust implementation is a bit behind compared to where kubo support currently is. You could check out https://docs.rs/ipfs-unixfs/latest/ipfs_unixfs/ to see if you could use it to fit your needs | 04:30:25 |
rklaehn | We (n0) got a decent gateway with unixfs read and write support including HAMT. I think the unixfs part is pretty complete. Where we had issues is 100% interop with go-ipfs regarding bitswap, due to some rust-libp2p issues beyond our control. | 04:35:16 |
mindtree | Thanks both for the SITREP! To give a little more context on my application - I'm working on a source code package manager similar to UnixFS struck me as a good fit, as I could just "insert" the package's directory when publishing, and "get" it when fetching source. I was hoping to use something compatible with go-ipfs, so that users could use existing tooling to easily mirror/pin registry packages that they care about, but it sounds like we might not have an implementation capable of both insertion and retrieval in a go-ipfs-compatible manner 🤔 For now I might even just try compressing the source directory into a slice of bytes and then Does this sound crazy? I'll likely revisit UnixFS again in the future, but at least this would get us started | 04:44:00 |
mindtree | rklaehn: I just watched your blake3 and bao overview earlier - thanks for sharing, was cool to get an overview of the guts :)
Is there somewhere I can read about what these issues were? It sounds like you guys are currently focused on maximal performance for p2p content-addressed storage in general. Is the idea that once you've got this lower-level implemented, you'll look into ipfs-compatibility on top? Or do you see iroh as a whole new alternative to IPFS? | 04:48:36 |
rklaehn | Well, we see it as fulfilling the original promise of ipfs - a global content-addressed file system. | 04:50:25 |
rklaehn | If someone wants to put unixfs on top, no problem. But go-ipfs compat is not planned. | 04:51:49 |
rklaehn | We spent a lot of time writing a high performance bitswap impl, only to notice that it can not be really high performance and follow the go-ipfs approach. We also had issues with go-ipfs and rust-libp2p interop at the stream level. At some point it was just too much... | 04:55:35 |
rklaehn | Don't know the exact details, since I was not doing that part. But for ipfs-embed by David and me we also found go-ipfs compat hard and finicky. | 04:57:05 |
rklaehn | Glad you liked the video. It was the first take where the recording worked, so it was not perfect. But I don't have time to redo it - have to write BAO stuff... | 04:59:30 |
mindtree | In reply to @rklaehn:matrix.org Yeah I can see where you're coming from for sure, and I'm excited to see what you folks come up with! The main concern I have for iroh is that I worry folks already associate "IPFS" with not just the idea, but the go implementation. There's already a lot of tooling around the go implementation like the CLI, the browser extension, etc, and when a lot of folks think IPFS I'm guessing they just think of those tools. If iroh is to dismiss compatibility with all of this tooling, that seems like a whole other level of ambition as I guess it puts you in a position where you're now competing not only against the go implementation, but against the whole ecosystem's tooling, its marketing, and the existing mindshare where "go-ipfs = IPFS"? If that's the plan, then I have a lot of respect for iroh in going for it! But I do worry that people will get confused when they're looking for an IPFS implementation, come across iroh which describes itself as an IPFS implementation, they try using it, and then wonder why it doesn't work with "all the other IPFS stuff". It would be cool to see a compatibility table between kubo and iroh's plans. E.g. is it just that the two will have different networks because of their DHT implementation differences? Or is it the case that even the CIDs for the same raw data are different between the two? | 05:12:01 |
rklaehn | We are figuring it all out right now. Expect more clarity by the time of ipfs thing in April. | 05:13:33 |
rklaehn | Our Cids are blake3 Cids, and our blocks can be of arbitrary size. Other than that there is nothing unusual about them. | 05:14:39 |
17 Mar 2023 | ||
@siman:matrix.org left the room. | 00:56:42 | |
David | W3F - OOO May 13 changed their display name from David to David - OOO (sick). | 06:10:12 | |
18 Mar 2023 | ||
Torir joined the room. | 22:27:55 | |
Torir changed their display name from torir to Torir. | 22:47:13 | |
20 Mar 2023 | ||
@musicmatze:beyermatthi.as left the room. | 15:47:21 | |
22 Mar 2023 | ||
David | W3F - OOO May 13 changed their display name from David - OOO (sick) to David | W3F. | 08:07:13 | |
23 Mar 2023 | ||
@chasev:matrix.org joined the room. | 00:19:20 | |
Games joined the room. | 14:29:29 | |
28 Mar 2023 | ||
@hamnox:matrix.org left the room. | 23:32:44 | |
5 Apr 2023 | ||
ivii joined the room. | 15:37:04 | |
6 Apr 2023 | ||
sergd85 joined the room. | 03:34:43 | |
7 Apr 2023 | ||
David | W3F - OOO May 13 changed their display name from David | W3F to David | W3F - OOO Apr 13. | 07:53:12 | |
10 Apr 2023 | ||
Hunter Beast | rklaehn: I found a version of Bao once that implemented batch slice verification, have you seen it? I forget it's name, but I figured you might know. Also, do you know if the existing crate is still being maintained? | 21:42:20 |
11 Apr 2023 | ||
rklaehn | Wdym batch validation? This would be quite easy to implement using my crate bao-tree, but one of the main attractions of BAO is streaming validation as data comes in | 11:27:08 |