| 11 Apr 2023 |
|  bajtos joined the room. | 11:44:58 |
 Hunter Beast | In reply to @rklaehn:matrix.org
Wdym batch validation? This would be quite easy to implement using my crate bao-tree, but one of the main attractions of BAO is streaming validation as data comes in
Perfect! That's exactly what I was looking for.
By "batching", I meant this:
"It also allows encoding not just single ranges but sets of non-overlapping ranges. E.g. you can ask for bytes [0..1000,5000..6000] in a single query."
Also, while I have you, shouldn't those be 1024 byte ranges? | 13:09:18 |
| Hunter Beast | I'm working on this project and noticed there were a few other things I needed.
https://github.com/diba-io/carbonado
I might look into switching to bao-tree instead, if you think that's a good idea.
bao-tree supports append-only logs so long as you use outboard encoding, correct? There's no way to do append-only with the combined format? | 13:12:55 |
 rklaehn | You could do a combined format that supports append only, but the default format is pre order, so the two hashes immediately below the root come first. | 16:17:35 |
| rklaehn | For iroh we want outboards because we want the files itself to remain unchanged. | 16:20:27 |
| rklaehn | Imagine adding a 300gb file and then having to mix it with hashes. | 16:20:45 |
| rklaehn | * You could do a combined format that supports append only, but the default format is pre order, so the two hashes immediately below the root come first. So obviously that won't work when appending. | 16:22:24 |
| rklaehn | bao-tree just defines the tree and a few iterators, so it is pretty flexible in terms of how you store the actual data. | 16:29:13 |
| rklaehn |
E.g. you can ask for bytes [0..1000,5000..6000] in a single query.
Yes, that is correct. The actual querying is always in terms of 1024 byte bao chunks. Byte queries have to be rounded up to chunks.
| 16:36:57 |
| rklaehn | I will do a presentation about our plans at ipfs thing on Saturday. | 16:54:36 |
| rklaehn | Carbonado looks interesting... | 16:58:17 |
| rklaehn |
I might look into switching to bao-tree instead, if you think that's a good idea.
Well, we are going to use bao-tree as the base for iroh. That means that we intend to maintain it. But be aware that there might be some instability in the coming month or two.
| 17:50:24 |
| rklaehn | I am pretty happy with the core, which is just the definition of the binary tree and a few iterators. But especially the io part will undergo some changes. It is really annoying to have so much duplication between the async and sync io stuff... | 17:51:31 |
| 12 Apr 2023 |
| Hunter Beast | In reply to @rklaehn:matrix.org
I am pretty happy with the core, which is just the definition of the binary tree and a few iterators. But especially the io part will undergo some changes. It is really annoying to have so much duplication between the async and sync io stuff... Yeah, async traits can't arrive soon enough... | 14:05:02 |
| Hunter Beast | In reply to @rklaehn:matrix.org
Carbonado looks interesting... Thank you! Iroh does as well! If there's any way you think we can collaborate on, say, Lightning pinning services or archival of files on Carbonado (as a replacement for Filecoin), I'd be very interested in any ideas that come to mind. | 14:06:15 |
| Hunter Beast | In reply to @rklaehn:matrix.org
Imagine adding a 300gb file and then having to mix it with hashes. Yeah, we've decided to just slice and encode files into 1MB segments... For 300GB that'd be 300k files, but at least it'd stream well in memory. The reason we decided to do that is because the entire file is encrypted at-rest, so it doesn't matter if the file is untouched or not. | 14:07:31 |
| Hunter Beast | In reply to @rklaehn:matrix.org
I will do a presentation about our plans at ipfs thing on Saturday. Where can I see that, btw? | 14:07:57 |
| 13 Apr 2023 |
|  David | W3F - Traveling changed their display name from David | W3F - OOO Apr 13 to David | W3F. | 07:30:49 |
| rklaehn | I think the presentations will be put on YouTube. Not sure if there is going to be a live stream. | 09:13:51 |
| Hunter Beast | In reply to @rklaehn:matrix.org
I think the presentations will be put on YouTube. Not sure if there is going to be a live stream. Well, be sure to drop a link once you have one! | 16:08:52 |
| 18 Apr 2023 |
|  @ewhokjvipjqjkd:matrix.org left the room. | 15:32:08 |
|  rvlobato joined the room. | 19:58:07 |
| 21 Apr 2023 |
|  @nukemandan:matrix.org removed their profile picture. | 03:41:26 |
| @nukemandan:matrix.org changed their display name from NukeManDan to Nuke 🙋. | 03:41:43 |
| @nukemandan:matrix.org invited  @nuke:parity.io. | 03:46:30 |
| @nuke:parity.io joined the room. | 03:47:08 |
| Hunter Beast | In reply to @rklaehn:matrix.org
I think the presentations will be put on YouTube. Not sure if there is going to be a live stream. How'd that go? | 18:26:58 |
| rklaehn | Nothing published yet. Here are the slides: | 21:53:19 |
| rklaehn | https://n0-computer.github.io/bao-docs/bao-thing.html | 21:54:05 |
| rklaehn | Was well received, although several people are disappointed that we don't aim for compatibility with ipfs anymore. | 21:56:28 |
