247 Members
Implementing ipfs in rust based on rust-libp2p at https://github.com/rs-ipfs/rust-ipfs37 Servers

Load older messages

21 Sep 2020
@dvc94ch:matrix.orgdvc94chYes for example16:31:51
@dvc94ch:matrix.orgdvc94chIt's a much more useful abstraction than unnamed recursive pins16:32:14
@rklaehn:matrix.orgrklaehnI see. So an alias is a named pin, and otherwise everything (refcounting etc.) remains the same?16:32:29
@dvc94ch:matrix.orgdvc94chWith additionally unaliased blocks being in a cache16:33:20
@dvc94ch:matrix.orgdvc94chSo they will be evicted over time16:33:44
@dvc94ch:matrix.orgdvc94chWhen buffering a video or retrieving a website, you don't alias it and it will be cleaned up when the cache is full16:34:39
@rklaehn:matrix.orgrklaehnThat would mean retaining some kind of lru data for blocks. Yes, sounds useful. But if you build stuff you should nevertheless not rely on stuff staying in the lru, but pin in the same txn where you add a tree.16:35:56
@dvc94ch:matrix.orgdvc94chWell then you need to manage cleanup manually16:36:43
@rklaehn:matrix.orgrklaehnNo, should work fine with the alias concept. I have a tree, do some changes, get a new tree (new root hash), and store that in the alias for the tree, which will start getting rid of the parts of the tree that I no longer need. At no point do I rely on the LRU.16:37:59
@dvc94ch:matrix.orgdvc94chBut yes, stuff you want to keep (<= cache size) needs to be inserted and then aliased to keep it around16:38:00
@dvc94ch:matrix.orgdvc94chThat also works yes, you can't do that easily with unnamed pins16:39:49
@dvc94ch:matrix.orgdvc94chWell if you don't want to rely on the cache at all inserting aliasing and unaliasing needs to be packed in transactions16:40:32
@rklaehn:matrix.orgrklaehnIn this case the size of the change would be limited, so I guess this would be no problem. But I don't know anything about sled transactions except that they exist and that they can span multiple trees...16:42:54
@dvc94ch:matrix.orgdvc94chThey are serializable16:43:54
@rklaehn:matrix.orgrklaehnAnyway, the alias concept makes a lot of sense, I was just initially confused by the name.16:44:22
@dvc94ch:matrix.orgdvc94chNo weirdness, unlike other dbs, the semantics of sled are what you'd expect16:44:49
@rklaehn:matrix.orgrklaehnSo sled does not have a single writer, like e.g. LMDB?16:48:02
@dvc94ch:matrix.orgdvc94chI'm don't think so, but it's behavior is like having a single writer16:49:59
@dvc94ch:matrix.orgdvc94chBut you should probably ask on the sled channel...16:50:16
@rklaehn:matrix.orgrklaehnQuite eager to see the next version of ipfs-embed... Have a nice evening.16:52:01
@dvc94ch:matrix.orgdvc94chThanks! You too16:52:18
@dvc94ch:matrix.orgdvc94chPR is ready for review: https://github.com/ipfs-rust/rust-ipld/pull/6017:48:22
@darkharmony9999:matrix.orgdarkharmony9999 joined the room.19:41:59
22 Sep 2020
@rklaehn:matrix.orgrklaehnDon't have time for a thorough review, but added some comments.07:22:37
@dvc94ch:matrix.orgdvc94chYes the network operations can timeout07:28:16
@dvc94ch:matrix.orgdvc94chI'm thinking of readding the concept of unnamed pins, with the difference that they aren't persisted to disk. This allows not depending on the cache (you could set the cache to 0)07:30:01
@dvc94ch:matrix.orgdvc94chThanks for your comments!07:30:19
@dvc94ch:matrix.orgdvc94chSo essentially they'd be a sort of lock07:32:50
@dvc94ch:matrix.orgdvc94chIt makes things more complicated and harder to understand, let's see if there is a need for it first08:02:48
@rklaehn:matrix.orgrklaehn👍️ keep it simple for now. Once the whole db issue is settled, it should be simple to add if really needed...08:04:24

There are no newer messages yet.

Back to Room List