!homDUJhmxHaZNgyLVA:matrix.org

binrw

162 Members
Official Matrix channel for the binread & binrw Rust libraries. Bridged to Discord. https://github.com/jam1garner/binrw/12 Servers

Load older messages


SenderMessageTime
16 Apr 2024
@vhdirk:matrix.orgDirk Van HaerenborghI'll have to make a standalone reproducer though15:44:38
@snover:matrix.orgsnover for an enum variant you’ll still need a separate struct that represents the bitfield and then enum Foo { BitVariant(BitStruct) }. 16:32:27
@vhdirk:matrix.orgDirk Van HaerenborghI see. I'll test it out, but my initial 'fear' is that it may make my API quite dense 16:36:47
@vhdirk:matrix.orgDirk Van HaerenborghEven so, how would I go about making an u8 enum that can only take up 2 bits or so?16:37:51
@snover:matrix.orgsnoverit’s not possible for anything to be smaller than one byte.16:53:42
@snover:matrix.orgsnoversome better guidance may be possible if you can say more about the structure of the data you are reading16:56:27
17 Apr 2024
@vhdirk:matrix.orgDirk Van Haerenborgh
In reply to @snover:matrix.org
it’s not possible for anything to be smaller than one byte.
I know that; I fully expect the enum to take up at least one byte when taking its value. Modular_bitfield has BitfieldSpecifier as a sort of helper to let any ambedding struct know it should be packed in a lower amount of bits
17:50:22
22 Apr 2024
@jacg:matrix.org@jacg:matrix.org left the room.07:09:47
24 Apr 2024
@kaiyou:tedomum.netkaiyou joined the room.21:04:27
13 May 2024
@jkorinth:matrix.orgjkorinth joined the room.07:51:09
@jkorinth:matrix.orgjkorinth hi *! is it possible to use binrw in no_std and without alloc? i can't figure out how to write into a &mut [u8; N]. 07:53:34
@jkorinth:matrix.orgjkorinth ah, nevermind. just had to cast to &mut [u8] and it worked. 👍️ 09:01:48
@jkorinth:matrix.orgjkorinth i need to read+write a data format that look something like this:
struct CrcBlock {
    crc32: u32,
    block: Block,
}
crc32 is the checksum over the bytes of block. verifying it may be possible with a #[br(temp, ...)], but how would i compute the checksum during write?
11:20:20
@kaiyou:tedomum.netkaiyouTo my understanding you need to use a `map_stream` to map the write stream and compute the CRC there, plus `stream` to grab the mapped stream into a variable. If your crc were written after the block, here is an example from the tests: https://github.com/jam1garner/binrw/blob/master/binrw/tests/derive/write/stream.rs If it is indeed written before, you can probably use a padding before your block, seek before the block when writing the CRC and restore position afterwards (all are implemented as bw attributes).15:40:47
14 May 2024
@jkorinth:matrix.orgjkorinthThx, will try that. Does sound a bit complicated, though, maybe I’ll just patch the crc in manually.05:13:03
@kaiyou:tedomum.netkaiyou

Indeed, deriving BinWrite does not expect or guarantee the steam is Read, only Write+Seek. Hence the need to provide a Write implementation that handles written bytes.

However, although I never used them this way, stream and map_stream together provide access to the concrete mapped type, not a trait object, so you could map to something that is Read+Write+Seek, e.g. some type that buffers writes and provide a Cursor-like interface. I don't think you will find this in std:: though.

Pretty sure it is less idiomatic, since it requires you buffer the entire written stream.

06:05:13
@kaiyou:tedomum.netkaiyou(Something like https://docs.rs/tee_readwrite/0.2.0/tee_readwrite/struct.TeeWriter.html could be of interest)06:10:24
@kaiyou:tedomum.netkaiyou* ~(Something like https://docs.rs/tee_readwrite/0.2.0/tee_readwrite/struct.TeeWriter.html could be of interest)~ nevermind, it is not `Seek`, sorry.06:15:50
@kaiyou:tedomum.netkaiyou* ~~(Something like https://docs.rs/tee_readwrite/0.2.0/tee_readwrite/struct.TeeWriter.html could be of interest)~~ nevermind, it is not Seek, sorry.06:16:03
@kaiyou:tedomum.netkaiyou* Edit : nevermind, I was suggesting a crate, but it does not provide Seek, sorry.06:18:19
@jkorinth:matrix.orgjkorinthMakes sense, most formats are optimized for reading, where checksum first then allows a single forward pass. But it might be nice to add a “backward patch” mechanism during write. Think it’s pretty common.17:00:51
15 May 2024
@snover:matrix.orgsnover jkorinth: sure, it is one of those things that have been thought about. It comes up pretty regularly with e.g. file offsets. I thought there was a ticket about it, but I can’t seem to find it right now :-) 05:24:01
@kaiyou:tedomum.netkaiyou

I am preparing a PR to allow SeekFrom specifiers in take_read_seek, this feels quite similar: it's mostly about having Seekable stream helpers where non Seek versions exist in std or popular crates.

Would a separate helper be accepted or would you prefer a note and example in the docs?

06:41:55
@snover:matrix.orgsnover kaiyou: I am not sure I understand. Can you say more? 20:12:02
16 May 2024
@kaiyou:tedomum.netkaiyou

jkorinth use case could be solved in a less verbose way with a Write wrapper that also provides read access afterwards instead of reimplementing the trait, or maybe a helper function or directive that quickly implement a Write+Seek wrapper with a write-tee.

I have a feeling this kind of helper exists already outside binrw, but rarely provide Seek. This seems to be the reason why binrw provides take_read_seek for instance.

My question was: would such a helper as I just described be welcome as part of binrw itself, or should we provide it in a separate crate and add documentation notes in binrw, much like the documentation about parsing bit fields?

11:54:36
@snover:matrix.orgsnover kaiyou: was that supposed to be directed toward me? TakeSeek is because std::io::Take implements only Read trait so isn’t compatible with binrw. If std::io::Take was also impl <T: Seek> Seek for Take<T> then TakeSeek would not exist. binrw provides NoSeek already for compatibility with non-seekable streams that guarantee to parse successfully. i still don’t know what you are talking about when you refer to adding SeekFrom specifiers to take_read_seek (I assume you mean take_seek)? 21:31:08
17 May 2024
@kaiyou:tedomum.netkaiyou

Agreed about TakeSeek, sorry about my poor writing.

As to SeekFrom, I am referring to std::io::Seek::seek(), which takes a SeekFrom. Binrw's TakeSeek could benefit from the object being Seek to allow restricting the stream to a relative position defined by a SeekFrom. I find it particularly useful to restrict the stream relative to the end for instance, e.g. to remove a fixed-size trailer.

This would require additional calls to seek(), so probably in a separate method.

05:04:47
@kaiyou:tedomum.netkaiyou* Agreed about TakeSeek, sorry about my poor writing. As to SeekFrom, I am referring to std::io::Seek::seek(), which takes a SeekFrom. Binrw's TakeSeek could benefit from the object being Seek to allow restricting the stream to a relative position defined by a SeekFrom. I find it particularly useful to restrict the stream relative to the end for instance, e.g. to remove a fixed-size trailer. This would require additional calls to seek(), so probably in a separate helper. 05:36:18
@kaiyou:tedomum.netkaiyou* Agreed about TakeSeek, sorry about my poor writing. As to SeekFrom, I am referring to `std::io::Seek::seek()`, which takes a `SeekFrom`. Binrw's `TakeSeek` could benefit from the object being Seek to allow restricting the stream to a relative position defined by a `SeekFrom`. I find it particularly useful to restrict the stream relative to the end for instance, e.g. to remove a fixed-size trailer. This would require additional calls to `seek()`, so probably in a separate helper.05:37:04
6 Jun 2024
@vladv:mozilla.orgvladv joined the room.17:24:37

There are no newer messages yet.


Back to Room ListRoom Version: 6