!homDUJhmxHaZNgyLVA:matrix.org

binrw

168 Members
Official Matrix channel for the binread & binrw Rust libraries. Bridged to Discord. https://github.com/jam1garner/binrw/12 Servers

Load older messages


SenderMessageTime
25 Jul 2024
@kaiyou:tedomum.netkaiyou(Sorry if this is unclear, maybe I also misunderstood your question :x)19:44:38
@kaiyou:tedomum.netkaiyou I am affraid bw(ignore) requires br(temp) though, I am not sure how you may use the id outside the parsing logic. 19:45:42
@snover:matrix.orgsnover got it. i think the best thing right now that you can do is to write a member function on the enum that returns the correct id for each enum variant. binrw does not have any thing right now that does this automatically. there are some other crates that you could use to derive. either case ends up requiring to write the magic value twice, which is unfortunate. you can also use import and pre_assert to pick an enum variant instead, but i do not think that any of this will avoid the current requirement to duplicate something—either you will have one unit enum with the magic numbers and then a second data enum with the same variants, using pre_assert, or you will have two things with the same magic numbers, using magic. 19:49:16
@krensito:matrix.orgckrenslehner
In reply to @kaiyou:tedomum.net

Here is an excerpt from real code (not published, sorry) :

    #[br(temp, restore_position)]
    #[bw(ignore)]
    op: Operation,
    data: Payload

The field is simply not written. When reading, I read the opcode, makes decisions based on it, then parse a Payload which is an enum using magics. When writing, it simply writes the enum and prepends the magic bytes.

Thanks I get this and I can also use this as a workaround. The only thing which bothers me is, that I need to use a kind of half init struct when I want to write the data. When I create a variable of this structure I need to pass a value to the enum and the id (even though the id kinda redundant) of course I can just use private fields and new and just live with it
19:49:27
@snover:matrix.orgsnover * got it. i think the best thing right now that you can do is to write a member function on the enum that returns the correct id for each enum variant. binrw does not have any thing right now that does this automatically. there are some other crates that you could use to derive. either case ends up requiring specifying variants twice, which is unfortunate. you can also use import and pre_assert to pick an enum variant instead, but i do not think that any of this will avoid the current requirement to duplicate something—either you will have one unit enum with the magic numbers and then a second data enum with the same variants, using pre_assert, or you will have two things with the same magic numbers, using magic. 19:49:35
@krensito:matrix.orgckrenslehner
In reply to @snover:matrix.org
got it. i think the best thing right now that you can do is to write a member function on the enum that returns the correct id for each enum variant. binrw does not have any thing right now that does this automatically. there are some other crates that you could use to derive. either case ends up requiring specifying variants twice, which is unfortunate. you can also use import and pre_assert to pick an enum variant instead, but i do not think that any of this will avoid the current requirement to duplicate something—either you will have one unit enum with the magic numbers and then a second data enum with the same variants, using pre_assert, or you will have two things with the same magic numbers, using magic.
Yes that's exactly what I am struggling with! In my original code I was using pre_assert. But then I realized that there is no round-trip anymore. Now I tried to smash everything together to avoid code duplication. I have the feeling that I might be able to achieve it using a custom stream but 3h+ of trying did not yet bring a real success. Basically I wanted to introduce a kind of trait bound which requires T: meta::WriteMagic and then I wanted to use ::Magic to get the actual value. But somehow I just don't succeed. Anyway it was a good learning experience so far :-D
19:54:58
@krensito:matrix.orgckrenslehner* Yes that's exactly what I am struggling with! In my original code I was using pre_assert. But then I realized that there is no round-trip anymore. Now I tried to smash everything together to avoid code duplication. I have the feeling that I might be able to achieve it using a custom stream but 3h+ of trying did not yet bring a real success. Basically I wanted to introduce a kind of trait bound which requires T: meta::WriteMagic and then I wanted to use `T as meta::WriteMagic::Magic` to get the actual value. But somehow I just don't succeed. Anyway it was a good learning experience so far :-D19:56:03
@krensito:matrix.orgckrenslehner* Yes that's exactly what I am struggling with! In my original code I was using pre_assert. But then I realized that there is no round-trip anymore. Now I tried to smash everything together to avoid code duplication. I have the feeling that I might be able to achieve it using a custom stream but 3h+ of trying did not yet bring a real success. Basically I wanted to introduce a kind of trait bound which requires T: meta::WriteMagic and then I wanted to use <T as meta::WriteMagic::Magic> to get the actual value. But somehow I just don't succeed. Anyway it was a good learning experience so far :-D19:56:31
@snover:matrix.orgsnoveri suppose you could write a macro to try to avoid duplicating the values19:56:41
@krensito:matrix.orgckrenslehner* Yes that's exactly what I am struggling with! In my original code I was using pre_assert. But then I realized that there is no round-trip anymore. Now I tried to smash everything together to avoid code duplication. I have the feeling that I might be able to achieve it using a custom stream but 3h+ of trying did not yet bring a real success. Basically I wanted to introduce a kind of trait bound which requires T: meta::WriteMagic and then I wanted to use `<T as meta::WriteMagic>::Magic` to get the actual value. But somehow I just don't succeed. Anyway it was a good learning experience so far :-D19:57:31
@snover:matrix.orgsnoveri have been writing c++ lately so i will not even attempt to provide a starter since it would be utterly broken but since it is just variant names and literal values it should be a mercifully easy macro_rules application20:02:31
@krensito:matrix.orgckrenslehnerGood point thanks. I will give this a try :-)20:06:00
26 Jul 2024
@krensito:matrix.orgckrenslehner
In reply to @snover:matrix.org
i suppose you could write a macro to try to avoid duplicating the values

https://github.com/Krensi/binrw-datapoints

Maybe someone could benefit from this example of how I control writing and reading only via the enum. I also played around with generics to support different types for the encoding format
size - id - value

value is the only item of actual interest which is the enum

with your tipp to use macro_rules it was actually very easy to get the ID - thanks again :-)

19:30:55
3 Aug 2024
@nefix:matrix.orgNéfix Estrada joined the room.09:41:20
@nefix:matrix.orgNéfix Estrada

Hello! Is it possible to use both a calc directive and an assert? This is complaining that *u_ack_vector_size doesn't exist

    /// A 16-bit unsigned value that specifies the size of the AckVector field
    /// in bytes. The maximum size of the ACK Vector is 2048 bytes.
    #[br(assert(u_ack_vector_size >= 2048, "u_ack_vector_size maximum size is 2048"))]
    #[bw(assert(*u_ack_vector_size >= 2048, "u_ack_vector_size maximum size is 2048"))]
    #[bw(calc = (ack_vector.len()) as u16)]
    pub u_ack_vector_size: u16,

Thanks! :)

09:43:14
@nefix:matrix.orgNéfix Estrada

And another question. I'm parsing packets. All of them have a header, which specifies the packet type:

/// UDP_PACKET_HEADER
/// This structure describes a UDP packet.
#[binrw]
#[brw(little)]
pub struct UdpPacketHeader {
    /// The packet type information, which can be one of the enumerations
    /// specified in 2.2.5.4.1.
    pub pkt_id: UdpPktType,

    /// Specifies the packet length excluding the length of UDP_PACKET_HEADER
    pub pkt_len: u16,
}

The UdpPktType is an enum:

#[binrw]
#[brw(repr(u16))]
pub enum UdpPktType {
    /// PKT_TYPE_CONNECT_REQ
    /// This constant is used to represent the CONNECT packet type sent by
    /// the client during the Connection Setup Phase (section 1.3.1.1.1).
    PktTypeConnectReq = 1,
...

And then, there's each packet, which has its own structure.

Is it possible to make a "generalized parser"? E.g. have a Packet union, which reads the header type and, depending on the packet type read one packet or another. Also, I'd like to have the header "embeded" in each packet, since there are some that depend on it for lenghts:

#[binrw]
#[brw(little)]
pub struct DataPkt {
    // HEADER HERE TO BE ABLE TO `count` the length of data 
    /// An array of BYTE containing the RDP UDP data.
    pub data: Vec<u8>,
}

I'd be nice to be able to simply write a packet and automatically get the header, so:

  1. Packet type is set
  2. Length is also set
09:50:11
@snover:matrix.orgsnover
In reply to @nefix:matrix.org

Hello! Is it possible to use both a calc directive and an assert? This is complaining that *u_ack_vector_size doesn't exist

    /// A 16-bit unsigned value that specifies the size of the AckVector field
    /// in bytes. The maximum size of the ACK Vector is 2048 bytes.
    #[br(assert(u_ack_vector_size >= 2048, "u_ack_vector_size maximum size is 2048"))]
    #[bw(assert(*u_ack_vector_size >= 2048, "u_ack_vector_size maximum size is 2048"))]
    #[bw(calc = (ack_vector.len()) as u16)]
    pub u_ack_vector_size: u16,

Thanks! :)

When you use calc there is no actual field, so assert on ack_vector.len() instead. From the docs:

Since the field is treated as a temporary variable instead of an actual field, when deriving BinRead, the field should also be annotated with #[br(temp)].

20:43:16
@snover:matrix.orgsnover
In reply to @nefix:matrix.org

And another question. I'm parsing packets. All of them have a header, which specifies the packet type:

/// UDP_PACKET_HEADER
/// This structure describes a UDP packet.
#[binrw]
#[brw(little)]
pub struct UdpPacketHeader {
    /// The packet type information, which can be one of the enumerations
    /// specified in 2.2.5.4.1.
    pub pkt_id: UdpPktType,

    /// Specifies the packet length excluding the length of UDP_PACKET_HEADER
    pub pkt_len: u16,
}

The UdpPktType is an enum:

#[binrw]
#[brw(repr(u16))]
pub enum UdpPktType {
    /// PKT_TYPE_CONNECT_REQ
    /// This constant is used to represent the CONNECT packet type sent by
    /// the client during the Connection Setup Phase (section 1.3.1.1.1).
    PktTypeConnectReq = 1,
...

And then, there's each packet, which has its own structure.

Is it possible to make a "generalized parser"? E.g. have a Packet union, which reads the header type and, depending on the packet type read one packet or another. Also, I'd like to have the header "embeded" in each packet, since there are some that depend on it for lenghts:

#[binrw]
#[brw(little)]
pub struct DataPkt {
    // HEADER HERE TO BE ABLE TO `count` the length of data 
    /// An array of BYTE containing the RDP UDP data.
    pub data: Vec<u8>,
}

I'd be nice to be able to simply write a packet and automatically get the header, so:

  1. Packet type is set
  2. Length is also set
Use a data enum with magic attribute.
20:43:45
@snover:matrix.orgsnover
In reply to @nefix:matrix.org

And another question. I'm parsing packets. All of them have a header, which specifies the packet type:

/// UDP_PACKET_HEADER
/// This structure describes a UDP packet.
#[binrw]
#[brw(little)]
pub struct UdpPacketHeader {
    /// The packet type information, which can be one of the enumerations
    /// specified in 2.2.5.4.1.
    pub pkt_id: UdpPktType,

    /// Specifies the packet length excluding the length of UDP_PACKET_HEADER
    pub pkt_len: u16,
}

The UdpPktType is an enum:

#[binrw]
#[brw(repr(u16))]
pub enum UdpPktType {
    /// PKT_TYPE_CONNECT_REQ
    /// This constant is used to represent the CONNECT packet type sent by
    /// the client during the Connection Setup Phase (section 1.3.1.1.1).
    PktTypeConnectReq = 1,
...

And then, there's each packet, which has its own structure.

Is it possible to make a "generalized parser"? E.g. have a Packet union, which reads the header type and, depending on the packet type read one packet or another. Also, I'd like to have the header "embeded" in each packet, since there are some that depend on it for lenghts:

#[binrw]
#[brw(little)]
pub struct DataPkt {
    // HEADER HERE TO BE ABLE TO `count` the length of data 
    /// An array of BYTE containing the RDP UDP data.
    pub data: Vec<u8>,
}

I'd be nice to be able to simply write a packet and automatically get the header, so:

  1. Packet type is set
  2. Length is also set
* Use a data enum with magic attribute on the variants.
20:43:57
@nefix:matrix.orgNéfix Estrada
In reply to @snover:matrix.org

When you use calc there is no actual field, so assert on ack_vector.len() instead. From the docs:

Since the field is treated as a temporary variable instead of an actual field, when deriving BinRead, the field should also be annotated with #[br(temp)].

I see, thanks!
20:44:31
@nefix:matrix.orgNéfix Estrada
In reply to @snover:matrix.org
Use a data enum with magic attribute on the variants.
Could you provide an example? Thanks! :)
20:45:53
@snover:matrix.orgsnover
#[binrw]
enum DataPkt {
  #[brw(magic(1u16))]
  UdpPktType { len : u16 }, 
  #[brw(magic(2u16)]
  PktType2 { len: u16 },
  ...
}
20:47:46
@nefix:matrix.orgNéfix EstradaI see, I think I've got it. Thanks! :)20:49:21
@nefix:matrix.orgNéfix Estrada
In reply to @snover:matrix.org
#[binrw]
enum DataPkt {
  #[brw(magic(1u16))]
  UdpPktType { len : u16 }, 
  #[brw(magic(2u16)]
  PktType2 { len: u16 },
  ...
}
And how can I make the len part of the enum? Or do I need to specify it in all the Packets? Thanks! :)
21:13:10
@nefix:matrix.orgNéfix Estrada

Right now it's looking as follows:

#[binrw]
#[brw(little)]
pub enum Packet {
    /// PKT_TYPE_HANDSHAKE_REQUEST
    /// This constant represents that the packet type is handshake request.
    #[brw(magic(0x1u16))]
    HandshakeRequest(HandshakeRequestPacket),

    /// PKT_TYPE_HANDSHAKE_RESPONSE
    /// This constant represents that the packet type is handshake
    /// response.
    #[brw(magic(0x2u16))]
    HandshakeResponse(HandshakeResponsePacket),

    /// PKT_TYPE_EXTENDED_AUTH_MSG
    /// This constant represents that the packet type is an extended
    /// authentication message.
    #[brw(magic(0x3u16))]
    ExtendedAuthMsg(ExtendedAuthPacket),
}
21:13:46
@snover:matrix.orgsnoverto use data enum like this would specify in all of them. you could use composition of inner struct if you want. a common alternative approach for this case is to use a wrapper outer struct and then inner enum and pre_assert on the id instead.21:14:58
@snover:matrix.orgsnoverpub struct DataPkt { id: u16, len: u16, #[br(args(id, len))] data: Packet } #[br(import(id: u16, len: u16)] enum Packet { #[br(pre_assert(id == 1))] HandshakeRequest(#[br(args(len))] HandshakeRequestPacket), … }21:17:37
@snover:matrix.orgsnoverwhoops. formatting.21:17:46
@snover:matrix.orgsnover *
pub struct DataPkt { 
  id: u16,
   len: u16, #\[br(args(id, len))\] data: Packet } #\[br(import(id: u16, len: u16)\] enum Packet { #\[br(pre\_assert(id == 1))\] HandshakeRequest(#\[br(args(len))\] HandshakeRequestPacket), … }
21:17:55
@snover:matrix.orgsnover *
pub struct DataPkt { 
  id: u16,
  len: u16,
  #[br(args(id, len))] data: Packet
} 


#[br(import(id: u16, len: u16)]
enum Packet { 
  #[br(pre_assert(id == 1))] HandshakeRequest(#[br(args(len))] HandshakeRequestPacket), 
  …
 }
21:18:36

Show newer messages


Back to Room ListRoom Version: 6