!OqDVTrmPstKzivLwZW:gitter.im

python-trio/general

751 Members
Discussion of Trio, a friendly Python library for async concurrency and I/O17 Servers

Load older messages


SenderMessageTime
5 Dec 2023
@vin-6241f55d6da0373984934e30:gitter.imvin (Vineet Kumar) oh heh the code I pasted never actually sets result 😳, but you get the idea. I also just found https://trio.readthedocs.io/en/latest/reference-core.html#custom-supervisors which has a nearly-identical example 22:07:12
@richardsheridan:matrix.orgrichardsheridanhm there might be a bug if those channels are buffered. someone could synchronously push items into two different channels, and two winners would wake up before the cancellation hits!22:10:13
@richardsheridan:matrix.orgrichardsheridanI mean, it's just one more reason unbuffered channels are better i guess22:10:34
@vin-6241f55d6da0373984934e30:gitter.imvin (Vineet Kumar)Ah yeah that's good to think about. In my case they were unbuffered.22:10:56
@vin-6241f55d6da0373984934e30:gitter.imvin (Vineet Kumar) looks like trio-util's wait_any would do what I want 22:19:52
@vin-6241f55d6da0373984934e30:gitter.imvin (Vineet Kumar)partially, anyway. I'd still have to define the function that reads from a channel and sets a nonlocal result, and loop over the channels creating partials. Not sure if the small savings are worth adding the dependency.22:24:12
@vin-6241f55d6da0373984934e30:gitter.imvin (Vineet Kumar)Yeah writing my own generic function for this feels better. Updated that gist with that plus more context on how the channels are created/filled: https://gist.github.com/vin/bd77191a7394e67f08e128e422e9999e 22:53:08
@vin-6241f55d6da0373984934e30:gitter.imvin (Vineet Kumar)Any comments/critiques appreciated!22:55:19
6 Dec 2023
@jakkdl-56365bc516b6c7089cb91462:gitter.imjakkdl (John Litborn)pytest does not seem overly excited about adding an extension [similar] to pytest.raises in a timely manner, so I'll go with adding it to trio.testing and then possibly poke pytest devs later to see if they like it once they can see it used in the wild. I think I'll go with my second version, https://github.com/pytest-dev/pytest/pull/11671, as that relies less on pytest internals. Only need to rely on pytest.ExceptionInfo, and that's a public export . My currently open PR for strict_exception_groups=True uses a copy of my first version, https://github.com/pytest-dev/pytest/pull/11656 in a pretty hacky way14:49:45
@jakkdl-56365bc516b6c7089cb91462:gitter.imjakkdl (John Litborn)so I'll write a PR that adds RaisesGroup + rewrites some current tests that uses it, and then once merged update #2886 so the diff once again gets a bit more manageable14:51:10
@jakkdl-56365bc516b6c7089cb91462:gitter.imjakkdl (John Litborn)https://github.com/python-trio/trio/pull/288614:51:18
@mikenerone:matrix.orgmikeneroneRedacted or Malformed Event15:55:32
@mikenerone:matrix.orgmikenerone * vin (Vineet Kumar): If it would be beneficial to be able to be more responsive to incoming messages, instead of the hardcoded ten-second sleep between read cycles, you could create a new_message event (as an instance attribute) and wait on that. _receive_messages() would then set the event when it receives a message. Note: Trio events aren't resettable, so after the wait you'd want to replace the object with a new one. 15:55:52
@mikenerone:matrix.orgmikenerone * vin (Vineet Kumar): If it would be beneficial to be able to be more responsive to incoming messages, instead of the hardcoded ten-second sleep between read cycles, you could create a new_message event (as an instance attribute) and wait on that. _receive_messages() would then set the event when it receives a message. Note: Trio events aren't resettable, so after the wait you'd want to replace the Event object with a new one. 15:56:08
@mikenerone:matrix.orgmikeneroneRedacted or Malformed Event16:08:30
@jakkdl-56365bc516b6c7089cb91462:gitter.imjakkdl (John Litborn)It's alive~ https://github.com/python-trio/trio/pull/289817:08:55
@goodboy:matrix.orglord_fomo
In reply to @vin-6241f55d6da0373984934e30:gitter.im
I started playing with it today and found myself having to write things that felt like they should already be somewhere. For example, "read from any of a set of channels, first-one-wins". It's straightforward to write, but I also feel like I shouldn't be writing it, like it should already be in a utility library. I'm not specifically asking for pointers to libraries either, just sharing some feedback and wondering if this is a common early-trio-user sentiment
food for thought: https://250bpm.com/blog:72/
18:38:44
@goodboy:matrix.orglord_fomo a soln to this i've found pretty simple is just push msgs onto the channel and then on the reader side do a match msg: and you avoid needing multiple channels 18:39:57
@goodboy:matrix.orglord_fomo
In reply to @vin-6241f55d6da0373984934e30:gitter.im
My first draft ignored the qos aspect: a worker for each queue, each writing to the same output channel, and a single method that would read off of that channel, and that worked as designed
so this orig approach i suppose
18:41:08
@goodboy:matrix.orglord_fomoyeah i think if i was doing this i'd just use one channel and key msgs with a prio and then sort on every new received msg an deliver the prio sorted msg set on every read iteration?18:50:15
@goodboy:matrix.orglord_fomo if looking for "critique" i don't think i've ever seen this nor would recommend it 😂 https://gist.github.com/vin/bd77191a7394e67f08e128e422e9999e#file-multi_queue-py-L9 18:51:21
@goodboy:matrix.orglord_fomoi also don't think making this method async and use await chan.send() is of any benefit if you don't limit the channel size18:53:21
@goodboy:matrix.orglord_fomohttps://gist.github.com/vin/bd77191a7394e67f08e128e422e9999e#file-multi_queue-py-L4818:53:21
@goodboy:matrix.orglord_fomothat will basically never block, so there's not really a point in making it async, especially since your read call is to seemingly sync code18:53:50
@goodboy:matrix.orglord_fomo actually given that the msg source isn't async i don't think it benefits you much at all to not just have a hot loop in another thread to get the ._sqs msgs into trio land? 18:56:33
@goodboy:matrix.orglord_fomoalso, wouldn't you potentially get hangs depending on which _sqs task returns first? if you have some lower prio source enter and block before some higher one, the loop should be halted on that call until the next low prio msg arrives no?18:59:14
@goodboy:matrix.orglord_fomounless i'm missing something?18:59:21
@goodboy:matrix.orglord_fomo i think if i was writing this, i'd spawn a thread per queue_url using .to_thread and then deliver all rxed msgs onto a common mem chan with prio keys for sorting on the trio side. 19:00:23
@goodboy:matrix.orglord_fomo
19:18:51
@goodboy:matrix.orglord_fomo on another note, Lura do you have a speak-easy lib for matrix client stuff? 19:19:07

There are no newer messages yet.


Back to Room ListRoom Version: 6