Sender | Message | Time |
---|---|---|
25 Apr 2024 | ||
oremanj (Joshua Oreman) | e.g. imagine that the context manager starts up a background task and wants to allow you to implicitly communicate with it | 19:00:21 |
oremanj (Joshua Oreman) | If you do:
then it is actively unhelpful for | 19:01:25 |
2 May 2024 | ||
mikenerone | I want a try/except specifically to detect an error during entry into an async context manager. Due to syntactic nesting, I don't see how to do this with the usual
The obvious solution is to use an
I know Trio goes to some great pains to prevent misnesting of CMs. I just wanted to double check that this try/except approach, which is similarly not possible with the normal nesting syntax, doesn't similarly violate any rules. | 14:01:09 |
agronholm (Alex Grönholm) | I don't see a problem with this | 14:03:56 |
mikenerone | Cool. I didn't think so. Thanks! | 14:06:34 |
TeamSpen210 (Spencer Brown) | That would be fine, since if the try succeeds, you entered the CM and then the exit stack will ensure you exit in the right order. If it fails, you didn't enter the CM at all. | 21:58:39 |
3 May 2024 | ||
arthur-tacca | Redacted or Malformed Event | 12:59:26 |
arthur-tacca | It's equivalent to this scope-respecting snippet (which you could equally use instead) so it doesn't violate any nesting rules. The reason it's equivalent boils down to what @TeamSpen210 (Spencer Brown) said: if the call to my_cm() or cm.__enter__() fails then .__exit__() does not get called regardless of whether using async with directly or AsyncExitStack like your snippet. | 12:59:55 |
arthur-tacca | In reply to@mikenerone:matrix.orgIt's equivalent to this scope-respecting snippet (which you could equally use instead) so it doesn't violate any nesting rules. The reason it's equivalent boils down to what @TeamSpen210 (Spencer Brown) said: if the call to my_cm() or cm.__enter__() fails then .__exit__() does not get called regardless of whether using async with directly or AsyncExitStack like your snippet. | 13:00:40 |
arthur-tacca | In reply to@arthur-tacca:matrix.orgActually the above is not true anyway if you want to suppress the error - with the original snippet, if you don't re-raise the exception (or any other exception or return) then body of the context manager will still execute, but just with the context manager not active | 13:02:46 |
mikenerone | Ah, yes. In my case, I don't want to suppress the error, so the effect is the same. | 14:35:03 |
mikenerone | Am I the only one that feels like the inability to syntactically wrap the entry of a CM with a try/except (and by "syntactically" I mean with syntax only, no separate state variable) is a bit of a design oversight? That's not to say I have a better idea off the top of my head. | 15:18:40 |
mikenerone | At least it's easy to work around (either of the above approaches, or by wrapping in a custom CM of your own (and not even that if the target CM is under your control)), but it feels like it should be able to be done syntactically. | 15:34:52 |
Daniel Alp joined the room. | 18:11:45 | |
arthur-tacca | In reply to@mikenerone:matrix.orgThe same is true of stack variable constructors in C++. You can work around that by using the heap and std:: unique_ptr , which is obviously very different from AsyncExitStack but acts a bit like it in this context. (I'm not disagreeing, it's just a point of composition.) | 20:00:33 |
arthur-tacca | The same is true of stack variable constructors in C++. You can work around that by using the heap and std::unique_ptr , which is obviously very different from AsyncExitStack but acts a bit like it in this context. (I'm not disagreeing, it's just a point of comparison) | 20:02:35 |
4 May 2024 | ||
Peter Sutton | Is there a good pattern for ensuring progress. I've got a collections of tasks that go async a lot then loop. In the middle of the loop, the task does a very expensive task (ffmpeg video encode). I'd like tasks to take turns at this point, round robin style, so that the tasks are all outputting files at roughly the same rate. | 20:07:23 |
Peter Sutton | * Is there a good pattern for ensuring progress. I've got a collection of tasks. Each task goes async a lot then loops. In the middle of the loop, the task does a very expensive task (ffmpeg video encode). I'd like tasks to take turns at this point, round robin style, so that the tasks are all outputting files at roughly the same rate. | 20:08:10 |
agronholm (Alex Grönholm) | how are you launching the ffmpeg thing? if you do it with async directives, as you should be, they would all run concurrently | 20:10:01 |
Peter Sutton | ffmpeg saturates all CPUs so I can only run one at once (I'm using a CapacityLimiter). | 20:14:46 |
agronholm (Alex Grönholm) | so what is it that you're aiming for here? that it would suspend one of the subprocesses somehow, in favor of another? | 20:16:01 |
Peter Sutton | Only one task can run ffmpeg at once. Each task should only be allowed to encode again once every other task waiting to encoding has encoded. | 20:17:57 |
agronholm (Alex Grönholm) | oh, okay, so you don't want to suspend the subprocesses, but instead let another task launch a new subprocess | 20:18:39 |
agronholm (Alex Grönholm) | that's perfectly doable | 20:18:56 |
agronholm (Alex Grönholm) | you need to run these tasks concurrently, and use a Lock | 20:19:26 |
agronholm (Alex Grönholm) | each task acquires a lock before running the subprocess, and then releases it | 20:19:43 |
Peter Sutton | Do locks act like FIFO queuea? I.e., the lock with be given to tasks in the order they attempt to acquire it. | 20:21:35 |
Peter Sutton | I guess they are: https://trio.readthedocs.io/en/stable/reference-core.html#fairness | 20:28:21 |
agronholm (Alex Grönholm) | on trio, you'll want this: https://trio.readthedocs.io/en/stable/reference-core.html#trio.StrictFIFOLock | 20:28:58 |
Peter Sutton | oh yer. Thanks that's exactly what I need. | 20:33:16 |