| 4 Jun 2021 |
|  Piers Wombwell joined the room. | 12:20:27 |
 Sam Sneddon [:gsnedders] | In reply to @jgraham_:matrix.org
Sam Sneddon [:gsnedders]: Well I was sure I didn't until I typed cargo init wptreport and it HALed me by complaining that the path already existed. So apparently I did write one in 2018. Anyway I've now updated it to some of the changes since then, and it passes the tests consisting of parsing some random wptreport.json files I had lying around, so https://github.com/jgraham/wptreport.rs see I think I had some vague memory of you writing this, plus trying to parse the manifest in Rust (which is inevitably harder now it's a trie) | 12:50:04 |
|  stevoooo joined the room. | 12:52:04 |
 jgraham | If you're talking about binary deps anyway I'm totally in favour of making the manifest generation a Rust library :) But I definitely don't have code for that one. | 12:59:40 |
| Sam Sneddon [:gsnedders] | nah, I'm debating doing other stuff, processing many wptreport files and wanting some sane data structure for it | 13:03:29 |
| Sam Sneddon [:gsnedders] | and without huge memory usage | 13:03:37 |
 foolip | Sam Sneddon [:gsnedders]: if you have many similar JSON files that you want to load into memory, I'm still pretty pleased with myself for the trick in https://github.com/Ecosystem-Infra/wpt-results-analysis/blob/3b8a4f726a5a8d544c15be544ed88962c8f3ed1b/lib/results.js | 13:05:30 |
| foolip | Just checked, the ~43k full runs wpt.fyi has is still only a 313MB repo. It's bigger than that if loaded into memory (no gzip) but you can definitely load thousands into memory at the same time at least. | 13:09:00 |
| jgraham | https://searchfox.org/mozilla-central/source/testing/web-platform/tests/tools/wptrunner/wptrunner/metadata.py puts some effort into having a memory-efficient representation of a large number of results | 13:10:20 |
| jgraham | (searchfox link just because it's easier than GitHib search) | 13:10:37 |
| Sam Sneddon [:gsnedders] | jgraham: even just json.load in Python takes a long time because it's really doing way too much | 13:11:30 |
| stevoooo | hello all. am having more problems with wpt's update-expectations and could do with a few pointers :-/ | 13:13:43 |
| jgraham | Well sure, I'm not claiming that some python solution is going to be super efficient compared with the platonic ideal. But the metadata.py code at least tries to store a single representation of all strings, and bitpacks the results data so the total memory usage is reasonable even when loading several hundred wptreport.json files simultaneously to update wpt results based on a m-c run | 13:13:44 |
| jgraham | (reasonable being "about 1Gb", iirc) | 13:14:17 |
| stevoooo | i'm trying to generate expectations data for our Flow browser and i'm getting an error from the script. however, if i try to generate expectation data for firefox then it works which clearly suggests that there's a problem with our integration somewhere, only i'm not really sure where to start looking | 13:14:38 |
| Sam Sneddon [:gsnedders] | update-expectations is pretty fragile and not that well tested, and has often been broken unless you use it exactly as Firefox does | 13:15:18 |
| stevoooo | the error i'm getting is ValueError: not enough values to unpack (expected 2, got 0) in File "/Users/stp/vbox-share/wpt/tools/wptrunner/wptrunner/manifestupdate.py", line 311, in build_conditional_tree properties, dependent_props = run_info_properties | 13:15:19 |
| stevoooo | we're getting an empty dictionary for run_info_properties | 13:15:50 |
| stevoooo | any ideas where the data that's supposed to be in there orginates? we've clearly got something missing in our flow.py stuff but i've no idea what | 13:16:39 |
| jgraham | stevoooo: See https://searchfox.org/mozilla-central/source/testing/web-platform/tests/tools/wptrunner/wptrunner/browsers/firefox.py#274. It's a list of properties to use for updates and then a dict of {property: [secondary property]}. So the properties are run info features that might distinguish different test results (usually for the same browser), and the secondary properties are properties that should only be used if there's still multiple different results after considering all the top-level properties | 13:20:34 |
| jgraham | it means that e.g. for Firefox we distinguish on os name before os version, even though the latter will generally be more specific than the former | 13:21:29 |
| jgraham | So it might work to just make an empty list and an empty dict, if you're running the tests in a single configuration. | 13:22:04 |
| stevoooo | ah right | 13:22:17 |
| jgraham | (these properties all have to appear as keys in the run_info dict) | 13:22:42 |
| stevoooo | thanks - i'll give that a try | 13:23:22 |
| stevoooo | ah brilliant, that's sorted it - i now have some expectation data! cheers | 13:25:58 |
| jgraham | Great! As Sam Sneddon [:gsnedders] points out this code is not exactly widely used, so it's a bit of a pain to get working | 13:29:22 |
| stevoooo | i'm sure i'll be back with further questions :) | 13:29:47 |
| stevoooo | essentially have wpt run working now... need to sort out expectation data and then automate some nightly runs and hopefully we'll be able to try and get this submitted | 13:30:45 |
| jgraham | Amazing! | 13:31:30 |
