!MNkZeDktHEshWITStt:matrix.org

OpenLMIS-Help

28 Members
1 Servers

Load older messages


Timestamp Message
3 Nov 2019
04:23:29@_slack_openlmis_UNSTNRVU7:matrix.orgYamato O. but I'm working backwards from proofOfDelivery
05:03:08@_slack_openlmis_UNSTNRVU7:matrix.orgYamato O. is there any way at all to query more than one shipment at a time?
4 Nov 2019
07:15:06@_slack_openlmis_U03QATSPZ:matrix.orgjoshzamor Hi Yamato O., it looks like there isn’t currently: https://github.com/OpenLMIS/openlmis-fulfillment/blob/master/src/main/java/org/openlmis/fulfillment/web/shipment/ShipmentController.java#L178
07:15:31@_slack_openlmis_U03QATSPZ:matrix.orgjoshzamor I’m a bit surprised by that as well.
07:18:01@_slack_openlmis_U03QATSPZ:matrix.orgjoshzamor Yamato O.: I’d recommend you follow the API requests that the web-ui makes. I know that’s a lot and less than ideal, however it’s the cleanest path forward given where the API is at today. That’ll be more than acceptable for the mobile-practicum, even if it’s not the performance we’ll need for the field, it’ll give us a starting point which we can iterate on later to improve the performance (improve the API for this use-case).
08:02:02@_slack_openlmis_UNSTNRVU7:matrix.orgYamato O. In the end I resolved my problem by reducing the number of API calls down to n+2
08:02:14@_slack_openlmis_UNSTNRVU7:matrix.orgYamato O. with n being the number of proof of deliveries received
08:03:30@_slack_openlmis_UNSTNRVU7:matrix.orgYamato O. first, i did api/proofsofdelivery to get all the pods, and then a call for every unique orderable found inside every pod at once, since api/orderable allows for repeat parameters
08:04:15@_slack_openlmis_UNSTNRVU7:matrix.orgYamato O. from that point i do api/proofsofdelivery/podid?expand=shipment.order for each pod, to get the order and shipment associated with the pod at once
08:04:25@_slack_openlmis_UNSTNRVU7:matrix.orgYamato O. (edited) ... shipment at ... => ... shipment associated with the pod at ...
08:07:16@_slack_openlmis_UNSTNRVU7:matrix.orgYamato O. if api/shipment had repeat parameters like api/orderable did, or if api/proofsofdelivery had the ability to expand for each pod in it, i could reduce the number of calls further, but performance seems acceptable right now based on the demo server
08:09:50@_slack_openlmis_UNSTNRVU7:matrix.orgYamato O. my concern is that api/orderables has a limit of how long the request params can be, so if it has to get more orderable ids than the max character limit, it will not work. Is there a way to make a request for specific orderables inside of a json body instead?
5 Nov 2019
10:51:39@_slack_openlmis_U03QATSPZ:matrix.orgjoshzamor Yamato O.: You can use /api/orderables/search, _however_ it looks like it requires the orderable’s version number. Do you have that?
10:52:51@_slack_openlmis_U03QATSPZ:matrix.orgjoshzamor This is an example of this request: https://github.com/OpenLMIS/openlmis-referencedata/blob/master/performance/tests/orderables.yml#L82
10:53:46@_slack_openlmis_U03QATSPZ:matrix.orgjoshzamor Or as: ``` { “identities”: [ { “id”: “7f7b83db-580d-4269-88d2-7f9c80a591a9”, “versionNumber”: “1” } ] }
10:53:50@_slack_openlmis_U03QATSPZ:matrix.orgjoshzamor (edited) ... { “identities”: [ { “id”: “7f7b83db-580d-4269-88d2-7f9c80a591a9”, “versionNumber”: “1” } ] } => ... { "identities": [ { "id": "7f7b83db-580d-4269-88d2-7f9c80a591a9", "versionNumber": "1" } ] } ```
10:54:56@_slack_openlmis_U03QATSPZ:matrix.orgjoshzamor Requiring the versionNumber feels like a limitation. sbrudzinski do you have any thoughts or know whom on the core team could help us understand this better?
10:59:21@_slack_openlmis_U195RKK4G:matrix.orgsbrudzinski This sounds right, and yeah - not requiring the versionNumber would be a good step
7 Nov 2019
13:45:38@_slack_openlmis_UBSGVMUD8:matrix.orgPaulina Buzderewicz joined the room.
13:47:19@_slack_openlmis_UBSGVMUD8:matrix.orgPaulina Buzderewicz Hi. I'm currently working on improving performance of requisition update. This is strongly related to search queries for ftaps which take about 50s for Essential Meds program. Now the query for ftap search by any params contains join statements on programs, latest orderables, program orderables and facility types. These joins are made just to check if program orderable is active for latest orderable related to a proper ftap. My question is, do we really need these joins? Especially in a situation when we search ftaps by specific version identities. Removing them makes search faster by 40 seconds (for EM program). joshzamor, sbrudzinski, llewczynski
13:48:46@_slack_openlmis_UBSGVMUD8:matrix.orgPaulina Buzderewicz (edited) ... EM program). => ... EM program). llewczynski
14:11:55@_slack_openlmis_U03QATSPZ:matrix.orgjoshzamor Uh - that’s horrible join performance. How is the join working? Does it have an index? What’s EXPLAIN tell us?
14:24:04@_slack_openlmis_UBSGVMUD8:matrix.orgPaulina Buzderewicz This is the output for 3 identities:
 Aggregate  (cost=646.54..646.55 rows=1 width=8)
   ->  Nested Loop  (cost=392.69..646.53 rows=1 width=0)
         Join Filter: (ftap.facilitytypeid = ft.id)
         ->  Nested Loop  (cost=392.69..645.51 rows=1 width=16)
               Join Filter: (ftap.programid = p.id)
               ->  Nested Loop  (cost=392.55..645.34 rows=1 width=48)
                     Join Filter: (ftap.programid = po.programid)
                     ->  Hash Join  (cost=392.27..644.12 rows=3 width=72)
                           Hash Cond: (orderables.id = ftap.orderableid)
                           ->  HashAggregate  (cost=368.11..468.84 rows=10073 width=24)
                                 Group Key: orderables.id
                                 ->  Seq Scan on orderables  (cost=0.00..317.74 rows=10074 width=24)
                           ->  Hash  (cost=24.12..24.12 rows=3 width=48)
                                 ->  Bitmap Heap Scan on facility_type_approved_products ftap  (cost=12.89..24.12 rows=3 width=48)
                                       Recheck Cond: (((id = '3412f30c-e47b-4e45-ae24-226b52c333ac'::uuid) AND (versionnumber = 1)) OR ((id = '75030425-817d-448f-8493-3b5fd4231f3c'::uuid) AND (versionnumber = 1)) OR ((id = 'c859b805-4080-4a61-8c39-7206764ef2ca'::uuid) AND (versionnumber = 1)))
                                       ->  BitmapOr  (cost=12.89..12.89 rows=3 width=0)
                                             ->  Bitmap Index Scan on facility_type_approved_products_pkey  (cost=0.00..4.30 rows=1 width=0)
                                                   Index Cond: ((id = '3412f30c-e47b-4e45-ae24-226b52c333ac'::uuid) AND (versionnumber = 1))
                                             ->  Bitmap Index Scan on facility_type_approved_products_pkey  (cost=0.00..4.30 rows=1 width=0)
                                                   Index Cond: ((id = '75030425-817d-448f-8493-3b5fd4231f3c'::uuid) AND (versionnumber = 1))
                                             ->  Bitmap Index Scan on facility_type_approved_products_pkey  (cost=0.00..4.30 rows=1 width=0)
                                                   Index Cond: ((id = 'c859b805-4080-4a61-8c39-7206764ef2ca'::uuid) AND (versionnumber = 1))
                     ->  Index Scan using program_orderables_orderableid_idx1 on program_orderables po  (cost=0.29..0.39 rows=1 width=40)
                           Index Cond: (orderableid = orderables.id)
                           Filter: ((active IS TRUE) AND ((max(orderables.versionnumber)) = orderableversionnumber))
               ->  Index Only Scan using programs_pkey on programs p  (cost=0.14..0.16 rows=1 width=16)
                     Index Cond: (id = po.programid)
         ->  Seq Scan on facility_types ft  (cost=0.00..1.01 rows=1 width=16)
18:08:32@_slack_openlmis_U195RKK4G:matrix.orgsbrudzinski I don't think the FTAP retrieval endpoints should take into consideration whether the ProgramOrderable for the program is active or not. It sounds to me like the FTAP endpoints should return whatever they have configured for the version that is requested and then it's up to the client of that endpoint to decide whether they need any additional validation checks or not (eg. against program orderables). They should however respect their own active flag, of course. If we were to validate against the program orderables, I don't even know how exactly we would want to approach that. Both FTAPs and Orderables are versioned, but the link from FTAP to Orderable is not. They just reference the orderable ID. The current check against the program orderable just takes the latest orderable version (see part Filter: ((active IS TRUE) AND ((max(orderables.versionnumber)) ) and performs a check against that. This is clearly wrong, as I should be able to retrieve a past version of an FTAP, e.g. for the legacy requisitions (per our redesign to drop snapshotting), no mater if the program is currently active for that requisition or not. Thoughts joshzamor Paulina Buzderewicz llewczynski?
8 Nov 2019
04:29:00@_slack_openlmis_U03QATSPZ:matrix.orgjoshzamor You’re right sbrudzinski. My first impression was of course an FTAP should factor in if the Orderable is active in the program, but you’re right that since it’s purposely un-versioned (from FTAP to Orderable), that there really shouldn’t be this expectation placed on it. We could reverse it and version the connection, and maybe someday that’s what we’ll do, but that too has trade-offs.
04:34:50@_slack_openlmis_U03QATSPZ:matrix.orgjoshzamor Lets make sure we describe this in the API documentation of FTAPs - that being active in the Program isn’t a condition that an FTAP ensures, and it’s up to the client to figure this out if it needs to. Basically we’re saying that an FTAP is an approval that the facility (type) can order that Orderable, along with a template of parameters if it does, but that it’s not a gurantee that the Orderable is currently something that may be Ordered in that Program.
04:35:25@_slack_openlmis_U03QATSPZ:matrix.orgjoshzamor This feels like a breaking semantic change - what other knock-on effects does it have?
04:37:05@_slack_openlmis_U03QATSPZ:matrix.orgjoshzamor
In reply to@_slack_openlmis_UBSGVMUD8:matrix.org
This is the output for 3 identities:
 Aggregate  (cost=646.54..646.55 rows=1 width=8)
   ->  Nested Loop  (cost=392.69..646.53 rows=1 width=0)
         Join Filter: (ftap.facilitytypeid = ft.id)
         ->  Nested Loop  (cost=392.69..645.51 rows=1 width=16)
               Join Filter: (ftap.programid = p.id)
               ->  Nested Loop  (cost=392.55..645.34 rows=1 width=48)
                     Join Filter: (ftap.programid = po.programid)
                     ->  Hash Join  (cost=392.27..644.12 rows=3 width=72)
                           Hash Cond: (orderables.id = ftap.orderableid)
                           ->  HashAggregate  (cost=368.11..468.84 rows=10073 width=24)
                                 Group Key: orderables.id
                                 ->  Seq Scan on orderables  (cost=0.00..317.74 rows=10074 width=24)
                           ->  Hash  (cost=24.12..24.12 rows=3 width=48)
                                 ->  Bitmap Heap Scan on facility_type_approved_products ftap  (cost=12.89..24.12 rows=3 width=48)
                                       Recheck Cond: (((id = '3412f30c-e47b-4e45-ae24-226b52c333ac'::uuid) AND (versionnumber = 1)) OR ((id = '75030425-817d-448f-8493-3b5fd4231f3c'::uuid) AND (versionnumber = 1)) OR ((id = 'c859b805-4080-4a61-8c39-7206764ef2ca'::uuid) AND (versionnumber = 1)))
                                       ->  BitmapOr  (cost=12.89..12.89 rows=3 width=0)
                                             ->  Bitmap Index Scan on facility_type_approved_products_pkey  (cost=0.00..4.30 rows=1 width=0)
                                                   Index Cond: ((id = '3412f30c-e47b-4e45-ae24-226b52c333ac'::uuid) AND (versionnumber = 1))
                                             ->  Bitmap Index Scan on facility_type_approved_products_pkey  (cost=0.00..4.30 rows=1 width=0)
                                                   Index Cond: ((id = '75030425-817d-448f-8493-3b5fd4231f3c'::uuid) AND (versionnumber = 1))
                                             ->  Bitmap Index Scan on facility_type_approved_products_pkey  (cost=0.00..4.30 rows=1 width=0)
                                                   Index Cond: ((id = 'c859b805-4080-4a61-8c39-7206764ef2ca'::uuid) AND (versionnumber = 1))
                     ->  Index Scan using program_orderables_orderableid_idx1 on program_orderables po  (cost=0.29..0.39 rows=1 width=40)
                           Index Cond: (orderableid = orderables.id)
                           Filter: ((active IS TRUE) AND ((max(orderables.versionnumber)) = orderableversionnumber))
               ->  Index Only Scan using programs_pkey on programs p  (cost=0.14..0.16 rows=1 width=16)
                     Index Cond: (id = po.programid)
         ->  Seq Scan on facility_types ft  (cost=0.00..1.01 rows=1 width=16)
Thanks for this Paulina Buzderewicz, it helps. It would be interesting to see this with EXPLAIN ANALYZE rather than plain EXPLAIN - that gives us a hint of actual cost and the state of the indices.
20 Nov 2019
22:58:15@_slack_openlmis_U063EBB47:matrix.orgbenleibert The cce_catalog_items table in the CCE service includes the following fields: storagetemperature (text) maxoperatingtemp (integer) minoperatingtemp (integer) Does anyone happen to know how the storagetemperature one is used? Given that it’s a text field and that it’s associated with models of CCE rather than specific ones, it can’t be intended to store live values. Values for the field on the UAT server include MINUS5, MINUS8, MINUS12, etc. which don’t make sense to me.
22:59:11@_slack_openlmis_U063EBB47:matrix.orgbenleibert (edited) ... (integer) Minoperatingtemp (integer) ... => ... (integer) minoperatingtemp (integer) ...

There are no newer messages yet.


Back to Room List