!FsygHZEAWANEgbeCMB:matrix.org

Navitia

70 Members
Welcome here to talk about Navitia (pronounce [navi-sia]) github.com/CanalTP/navitia and related repos - You can also check groups.google.com/d/forum/navitia for navitia.io API & data & usage | doc.navitia.io for integration5 Servers

Load older messages


SenderMessageTime
9 Apr 2021
@maelito:matrix.orgmaelito
In reply to antoine_de
hum si c'est du vol d'oiseau à l'arrivée et au départ c'est qu'il y a un raté non? vous avez bien activé valhala ?
Seulement à l'arrivée, le départ c'est bien un trajet à vélo
16:30:41
@maelito:matrix.orgmaelito* Seulement à l'arrivée, le départ c'est bien un trajet à vélo 17:24:49
14 Apr 2021
@shakedo:matrix.orgShaked Ofek joined the room.11:52:27
@shakedo:matrix.orgShaked OfekHi there - I used Navitia docker-compose before to setup a server for querying GTFS feed and generating Isochrones. That was amazing! I want to go to the next level and provide changes to the feed, then see how the isochrones are affected by that. In my previous configuration, that would require waiting 20 minutes for a new GTFS file to be processed by gtfs2ed (GTFS of 300MB~). I will work with much smaller GTFS files, but can you recommend on a way to make this processing time shorten to just several seconds? Can Navitia be provided with incremental changes? Or maybe use GTFS-RT (I don't know too much about this protocol yet) in order to achieve these feed changes? 11:55:38
@stifoon:matrix.orgstifoon
In reply to @antoine_de:matrix.org
ah c'est chelou qu'il t'ai proposé un train annulé, normalement ils chargent les info temps réel de la SNCF (c'est bien toujours le cas stifoon ?)
Le lent... J'avais pas vu ton message, mais normalement on a bien les data temps réel SNCF. Il faut ajouter le paramètre &data_freshness=realtime
12:11:06
@stifoon:matrix.orgstifoon
In reply to @shakedo:matrix.org
Hi there - I used Navitia docker-compose before to setup a server for querying GTFS feed and generating Isochrones. That was amazing!

I want to go to the next level and provide changes to the feed, then see how the isochrones are affected by that. In my previous configuration, that would require waiting 20 minutes for a new GTFS file to be processed by gtfs2ed (GTFS of 300MB~).

I will work with much smaller GTFS files, but can you recommend on a way to make this processing time shorten to just several seconds?
Can Navitia be provided with incremental changes? Or maybe use GTFS-RT (I don't know too much about this protocol yet) in order to achieve these feed changes?
There are many ways to change data, it depends on your change. If you want to change the stop_point referential or lines descriptions, then you will have to push another GTFS
12:13:54
@stifoon:matrix.orgstifoonIf you want to compare 2 networks, you could make a single GTFS with the 2 networks, and play with the "forbidden_id" parameter to deactivate one network or the other. It is a usefull feature to compare in realtime 2 states of the data12:15:28
@stifoon:matrix.orgstifoonBut if you want to change only realtime stop_time, then you will have to push GTFS-RT data, and then it is a bit more difficult :)12:16:05
@stifoon:matrix.orgstifoonYou will have to plug Kirin with Navitia12:16:17
@stifoon:matrix.orgstifoon(btw 20min to feed Navitia with a 300Mo GTFS data seems to be long 🤔 )12:17:01
@shakedo:matrix.orgShaked OfekThanks for the prompt reply :) I'm not really interested in RT, I was just suggesting it as a workaround.12:17:37
@shakedo:matrix.orgShaked Ofek
In reply to @stifoon:matrix.org
(btw 20min to feed Navitia with a 300Mo GTFS data seems to be long 🤔 )
Hmm I was using a slow machine back then, so probably that's why. It was a once-in-a-month thing.
But let's say I'm using a strong machine, how much time is it reasonable to process a new feed of 10-20MB? Even less
12:22:46
@shakedo:matrix.orgShaked OfekCan it go down to seconds?12:23:16
@stifoon:matrix.orgstifoonunfortunaly, not 😬12:30:13
@stifoon:matrix.orgstifoonYou want to change "stop_points and lines" or only stop_times ?12:31:03
@stifoon:matrix.orgstifoonIf you want to change every kind of data (to test a new line for example, or to compare 2 networks), then the solution with "a single GTFS with the 2 offers in it" could be the best solution > you just have to activate lines or networks you want to activate in journeys parameters12:33:12
@stifoon:matrix.orgstifoon * If you want to change every kind of data (to test a new line for example, or to compare 2 networks), then the solution with "a single GTFS with the 2 offers in it" could be the best solution > you just have to activate lines or networks you want to activate in journeys parameters12:34:02
@shakedo:matrix.orgShaked OfekOK cool, I'll try playing around with GTFS load times and I will see if that works for me. Thanks!12:39:57
@shakedo:matrix.orgShaked OfekI've just cloned the docker-compose repo and ran "docker-compose up", but the rabbit mq fails right away with ""Too short cookie string" error. Before, I had an error about insufficient disk page, but I allocated more disk space for docker to use13:06:58
@pbougue:matrix.orgpbougue Shaked Ofek: The fact that you already have loaded OSM data on the side (I guess) can affect processing time, part of OSM processing is not re-done, but part of it is. 20:18:10
@pbougue:matrix.orgpbougue * Shaked Ofek: The fact that you already have loaded OSM data on the side (I guess) can affect processing time 20:18:18
@pbougue:matrix.orgpbougue * Shaked Ofek: The fact that you already have loaded OSM data on the side (I guess) can affect processing time, part of OSM processing is not re-done, but part of it is. 20:18:49
15 Apr 2021
@kinou:matrix.orgkinou

data integration was never really built to be fast, so seconds won't be possible. One way to speed thing a bit might be to update transport data in the intermediate database in place of using a new gtfs, obviously it's more work on your side as documentation on ED is probably non-existent.
as already said, if you do not need OSM data remove them, it will speed thing a bit.
it will be interesting to know what step is slow, in logs you should be able to extract timing for these step:

  • gtfs2ed
  • ed2nav
  • kraken loading
05:55:45
@kinou:matrix.orgkinoukraken loading will be hard to optimize, except using realtime, but it's more components to setup05:57:34
@kinou:matrix.orgkinouone note on my first "idea", manually updating the ed database is a bad idea on a production system as no guarantee are made on it's "interface" but if it's only for experimenting it's possible, schema is inspired by transmodel but a (very) simplified version: http://www.transmodel-cen.eu/06:03:14
@pbougue:matrix.orgpbouguePlease also note that gtfs2ed is deprecated and now it's supposed to be gtfs2ntfs (from https://github.com/CanalTP/transit_model/blob/master/gtfs2ntfs/README.md) -> ntfs2ed (so provide navitia a NTFS). (This is not heavily advertised 🙏)10:25:03
@pbougue:matrix.orgpbougue * Please also note that gtfs2ed is deprecated and now it's supposed to be gtfs2ntfs (from https://github.com/CanalTP/transit_model/blob/master/gtfs2ntfs/README.md) -> ntfs2ed (so provide navitia a NTFS). (This is not heavily advertised 🙏)10:27:06
18 Apr 2021
@shakedo:matrix.orgShaked Ofek Thanks pbougue & kinou for the ideas!
I'll start with experimenting and seeing how much time it takes to process a new small GTFS (or NTFS actually), as this would be the case usually.
06:56:35
@shakedo:matrix.orgShaked Ofek * Thanks pbougue & kinou for the ideas!
I'll start with experimenting and seeing how much time it takes to process a new small GTFS (or NTFS actually), as this would be the case usually.
10:36:47
@shakedo:matrix.orgShaked Ofekimage.png
Download image.png
10:37:19

There are no newer messages yet.


Back to Room List