Repository navigation
Add NewRoutes to GTFS-RT to support dynamic service (#606) - #621
cleveramarquet wants to merge 14 commits into
Conversation
Add NewRoutes to proto
|
|
||
| // See definition of routes.route_type in (CSV) GTFS. | ||
| // Required | ||
| optional RouteType route_type = 6; |
There was a problem hiding this comment.
As I did in the Google doc, I'm requesting this to be changed to an int so it can reflect the full range of extended route types. As a benefit this enum would then not have to be kept in sync with static GTFS.
There was a problem hiding this comment.
I see your point about the extended route types. However, I used an enum here to stay consistent with how the rest of the current GTFS-RT specification handles route types. Perhaps we could stick with the enum for this PR to keep the scope focused on NewRoutes, and then address a global move to int32 for route types across the entire spec in a separate proposal?
There was a problem hiding this comment.
Well, since GTFS has ironclad backwards-compatibility guarantees, I don't think I will be able to convince the community that there are two ways to define the mode.
There was a problem hiding this comment.
100% agree with @leonardehrenfried.
We need extended route types support from the beginning so the feature can be useful.
(off topic: IMO the current GTFS route types should be deprecated and replaced with extended route types)
There was a problem hiding this comment.
I've updated route_type to int32 to support extended types as suggested. Thanks for the catch!
There was a problem hiding this comment.
The move to int32 (should probably be uint32?) doesn't bother me, but it's really not obvious to me that extended route type is the way to go. There's been discussion in the past about this and it was contentious
There was a problem hiding this comment.
I agree that the status of the extended route types have been pretty frustrating for many years.
I'm aware that they are pretty unimportant in North America (correct?) but in Europe people very much like to separate their services into distinct types. From the perspective of a routing algorith, it may not matter much, but regular commuters care quite a lot about the difference between regional rail and suburban rail, for example.
I'm in favour of these distinctions, too, and would like to hear your opinions. If @gcamp, @felixguendling and I agree that it's worthwhile, we could try another attempt at standardising.
There was a problem hiding this comment.
we could try another attempt at standardising
I would support this as it's a very obvious shortcoming of GTFS compared to NeTEx and commercial proprietary data formats like HRDF, DIVA, ISA, etc. And it should be easy to fix because extended route types are used in practice in a lot of places already.
Change route_type to int32 to support extended route types per PR feedback
|
@cleveramarquet You're missing the reference.md updates! |
|
Thanks, @cleveramarquet, for yielding to my request. |
|
Hi @cleveramarquet, Just a quick reminder based on CHANGES.md, a few conditions need to be met before initiating a vote:
(Counted from when the PR is opened)
("Development in progress" does not meet this requirement) Given that these conditions are not currently met, we consider the current vote invalid. You may initiate another vote once all requirements are met. |
Adding documentation for the Route message including route metadata, sort order, and activation dates to align with the NewRoutes proposal
Updated gtfs-realtime.proto to include: route_sort_order (uint32) for display sequencing. start_date and end_date (string) for route activation periods. Added comments referencing the corresponding (CSV) GTFS definitions to maintain specification consistency.
|
Thanks @leonardehrenfried and @felixguendling for the feedback. I have updated route_type to int32 in the .proto file and updated the documentation in reference.md to reflect this. This ensures we support the full range of extended route types from the start. @gcamp I’ve just pushed an update that includes the reference.md changes. It now contains the full definitions for the Route message, including the route_sort_order, start_date, and end_date fields. @tzujenchanmbd Thank you for the reminder on the CHANGES.md requirements. I’ve retracted the formal 'Call for Vote' for now. I will leave the PR open for the required discussion period. In the meantime, I am working with the Transit App team to move their implementation from 'in-progress' to a state that meets the voting requirements. I'll provide the updated links once those are ready. Regarding the route_type discussion, I’ve updated my PR to int32 to ensure NewRoutes is compatible with extended types immediately. However, to avoid overloading this proposal with a broader debate on global transit mode definitions, I suggest we move any further discussion on standardizing route_type across the rest of the spec to Issue #310 as suggested by @tzujenchanmbd. |
etienne0101
added
GTFS Realtime
Updated agency_id, route_short_name, and route_long_name to match the requirements in the GTFS-NewRoutes proposal document.
|
The |
That's correct @felixguendling, this should not stop us from making progress here. We can take care of standardizing more values for route_type in another pull request. We are very close to having both public endpoints in between Hitachi (Clever Devices) and Transit running so we can officially put this pull request for a vote soon. |
…#34) Repoints the gtfs-spec submodule from the bare head of google/transit#621 to TransitApp/transit@vendored-with-proposals, which is upstream master plus a merge commit per proposal we adopt early: #611 cyclist positions VehiclePosition 12/13, CarriageDetails 6/7 #621 NewRoutes Route, FeedEntity.route, TripProperties.route_id Previously the pin could only carry one in-flight proposal at a time, since a submodule pin is a single commit and each PR head sits on its own base. Pinning an integration branch lifts that limit and also recovers the 15 master commits both heads were missing (Alert.communication_period, Alert.impact_period, Cause.SPECIAL_EVENT). The pin stays fetchable through the existing .gitmodules URL: GitHub serves any commit in a public fork network by SHA from the parent repo, which is the same mechanism that made the previous pin work. No .gitmodules change and no `git submodule sync` needed. Node bindings regenerated (CJS + ESM + both .d.ts); version 1.11.0. The diff against 1.10.0 is purely additive - no types removed. Refreshing the branch: reset it to upstream master and re-merge each still-open PR head. Never merge master into it - google/transit squash-merges, so a landed PR arrives as a new commit that git will not recognise as already applied. Deleting it: once #611 and #621 have both landed upstream, repin gtfs-spec to google/transit master and delete the branch. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
Hi @tzujenchanmbd /@gcamp, whenever you have a moment, could you please approve the pending workflow runs on this PR so the security scan can execute? Thank you! |
skalexch
left a comment
skalexch
left a comment
There was a problem hiding this comment.
@cleveramarquet I left one comment, route_id is missing from the table of TripProperties.
Also, there are mentions of route_id in reference.md. To me, it seems like all of these mentions are related to TripDescriptor and EntitySelector (e.g. lines 472 and 480), and the NewRoutes are only defined in TripProperties.
-
Should we explicitly mention that all
route_idsin TripDescriptor and EntitySelector should only reference staticroute_ids, and that dynamically createdroute_idsare only referenced viatrip_properties?- I saw that the Google Doc of the proposal suggests referencing dynamically created routes in EntitySelector, but maybe that was removed from the scope of this proposal? I think if you're bringing it back, it opens a new conversation on whether it makes sense to link an alert to a route that isn't statically defined.
-
Just a governance remark for future proposals, please make sure to include as much technical details as possible about the proposal in the body of the PR. Such as the details regarding inclusion in TripProperties, as well as one or two examples from the Google Doc. The doc should only be a complement to the information mentioned in the proposal.
-
@tzujenchanmbd I think, at some point, we should elaborate on the definition of TripProperties in
reference.mdto make sure that the way it functions is clearer.
|
@cleveramarquet thanks for adding |
Co-authored-by: Skander Chouchene <skander@mobilitydata.org>
|
Call for Vote Producer Implementation: Hitachi Test Feed Consumer Implementation: Transit App I am officially initiating a Call for Vote for the adoption of NewRoutes. Per the contribution guidelines, this voting window will remain open for at least 7 calendar days. The voting period will span 1 week and will end on 2026-11-09 at 23:59:59 UTC. |
|
@cleveramarquet you need to put an exact end date and time to the vote to make sure we know which votes are valid. Here's an example. |
|
Sorry for posting feedback comments during the voting period - if these are addressed Swiftly will vote +1. |
|
@phil-swiftly @cleveramarquet if the comments change the proposal substantially and are not just for the format, then the change cannot be made during the vote since it changes the scope of what's being voted on. Also I cannot see the comments, did you submit the review? |
|
Hi everyone, Thank you again for all the progress and feedback on this proposal! @phil-swiftly — Could you please share the specific points or comments you were referring to? I want to make sure everything is fully addressed. Since we didn't quite reach the required quorum of supporting votes before the initial window closed, the vote is considered as failed. I am starting a new voting period to give everyone a bit more time to review and vote. New Voting Window: Through 2026-11-17 at 23:59:59 UTC @BethBeveridge, @felixguendling, and @leonardehrenfried — when you have a moment, would you mind taking a look and casting your votes? Thank you all so much for your help getting NewRoutes across the finish line! |
|
+1 MOTIS (consumer) |
|
@cleveramarquet could you please rebrand this as a new vote? It's fine for the time period, but once the first vote ended, it failed. You can just announce it as: "we closed the first vote on the 9th (failed) and opened a new vote that ends on the 17th)." |
@skalexch, yes of course. I updated the comment to reflect the new voting period. Thanks |
phil-swiftly
left a comment
phil-swiftly
left a comment
There was a problem hiding this comment.
Here are the comments, sorry about the delay
| | **trip_modifications** | [TripModifications](#message-tripmodifications) | Conditionally required | One | List of trips affected by a particular modifications, such as a detour. <br><br>**Caution:** this field is still **experimental**, and subject to change. It may be formally adopted in the future. | | ||
|
|
||
| | **trip_modifications** | [TripModifications)(#message-tripmodifications) | Conditionally required | One | List of trips affected by a particular modifications, such as a detour. <br><br>**Caution:** this field is still **experimental**, and subject to change. It may be formally adopted in the future. | | ||
| | **route** | [Route](#message-route) | Conditionally required | One | Data about a new route added to the feed dynamically, such as a bus bridge. <br><br>**Caution:** this field is still **experimental**, and subject to change. It may be formally adopted in the future. | |
There was a problem hiding this comment.
In order to clarify the Conditionally required status of this FeedEntity, I think route needs to be added to the list in the FeedEntity description of which at least one entity type must be entered. Link here
There was a problem hiding this comment.
@phil-swiftly, thank you for the thorough review! I've added route to the FeedEntity conditionally required entity types list in the latest commits to reference.md. Let me know if this can be resolved.
| | **shape_id** | [string](https://protobuf.dev/programming-guides/proto2/#scalar) | Optional | One | Specifies the identifier of the shape of the vehicle travel path when the trip shape differs from the shape specified in (CSV) GTFS or to specify it in real-time when it's not provided by (CSV) GTFS, such as a vehicle that takes differing paths based on rider demand. See definition of `trips.shape_id` in (CSV) GTFS. <br>If a shape is neither defined in (CSV) GTFS nor in real-time, the shape is considered unknown. This field can refer to a shape defined in the (CSV) GTFS in shapes.txt or a `Shape` in the same (protobuf) real-time feed. The order of stops (stop sequences) for this trip must remain the same as (CSV) GTFS. If it refers to a `Shape` entity in the same real-time feed, the value of this field should be the one of the `shape_id` inside the entity, and _not_ the `id` of `FeedEntity`.<br>Stops that are a part of the original trip but will no longer be made, such as when a detour occurs, should be marked as schedule_relationship=SKIPPED or more details can be provided via a `TripModifications` message.<br><br>**Caution:** this field is still **experimental**, and subject to change. It may be formally adopted in the future. | | ||
| | **trip_headsign** | [string](https://protobuf.dev/programming-guides/proto2/#scalar) | Optional | One | Specifies the headsign for this trip when it differs from the original. <br><br>**Caution:** this field is still **experimental**, and subject to change. It may be formally adopted in the future. | | ||
| | **trip_short_name** | [string](https://protobuf.dev/programming-guides/proto2/#scalar) | Optional | One | Specifies the name for this trip when it differs from the original. <br><br>**Caution:** this field is still **experimental**, and subject to change. It may be formally adopted in the future. | | ||
| | **route_id** | [string](https://protobuf.dev/programming-guides/proto2/#scalar) | Optional | One | Specifies the route associated with the trip, if the route has been added dynamically. The associated `Route` message should be included in the same feed. <br><br>**Caution:** this field is still **experimental**, and subject to change. It may be formally adopted in the future.| |
There was a problem hiding this comment.
I think we should also note that this field is forbidden when schedule_relationship is not NEW. This is in the protobuf comments, but should also be included here.
There was a problem hiding this comment.
@phil-swiftly, thank you for the thorough review! I've documented that route_id in TripProperties is optional when schedule_relationship is NEW and forbidden otherwise in the latest commits to reference.md. Let me know if this can be resolved.
|
|
||
| | _**Field Name**_ | _**Type**_ | _**Required**_ | _**Cardinality**_ | _**Description**_ | | ||
| | :--- | :--- | :--- | :--- | :--- | | ||
| | **route_id** | [string](https://protobuf.dev/programming-guides/proto2/#scalar) | Required | One | Identifier of the route. Must be different than any route_id defined in the static GTFS. | |
There was a problem hiding this comment.
Can we include the note from the Google Doc that route_id
Should not be reused for different disruptions or car pools on the same service day.
There was a problem hiding this comment.
@phil-swiftly, thank you for the thorough review! I've added the cautionary note regarding route_id reuse for different disruptions or car pools on the same service day. in the latest commits to reference.md. Let me know if this can be resolved.
Co-authored-by: phil-swiftly <philip.cline@goswift.ly>
Clarified the shape entity to also document route as a choice
Addressed review feedback from @phil-swiftly: - Included `route` in the `FeedEntity` description list of conditionally required entity types. - Noted that `route_id` in `TripProperties` is forbidden when `schedule_relationship` is not `NEW`. - Added a cautionary note that `route_id` should not be reused for different disruptions or car pools on the same service day.
Added a cautionary note that `route_id` should not be reused for different disruptions or car pools on the same service day.
cleveramarquet
left a comment
cleveramarquet
left a comment
There was a problem hiding this comment.
Addressed review feedback from @phil-swiftly on PR #621:
- Included
routein theFeedEntitydescription list of conditionally required entity types. - Noted that
route_idinTripPropertiesis optional ifschedule_relationshipisNEW, and forbidden otherwise. - Added a cautionary note that
route_idshould not be reused for different disruptions or car pools on the same service day.
|
@cleveramarquet according to the RT governance, "During voting period only editorial changes to the proposal are allowed (typos, wording may change as long as it does not change the meaning)." The changes that were made during the vote constitute more than that. Unfortunately, that means the current vote is not valid. Therefore, it is better to stop it. We recommend keeping the proposal open for a week just to make sure the changes you made are visible to community members, then you can start the vote to adopt NewRoutes. Tagging @phil-swiftly (for having requested the changes) and @felixguendling (for having voted). |
|
Based on the governance feedback from @skalexch regarding the substantive updates made during the active voting window, we are officially canceling the current vote. Per the GTFS contribution guidelines, I am placing the PR back into a 1-week review mode effective today so that community members can review the latest changes to Thank you everyone for your patience and feedback! |
|
If we're back to review mode anyhow: when processing the Protobuf file, it's most efficient to go through it linearly from begin to end in a single pass. If trip updates use the new route, it would help for consumers if the route has already been parsed when it's referenced. So I would propose to add a rule that requires that all entities that are referenced to be earlier in the list than the first entity that references it. This would also be nice to have for |
Description
This proposal introduces a
NewRoutesmessage to theFeedEntityin GTFS Realtime. As discussed in Issue #606, the current specification lacks a standard way to define route metadata for services that do not exist in the static GTFS schedule.The full technical specification and field definitions can be found in the GTFS-NewRoutes Google Doc.
By allowing a
Routeto be defined directly within the real-time feed, consumers can provide a complete and accurate experience for riders without requiring agencies to "reuse" unrelated static IDs.Implementations
Proposal Update Tracker
Date Update Description
(2026-04-09) (Raised proposal)
(2026-10-19) (Discussion period started)
(2026-11-02) (Review period started)
Checklist
I have read and understood the GTFS Change process.
I have read and understood the GTFS Guiding Principles.
I understand the Role of Advocate and the Responsibilities attached to it.
I have signed the Contributor License Agreement (CLA).
I have ensured that this proposal is not covered by an existing Pull Request.