night199uk - is there specific support for canb...
is there specific support for canboat anywhere? I'm working to add support to canboat, canboatjs, and eventually signalk-nmea2000 for the propetiary raymarine autopilot PGNs. 130918 and 130848. this should help signalk to control raymarine autopilot heads like the raymarine p70, I hope.
raymarine PGNs seem to contain an string type that I cannot find support for in canboat today.
it is a fixed maximum number of bytes (i.e. 16 bytes), but shorter string are allowed & NULL-terminated. in canboat pgn.h I see either STRING_FIXED (which reads and displays all 16 bytes, including the NULL and garbage after the NULL), or variable length types like STRINGLZ, which is a string with length encoded.
is there a CANBOAT type for "NULL terminated string, with maximum length", that I'm not finding?
165 Replies
Assuming you’re aware of the documentation. https://canboat.github.io/canboat/canboat.html this shows what’s been developed so far and which PGNs have already been reverse engineered
@Tony yup; thanks - I do see that. i'm wondering which files need to be modified when adding support for these new PGNs?
i guess a different question would be - what is the purpose of each of canboat.json, canboat.xml, canboat.html?
If you start with the Readme on the GitHub repo it will walk you through everything including contributing
https://github.com/canboat/canboat/wiki
The readme takes you to the link I posted
Specifically this page explains extending to more pgns https://github.com/canboat/canboat/wiki/Extending
Regarding the definitions (q2) https://github.com/canboat/canboat/wiki/Using-the-definitions-in-other-software
There is already support for the Raymarine autopilots
There is support for the simple -1, +1 type autopilot. But not for Waypoint tracking.
Yes. True.
Not sure if the modern Raymarine EV-1s support the 129284 navigationData PGNs, but at least the older X30/P70 control units don't, and only use the 130848 and 130918 Raymarine custom PGNs for route and waypoint information.
If you need help, best to open issues in the canboat git repository
I think I got it all figured out. Seems the xml & json are auto-generated from pgn.h.
Kees and I can help you there
I just raised:
https://github.com/canboat/canboat/pull/488
Yes
And I raised an issue on canboatjs to update that project. I think that's the right way to get things done, let me know if not 🙂
once i can hook all this up, this should get full waypoint tracking autopilot working for raymarine.
Yep. Once Kees merges that and does a canboat release. Then I can do a canboatjs release.
my goal is to have e.g. aqua maps be able to full drive my raymarine autopilot via signalk.
If you want to start testing things now, you can copy that generated canboat.json file into your @canboat/pgns module
awesome! let me try that.
that's the only change required in canboatjs?
Yes
perfect, thanks!
i want to move to the signalk side now and have it start generating those PGNs so I can test fully
is there a simple tool to generate NMEA-2K PGNs from the CLI? i looked at cansend, but that seems to be raw CAN frames and much more raw
maybe i need to spend more time on cansend
Easiest is through the data fiddler in the sk server
You can enter the canboat json and send that out to the canbus
oh, nice. is there an example of sending an nmea-2k frame with the data fiddler?
It’s just the canboat json format
oh wow, shit. that's awesome.
Examples here
GitHub
canboatjs/test/pgns at master · canboat/canboatjs
Native javascript NMEA 2000 decoder and encoder. Contribute to canboat/canboatjs development by creating an account on GitHub.
The “expected” in those
so first i'd have to add the support to canboat, canboatjs (copying the json file), then restart my sk server then i should be able to send raw json like this:
{"timestamp":"2025-03-09-20:54:28.297","prio":7,"src":6,"dst":255,"pgn":130848,"description":"SeaTalk: Waypoint Information","fields":{"Manufacturer Code":"Raymarine","Industry Code":"Marine Industry","SID":191,"Waypoint Name":"Waypoint 134Qwer","Waypoint Sequence":"0001","Bearing to Waypoint, True":10.8,"Bearing to Waypoint, Magnetic":20.0,"Distance to Waypoint":1915725.00}}
{"timestamp":"2025-03-09-20:54:28.303","prio":7,"src":6,"dst":255,"pgn":130918,"description":"SeaTalk: Route Information","fields":{"Manufacturer Code":"Raymarine","Industry Code":"Marine Industry","Current Waypoint Sequence":1,"Current Waypoint Name":"Waypoint 134Qwer","Next Waypoint Sequence":2,"Next Waypoint Name":"Waypoint 746;�","Distance to Waypoint":4912407.04,"Bearing to Waypoint, True":85.1,"Bearing to Next Waypoint, True":79.2}}
Yes
that's awesome. alright, let me go fiddle with that.
thanks!
sorry - i didn't see a response to my original question on this thread
do you know if there is a "null terminated with maximum length" string type in canboat?
i think there is not.
I thought so
Give me a few minutes, I can check…
sure. these raymarine strings are always 16 bytes, but if the contents is < 16 bytes long, it is null terminated and the rest of the bytes filled with junk.
e.g. in the json above you see:
*Waypoint 746;�
Waypoint 746 is 13 characters + 1 byte for NULL, the rest of the characters up to 16 bytes are filled with junk.
using STRING_FIXED expects all 16 characters to be "relevant".
sorry, hard to explain what I mean by this
I think you are right
there;s no type for that
@Kees Verruijt ?
easy enough to add, but should confirm with Kees, probably best to open an issue so we can track it
I’ve just reread the description for STRING_FIXED and although I think Raymarine is being sloppy here I don’t think there is harm if we terminate on the first zero byte. Please open an issue!
i can work on an implementation. it's preventing me moving further now as it's messing up the json parser due to binary in the stream.
would you prefer to fix the current implementation of STRING_FIXED to terminate on \0, or add a new type, like, STRING_NULL(xxx, MAX_BYTES)?
Error: Invalid JSON (Unexpected "\u0002" at position 336 in state STOP)
etc
Where is that invalid json coming from?
candump can0 | candump2analyzer | analyzer -json | bin/n2k-signalk
can you try with canboatjs?
so i guess i could work around it, but still, it seems like being thorough i should fix this on the way through
@Scott Bender what would be the comparaible to candump can0 | candump2analyzer | analyzer -json?
i notice the output of canboatjs is marginally different
is canboatjs can0 | bin/n2-signalk enough?
candumpjs can0
k, thx
Obviously, canboat needs fixed. But just interested in how canboatjs is handling it…
canboatjs works good
oh wait
canboatjs is quoting it
{"canId":503277062,"prio":7,"src":6,"dst":255,"pgn":130918,"timestamp":"2025-03-10T01:08:29.801Z","input":["2025-03-10T01:08:29.796Z,7,130918,6,255,8,60,2f,3b,9f,01,00,57,61","2025-03-10T01:08:29.797Z,7,130918,6,255,8,61,79,70,6f,69,6e,74,20","2025-03-10T01:08:29.797Z,7,130918,6,255,8,62,31,33,34,51,77,65,72","2025-03-10T01:08:29.798Z,7,130918,6,255,8,63,02,00,57,61,79,70,6f","2025-03-10T01:08:29.799Z,7,130918,6,255,8,64,69,6e,74,20,37,34,36","2025-03-10T01:08:29.800Z,7,130918,6,255,8,65,00,3b,a7,02,00,bf,47","2025-03-10T01:08:29.801Z,7,130918,6,255,8,66,1d,00,3a,07,36,d3,ff"],"fields":{"Manufacturer Code":"Raymarine","Industry Code":"Marine Industry","Current Waypoint Sequence":1,"Current Waypoint Name":"Waypoint 134Qwer","Next Waypoint Sequence":2,"Next Waypoint Name":"Waypoint 746\u0000;'\u0002","Distance to Waypoint":4912412.16,"Bearing to Waypoint, True":1.4848,"Bearing to Next Waypoint, True":1.3831},"description":"SeaTalk: Route Information"}
note: "Waypoint 746\u0000;'\u0002"
Ok. Not surprised.
so waypoint 746 is a 12-byte null-terminated string in a fixed 16-byte field
At least it’s valid json 🤣
yeah
there are some other things i see in this raymarine data that is just yucky.
when there is no waypoint name, they just pad the whole 16-byte field with 0xff 🤦♂️
How does canboatjs handle that?
(And trust me, I’ve done a lot of reverse engineering of Raymarine stuff, always fun!)
will test shortly and confirm
this is my first can/n2k/nmea decoding rodeo but not my first reverse engineering obviously, so yeah, always fun
I have an EV-1, so this never came up for me.
does the ev-1 honour the standard 129284 PGNs?
I think so, but I’m not sure.
that has a p70s control head?
Yes
i'm not convinced the p70s honours them
The p70 has nothing to do with it
can you use "track" mode to track a route on your p70s with routes provided by e.g. signalk?
P70 is just the control head
You don’t even need that
I have not tried to provide routes from signalk. Cause I use an Axiom for that.
ahhh, there 🙂
yeah, my axiom upgrade is in the post
i bought the boat a year ago and i'm just finishing my first season in the bahamas and heading back to the US, and have a big pile of upgrades sat waiting for me.
but still, i'd prefer to be able to send routes from aqua maps these days than rely on the mfds.
this boat currently has 2x e126 MFDs that are TERRIBLE and slow.
so i've become accustomed to navionics and aqua maps and i'd just rather be able to drive the whole thing from aqua maps than rely on the MFDs.
just far more intuitive these days to sit in bed and program a route on aqua maps on the ipad and then get up the next day and sail it.
I’ve been very happy with Navionics+Axiom. But it will be very cool to get this working.
yeah, i think it's easier to move GPX based routes from navionics to axiom?
getting a route from navionics to my e126 is terrible
It’s pretty easy
anyway, never let a good problem go unsolved. might as well fix this while i'm here. 🙂
this should work for your axiom as well
Agreed. I can try to get it working with with the EV-1 too. For fun.
so now i'm going to try and get signalk->nmea2000 generating these pgns
I can work on making canboatjs handle these strings tomorrow…
An example of the empty, 0xff one would help.
how do you want it?
what format?
don't suppose you use 010 editor?
No
The output you pasted about will work.
It has the “input” data in it
okay, will have to regenerate it, means faffing with my e126 for ages
i have a binary stream of about 20 different PGNs i'm working on
would be easy just to send you that
That’s fine
got an email address?
scott@scottbender.net
sent
Oh. I thought it was going to be candump…
Definitely easier for me if in a format canboat understands
ahhh yeah no worries. i'll have to recollect all the data.
spent all day collecting data and generating test cases from my e126. was slow going to build all these samples. will have to rebuild it in candump format tomorrow if you need it that way.
can send it to you tomorrow.
I have enough from above to work on the 0 terminated string. And I can hack together the other one.
I’ll let you know if I need more.
yeah, i mean it's pretty simple.
there are three cases:
1) 16 byte string in a 16 byte field -> all valid, no null terminator: Waypoint 134Qwer -> 57 61 79 70 6F 69 6E 74 20 31 33 34 51 77 65 72
2) 12 (or shorter) byte string in a 16 byte field -> only 12 valid bytes, null terminator and uninitialized junk after the null: Waypoint 166 -> 57 61 79 70 6F 69 6E 74 20 31 36 36 00 30 D7 3F (30 d7 3f are junk)
3) no valid waypoint, string field padded with nothing (0xff) -> FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
GitHub
fix: issues with STRING_FIX field type decoding by sbender9 · Pull ...
Handle 0 terminated strings (fixes issues with some proprietary Raymarine pgns)
Code to find end of strings did not work properly with some data
nice. question, does string_fix also now append the zero byte when creating strings?
i'm just working on the signalk-to-nmea2000 plugin code to generate PGNs 130848 and 130918
having problems right now but will figure that out
No. It does is according to the n2k standard. It fills with 0xff
interesting. let me test this and see how the raymarine units deal with it.
it seems like the RM stuff is in breach of the spec from the sounds. I don't have the NMEA2K spec
We’re going to have to figure out a way to zero terminate these specific ones.
Definitely don’t want to change it to do that by default.
yeah. perhaps what we need is STRING_FIX_RAYMARINE or some such to keep STRING_FIX N2K spec compliant
otherwise you'll find some issue with N2K compliant units elsewhere
Yep
question: is any modification needed to canboatjs to generate my new 130848 PGNs with toPgn?
i've added code to signalk-to-nmea2000 to generate these new PGNs, but my new PGNs are always coming back empty
Raymarine Waypoint Information (130848) test # 0/0 works:
AssertionError: expected { Object (prio, pgn, ...) } to have the same json representation as { Object (prio, pgn, ...) }
+ expected - actual
{
"dst": 255
- "fields": {}
+ "fields": {
+ "SID": 136
+ }
"pgn": 130848
"prio": 2
}You updated canboat.json?
yup, updated canboat.json
so you updated it in node_modules/@canboat/pgns/ inside your signalk-to-nmea2000 ?
yup
hrm, okay, i found something broken. sec.
and you shouold probably test these strings the way they will be encoded now. It's possible the Raymarine stuff will hande it if it's not zero terminated.
yeah, i will. hrm.
i just cannot get signalk-to-nmea2000 to generate these PGNs properly.
fixed a few issues i could find.
- "fields": {}
The PGNs I get back during npm test are just empty of fields.
can you email the canboat.json and the signalk-to-nmea2000 changes?
yeah, just debugging it. let me look first before wasting your time. looks like toPgn is working good actually.
whoop, got it working. briefly saw an autopilot track on my e126 head unit 🙂
yay! just hit "track" on my autopilot and it now prompts me to follow a track / route sent from signalk to the raymarine. 😄
my e126 MFDs also show me as tracking a route 😄
will be pretty cool. my test case is sending route information from aqua maps to signalk via it's TCP NMEA0183 interface with the autopilot beta control turned on. then signalk forwards that to NMEA.
can control my whole route from aqua maps.
rather, can program a route in aqua maps and then just tell the boat autopilot to track it. no need to turn my raymarine MFDs on any more 😄
where does the course-provider in signalk get its data?
i see my aqua maps tcp-client populating, e.g. navigation.courseRhumbline.nextPoint.position.
then i see courseApi populating navigation.courseGreatCircle.; with almost the same data as courseRhumbLine.
then i see the course-provider plugin populating navigation.course.calcValues.
I’m not familiar with that stuff. Going to need @AdrianP for that.
thanks. i see the conversions for 129284 use data from courseRhumbLine and courseGreatCircle as appropriate. but I'm not sure what are the correct course data values to send in 130848 and 130918. i don't know if those raymarine packets want great circle or rhumb line data, essentially.
and can i assume that if aqua maps is populating courseRhumbLine, the courseApi will correctly populate courseGreatCircle automatically from that data?
currently i have this conversion using the data in navigation.course.calcValues.* and what i see on my plotter looks broadly correct.
See https://demo.signalk.org/documentation/develop/rest-api/course_calculations.html
The course API sets values under the path
vessels/self/navigation/course and the calculations are made from these values.
Course-provider emits deltas for both the version 1 paths courseRhumbline, etc as well as the calcValues paths
See also https://demo.signalk.org/documentation/develop/rest-api/course_api.htmlthanks Adrian. I read both of those.
I think my confusion perhaps is that when I look at the signalk-to-nmea plugin and the 129284 PGN, that is using the values from e.g.
navigation.courseRhumbline.bearingToDestinationTrue &
navigation.courseGreatCircle.bearingToDestinationTrue
Then there are also updated variables in:
navigation.course.calcValues.bearingMagnetic
navigation.course.calcValues.bearingTrue
So I think my confusion is around which is the "right" place those PGNs should be sourcing the true and magnetic path to the destination from.
i'm approaching this from zero and I'm not an expert in great circle vs rhumbline or course APIs in general, so I'm probably missing a bunch of context and history
i guess the key question for me is:
which variables should PGNs 130848 and 130918 (Raymarine Waypoint Information & Raymarine Route Information) use to populate the magnetic and true bearings?
If I use the code in 129284 as an example, it should either be:
navigation.courseGreatCircle.bearingToDestinationMagnetic
navigation.courseGreatCircle.bearingToDestinationTrue
or:
navigation.courseRhumbLine.bearingToDestinationMagnetic
navigation.courseRhumbLine.bearingToDestinationTrue
But I do not know if the raymarine expects RhumbLine or GreatCircle in those PGNs and have no knowledge of the internals of the Raymarine so don't know how I could figure out the answer to that.
The course API does listen for
courseRhumbline.nextPoint.position and courseGreatCircle.nextPoint.position deltas and will treat this in the same way as an API request to set the destination and populate all the associated paths accordingly.
The approach of the course API is to manage all the associated Signal K paths, to basically tell me where you want to go and it will facilitate all the relevant calculations.
Clients then consume this information via signal K deltas or NMEA via the sk to NMEA pluginssure. i guess that doesn't help me (sorry)
i'm sort of confused about the different between rhumb line and great circle. i understand the basic principle i guess; but i don't understand practically how to apply it.
say today, aqua maps is sending a route to singalk using RMB; which ends up with the courseApi populating a RhumbLine route:
does that mean what should be sent out in 130848 is the bearingToDestinationMagnetic and bearingToDestinationTrue from the Rhumbline route?
and now i have Course Data Provider set to "Great Circle" and it is similarly making separate calculations. Is it calculating a Great Circle route to the same lng, lat in parallel to the Rhumbline route?
So as it stands the RMB sentence should be translated to
courseRhumbline.nextPoint.position by the nmea0183-to-signalk plugin.
The course API will see this and should set the destination from this.I see
I think I got it. so assuming we have position (lat, lng) we can calculate either a great circle or a rhumb line route to that destination lat/lng?
(or calculate both)
The course provider plugins will calculate one or the other based on the config
but then it's entirely user preference which set of bearings (rhumb line or great circle) we send to the autopilot
i.e depending on which one of the two routes they want to follow
okay, i think i understand what's going on now.
In the code for 129284, because there are two paths - GreatCircle and RhumbLine, depending on which nodes are populated in signalk only one 129284 will be sent. either great circle or rhumb line.
so in my case if Aqua Maps is sending a Rhumb Line route and populating those paths, the code that sends the 129284 will pick up the populated rhumb line paths and send those in the 129284 packet.
If you've got some example raw data I'm happy to create my own issue and fix it.
@night199uk I released a new canboatjs with the string fix
And I merged the string fix as well in canboat.
No release as of yet but you can just use head.
https://github.com/SignalK/signalk-to-nmea2000/pull/117
Combined with the canboat PR that already got merged, this should get Raymarine autopilots (like P70/X30, and EV1/ACU-400) to enable "track" mode to track a route sourced by SIgnalK.
example PGNs:
{
"timestamp": "2025-03-12-01:27:50.397",
"prio": 3,
"src": 100,
"dst": 255,
"pgn": 130848,
"description": "SeaTalk: Waypoint Information",
"fields": {
"Manufacturer Code": "Raymarine",
"Industry Code": "Marine Industry",
"SID": 136,
"Waypoint Name": "Waypoint 1",
"Waypoint Sequence": "0001",
"Bearing to Waypoint, True": 221.2,
"Bearing to Waypoint, Magnetic": 212.0,
"Distance to Waypoint": 1243.28
}
}
{
"timestamp": "2025-03-12-01:27:50.402",
"prio": 3,
"src": 100,
"dst": 255,
"pgn": 130918,
"description": "SeaTalk: Route Information",
"fields": {
"Manufacturer Code": "Raymarine",
"Industry Code": "Marine Industry",
"Current Waypoint Sequence": 1,
"Current Waypoint Name": "Waypoint 1",
"Next Waypoint Sequence": 255,
"Distance to Waypoint": 1243.28,
"Bearing to Waypoint, True": 221.2
}
}
I'm moving the boat on a long route tomorrow and will try it out on a route sourced from Aqua Maps.
at least right now it's doing the right things sat here on anchor.
hrm. the raymarine plugin works okay (except some string handling things, which I need to pull in @Scott Bender 's PR to look at). my raymarine head can now "track" directly to waypoints provided by signalk.
but i'm interested to understand the intricacies / thinking behind the different course data.
with the current version of the PR i'm using data provided by this plugin:
https://github.com/SignalK/course-provider-plugin?tab=readme-ov-file#course-provider-plugin
specifically bearingTrue and bearingMagnetic. however this seems to be sometimes significantly off from the data in navigation.course.Rhumbline.*.
so i guess i'm trying to understand the discrepency to make sure i'm using right navigation data in the PGNs.
i think what i'm seeing is that the data in the course provider always wants to keep you on the track line, i.e. the perfect line from a->b first.
i thought that's what:
navigation.course.calcValues.bearingTrackMagnetic & navigation.course.calcValues.bearingTrackTrue
should provide, but even bearingTrue and bearingMagnetic seem to show the same behaviour.
e.g. i'm on the water right now and navigation.course.calcValues.bearingMagnetic is almost 20degrees less than navigation.courseRhumbline.bearingToDestinationMagnetic.
for a waypoint just 2-3nm away i wouldn't expect this discrepency.
The course provider uses the values in
course.nextPoint.position and course.previousPoint.position which are populated by the course API.
As per the spec ....
bearingTrackTrue is calculated using previousPoint and nextPoint
bearingTrue is calculated using the vesselPosition and nextPoint
Can you confirm these paths have the expected values as they AFAIK only populated when the course API methods are used.
NMEA to Signalk plugins do not populate these paths, they populate courseRhumbline.nextPoint.position.weird. those paths are getting populated.
so I think the issues I was seeing are due to xte. It seems like by default, when you're offline the course line from a->b, the raymarine autopilot uses XTE first to track quickly back onto the a->b course line, before turning forward towards b.
i'm new to mfd routing and autopilots in general, so i didn't really know about xte. but given the xte behavior this is working as expected.
i saw one other instance where the autopilot didn't do what i expected, but i didn't dig into that one too much. otherwise i used this plugin for an 8 hour route today.
the only major down side right now is that because RMB is "waypoint by waypoint" the raymarine turns the autopilot off at the end of each waypoint as that is also the end of the route. right now aqua maps doesn't support programming the whole forward route I guess.
so i'm going to do some more work:
i'm going to write a signalk plugin to import routes directly from your public folder in aqua map via gpx to the route api. then i need to do some changes here to expose the future waypoints to the raymarine.
tl;dr, i need to get some routes into signalk so that I can make this pgn support full routes, not just single waypoints.
i also emailed aqua maps to see if they would consider programming full routes into signalk via the routes API. that would obviate the need for a signalk plugin to retrieve routes.
i'm halfway done on the plugin to retrieve gpx files from aqua maps. i have it working in python already. just need to convert it to JS.
It may not help but I'll mention it anyway, the course API allows a route to be "activated" if the route is available via
/signalk/v2/api/resources/routesyup. saw that.
hey all. adding support for the raymarine PGNs worked great and I can track waypoints originated via RMB from Aqua Maps. it's sort of clunky due to the limitations of the NMEA-0183 PGNs, but it works.
I now created a plugin to import routes from aqua maps so they can be run by the course API which will help to remove some of the clunkiness of the above approach.
but i'm running into an issue. here's the problem: the PGN129284 is never sent for courses being run by the course API.
looking here:
https://github.com/SignalK/signalk-to-nmea2000/pull/63
it seems like there is a course API v1 and a course API v2?
when i follow a course sent in via an external device (via e.g. RMB over NMEA0183); i see navigation.courseRhumbLine.* updated by the external device PGNs. currently 129284 tracks only those paths.
when I activate a route in the course API using e.g. freeboard-sk, and follow it, I notice that we only update the course API paths:
course.navigation.*
PGN 129284 is never sent which external autopilots like my Raymarine require.
i'd like to try and fix this, by having 129284 generated for navigation.course.* paths, essentially completing PR#63 above, but wanted some input on the right approach to do that?
@AdrianP i think the above is your PR so maybe you have most input 🙂
The signalk-to-n2k plugin is responsible for emitting the PGNs, so as long as the Signalk paths are populated and a map exists for the PGN it should be all good.
The course provider plugins populates the V1 paths so that part should be ok.
I'll have a look when I get a chance
yeah, i spent the day working on this.
i came up with a newer version of https://github.com/SignalK/signalk-to-nmea2000/pull/63 that pulls data from the course-data-provider instead of the old v1 paths.
what i'm trying to understand is the heritage i guess and how much needs to be preserved
i.e. why did https://github.com/SignalK/signalk-to-nmea2000/pull/63 never land?
i have it working great here, with 129284 & 130848 & 130918 migrated to the course-data-provider paths.
but the question is what should happen to the old navigation.course.Rhumbline & navigation.course.GreatCircle paths?
They are considered deprecated from a version 2 perspective, and will remain for the momernt for backwards compatibility.
so if I raise a PR that moves 129284 to just read from the newer course-data-provider paths, that would make sense?
I think that should be fine.
thanks. in that case, since i'm here should i look at the incoming NMEA0183 PGNs as well? RMB, XTE, etc.
otherwise this becomes a breaking change, potentially. if NMEA0183 -> RMB writes to navigation.course.Rhumbline, but 129284 is not reading from that then i guess stuff will break.
i'm new to signalk and don't know a lot of history here, so i don't know where the bodies are buried sort of thing..
there are no bodies here. just love.
what would y'all think of a PR to expose the full details of the active route in navigation.course.ActiveRoute?
he's the problem: for Raymarine PGN130918; they want to know the name and bearing not just to nextPoint, but to subsequentPoint (nextPoint+1). this is to know that when we reach the next waypoint (nextPoint) whether we are at the end of the route or not; and how to draw the forward path.
as i understand it right now, i have two routes for how to do it:
- read the route from the url in href
- add the full route to activeRoute
since PGN130918 are sent regularly (same as 129284). can be as often as 500ms) querying the route via href just to get details about subsequentPoint would be quite expensive to do at that cadence.
so my thinking is to show the whole active route into course.Navigation.activeRoute?
(going to start floating some PRs for all this in the next day or two)
A reference would be the preferred approach as it insulates against ensuring it is being updated when the route is modified. Also as nextPoint (and nextPoint + 1) values only change as you anvigate through the route so not sure why regular calls would be required to the route href.... only need to call when a change in nextPoint value is detected.
@AdrianP when you say a reference - you mean storing the href?
"why regular calls would be required to the route href.... only need to call when a change in nextPoint value is detected"
how would the conversion code cache the subsequentPoint to send the 130918 PGN? each time the PGN is generated we need to include data about subsequentPoint, so as I understand it we either need to get the data via an expensive http request or cache it somehow?
maybe there is a way to cache "subsequentPoint" in conversions.js? not sure
or rather cache the whole route? i guess we can cache the "activeRoute" via the href and only update it when activeRoute changes?
You can get any data from
signalk via app.getSelfPath. No need for http call.
ooh. got an example?
let me grep
Yeah, check the other conversations
oh wait. app.getSelfPath is just for "data" though? can i get a route object this way?
e.g. i see:
server.getSelfPath('navigation.position');
I think so? That’s in the full tree, right?
i don't think it is is it? that's the reason for my question actually 🙂
unless i'm misunderstanding
Ahh. Getting into stuff I am not familiar with…
Let’s see what @AdrianP says…
like this
so i can get navigation.course.activeRoute via getSelfPath. but then I think i still need to do an http call to resolve the href to the actual route data?
i think that's why i was asking if activeRoute should contain the "full" route object of the current route, so we can query it quickly via app.getSelfPath, instead of making the fetch call to the href.
if there is a way to do app.getSelfPath to resolve get the full route object via getSelfPath from the href though, that would work too...
for 130918 i need to get "subsequentPoint" lat & lng and calculate the "next" angle between nextPoint and subsequentPoint so the raymarine knows where to turn after the completion of the current waypoint.
without a subsequentPoint in 130918, the raymarine thinks the route has ended when the current waypoint is reached and turns off tracking. "subsequentPoint" as i call it seems to be the way the raymarine knows the route is not finished and continues after "nextPoint" is reached.
Server plugin API exposes resource API methods to access and write resource entries without the need for http request.
https://demo.signalk.org/documentation/develop/plugins/server_plugin_api.html#resources-api-interface
fyi
https://github.com/SignalK/signalk-to-nmea2000/pull/118
@AdrianP I think this should be a newer and slightly more extensive version of PR#63:
https://github.com/SignalK/signalk-to-nmea2000/pull/63
@Scott Bender just testing with the fix for #489, seems broken. working on it here. with that fix it breaks output to jq 😦
jq: parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 34, column 336
samples of the offending packets:
2025-03-18-14:54:19.336,7,130918,6,255,8,00,2f,3b,9f,01,00,57,61
2025-03-18-14:54:19.337,7,130918,6,255,8,01,79,70,6f,69,6e,74,20
2025-03-18-14:54:19.338,7,130918,6,255,8,02,31,33,34,51,77,65,72
2025-03-18-14:54:19.339,7,130918,6,255,8,03,02,00,57,61,79,70,6f
2025-03-18-14:54:19.340,7,130918,6,255,8,04,69,6e,74,20,37,34,36
2025-03-18-14:54:19.341,7,130918,6,255,8,05,00,0b,95,be,00,e2,3a
2025-03-18-14:54:19.342,7,130918,6,255,8,06,1c,00,8b,09,36,d3,ff
2025-03-18-14:54:20.344,7,130918,6,255,8,20,2f,3b,9f,01,00,57,61
2025-03-18-14:54:20.344,7,130918,6,255,8,21,79,70,6f,69,6e,74,20
2025-03-18-14:54:20.345,7,130918,6,255,8,22,31,33,34,51,77,65,72
2025-03-18-14:54:20.346,7,130918,6,255,8,23,02,00,57,61,79,70,6f
2025-03-18-14:54:20.347,7,130918,6,255,8,24,69,6e,74,20,37,34,36
2025-03-18-14:54:20.348,7,130918,6,255,8,25,00,0b,95,be,00,e2,3a
2025-03-18-14:54:20.348,7,130918,6,255,8,26,1c,00,8b,09,36,d3,ff
What’s jq?
the json processing tool
e.g.
candump can0 | candump2analyzer | analyzer -json | jq 'select(.pgn == 130918)'
kind of like xslt or something for json. super handy.
candump can0 | candump2analyzer | analyzer -json | jq 'select(.src == 6 and .pgn == 130918)'
etc
but it's complaining of invalid json in the stream with the fix for #498 basically. looking now, have to go out in a bit tho.
going to have to work with @Kees Verruijt in that
does it work with the latest canboatjs?
i'm wondering if I need to create a new STRING_FIXED_RM for Raymarine, as we'll also need to be able to encode Raymarine compatible strings.
The default encoding might work, have you tried it?
i notice a bunch of things on the RAymarine are expecting null terminated strings in NMEA packets. I was looking at AIS data the other day and the Raymarine is reading junk from the end of the strings because they're not null terminated.
That’s very strange, can’t image they would required their messed up encoding for AIS pgns
yeah, i tried it. the raymarine reads junk for the 0xffs. raymarine even seem to be parsing some of the standard NMEA-2K PGNs incorrectly in this regard and are expecting null terminated strings.
yeah. i'll find some examples. i just updated my e126 to lighthouse v19 from v11 (yeah, old) to see if they'd fixed any of the bugs. need to play a little more.
I have AIS from a different vendor and the Raymarine stuff reads it fine
yeah. it's not all fields that are affected. i noticed it in some random fields.
duh. my bad
https://github.com/canboat/canboat/pull/491
@Kees Verruijt
fyi,
https://github.com/SignalK/signalk-to-nmea2000/pull/117
and
https://github.com/SignalK/signalk-to-nmea2000/pull/118
should be ready to go.
I will start to look at support for PGN129285 for more route information shortly.
@Scott Bender @Kees Verruijt example of Raymarine displaying garbage when strings aren’t null terminated in regular non proprietary PGNs. Will do a little more debug on this.
This is an E126 running lighthouse v19 which is the latest from 2017. So already 7 years old, hardly up to date. Maybe a big don’t care 🙂
Ok. I’ll take a look tomorrow. And I can test with an Axiom with the latest software.
Yeah. V19 is the latest you get for E126. I’m replacing them soon with Axiom anyway. I’m just a stickler for bugs. 😂 I will dig more too.
I have to figure out why my SignalK is looping back class A PGNs received over N2K too. I don’t have other sources of vessels right now.
Not sure what you mean “looping back”
Yeah ignore me. Maybe these were received over NMEA0183 and sent out N2K
hrm, i'm stuck
i need to call app.resourcesApi.getResource('routes', routeId) in my sk->nmea-2k converter (signalk-to-nmea2000).
however, because getResource returns a promise, it's async and i cannot for the life of me figure out how to do this.
i tried to "await" the value but that requires making the callback itself async/a promise; which breaks the n2k converter around toPgn, as it doesn't expect a promise.
to be able to truely convert route data into PGNs, we need to be able to get the current route in the N2K converter and I cannot figure out how to do that synchronously.
any ideas?
Probably need to make a change to sk-to-n2k internals so the converter calls can be async. (If it’s not there already). I’ll look into it…
yeah. i took a look but my JS is not up to scratch enough to know where to start with it.
This use case triggers the question of whether there needs to be the capability to "route" decoded NMEA data to either the data model or an API.
Alarm PGNs are a good example if we are to implement an Alarm Manager on the Server. The messages should be directed to the Alarm API which then emits the appropriate signalk deltas to allow for silencing, acknowledgement, etc.
Autopilot API would also benefit from this as would the Course API based on the current use case.
yeah, i was thinking about this the other day. there are multiple use cases for signalk. as a simple proxy from nmea0183, as a full fledged mfd, and sometimes half way in between.
i have been juggling that in some of the things i'm looking at. aqua maps sends nmea0183 commands to program routes, which are incomplete. those in turn program the course api. so you end up with two possible courses - the one from aqua maps or the one from the course api. it depends if "aqua maps" is your primary "driver" and you want to trust it for everything, or just have aqua maps program the course and have signalk be the brains and do the routing.
there are places i'm finding right now where it's not clear which paradigm you want
With a routing system in place it can become as simple as a configuration setting.....
@Teppo Kurki any thoughts on this?
why would activeRoute = app.getSelfPath('navigation.course.activeRoute')
return undefined despite there being valid data in that delta? does getSelfPath consider a timeout/age of the data and ignore data over a certain age?
hrm. when I query getSelfPath("navigation") and dump all the data I get back, course.activeRoute is missing. only calcValues exists under course. But when I look at the web UI data browser, I see navigation.course.activeRoute clearly there.
why do I see different results in getSelfPath vs the web UI data browser?
@Scott Bender fyi, i updated the raymarine autopilot support and added support for promises returned from converters as a separate commit in there:
https://github.com/SignalK/signalk-to-nmea2000/pull/117
we could break that out into a separate feature/PR if it's preferable. not sure how y'all roll.
yes, please break into separate PRs
cool, will do it tomorrow. i'm no javascript guru so let me know any comments on the implementation. it's working well here.
FYI v2 delta messages are not written to the data model, so will not be available via getSelfPath, etc.
You need to fetch the values via app.resourcesApi methods.
ahh yeah, I found it under app.courseApi in the end
It's quite confusing that they appear in the webpage data browser and not the model though
and also it's hit and miss. SOME courseApi deltas are there some aren't
might be worth documenting that somewhere. I didn't find it mentioned and it was surprising. I spent quite a lot of time trying to figure it out 😦
FYI....
http://192.168.86.32:3000/documentation/develop/developer_notes.html
Stream Interface
Currently, when v2 REST APIs emit deltas that contain v2 paths and structure, but they do not end up in the full model. This means that these paths and values are only available via API GET requests.
https://github.com/SignalK/signalk-to-nmea2000/pull/119
Broke out converter Promise support into it's own PR.
This should be landed before https://github.com/SignalK/signalk-to-nmea2000/pull/117
as #117 now relies on the promise support in #119.
hrm.
in signalk-to-nmea2000:
is there a way to easily test "timer" type conversions? onValueChange type conversions take their keys as arguments to the callback, so they are easy to mock.
but timer type services I'm having to do app.getSelfPath to retrieve the keys I need; which i don't know how to mock.
i was thinking to modify "timer" type services in signalk-to-nmea2000 so they could also specify a "keys" variable, like "onValueChange" conversions. Then, we resolve the keys and supply them to the callback before calling it. that way it makes the timer conversions themselves easier to mock and test.
does that make sense as a change?
Did you look at how the systemTime test works?
yeah, it sort of cheats
var dateObj = date ? date : new Date();
so it checks if the arg is null and if it is just creates new Date() object.
This leads to a lot of:
arg = arg ? arg : app.getSelfPath("thing")
to deal with testing.
my converter has 8 arguments, so it's a lot of boiler plate code.
hence the idea to move the key resolution out. i'm working on a PoC now. it works, but it doesn't solve my whole problem right now.
i've modified it so that 'timer' tests can also specify their keys, just like onValueChange tests. they're resolved externally and passed in. this means we can test them exactly the same way we test others. it also removes all the boiler plate:
arg = arg || app.getSelfPath("some.variable")
it's much cleaner.
I will PR it. it solves some of my problem but not all, but I still think it helps.
because navigation.course.activeRoute is not exposed in the data tree, we have to use app.courseApi to look it up. problem is app.courseApi is not mocked for the npm tests, so a converter that depends on the courseApi or navigation.course.activeRoute is untestable right now as far as I can tell.
@AdrianP maybe as you helped me with the course API stuff before
You could pass a dummy app object in
yeah, will have to create a mock app & courseApi
hey folks, just did some more polish on:
https://github.com/SignalK/signalk-to-nmea2000/pull/117
https://github.com/SignalK/signalk-to-nmea2000/pull/118
and broke out the other work I mentioned above into:
https://github.com/SignalK/signalk-to-nmea2000/pull/119
https://github.com/SignalK/signalk-to-nmea2000/pull/120
wondered what it might look like to get these landed please?
i'm going to start working on pgn129285 now
Initial implementation of 129285 done ✅